Data Warehouse Engineer

Job Description

We have 2 positions for a Data Warehouse Engineer with one of our clients in Atlanta, GA for an initial contract duration of 3 months.

Client is looking for a Data Engineer to join our Team. He/she will aid in the optimization of operations by manipulating and aggregating the disparate operational and back office data sources into a format that is easily digestible by both data scientists and statistically adept colleagues. His/her core responsibility will be to combine large volumes of disparate complex data, conduct quality checks on the data, manipulate the data and ensure continuous access to a clean format of the operational data for data scientists and other stakeholders. In addition, he/she will also assist in developing the data pipeline to ensure ongoing data collection, consolidation, and management

Responsibilities

• Design and Develop data ingestion pipelines and processes based on requirements in Python
• Create error handing, exception management and data quality routines to expose the anomalies in the data
• Profile and analyze data to identify gaps and potential data quality issues
• Identifies relationships between disparate data sources
• Uses Python, Databricks and Spark to code the data Engineering routines
• Perform unit and integration testing
• Works with the group of data scientists and business SMEs to get the requirements and present the details in data.
• Designs and jointly develops the data architecture with data architect and ensures security and maintenance
• Explores suitable options, designs, and creates data pipeline (data lake / data warehouses) for specific analytical solutions
• Identifies gaps and implements solutions for data security, quality and automation of processes
• Builds data tools and products for effort automation and easy data accessibility
• Supports maintenance, bug fixing and performance analysis along data pipeline
• Diagnoses existing architecture and data maturity and identifies gaps
• Gather requirements, assess gaps and build roadmaps and architectures to help the analytics driven organization achieve its goals