Sr. Data Engineer

Post Date

Feb 15, 2024


North Carolina

ZIP/Postal Code

Jun 23, 2024 Insight Global

Job Type



Database Administrator (DBA)

Req #


Pay Rate

$120k - $180k (estimate)

Job Description

This position reports to the Data Partnerships, Director of Data and Analytics Platforms. This individual will be primarily responsible for the development of data integration and delivery pipelines while also expanding the FHIR-based content stored within the data lake. These solutions will capitalize on technologies to improve the value of analytical data, improve effectiveness of information stewardship, and streamline the flow of data in the organization. Solutions will focus on using state of the art data and analytics tools including traditional and near real-time integrations, big-data, and delta lake architecture using both extract, load, transform (ELT) toolsets as well as REST APIs and FHIR. The ideal candidate will also be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools.

Job Responsibilities:

* Create and maintain optimal data pipeline architecture

* Develop a data lake on Microsoft Azure using the medallion architecture leveraging a delta lake format for the silver layer

* Assemble large, complex data sets that meet functional / non-functional business requirements

* Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

* Recommend design of analytics solutions which improves data integration, data quality, and data delivery with an eye towards re-useable components

* Articulate differences, advantages, and disadvantages between architectural solution methods

* Work with Agile team members to document and execute test plans and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes

* Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported

* Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc.

* Accountable for conforming to established architectural, developmental, and operational standards and practices

* Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics

* Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs

* Evaluate and recommend development tools

* Assist in application and data operations performance tuning

* Participate in system architecture design

* Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader

* Work with data and analytics experts to strive for greater functionality in our data systems

* Share troubleshooting and maintenance duties

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to

To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: .

Required Skills & Experience

* Python development experience

* Experience implementing data lakes on Microsoft Azure

* Design and implementation experience

* Experience with data pipeline and workflow management tools such as Azure Data Factory, Synapse Analytics pipelines

* Experience with Cloud-based analytics platforms such as Azure Synapse Analytics

* HIPPA experience

Nice to Have Skills & Experience

* Experience with Healthcare Data Lakes

Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.