We are seeking a skilled Data Engineer with a minimum of 5 years of experience developing data ingestion methods and pipelines in support of Business Intelligence (BI) solutions using Microsoft Azure, Azure Data Factory (ADF), Databricks, Python, and Spark SQL. The ideal candidate will be responsible for designing, building, and maintaining efficient data pipelines and architectures that support our data analytics and business intelligence initiatives.
**Key Responsibilities:**
- Develop and maintain data ingestion methods and pipelines using Azure Data Factory, Databricks, and Spark SQL.
- Collaborate with data scientists and analysts to understand data requirements and ensure data quality.
- Design, implement, and optimize data models for analytics and reporting using Python and Spark SQL.
- Monitor and manage data pipeline performance, ensuring reliability and scalability.
- Document processes and maintain data pipeline architecture for future reference.
- Troubleshoot and resolve issues related to data integrity and pipeline performance.
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
Human Resources Request Form. The EEOC "Know Your Rights" Poster is available
here.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
- Bachelors degree in Computer Science, Information Technology, or a related field.
- Minimum of 5-7 years of proven experience with Microsoft Azure services, especially ADF and Databricks, focusing on BI solutions.
- Strong programming skills in SQL, Python, and Spark SQL.
- Experience with data warehousing concepts and data modeling.
- Familiarity with data governance, security, and compliance best practices.
- Excellent problem-solving skills and attention to detail.
- Excellent communication skills and experience working with cross functional teams
- Experience with Azure Synapse Analytics.
- Knowledge of big data technologies.
- Familiarity with Agile methodologies.
- Any functional experience in Manufacturing/Supply Chain, Sales and Marketing, Finance, or R&D
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.