Data QA Engineer

Day to Day:
We are seeking a qualified candidate for the role of QA Engineer, who will be responsible for developing and executing automated test scripts in a big data environment. The Senior Data Quality Engineer will support the development of secure, scalable data integration and analytics solutions within the People Enablement Technology team. This role focuses on validating data pipelines built using Databricks, Azure Data Factory, Snowflake, and related technologies. The ideal candidate will have a strong background in ETL testing, data validation, and automation, with a deep understanding of data governance, encryption, and automation practices.

You will collaborate closely with data engineers, architects, analysts, and business stakeholders to ensure that data solutions meet business requirements and maintain high standards of quality and compliance. This role focuses on validating data pipelines built using Databricks, Azure Data Factory, Snowflake, and related technologies. The ideal candidate will have a strong background in ETL testing, data validation, and automation, with a deep understanding of data governance, encryption, and DevSecOps practices.

You will collaborate closely with data engineers, architects, analysts, and business stakeholders to ensure that data solutions meet business requirements and maintain high standards of quality and compliance.

Principal Duties and Responsibilities:

Design and implement test strategies for complex data pipelines and workflows using Databricks, Azure Data Factory, and Snowflake
Develop and maintain automated test suites for data validation, transformation logic, and data quality checks using Python, SQL, and PySpark
Collaborate with cross-functional teams to define test cases, acceptance criteria, and quality benchmarks
Perform root cause analysis and support troubleshooting of data issues in production and non-production environments
Maintain detailed test documentation, prepare test data sets, and contribute to knowledge sharing across the QA and engineering teams

We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here.

To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .

Must Haves:
5+ years in Data Engineering with a strong understanding of ETL QA testing
Proficiency in Python and PySpark for developing and executing automated test scripts in a big data environment, along with experience in performance testing.
Strong SQL skills to perform data validation and verification during testing.
Expertise in Selenium
Familiarity with cloud platforms such as Azure, and hands-on experience with Databricks for data processing and testing.

Plusses:
Understanding of data encryption, handling sensitive data, and governance using Databricks Unity Catalog.
- Experience with People data
Power BI for visualizing and analyzing data, aiding in the identification of data quality issues and anomalies.
Experience in a retail environment

Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.