Senior Databricks Engineer

Role Overview
We are looking for a senior, hands-on Databricks Engineer to support data migration initiatives, with Databricks serving as the primary orchestration tool. This role is heavily execution-focused and best suited for a strong engineer with deep data experience who is comfortable owning work end to end in a fast-paced environment.
The team is setting up and operationalizing Databricks to orchestrate ETL pipelines and database-to-database migrations, primarily pulling data from Azure SQL databases and moving it into target data platforms. This role will work alongside senior architects, taking ownership of delivery and execution.

Key Responsibilities

Design and implement Databricks-based orchestration for data migration workflows.
Build and manage ETL pipelines, moving data from one SQL database to another.
Develop and coordinate Databricks jobs using Python and SQL.
Pull and transform data from Azure SQL Database sources.
Work with Azure Data Lake environments and understand how data is stored, accessed, and migrated.
Leverage Unity Catalog (experience highly preferred) for governance and data management within the lake.
Own tasks end to end, from development through execution and completion.
Collaborate with solution architects while operating independently on hands-on delivery.
Work in a fast-paced environment, meeting deadlines and following through on execution without heavy oversight.

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Required Qualifications

8+ years of total software engineering experience.
4+ years of hands-on Databricks experience.
Strong Python background (production-level development).
Advanced SQL experience, including complex transformations and database-to-database migrations.
Solid experience working with SQL Database as a source system.
Strong understanding of ETL pipeline design and execution.
Proven ability to work independently and take full ownership of deliverables.
Strong engineering mindset with a bias toward execution and results.

Nice to Have (Not Required)

Experience with Azure Data Lake (ADLS Gen2).
Familiarity with Unity Catalog.
Prior experience using Databricks as an orchestration layer for migrations.

Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.