Job Description
We are seeking a Lead Software Engineer to drive large-scale data platform modernization and migration initiatives. This individual will provide technical leadership for the migration of legacy data and application workloads to a modern cloud-based architecture built on Databricks. The ideal candidate will have deep expertise in data engineering, cloud technologies, platform migrations, and software development best practices.
This role will partner with Product Owners, Architects, Data Engineers, and business stakeholders to design scalable solutions, establish engineering standards, and lead a team through the full migration lifecycle from assessment and planning through implementation and production deployment
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
8+ years of software engineering experience with progressive technical leadership responsibilities.
3+ years of hands-on experience with Databricks.
Proven experience leading large-scale data, application, or cloud migration projects.
Strong experience developing solutions using Apache Spark (PySpark or Scala Spark).
Experience building and optimizing ETL/ELT pipelines.
Strong SQL development and database experience.
Experience working with cloud platforms such as Azure, AWS, or GCP.
Expertise with modern software development practices including CI/CD, source control, automated testing, and code reviews.
Experience designing scalable, high-performance distributed systems.
Strong stakeholder communication and leadership skills.
Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
Nice to Have Skills & Experience
Azure Databricks experience in a large enterprise environment.
Experience migrating from traditional data warehouse platforms to Databricks.
Experience with Delta Lake, Unity Catalog, and Medallion architecture.
Experience with Azure Data Factory, Synapse, or similar data integration tools.
Knowledge of Power BI and enterprise analytics platforms.
Experience leading Agile development teams.
Exposure to data governance, security, and regulatory compliance frameworks.
Databricks, Azure, or cloud certifications.
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.