Job Description
Insight Global is seeking a Databricks Platform Services Engineer for a leading healthcare organization in the enterprise data and AI space. This individual will sit at the intersection of cloud engineering and data platform governance, owning the design, reliability, and security of a large-scale Databricks environment on Azure.
The ideal candidate brings a strong platform engineering mindset, treating infrastructure as a product with a focus on automation, scalability, and least-privilege access. They will play a key role in enabling data and ML teams by delivering a secure, well-governed, and highly observable platform. This role is perfect for someone who thrives in complex enterprise environments and enjoys building streamlined, automated solutions that balance governance with engineering agility. Day-to-Day includes:
- Design and manage Databricks platform infrastructure on Azure
- Oversee workspace lifecycle management (dev/test/prod environments)
- Implement and enforce Unity Catalog governance and data access policies
- Manage IAM, managed identities, and secure networking configurations
- Monitor platform health, reliability, and compute usage
- Build dashboards for observability and reporting (availability, job performance, cost)
- Serve as escalation point for platform issues and maintain runbooks
- Conduct access reviews and enforce compliance standards
- Partner with data, ML, and analytics teams for onboarding and support
- Automate infrastructure and workflows using Terraform and CI/CD tools
- Maintain audit logging, lineage tracking, and governance reporting pipelines
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
3–5 years of experience in data engineering, platform engineering, or similar enterprise role
2+ years of Azure cloud experience (ADF, ADLS Gen2, Azure Networking, Managed Identities, Entra ID)
2+ years of hands-on Databricks experience (workspace administration, cluster policies, Unity Catalog)
2+ years of Python development with PySpark and Databricks notebooks
Strong experience with IAM, least-privilege access, and cloud security principles
Experience managing workspace lifecycle (onboarding, configuration, promotion, decommissioning)
Knowledge of data governance, access controls, and compliance practices
Ability to work independently and drive solutions in ambiguous enterprise environments
Bachelor’s or Master’s degree in Computer Science, Information Systems, Mathematics, or related field
Nice to Have Skills & Experience
Databricks certification (Platform Administrator or Data Engineer Associate)
2+ years of Terraform / Infrastructure-as-Code experience
Experience with CI/CD pipelines (GitHub Actions or similar)
Experience implementing Unity Catalog at enterprise scale across multiple workspaces
Familiarity with data governance tools (Purview, Collibra, Alation)
Experience with Azure DevOps, Pulumi, or Crossplane
Background collaborating with cloud security teams (RBAC, private endpoints, segmentation)
Experience with cost management tools (Azure Cost Management, Databricks usage dashboards)
Strong documentation, reporting, and stakeholder communication skills
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.