Job Description
Insight Global is seeking a Data Pipeline Engineer to support Palantir Foundry implementations for U.S. Government customers. This role is a hybrid Data Engineer / Software Engineer / Incident Response position focused heavily on maintaining the health and reliability of production data pipelines across multiple customer environments.
The day-to-day work is predominantly incident-driven, with engineers spending the majority of their time responding to data health failures, debugging pipeline issues, and identifying root causes across complex, large-scale data systems. While some development work is involved, it is primarily focused on patching, optimizing, and refactoring existing pipelines rather than building net-new solutions.
This individual will support hundreds of pipelines across multiple customer stacks and must be comfortable navigating unfamiliar systems, diagnosing issues quickly, and collaborating with internal teams, data providers, and government stakeholders. Success in this role requires strong problem-solving skills, comfort working in production environments, and the ability to take ownership of incidents end-to-end.
Key Responsibilities:
Respond to, triage, and debug data pipeline failures and incidents across multiple customer stacks (primary responsibility)
Monitor pipeline health, review alerts, and ensure timely response to data issues (targeting first review within 24 hours)
Analyze root causes of failures (e.g., data quality issues, pipeline breaks, infrastructure constraints, OOM errors) and implement fixes where applicable
Read, modify, and patch existing code (primarily Python/PySpark) to restore pipeline functionality and improve stability
Support and maintain pipeline schedules, re-run failed jobs, and ensure proper data delivery to downstream systems
Coordinate with upstream data providers and internal teams to resolve ingestion issues, certificate expirations, or connectivity problems
Escalate issues when ownership or permissions fall outside of scope, while maintaining accountability for resolution
Communicate pipeline outages, data inconsistencies, and resolutions to stakeholders as needed
Contribute to ongoing improvements in monitoring, alerting, and pipeline reliability
Pay Range: Pay will be hourly and varies by level of experience. Average Range: ~$55–99/hr. Exact compensation may vary based on several factors, including skills, experience, and education. Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
Active DoD Secret Security Clearance or higher
5–10+ years of experience in incident response, site reliability, application support, or data engineering environments
Strong hands-on experience with data pipelines, ETL/ELT processes, and distributed data systems
Strong coding experience in Python, with PySpark highly preferred
Familiarity with debugging and optimizing Spark-based pipelines, including handling memory issues (e.g., OOM errors)
Experience performing root cause analysis and documenting production incidents
Understanding of relational databases, data ingestion patterns, and dataframe manipulation
Basic knowledge of networking concepts and REST APIs (for troubleshooting external data sources)
Nice to Have Skills & Experience
Experience with Palantir Foundry or similar data platforms
Experience with distributed compute frameworks and large-scale data processing
Familiarity with visual ETL or pipeline tools (e.g., Microsoft Fabric, Informatica, Dataiku, etc.)
Prior experience supporting customer-facing data platforms or products
Background in government, defense, or cleared environments
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.