Job Description
• Own and develop end‑to‑end data pipelines that move data from multiple internal systems into Snowflake for consumption by analytics and AI agents.
• Design, build, and maintain production‑grade Python data engineering code, including reusable libraries and services.
• Work extensively with unstructured and semi‑structured data (logs, system outputs, text‑based data) generated by hardware and test systems.
• Prepare and structure data so it can be consumed by AI agents, including supporting semantic search and downstream AI use cases.
• Collaborate closely with data engineers, software engineers, data scientists, software architects, and hardware engineers.
• Participate in system design discussions, clearly explaining trade‑offs, architectural decisions, and pipeline design choices.
• Operate daily in Linux environments, using shell tools to interact with systems, debug issues, and support data movement.
Contribute to a fast‑moving, AI‑driven initiative with a bias toward ownership, reliability, and continuous improvement.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
• 5+ years of hands‑on data engineering experience, including designing and owning end‑to‑end data pipelines in production.
• Strong Python expertise for building and maintaining production data systems (not notebook‑only or academic use).
• Experience working in AWS‑based environments, supporting data pipelines and system integrations.
• Strong Linux and shell experience; must be comfortable working daily in Linux systems (Windows‑only backgrounds not suitable).
• Experience handling unstructured and semi‑structured data, including schema drift, parsing, and normalization.
• Experience ingesting data into Snowflake and working with tables and data models.
• Strong system design fundamentals, with the ability to reason through pipeline architecture given ambiguous requirements.
• Ability to clearly explain technical decisions to both technical and non‑technical stakeholders.
• Understanding of how data is consumed by AI systems or agents (conceptual understanding is sufficient; not an ML role).
• Willingness and ability to start quickly and work in a fast‑paced, delivery‑focused environment.
Nice to Have Skills & Experience
• Experience with embeddings, vector databases, or semantic search use cases.
• Exposure to streaming or near‑real‑time data systems (e.g., Kafka), even if primarily batch‑focused.
• Familiarity with modern data DevOps practices, including CI/CD, Git‑based workflows, and infrastructure as code.
• Experience using AI tools to assist development, with the ability to review, validate, and explain AI‑generated code.
• Time‑zone flexibility to support collaboration across East Coast and West Coast teams (Central or Mountain preferred).
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.