Job Search Results for data engineer

Insight Global partners with a large local utility company and is looking for a Principal Data Engineer to serve as a technical leader responsible for shaping, designing, and advancing... More enterprise-scale data infrastructure and data engineering solutions. This position leads complex, high-impact data and analytics initiatives and provides technical expertise across data architecture, data quality, metadata management, and platform scalability. The Principal Data Engineer works cross-functionally to translate complex business requirements into integrated data solutions, drives data engineering standards and best practices, and leads continuous improvement efforts through performance optimization and automation. While this is not a people management role, the engineer will lead virtual teams, mentor other engineers, and provide hands-on guidance for complex, multi-platform technical challenges, including the deployment of advanced analytics and machine learning solutions at scale.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Insight Global is hiring a Data Engineer to join our Fortune 100 Financial Services client in Dallas, TX. This role will be supporting the firm's research division and responsibilities will include... More horizontal enablement, data solution delivery, semantic modeling, business metrics and analytics, data visualization, software craftsmanship, and version control, and knowledge of SDLC. This is a contract role requiring 5 days onsite at their Dallas, TX location.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

?-Must have an active Secret clearance/T1 background investigation or higher-Minimum 7 years experience-Bachelor's Degree in Computer Science, Information Technology Management or Engineering, or... More other comparable degree or experience-Eligible to obtain an active Secret clearance/TI background investigation-Experience in Data/ML engineering-Strong Experience with ETL, data labeling, and data preparation-Experience with one or more of the following technologies: Cloudera, DataBricks, Snowflake, NiFi, Python, SQL, Kafka-Strong experience designing, developing, and deploying data solutions-Strong experience designing, implementing, and maintaining data architecture and services to be used for AI/ML. Additionally, operationalizing and maintaining AI/ML models in production.-Strong communication skills, ability to communicate technical concepts in both layman and technical terms to technical and non-technical stakeholdersWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

The Data Engineer is responsible for architecting, developing, and maintaining enterprise-scale data platforms that support analytics, operational reporting, and machine learning initiatives. This... More role requires deep technical expertise in distributed systems, cloud platforms, and data modeling, combined with strong communication and leadership capabilities.Design and implement large-scale ETL/ELT pipelines using Python, Spark, and distributed processing frameworksDevelop and maintain big data infrastructure leveraging Hadoop, Spark, Kafka, and Kafka StreamsArchitect cloud-native data solutions on AWS or Azure, including serverless componentsBuild and optimize Snowflake data warehouses using dimensional modeling best practicesConduct data discovery and source analysis to support new integrations and transformationsModel complex datasets using normalized, denormalized, star, and snowflake schemasIntegrate external systems through RESTful APIs and automated ingestion frameworksImplement DevOps practices including CI/CD, containerization, and orchestrationDevelop real-time streaming applications using Kafka, Storm, Kinesis, or Pub/SubOperationalize machine learning models in collaboration with data science teamsOptimize performance across queries, pipelines, and distributed workloadsTroubleshoot and resolve complex data issues across upstream and downstream systemsMaintain comprehensive documentation for data pipelines, models, and integrationsWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

A client of Insight Global is looking for an experienced Palantir Sr. Data Engineer. Senior Data Engineer- Palantir Foundry will be responsible for designing, building, and scaling enterprise data... More products and operational workflows using Palantir Foundry as part of the client's connected data initiative. The role serves as a senior individual contributor, translating complex business processes into governed, production-grade data pipelines, ontologies, and applications that enable analytics, automation, and decision support across the organization. Compensation for this role will be a range of $130,000-$150,000/yr.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

The Data Engineer is responsible for architecting, developing, and maintaining enterprise-scale data platforms that support analytics, operational reporting, and machine learning initiatives. This... More role requires deep technical expertise in distributed systems, cloud platforms, and data modeling, combined with strong communication and leadership capabilities.Design and implement large-scale ETL/ELT pipelines using Python, Spark, and distributed processing frameworksDevelop and maintain big data infrastructure leveraging Hadoop, Spark, Kafka, and Kafka StreamsArchitect cloud-native data solutions on AWS or Azure, including serverless componentsBuild and optimize Snowflake data warehouses using dimensional modeling best practicesConduct data discovery and source analysis to support new integrations and transformationsModel complex datasets using normalized, denormalized, star, and snowflake schemasIntegrate external systems through RESTful APIs and automated ingestion frameworksImplement DevOps practices including CI/CD, containerization, and orchestrationDevelop real-time streaming applications using Kafka, Storm, Kinesis, or Pub/SubOperationalize machine learning models in collaboration with data science teamsOptimize performance across queries, pipelines, and distributed workloadsTroubleshoot and resolve complex data issues across upstream and downstream systemsMaintain comprehensive documentation for data pipelines, models, and integrationsWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Pipeline Development?Develop and maintain ETL/ELT pipelines in Databricks using Python (PySpark) and SQL?Build data pipelines aligned to medallion architecture (Bronze, Silver, Gold... More layers)?Implement transformations, aggregations, and data enrichment logic to support downstream analytics?Ensure all pipelines are scalable, efficient, and production-readyWorkflow Migration (Alteryx ? Databricks)?Convert Alteryx workflows into Databricks-native pipelines?Translate legacy business logic into optimized Spark-based transformations?Support migration of both: oSimple workflows (standard transformations)oComplex workflows (multi-step logic, flat-file dependencies)?Validate output to ensure high parity with legacy systemsData Ingestion & Processing?Build and maintain ingestion pipelines for: oStructured data sourcesoFlat files and semi-structured datasets?Implement robust ingestion frameworks to handle schema changes and automate data loads?Optimize ingestion processes for performance, cost, and reliabilityTesting, Quality & Optimization?Execute unit testing, SIT, and regression testing for pipelines?Support data validation by comparing legacy vs. new system outputs?Identify and resolve defects, data discrepancies, and performance issues?Optimize pipeline performance (runtime, cost, scalability)Documentation & Best Practices?Write clear technical documentation (code comments, runbooks, README files)?Follow established coding standards, version control, and CI/CD processes?Contribute to continuous improvement of development standards and reusable frameworksCollaboration & Agile Delivery?Work within an Agile Scrum environment (2-week sprints)?Partner with Data Architects, Product teams, and SMEs to understand requirements?Participate in sprint ceremonies (standups, planning, retrospectives)?Support backlog refinement and provide technical input on implementation approachesWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

We are seeking a Data Architect / Machine Learning Advisor to join the Wildfire Consequence Modeling team. This role is ideal for a technical leader who can bridge the gap between research and... More practical implementation by combining expertise in data architecture, machine learning, cloud solution design, and modern data technologies.The ideal candidate will possess deep experience with AWS and emerging cloud-native technologies, enabling them to design scalable, secure, and production-ready solutions. They will be comfortable collaborating with academically oriented researchers while bringing a strong software engineering and architectural mindset to model development, deployment, and operationalization.Ideal Candidate ProfileProven experience architecting, developing, and deploying machine learning and data solutions within AWS cloud environments.Deep understanding of modern cloud-native architectures, emerging technologies, and best practices for scalable AI/ML platforms.Strong background in both solution architecture and hands-on machine learning engineering.Ability to evaluate business requirements and design practical, scalable, and maintainable technical solutions.Experience translating research and analytical concepts into production-grade applications and data products.Strong consultative skills with the ability to advise leadership on technology strategy, architecture decisions, and implementation approaches.Excellent communication and stakeholder management skills, with the ability to effectively partner with Product Managers, researchers, engineers, and leadership teams.Strong software engineering foundation and experience building enterprise-scale data and machine learning systems.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

This Data Engineer will serve as the technical owner of the data ingestion and transformation platform supporting a large-scale Dynamics 365 financial system implementation. The individual will be... More responsible for building and maintaining Microsoft Fabric Lakehouse pipelines, managing Dataverse/Fabric Link integrations, and ensuring enterprise data flows reliably from source systems into reporting environments. The role requires hands-on expertise with data pipeline development, monitoring, data quality validation, and Azure-based analytics technologies. This person will work closely with Business Data Analysts and BI teams to deliver scalable, well-governed data solutions that power mission-critical reporting and analytics across the organization.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Job Summary:The Data Engineer role will be responsible for implementing and managing the business's data infrastructure to support operations and strategic initiatives. This position plays a crucial... More part in developing and maintaining a robust, scalable data architecture that ensures data integrity, security, and accessibility across the organization. A typical day involves designing and building ETL/ELT pipelines, collaborating with the Data Architect and business stakeholders, ensuring high data quality, and exploring opportunities for AI and automation. Additionally, this role leverages new technologies to continually enhance the overall data ecosystem and support business growth.Only applicants living in Southern California will be considered.Essential Duties and Responsibilities: Design, build, and optimize ETL/ELT pipelines to ingest data from various sources into the data ecosystem.Implement processes and tools for data validation, cleaning, and quality assurance to maintain a reliable and accurate data foundation.Work closely with the Data Architect to design and implement a scalable and secure data infrastructure, including data lakes and data warehouses.Collaborate with business stakeholders to understand their data requirements and deliver solutions that enable data-driven decision-making.Utilize container technologies like Docker and Kubernetes to deploy scalable data services and machine learning environments.Implement and enforce data governance and security policies to ensure data privacy and compliance throughout the data lifecycle.Work with business users to assess opportunities and support AI automation with LLMs and agents.Stay current with emerging technologies and best practices in data engineering to continuously improve the data infrastructure.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less