Job Search Results for big data

REMOTE Sr. Data Engineer

Insight Global is seeking a Sr. Data Engineer to join our actuarial consultant customer 100% remotely. In this position as a Sr. Data Engineer of our client?s Data Platform, you will be responsible... More for designing and implementing robust Data Platform solutions that meet business objectives while ensuring compliance with industry-leading data privacy standards. You will collaborate closely with cross-functional agile teams to drive data architecture decisions, implement best practices, and contribute to the success of our client?s projects. Responsibilities include: ?Data Platform: Creation of a Databricks Data Warehouse(s) and Lakehouse solutions for a healthcare data focused enterprise. oData Governance: Configuring and maintaining unity catalog to enable enterprise data lineage and data quality oData Security: Building out Data Security protocols and best practices including the management of identified and de-identified (PHI/PII) solutionsoExternal Data Products: Building data solutions for clients while upholding the best standards for reliability, quality, and performanceoETL: Building solutions within Delta Live Tables and automation of transformationsoMedallion Architecture: Building out performant enterprise-level medallion architecture(s) oStreaming and Batch Processing: Building fit-for-purpose near real-time streaming and batch solutions oLarge Data Management: Building out performant and efficient enterprise solutions for internal and external users for both structured and unstructured healthcare dataoPlatform Engineering: Building out Infrastructure as Code using Terraform and Asset BundlesoCosts: Working with the business to build cost effective and cost transparent Data solutions?Pipeline/ETL Management: You will help architect, build, and maintain robust and scalable data pipelines, monitoring, and optimizing performanceoExperience working with Migration tools i.e., AWS DMS, AWS Glue, Fivetran, integrate.iooIdentify and implement improvements to enhance data processing efficiencyoExperience with building out effective pipeline monitoring solutionsoBuild the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Delta Live Tables, Python, Scala, and cloud-based ?big data? technologies.?Data Modeling: Lead design, implementation, and maintenance of standards based (FHIR, OMOP, etc.) and efficient data models for both structured and unstructured dataoAssemble large, complex data sets that meet functional and non-functional business requirementsoDevelop and maintain data models, ensuring they align with business objectives and data privacy regulations?Collaboration: Partner internally with key stakeholders to ensure we are providing meaningful, functional, and valuable dataoEffectively work with Data, Development, Analysts, Data Science, and Business team members to gather requirements, propose, and build solutions. oCommunicate complex technical concepts to non-technical stakeholders and provide guidance on best practices.oEnsure that technology execution aligns with business strategy and provides efficient, secure solutions and systems?Processes and Tools: Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.oBuild analytics tools that utilize the data pipeline to provide actionable insights into operational efficiency and other key business performance metrics.oCreate data tools for clinical, analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Engineer

In role, the candidate is expected to develop core components for the Data Warehouse. Work in this role is data intensive and includes understanding of requirements, software development, unit... More testing, and related documentation.?Evaluate and interpret documented technical requirements and translate those into software deliverables.?Develop database software components (procedures, packages, functions, etc.) that meet documented requirements.?Identify unit test cases required to validate software components.?Execute unit test cases against software components to confirm accuracy of development.?Peer review development deliverables for team members to confirm standards and best practices.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Senior Data Engineer

Insight Global is looking to add a 2 Data Engineers to our client's team who are headquartered in the St. Louis area but can sit remotely. They will be responsible for moving applications from one... More environment to another in addition to ensuring every component has been upgraded and will also be responsible for modernizing the framework as well. They will need someone to come in and support development efforts across the enterprise and will interact with multiple different application owners & software vendors so communication is very important. They will be using Apache Spark, Apache Iceberg, and Apache Airflow for ETL pipelines. This person will transform and integrate EBCDIC Mainframe data into Hive and Impala tables using Precisely Connect for Big Data, optimize data transformation processes for performance, scalability, and reliability, and ensure data consistency, accuracy, and quality across the ETL pipelines. This person will function as the point of escalation for ETL loads, data quality issues, and production support issues, monitor ETL workflows and troubleshoot issues to ensure smooth production operations and implement proactive monitoring, alerting, and reporting solutions to improve pipeline reliability. This individual will be responsible for supporting and rebuilding legacy ETL jobs (currently not using ACID transactions) with modern solutions using Apache Spark and Apache Iceberg to support ACID transactions. They will be overseeing design, and implement workflows for automating data pipelines using Apache Airflow as well as establishing and enforcing best practices for ETL code development, version control, and deployment using Azure DevOps.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Labeling SME

We are seeking a data labeling SME. The Data Labeling Subject Matter Expert will be responsible for overseeing and ensuring the accuracy and quality of data labeling processes. This role requires a... More deep understanding of HIPPA, PII, and data annotation techniques, tools, and best practices. The targeted pay range is $50-65/hour.We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here. To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .Less

REMOTE Staff Data Engineer

Insight Global is seeking a Staff Data Engineer to join our actuarial consultant customer 100% remotely. In this position as a Staff Data Engineer of our client?s Data Platform, you will be... More responsible for designing and implementing robust Data Platform solutions that meet business objectives while ensuring compliance with industry-leading data privacy standards. You will collaborate closely with cross-functional agile teams to drive data architecture decisions, implement best practices, and contribute to the success of our client?s projects. What you will be doing:?Acts as a subject matter expert and thought leader within the Data Platform Domain?Data Strategy: Serves as a thought leader in data processing design and implementation, defining advanced structure for moving, storing, and maintaining high-quality data.?Team Leadership: Leads projects by managing timelines, coordinating teams, and communicating project statuses. Influences organizational direction through effective leadership and strategic collaboration?Data Governance and Security: Serves as a subject matter expert on governance standards, continuously aligning data practices with evolving industry best practices and requirements?Project Management and Scope of Work: Contributes to defining the overall vision and strategy for data engineering within the organization, ensuring alignment with organizational goals and long-term objectives?Results Orientation: Establishes visionary goals, advises on strategic plans, employs advanced monitoring, influences high-level stakeholders, and delivers transformative results?Data Platform: Expansion of our Data Warehouse(s) and Lakehouse solutions for a healthcare data focused enterprise?Data Governance: Configuring and maintaining unity catalog to enable enterprise data lineage, data quality, auditability and data stewardship?Data Security: Building out Data Security protocols and best practices including the management of identified and de-identified (PHI/PII) solutions?Access Management: Always ensure a policy of least privilege is followed for anything being implemented?External Data Products: Building data solutions for clients while upholding the best standards for reliability, quality, and performance?ETL: Building solutions within Delta Live Tables and automation of transformations?Medallion Architecture: Building out performant enterprise-level medallion architecture(s) ?Streaming and Batch Processing: Building fit-for-purpose near real-time streaming and batch solutions ?Large Data Management: Building out performant and efficient enterprise solutions for internal and external users for both structured and unstructured healthcare data?Platform Engineering: Building out Infrastructure as Code using Terraform and Asset Bundles?Costs: Working with the business to build cost effective and cost transparent Data solutions?Pipeline/ETL Management: You will help architect, build, and maintain robust and scalable data pipelines, monitoring, and optimizing performance?Experience working with Migration tools i.e. Fivetran, AWS technologies and custom solutions?Identify and implement improvements to enhance data processing efficiency?Design and implement reliable and resilient Event Driven data processing?Experience with building out effective pipeline monitoring solutions?Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Delta Live Tables, Python, Scala, and cloud based ?big data? technologies?API Development: Drive our design and implementation of internal APIs for integrating data between different systems and applications?Integration with external systems utilizing API driven processes to ingest data?Develop APIs built on top of datasets for internal systems to consume data from Databricks?Experience integrating with external APIs including but not limited to Salesforce, Financial systems, HR systems and other external systems?Data Modeling: Lead design, implementation, and maintenance of standards based (FHIR, OMOP, etc.) and efficient data models for both structured and unstructured data?Assemble large, complex data sets that meet functional and non-functional business requirements?Develop and maintain data models, ensuring they align with business objectives and data privacy regulations?Collaboration: Partner internally and externally with key stakeholders to ensure we are providing meaningful, functional, and valuable data?Effectively work with Data, Development, Analysts, Data Science, and Business team members to gather requirements, propose, and build solutions. ?Communicate complex technical concepts to non-technical stakeholders and provide guidance on best practices.?Ensure that technology execution aligns with business strategy and provides efficient, secure solutions and systems?Gather requirements and build out project plans to implement those requirements with forecasted efforts to implement?Processes and Tools: Identify, design, and implement internal process improvements: oautomating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.?Build analytics tools that utilize the data pipeline to provide actionable insights into operational efficiency and other key business performance metrics.?Create data tools for clinical, analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.?Lead investigation of new tooling, develop implementation plans, and deployment of necessary toolingWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

INTL - MX - GCP Data Engineer

Day to Day:In this role, you?ll leverage your expertise in GCP and data engineering to modernize legacy applications and build scalable cloud analytics platforms. You?ll collaborate with global... More engineering teams to define and implement enterprise data strategies, working closely with stakeholders to align solutions with business needs and regulatory requirements. Your responsibilities will include designing and delivering a unified data platform on GCP, guiding teams on data modeling and architecture, and advising on best practices for cloud security, DevOps, and data mesh. You?ll also support proof-of-concepts, product evaluations, and integration efforts, while enabling insights through AI/ML platforms and modern data solutions.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

INTL - MX - GCP Data Engineer

Day to Day:In this role, you?ll leverage your expertise in GCP and data engineering to modernize legacy applications and build scalable cloud analytics platforms. You?ll collaborate with global... More engineering teams to define and implement enterprise data strategies, working closely with stakeholders to align solutions with business needs and regulatory requirements. Your responsibilities will include designing and delivering a unified data platform on GCP, guiding teams on data modeling and architecture, and advising on best practices for cloud security, DevOps, and data mesh. You?ll also support proof-of-concepts, product evaluations, and integration efforts, while enabling insights through AI/ML platforms and modern data solutions.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Senior Data Engineer

The client is seeking a Data Engineer to join their fast-growing data team and play a critical role in building, maintaining, and scaling a modern AWS-based data lake house. This individual will... More focus heavily on ingesting, transforming, and modeling geospatial and tabular data?bringing spatial data such as points, polygons, routes, and raster datasets into production-grade pipelines and databases rather than working solely within desktop GIS tools. The Data Engineer will design and maintain robust data pipelines, develop spatially aware data models, and partner closely with cross-functional teams to deliver reliable, scalable data solutions while leading technical initiatives in an Agile environment. The salary range for this position will range between 100-115K+ annually once converted from contract depending on level of experience.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Engineering Manager

?Lead a team of 2-5 Data Engineers with varied experience level to design and build data pipelines from various data sources to a target data warehouse using real-time and batch data load strategies... More utilizing cutting edge cloud technologies.?Serve as a people leader by directly mentoring and coaching engineers, supporting formal performance review cycles, and providing regular, actionable feedback to drive individual growth and team success.?Work with a cross functional team of business stakeholders, engineering leaders, data analysts and data scientists to formulate both business and technical requirements.?Partner with business and data leaders to understand broader business strategies and define a roadmap outlining how the team will support and enable key initiatives?Establish and enforce standards for documenting data platform designs, including logical and physical data models, metadata, ETL specifications, and end-to-end integration workflows.?Design processes for ensuring data quality and integrity while providing timely responses to ad hoc requests.?Lead and execute proof-of-concepts where appropriate to assess, validate, and improve technical processes and approachesWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Administrator

A Fortune 500 company is looking for a remote Data Administrator to support the organization's Retail Food Safety Operations team. This role focuses on keeping our food safety operations running... More smoothly by validating and monitoring data across multiple systems to make sure everything is accurate, consistent, and useful for business decisions. Day to day, you?d be managing and organizing our SharePoint sites, Teams channels, Outlook workflows, and document repositories so that critical food safety documentation, compliance records, and operational standards are easy to find, properly categorized, and aligned with corporate requirements. You?d support a fully remote team of 22 regional food safety and public health advisors, helping them collaborate efficiently, reference historical materials, and maintain clean, structured information libraries. A big part of the work involves auditing processes, organizing and labeling documents, and making sure our Non-Partner Workforce?related materials and accommodations are categorized correctly?without needing to be the subject-matter expert on the content itself. Overall, it?s a blend of technical administration, process upkeep, and thoughtful information organization to ensure the team has what they need at their fingertips.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less