Job Search Results for big data

Senior Data Engineer

Insight Global is looking to add a 2 Data Engineers to our client's team who are headquartered in the St. Louis area but can sit remotely. They will be responsible for moving applications from one... More environment to another in addition to ensuring every component has been upgraded and will also be responsible for modernizing the framework as well. They will need someone to come in and support development efforts across the enterprise and will interact with multiple different application owners & software vendors so communication is very important. They will be using Apache Spark, Apache Iceberg, and Apache Airflow for ETL pipelines. This person will transform and integrate EBCDIC Mainframe data into Hive and Impala tables using Precisely Connect for Big Data, optimize data transformation processes for performance, scalability, and reliability, and ensure data consistency, accuracy, and quality across the ETL pipelines. This person will function as the point of escalation for ETL loads, data quality issues, and production support issues, monitor ETL workflows and troubleshoot issues to ensure smooth production operations and implement proactive monitoring, alerting, and reporting solutions to improve pipeline reliability. This individual will be responsible for supporting and rebuilding legacy ETL jobs (currently not using ACID transactions) with modern solutions using Apache Spark and Apache Iceberg to support ACID transactions. They will be overseeing design, and implement workflows for automating data pipelines using Apache Airflow as well as establishing and enforcing best practices for ETL code development, version control, and deployment using Azure DevOps.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Labeling SME

We are seeking a data labeling SME. The Data Labeling Subject Matter Expert will be responsible for overseeing and ensuring the accuracy and quality of data labeling processes. This role requires a... More deep understanding of HIPPA, PII, and data annotation techniques, tools, and best practices. The targeted pay range is $50-65/hour.We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here. To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .Less

Remote GCP Data Engineer

Insight Global is seeking a skilled GCP Data Engineer with strong expertise in Python, SQL, and GCP/Big Query. The ideal candidate will design, develop, and maintain data pipelines, ensuring... More efficient data processing and storage. Responsibilities include optimizing data workflows, implementing data security measures, and collaborating with cross-functional teams to support data-driven decision-making. Prior experience with cloud-based data solutions and a passion for innovation are essential.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

SR. Data Engineer

A client of Insight Global, a leading telecommunications provider serving millions of customers, is seeking a Senior Data Engineer to join its Network Analytics team. This role is pivotal in... More designing and maintaining the data infrastructure that powers enterprise-scale network operations. You will lead efforts to build scalable ETL pipelines using Spark, optimize data storage solutions, and ensure data quality and availability for advanced analytics. Leveraging AWS cloud technologies, you will implement robust monitoring and alerting systems, manage deployment pipelines for continuous integration, and collaborate across teams to align data solutions with business objectives. Beyond technical execution, you will provide strategic guidance on data engineering best practices, document standards, and communicate insights effectively to stakeholders with diverse technical backgrounds. This position offers the opportunity to drive innovation, enhance operational efficiency, and influence data-driven decision-making at scale.This role will pay between $60-70 per hour based on experience.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Engineering Manager

?Lead a team of 2-5 Data Engineers with varied experience level to design and build data pipelines from various data sources to a target data warehouse using real-time and batch data load strategies... More utilizing cutting edge cloud technologies.?Serve as a people leader by directly mentoring and coaching engineers, supporting formal performance review cycles, and providing regular, actionable feedback to drive individual growth and team success.?Work with a cross functional team of business stakeholders, engineering leaders, data analysts and data scientists to formulate both business and technical requirements.?Partner with business and data leaders to understand broader business strategies and define a roadmap outlining how the team will support and enable key initiatives?Establish and enforce standards for documenting data platform designs, including logical and physical data models, metadata, ETL specifications, and end-to-end integration workflows.?Design processes for ensuring data quality and integrity while providing timely responses to ad hoc requests.?Lead and execute proof-of-concepts where appropriate to assess, validate, and improve technical processes and approachesWe are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

SR. Data Engineer- INTL India

A client of Insight Global, a leading telecommunications provider serving millions of customers, is seeking a Senior Data Engineer to join its Network Analytics team. This role is pivotal in... More designing and maintaining the data infrastructure that powers enterprise-scale network operations. You will lead efforts to build scalable ETL pipelines using Spark, optimize data storage solutions, and ensure data quality and availability for advanced analytics. Leveraging AWS cloud technologies, you will implement robust monitoring and alerting systems, manage deployment pipelines for continuous integration, and collaborate across teams to align data solutions with business objectives. Beyond technical execution, you will provide strategic guidance on data engineering best practices, document standards, and communicate insights effectively to stakeholders with diverse technical backgrounds. This position offers the opportunity to drive innovation, enhance operational efficiency, and influence data-driven decision-making at scale.This role will pay between 30-40 LPA based on experience.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

SR Data Scientist- INTL India

A client of Insight Global is a major telecommunications firm providing connectivity services to millions of customers. As part of the Network Analytics team, the Sr Data Scientist role is crucial in... More executing advanced analytic techniques to enhance the security and operational efficiency of a client of Insight Global?s enterprise networks. This position focuses on identifying anomalies that may affect employee experience (loss of connectivity, latency, etc.) and detecting insider threats or other malicious activities. The role requires expertise in statistical analysis and machine learning, utilizing generative AI to build data-driven, automated systems that enhance operational efficiency. Additionally, the position demands the ability to prioritize efforts, synthesize actionable insights, and effectively communicate findings to audiences with diverse agendas and varying levels of technical expertise. The Sr Data Scientist will manage the entire analytics life cycle, including algorithm productionalization, model validation and prototyping, data exploration and grooming, problem formulation, and requirements gathering. This role involves surveying diverse data sources?such as external APIs, flat files, relational databases, and distributed file systems?to identify analytic relevance. The Sr Data Scientist will interpret and synthesize results to drive actionable changes within the organization, collaborate across teams to integrate analytic products into production architectures, and develop, evaluate, and communicate courses of action. Additionally, the position requires teaching and explaining analytic techniques and tools to business-intelligence, data, and analytics professionals with varied backgrounds.This position will pay between 30-40 LPA based on experience.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Data Engineer

Overview:Insight Global is seeking a Data Engineer to support the migration from WhereScape to Coalesce and build scalable, production-grade data pipelines on Snowflake. This role is deeply technical... More and focused on ELT development, refactoring legacy workflows, and modernizing the data platform.Responsibilities:? Build, refactor, and optimize Snowflake ELT pipelines using Coalesce.? Translate existing WhereScape jobs into Coalesce components, ensuring logic parity and improved maintainability.? Develop efficient, reusable Python scripts for automation, data processing, and orchestration.? Implement scalable data models, transformations, and performance-optimized SQL.? Partner with architects and data engineers to align pipelines with new platform standards.? Support testing, validation, documentation, and productionizing of migrated pipelines.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

Vulnerability Data Engineer

Lead the collection, normalization, and integration of data from Palo Alto firewalls, F5 BIG-IP load balancers, and Qualys vulnerability scans. Design and build a structured SQL dataset that... More consolidates these sources into a clean, relational model for reporting and analytics. Develop ETL workflows, validate log interpretation with network/security teams, and document schemas, data flows, and source mappings to support the Client?s visibility and reporting needs.We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.Less

INTL- MEXICO Data Engineer

Insight Global is seeking a Data Engineer in Mexico for one of our top manufacturing clients. This candidate can sit anywhere fully remote in Mexico. In this role you will be joining the Corporate... More Enterprise Data Engineering Team. As an engineer you will need to have professional experience in data integration, data transformation and developing data pipelines. You will be designing data models and data storage solutions in an Azure cloud environment, need to have proficiency in Python as a programming language along with Scala and some Java, be able to build processes to support data transformation, data structures, metadata, dependency, and workload management, experience with API based data management and acquisition and communicate technical and business acumen in both Spanish and English proficiently. Day to Day: * Design data models and data storage solutions in the cloud (Azure) * Program with languages commonly used in data engineering: Python (primary), Java, or Scala * Build processes supporting data transformation, data structures, metadata, dependency, and workload management * API based data acquisition and management * Conduct data storage and processing technologies including (Hadoop, Spark, Kafka, SQL databases) * Work with data scientists to facilitate technical design of complex data sourcing, transformation, and aggregation logic, ensuring business analytics requirements are met * Work with big data pipelines to develop streaming analytics * Utilize business intelligence data visualization tools and techniques to translate business analytics into solutions * Work with parallel processing / multithreading and compute optimization * Communicate in an articulate and poised manner technical / business information Collaborate with management to prioritize business and information needsWe are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to Human Resources Request Form. The EEOC "Know Your Rights" Poster is available here. To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .Less