Insight Global is looking to add 2 Data Engineers to our client's team who are headquartered in the St. Louis area but can sit remotely. This individual will be working with a team of 2 Data Engineers and a hands on Lead Data Engineer. This individual will be supporting a brand new project standing up an Enterprise Data warehouse. They will be responsible for modernizing ETL processes within an on-premises Cloudera Data Platform (CDP) environment, leveraging technologies like Apache Spark, Apache Iceberg, and Apache Airflow for scalable, efficient, and reliable data transformation and management. Daily, they will be responsible for ETL development and troubleshooting needs, along with participating in production support environments. They will contribute to development efforts for ETL pipelines in the Enterprise Data Warehouse (EDW). They will support and rebuild legacy ETL jobs (currently not using ACID transactions) with modern solutions using Apache Spark and Apache Iceberg to support ACID transactions. This person will transform and integrate EBCDIC Mainframe data into Hive and Impala tables using Precisely Connect for Big Data. This individual will optimize data transformation processes for performance, scalability, and reliability and ensure data consistency, accuracy, and quality across the ETL pipelines. Development responsibilities will be 50% of this role. Additionally, this person will be sharing weekly 24/7 production support with the managed service vendor on a 4-week rotation, monitoring, ETL workflows and troubleshooting issues to ensure smooth production operations along with research and resolve user requests and issues. Production support is 20% of this role.
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
Human Resources Request Form. The EEOC "Know Your Rights" Poster is available
here.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
Hands-on experience with Cloudera Data Platform (CDP), including Hive and Impala
Extensive knowledge of Precisely Connect for Big Data or similar tools for mainframe data transformation.
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.