Job Description
A Consulting client of Insight Global is hiring for a Senior Data Engineer to join their team. This Data Engineer will work on a product development team responsible for data lakes and architecture of a healthcare application in an AWS environment. Responsibilities include:
- Provide innovative AWS data lake architecture solutions and recommendations
- Setup, configure, and troubleshoot AWS data migration services
- Catalog, setup, configure, and troubleshoot Glue or DBT - Perform heavy ETL work - Create tables and manage a Hudi database
- Provide SQS development
- Work extensively with Event Bridge Schedule
- Understand creation of tables/views/MV’s, and provide performance and tuning within RedShift
- Work within Aurora MySQL database management:
- Develop by writing Stored Procedures
- Setup Read replication
- Troubleshoot and monitor AWS resources for performance improvement
This is a fully remote role, ideally starting ASAP. The pay rate for this role varies depending on level and years of experience and will range from 27-36 LAKH.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
- 7-10+ years of data engineer experience
- Strong hands on engineering experience designing and operating Data Lakes on AWS
- AWS Glue or DBT experience
- Setup and configuration of Glue Catalog/metadata management
- Troubleshooting schema, crawler, and catalog issues
- Strong developer skills for building scalable ETL/ELT pipelines including job setup, configuration, and orchestration
- Amazon Redshift experience with development and operations
- AWS SQS - designing and -developing message-driven data pipelines
- Amazon Event Bridge Schedule experience for event-driven orchestration of data workflows and scheduling and dependency management.
- Experience owning end-to-end monitoring and optimization of AWS-based data pipelines, including performance, cost, and reliability, with strong root-cause analysis, proactive logging, alerting, and continuous performance improvements.
Nice to Have Skills & Experience
- AWS data migration services
- Apache Hudi database experence
- AWS Aurora MySQL database management
- Python or Scala experience
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.