Job Description
The AWS Senior Data Engineer will own the planning, design, and implementation of data structures. This role will be responsible for incorporating all internal and external data sources into a robust, scalable, and comprehensive data model within AWS to support business intelligence and analytics needs throughout the company. This person must have experience leading the environment and providing maintenance support when outages occur.
• Collaborate with cross-functional teams to understand and define business intelligence needs and translate them into data modeling solutions
• Develops, builds and maintains scalable data pipelines, data schema design, and dimensional data modelling in Databricks and AWS for all system data sources, API integrations, and bespoke data ingestion files from external sources. Includes Batch and real-time pipelines.
• Responsible for data cleansing, standardization, and quality control
• Create data models that will support comprehensive data insights, business intelligence tools, and other data science initiatives
• Create data models and ETL procedures with traceability, data lineage and source control
• Design and implement data integration and data quality framework
• Implement data monitoring best practices with trigger based alerts for data processing KPIs and anomalies
• Investigate and remediate data problems, performing and documenting thorough and complete root cause analyses. Make recommendation for mitigation and prevention of future issues.
• Work with Business and IT to assess efficacy of all legacy data sources, making recommendations for migration, anonymization, archival and/or destruction.
• Continually seek to optimize performance through database indexing, query optimization, stored procedures, etc.
• Ensure compliance with data governance and data security requirements, including data life cycle management, purge and traceability
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
• At least 4 years’ experience designing and managing data pipelines, schema modeling, and data processing systems.
• Hands-on expertise with ETL tools to drive scalable data solutions.
• Proficient in Python, with a track record of solving real-world data challenges and advanced automation
• Advanced SQL skills, including experience with database design, query optimization, and stored procedures.
• Strong experience with AWS and loading data into S3 buckets
• Terraform experience
Nice to Have Skills & Experience
- Databricks
- SAP
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.