The Data Engineer will be responsible for building, growing and optimizing our data architecture, including database management, data pipeline/ ETL creation and management, and overall data infrastructure management. You will also optimize data flows, maintaining, improving, cleaning, and manipulating data in operational and analytics databases, and working with analytics teams to use databases to build custom analytics for business partners. This leader will be responsible for ensuring that the client's core data reporting systems are meeting the needs of the business. This team will have a central role in strategy, requirements definition and prioritization for Data Ingestion, Data Accessibility, and Business Intelligence.
Responsibilities:
- Build and maintain large scale data structures and pipelines to organize data for new and existing projects and data products
- Build scalable infrastructure required for optimal ETL/ELT of data from a wide variety of data sources using SQL, GCP, BigQuery, Python, and AWS technologies.
- Monitor and optimize data delivery and develop tools for real-time and offline analytic as well as recommend ways to constantly improve data quality and reliability.
- Participate in the assessment, selection, and integration processes of our big data platform required to satisfy business needs ensuring that all systems meet business objectives.
- Design, construct and maintain disaster recovery procedures.
- Collaborate with business intelligence analysts to appropriately use data structures.
- Support agile project management processes in a rigorous, results driven environment.
- Collaborate in cross-divisional planning and develop accurate work level assessments and timelines and help translate product and feature needs into delivery plans.
- Partner with other data engineers and developers to insure data and event tracking specifications, review certified and ad hoc SQL and query optimization.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
HR@insightglobal.com.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
- 6+ years of experience building scalable applications using Python, SQL and GCP, and Big Query
- Experience with GCP tech stack; pulling data from different sources, maintaining history of it, and creating a semantic layer for the data
- Strong SQL and Python knowledge for data quality and validation
- Experience with real-time (RT) or near real-time (NRT) streaming data
- In depth knowledge of cloud database architectures, schema development and data modeling with tools including Big Query, Redshift and/or MongoDB
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.