Insight Global is seeking a Data Engineer to join a large retail client on a remote 6 month contract. As a Data Engineer, you'll join a large retail client on their Data and Analytics team which play a key role in developing reporting and analytics for key business units. The data engineer will join the Data Platform Squad which is currently supporting the generation of a large data lake to host guest data. This data lake will store guest data from various sources such as Salesforce, social media campaigns on Facebook, tik tok, etc, and clickstream data from their website.
On the day to day, the Data Engineer will take data from various sources, clean and ingest it to get onto the Azure data platforms in order to produce data models for the business stakeholders. They will be designing and implementing complex data pipelines from different internal/external sources using the Microsoft stack and Azure BI services. Leveraging Databricks Unity Catalog, they will also manage and govern machine learning models and associated data assets. There will also be close collaboration with data architects to transform requirements into code and prepare database structures needed for reporting, analytics, and visualizations. They will also create recommendations to advise on optimal data strategies for data ingestion, integration, and visualization to improve code performance and query optimization to meet demanding data analytics and visualization needs.
We are a company committed to creating inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity employer that believes everyone matters. Qualified candidates will receive consideration for employment opportunities without regard to race, religion, sex, age, marital status, national origin, sexual orientation, citizenship status, disability, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
Human Resources Request Form. The EEOC "Know Your Rights" Poster is available
here.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
- 5+ years of technical development experience on Microsoft Azure.
- Extensive experience with Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Proficient with Databricks Unity Catalog for model and data asset management.
- Experience in writing and optimizing complex SQLs, and scripting in python
- Comprehensive understanding of ETL and ELT processes with the ability to implement end-to-end solution flows for complex requirements.
- Ability to create intricate data flow diagrams integrating multiple systems with data platforms for product creation.
- Experience integrating NoSQL databases into data pipelines.
- Experience optimizing machine learning model training and deployment pipelines
- Experience w/retail and/or sporting goods industry experience.
- Experience working with digital customer data such as clickstream data (aka did that click lead to a purchase did it not), social media data (aka did that campaign engagement lead to a purchase or did it not)
- Machine Learning Predictive Analysis Experience (for example with SparkMLib)
- Experience with Access Control Lists (ACLs) for Data Security
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.