Insight Global is seeking a Senior Data Engineer to join a global housing market data insights and solutions firm. As a Senior Data Engineer, you will play a crucial role in designing, implementing, and optimizing our data infrastructure. You will be embedded in an application scrum team working on a large-scale application. Your expertise in SQL Server query optimization, Snowflake, and performance tuning will be instrumental in ensuring the efficiency and reliability of our data systems.
Database Design and Development:
Design and implement robust, scalable, and performance-optimized database solutions using SQL Server and Snowflake.
Develop and maintain database procedures, schemas, tables, indexes, and views to support application requirements.
SQL Server Query Optimization:
Analyze and optimize SQL queries for maximum efficiency and performance.
Identify and resolve performance bottlenecks in database systems through the use of query plans.
Collaborate with application developers to enhance query performance and database responsiveness.
Data Modeling:
Work closely with data engineering to design and implement data models that align with business requirements.
Ensure data integrity and consistency across the database environment.
ETL Processes:
Design, implement, and maintain ETL processes for efficient data extraction, transformation, and loading.
Collaborate with cross-functional teams to integrate data from various sources into the SQL Server database.
Monitoring and Maintenance:
Implement effective monitoring solutions to proactively identify and address potential issues.
Perform routine maintenance tasks such as backups, restores, data purges, and index de-fragmentation.
Experience with Unix and Powershell scripting technologies (running back scripts, basic) --> SSIS for defining ETL pipelines so has call command lines and wil lcall from powershel, backfile, move datafile into S3, scripting simple stuff, not advanced
Collaboration and Documentation:
Collaborate with cross-functional teams, including data scientists, product analysts, and application developers, to understand data requirements.
Participate in architectural discussions and technical decision-making processes.
Effectively communicate technical concepts to non-technical stakeholders.
Document database design, configurations, and procedures.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to
HR@insightglobal.com.
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:
https://insightglobal.com/workforce-privacy-policy/ .
Bachelor's degree in Computer Science, Information Technology, or a related field.
Proven experience as a Data Engineer with a focus on SQL Server and Data Warehouses like Snowflake.
In-depth knowledge of SQL Server database design, query optimization, and performance tuning.
Proficiency in writing complex SQL queries and stored procedures.
Experience with database scalability and writing deep queries accessing large datasets
Experience with ETL tools and processes (eg. SSIS (must have but its going away), Airflow, AWS Glue, Talend, or similar).
Experience with cloud technologies (S3 [AWS], EC2, Linux [some exposure, not looking for admin, some scripting in linux just using it for deployment])
Experience with GitHub and CICD systems
Experience with database security best practices.
Excellent problem-solving and analytical skills.
Effective and proactive communication skills and ability to work collaboratively in a team environment.
Knowledge of cloud-based database solutions (e.g., Amazon RDS (SQL Server), Azure SQL Database).
Experience with cloud Orchestration tools (e.g. Airflow, Talend, ADF, etc)
Experience with other database technologies (e.g., MySQL, PostgreSQL).
Certification in SQL Server or related technologies.
Experience of Geo Spatial data and queries
Citusdata
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.