Job Description
The Senior Data Engineer / Streaming Data Engineer will design, build, and support enterprise-scale streaming and big data pipelines across AWS, Spark, Hadoop, Kafka, and cloud-native ingestion platforms. This role is hands-on and production-focused, with responsibility for reliable real-time data movement, ingestion modernization, distributed systems engineering, and scalable data processing in a large enterprise environment.
• Hands-on streaming and real-time data engineering using Kafka, Spark, AWS Kinesis, and cloud-native data services.
• Modernization opportunity focused on moving legacy ingestion patterns to scalable AWS-native services.
• Production engineering role requiring strong troubleshooting, operational ownership, and distributed systems depth.
• Enterprise-scale environment with complex data warehousing, big data, and cross-platform integration needs.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
• 10+ years of experience in data engineering, software engineering, or related fields.
• Strong expertise with AWS cloud platform services and cloud-native data engineering patterns.
• Hands-on experience with Apache Spark using Scala and PySpark for large-scale data processing.
• Deep working knowledge of the Hadoop ecosystem and distributed data processing architectures.
• Strong experience with Kafka and streaming technologies, including real-time data pipeline design and support.
• Hands-on experience with data ingestion platforms such as Flume, AWS Kinesis, Kinesis Firehose, or similar tooling.
• Strong programming experience in Python, Scala, and Java.
• Deep understanding of enterprise data warehousing, big data architectures, distributed systems, and large-scale enterprise operating environments.
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.