Job Description
Job Description:
Insight Global is seeking a Quality Engineer, AI-Native for a top FinTech / intelligent automation client. This candidate will play a critical role in validating AI-driven workflows, agent behavior, and prompt outputs in an environment where testing goes far beyond traditional QA. They will help ensure that business-critical automation systems behave reliably, even when small changes create large output variation. The ideal candidate brings strong automation testing experience, comfort with ambiguity, and a modern approach to quality engineering in AI-powered systems.
Day-to-Day:
Test and validate AI/agentic systems and business workflows
Evaluate prompt outputs for consistency, reliability, and decision accuracy
Build and execute automated test strategies for probabilistic systems
Validate database interactions, API flows, and agent behaviors across workflows
Detect inconsistencies caused by prompt or workflow changes
Measure deviations in system performance and support regression testing efforts
Partner with engineering teams to ensure AI-enabled workflows meet accuracy and quality targets
Help define scalable QA approaches for testing prompts and agent behaviors at scale
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.
Required Skills & Experience
Strong QA/SDET background with automation experience
Experience with API and UI testing
Hands-on experience using Playwright or similar modern QA automation frameworks
Ability to validate complex systems and test ambiguity/non-deterministic outputs
Experience testing database interactions and backend workflows
Understanding of prompt engineering, prompt output validation, or AI system behavior
Ability to think statistically about validation, regression, variation, and reliability
Strong communication skills around testing strategy and quality risks
Nice to Have Skills & Experience
Experience testing AI/LLM systems or agentic workflows
Familiarity with LangChain, LangGraph, or LangSmith
Experience with model evaluation, prompt testing, or AI quality monitoring at scale
Background in data validation, complex workflow testing, or intelligent automation
Understanding of regulated or business-critical workflows such as underwriting, funding, or support operations
Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.