• Define and lead the end-to-end testing strategy for Outreach’s GenAI platform, including agentic workflows, LLM tool calls, LangGraph orchestration, and supporting ML pipelines. • Design and implement evaluation systems that handle both deterministic and non-deterministic outputs. • Own testing across Outreach’s suite of AI agents. • Work closely with Data Science, MLOps, and platform engineers to ensure testability is designed in from the start. • Integrate evaluation pipelines into CI/CD workflows. • Establish and track metrics that matter for AI systems: answer quality scores, tool invocation accuracy, hallucination rates, latency, and regression trends over model and prompt changes. • Define standards for AI testing across the org — including prompt regression testing, retrieval quality evaluation, and agent behavior contracts. • Raise the quality bar across engineering teams by mentoring engineers. • Actively track developments in AI evaluation tooling, LLM benchmarking, and testing research.

• Own end‑to‑end quality for assigned features, components, or services • Design and execute functional, integration, system, and regression testing • Perform exploratory testing to uncover edge cases and failure scenarios • Identify, document, and track defects with clear reproduction steps and impact analysis • Prioritize effectively, stay focused, and drive tasks through to completion • Design, develop, and maintain automated test cases for APIs, backend systems, and/or UI • Actively participate in CI/CD pipelines to support continuous testing • Work closely with developers, product managers, and other QA engineers • Participate in requirement and design reviews to ensure testability and risk coverage • Collaborate with developers to debug issues and validate fixes • Analyze test results and provide quality insights to support release decisions • Contribute to improving test coverage, tools, and processes

• Design and maintain test cases and test suites for product features • Participate in agile ceremonies including discovery, refinement, decomposition, sprint planning, and demos • Design, develop, and maintain automated test suites for APIs and mobile UI on iOS and Android platforms • Implement shift-left testing practices • Build and maintain test frameworks and tools for QA engineers and integrate them into CI/CD pipelines • Ensure the reliability and stability of testing infrastructure • Collaborate with TechOps and other teams to improve quality management processes and testing workflows • Investigate production issues and perform root cause analysis • Collaborate with backend and mobile developers to prevent regressions • Track quality metrics and automate routine QA activities • Advocate for product testability and mentor QA team members

• Design, implement, and maintain a comprehensive automation framework for functional, regression, and integration testing. • Build automated test scripts for APIs, UI workflows, data pipelines, and AI agent interactions using tools such as PyTest or Selenium. • Integrate automated tests into CI/CD pipelines (Jenkins, GitHub Actions, or Cloud Build) to support continuous delivery. • Ensure test scripts cover both core platform functionality and partner integrations (Veeva Vault, Vertex AI, Snowflake). • Develop and execute test cases for new features, enhancements, and bug fixes. • Conduct regression testing prior to each release to ensure platform stability. • Work with the AI Validation and Compliance teams to evaluate AI output quality (accuracy, hallucination, tone, compliance alignment). • Track and report system reliability metrics (latency, failure rate, API response times).