Agentic QA Engineer
Location
India
Posted
2 days ago
Salary
₹2,500K - ₹4,000K / year
Seniority
Mid Level
No structured requirement data.
Job Description
Agentic QA Engineer
Weekday
Role Description We are looking for an innovative Agentic QA Engineer to define and build the next generation of quality assurance practices for AI-native platforms. This is far more than a traditional QA role—you will combine software testing, automation engineering, and AI system validation to ensure platform reliability, scalability, and intelligent decision-making. You'll be responsible for validating the platform's behavior, including its AI-driven orchestration, routing, and multi-agent workflows. Rather than evaluating customer-built AI agents, your focus will be on ensuring the platform consistently makes reliable, secure, and efficient decisions under real-world conditions. Key Responsibilities - Platform Functional Testing - Design and execute end-to-end functional tests covering agent onboarding, deployment, orchestration, and request routing. - Validate platform behavior across multiple supported agent frameworks and deployment scenarios. - Test failure handling, recovery mechanisms, and edge cases throughout the platform lifecycle. - QA Automation & Reliability - Build and maintain automated regression test suites integrated into CI/CD pipelines. - Develop automated quality gates that prevent deployments when reliability metrics fall below acceptable thresholds. - Perform resilience testing by intentionally introducing failures such as service outages, database interruptions, network failures, and high traffic loads. - Define and monitor Service Level Objectives (SLOs) for platform reliability and availability. - Agentic Testing - Develop testing methodologies for AI-driven platform behavior where deterministic outputs are not possible. - Validate intelligent routing decisions across multiple AI agents. - Test multi-agent orchestration to ensure correct information flow, execution limits, cost controls, and successful task completion. - Verify platform safeguards against infinite agent loops, resource overconsumption, security risks, and cross-tenant data leakage. - Build and maintain a suite of deterministic test agents with predictable behaviors to evaluate platform decision-making accurately. - Quality Engineering - Create adversarial ("red team") testing strategies to uncover weaknesses in AI workflows and orchestration logic. - Convert execution traces into measurable platform health metrics such as routing accuracy, task success rate, execution cost, failure rates, and stuck-loop detection. - Investigate failure patterns and develop clear troubleshooting guides and operational playbooks. - Own quality assurance processes for platform releases and continuously improve testing strategies as the platform evolves. - Contribute toward developing platform testing capabilities that customers can leverage for validating their own AI agents. Qualifications - Strong experience in QA Engineering, SDET, Software Testing, or Site Reliability Engineering. - Proven expertise in building automated testing frameworks and integrating them into CI/CD pipelines. - Hands-on experience with test automation using Python or similar scripting languages. - Strong understanding of software quality principles, distributed systems, and cloud-native architectures. - Familiarity with AI/ML concepts and an understanding of how AI-powered systems behave in production environments. - Excellent analytical and problem-solving skills with an adversarial mindset for identifying edge cases and failure scenarios. - Strong communication skills with the ability to explain technical quality issues to engineering, product, and leadership teams. Preferred Qualifications - Experience working with Large Language Models (LLMs), AI agents, or autonomous AI systems. - Familiarity with agent frameworks such as LangGraph, CrewAI, AutoGen, or similar orchestration platforms. - Knowledge of distributed systems reliability and control plane architectures. - Experience with observability platforms such as OpenTelemetry, Grafana, Datadog, or similar monitoring tools. - Contributions to open-source QA, AI, or developer tooling projects. Benefits - Help define an entirely new discipline at the intersection of Quality Engineering, AI Systems, and Platform Reliability. - Build innovative testing methodologies for intelligent, AI-native platforms. - Work in a highly collaborative environment with significant ownership and technical autonomy. - Make a direct impact on the quality, scalability, and reliability of next-generation AI products. - Enjoy a flexible, remote-friendly work environment with competitive compensation and growth opportunities. Must-Have Skills - QA Automation - Agentic Testing - Test Automation - Python Good-to-Have Skills - Large Language Models (LLMs) - LangGraph - CrewAI - AutoGen - Site Reliability Engineering (SRE) - Distributed Systems - OpenTelemetry - Grafana - Datadog
Related Guides
Related Categories
Related Job Pages
More QA Automation Engineer Jobs
Role Description We are seeking a Senior Forward Deployed Engineer, Test Automation to join our Solutions Delivery team, to lead the technical design of enterprise test automation solution and lead implementation with a hands-on approach. In this forward-deployed role, you will be the technical and strategic anchor for solution delivery, involving TestPhi™ – Gradera’s AI-First Enterprise test automation platform, leading client-specific implementation, configuration & enhancements. We are looking for a seasoned professional who can strategically design solutions to integrate the platform for customers, lead the technical team, taking full ownership of the end-to-end solution delivery from strategy to production-grade deployment, independently with minimal supervision. Key Responsibilities - Understand client goals for test automation and centralized COE objectives. - Understand Gradera’s TestPhi platform capabilities, vision & be in awareness of the continuously evolving roadmap. - Assess current test automation initiatives, tool stack, processes & methodologies, at client. - Collaborate with client to design the future state for enterprise-grade test-automation framework and operating model, leveraging Gradera’s TestPhi platform. - Implement a pilot using Gradera’s TestPhi platform, configure and enhance. - Lead and scale implementation phase to at least 80% test coverage. - Define blueprint for enterprise test automation foundation, including Test data management, tool integration, CI/CD, process orchestration, quality management, reporting & governance. - Act as a technical beacon for the implementation team, performing hands-on automation tasks, conducting rigorous script reviews and setting the standard for quality engineering. - Monitor and model the maintenance plan for maximum up-time and efficiency of the test automation solution. - Create adoption strategy and intake process, to automate testing for other enterprise applications as part of long-term roadmap. - Contribute to platform development within Gradera, as needed. Qualifications - At least 8 years of professional experience in Software Quality Engineering successfully deploying test automation solutions in Enterprise environments. - At least 2 years of relevant experience with Playwright web test automation, including headless, automated test generation, and Playwright native agents. - At least 2 years of experience working with UiPath, automating tests & processes. - Must-have experience designing & implementing test automation for Enterprise commercial applications, Oracle ERP products or equivalent. - Proven ability to work independently and lead software quality initiatives with minimal supervision. - Experience operating in a Testing COE model, enabling software teams for automated testing, governance, standardization. - Experience defining quality standards & best practices, detailed review of automation tools/technologies. - Expert-level experience in Test data management, configuring for different line of businesses & environments. - Experience resolving test flakiness, increasing test execution success rate. - Experience defining approach to increasing overall test coverage – UI, API and integration tests. - Experience leading & participating different test cycles such as UAT, SIT. - Experience integrating automated tests as part of the DevOps, CI/CD process, with Azure DevOps. - Experience creating quality reports in Grafana, Allure or equivalent technologies. - Experience with mobile app test automation technologies. - Strong experience with BDD, designing solutions that leverage the BDD concepts, Gherkin format, working with business to adopt BDD practices. Additional Qualifications - Industry Experience: Experience in Energy & Utilities domain and understanding of B2B and B2C businesses process in this domain is a significant value-add.
Role Description We are seeking a talented and detail-oriented Quality Assurance Engineer (Mid-Level) to join our Agile delivery team and help ensure the successful delivery of high-quality solutions built on the SAP Business Technology Platform (SAP BTP). In this role, you will be responsible for designing, executing, and maintaining manual test scenarios and test cases while validating functional, integration, and regression testing outcomes. You will work closely with Product Owners, developers, customer stakeholders, and the Quality Manager to ensure complete test coverage, requirement traceability, and compliance with SAP quality standards and delivery practices. Key Responsibilities - Participate actively in Agile/Scrum ceremonies, including sprint planning, backlog refinement, reviews, and retrospectives. - Design, execute, and maintain manual test scenarios and test cases for SAP BTP applications and business processes. - Perform functional, integration, and regression testing to validate end-to-end business workflows. - Review and refine customer-provided test cases to align with business requirements, user stories, and acceptance criteria. - Ensure comprehensive test coverage and traceability across requirements, user stories, and testing artifacts. - Coordinate and support MIT (Integrated Testing Phase) activities, including execution tracking, defect management, and validation reporting. - Collaborate with Product Owners, developers, customer teams, and Quality Managers throughout the testing lifecycle. - Identify, document, track, and communicate defects, risks, and testing progress effectively. - Support test automation initiatives using SAP-recommended frameworks and tools. - Ensure adherence to SAP quality standards, testing methodologies, and delivery best practices. Qualifications - Proven experience in Software Quality Assurance with a strong focus on manual testing. - Solid understanding of QA methodologies, test planning, execution, defect management, and regression testing. - Experience working within Agile/Scrum delivery environments. - Strong ability to interpret business requirements, user stories, and acceptance criteria and translate them into effective test coverage. - Experience collaborating with cross-functional teams, including Product Owners, developers, and business stakeholders. - Excellent communication, analytical, documentation, and problem-solving skills. Tools & Technologies - SAP Business Technology Platform (SAP BTP) - Jira, Xray, or similar test management and traceability tools - Defect tracking and QA reporting tools - API testing tools such as Postman, Bruno, or similar - Exposure to Selenium, Cypress, or SAP-relevant test automation frameworks Nice to Have - Experience supporting test automation initiatives. - Knowledge of CI/CD pipelines and automated test integration. - Familiarity with SAP testing methodologies, SAP Activate, SAP Cloud ALM, or SAP Solution Manager. - Exposure to AI-driven testing approaches, intelligent test automation, or test generation tools. - Experience with performance testing tools such as JMeter.
Role Description We are looking for a Test Automation Engineer to make quality a first-class, automated part of how we build software. This is a high-impact opportunity to establish our test automation practice and own it end to end - building fast, reliable test suites with Playwright, embedding them into our pipelines, and giving every engineer the confidence to ship and the evidence that what they shipped works. - Own the automation framework. - Make tests part of every release. - Set the quality bar. - Protect the migration. - Test beyond the happy path. - Make quality visible. - Raise everyone’s game. - Work smart. - Build the standards. - Work as one team. Qualifications - Proven experience in a Test Automation, SDET or QA Engineering role, building and maintaining automated test suites in a modern product environment. - Strong hands-on experience with Playwright (or a close equivalent such as Cypress/Selenium, with a willingness to work in Playwright), including end-to-end and integration testing of web applications. - Solid scripting/programming ability, ideally in JavaScript/TypeScript, and comfort working closely with a React and Node.js codebase. - Experience integrating automated tests into CI/CD pipelines (e.g. GitHub Actions) and treating test code with the same rigour as production code. - A track record of raising test coverage from a low base in a pragmatic, risk-prioritised way - and of keeping suites fast and free of flakiness. - Good understanding of testing strategy across the pyramid: unit, integration, end-to-end, regression, and where each adds the most value. - Experience with API testing, test data management and test environments; exposure to performance/load testing is welcome. - A pragmatic, bias-to-action mindset - you ship meaningful coverage quickly and improve it continuously, rather than waiting for the perfect framework. - Awareness of operating within compliance frameworks such as ISO 27001 and Cyber Essentials, and why evidenced, repeatable quality matters. - Great communication skills, fluent in written and spoken English, and the ability to influence quality across a remote team. Requirements - Experience with Playwright or equivalent testing frameworks. - Proficiency in JavaScript/TypeScript. - Experience with CI/CD tools, specifically GitHub Actions. - Ability to work in a cross-functional agile team. Benefits - Remote working within the UK. - Home office budget of £200 annually. - 25 days’ annual leave, increasing with tenure, plus the option to purchase up to 5 additional days each year. - Birthday off. - Collaborative team bonus scheme. - Pension scheme access. - Perkbox salary sacrifice schemes. - Medical cashback scheme. - Enhanced parental leave. - Regular team connection opportunities, including an annual Summer Conference.
Quality Assurance Automation Engineer
Cumberland LabsEarly stage blockchain business builder and incubator from Cumberland & DRW
• You own quality and test automation at Cashen. • You will build the QA infrastructure from the ground up — automated test suites, CI integration, regression coverage, and release validation — for a platform where bugs have financial consequences. • You work directly with the CTO and the development team to ensure that every release meets the correctness bar required for institutional financial software. • As the platform expands from off-chain marketplace to on-chain DAML smart contracts, you will extend test coverage across both layers.


