Nagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
Senior QA Automation Engineer, AI & LLM Testing
Location
Qatar
Posted
4 days ago
Salary
0
Seniority
Senior
Job Description
Senior QA Automation Engineer, AI & LLM Testing
Nagarro
• Design, develop, and maintain automated test suites for web applications, APIs, and AI-powered features. • Evaluate LLM outputs for response accuracy, relevance, consistency, and hallucination risks. • Create prompt tests and quality checks using Promptfoo or similar LLM evaluation frameworks. • Build end-to-end UI automation using Playwright, following page object models and maintainable test architecture principles. • Develop reusable Python scripts for API testing, automation utilities, and LLM quality evaluation. • Create and maintain Postman collections for REST API testing, including environments, assertions, and CI/CD integration. • Use GitHub Copilot or similar AI coding assistants to improve test coverage, speed up automation development, and identify quality gaps. • Collaborate closely with product managers, developers, and stakeholders to define test strategies, communicate risks, and improve quality practices. • Actively contribute to Agile ceremonies, including refinement sessions, sprint planning, retrospectives, and daily collaboration.
Job Requirements
- Strong experience in QA Automation and software testing.
- Hands-on experience testing AI or LLM-based features, including prompt testing, output validation, response accuracy, and hallucination mitigation.
- Experience with Promptfoo or similar LLM evaluation tools/frameworks.
- Strong proficiency in Playwright for end-to-end UI automation.
- Solid knowledge of page object patterns, reusable test components, and scalable test architecture.
- Strong Python scripting skills for automation, API helpers, and custom quality checks.
- Experience with Postman and REST API testing, including collections, assertions, environments, and CI/CD integration.
- Practical experience using GitHub Copilot or similar AI coding assistants.
- Experience working in Agile environments such as Scrum or Kanban.
- Strong communication skills, with the ability to explain test strategies and quality risks to technical and non-technical stakeholders.
- Nice to have: Experience with mobile test automation using Appium, XCUITest, or similar tools.
- Understanding of iOS and Android platform-specific testing considerations.
- Experience integrating automated tests into CI/CD pipelines.
- Familiarity with MCP or agent-assisted testing approaches in Playwright.
Benefits
- Employees can work remotely
Related Guides
Related Categories
Related Job Pages
More SDET Jobs
• Seek highly skilled Testing Tools Development Engineer to design, develop, and maintain advanced testing frameworks and automation tools. • Build scalable, reusable, and integrated testing infrastructure that supports functional, performance, and security testing across diverse platforms and environments. • Develop reusable libraries, plugins, and CLI tools to support test execution, reporting, and integration with CI/CD pipelines. • Build scalable test harnesses for distributed systems, microservices, mobile native, and cloud SaaS applications. • Write robust, maintainable code in Java, JavaScript/TypeScript to automate test workflows. • Integrate tools with version control systems (Git), CI/CD platforms (Jenkins, Harness), and artifact repositories. • Containerize tools using Docker and orchestrate test environments using Kubernetes. • Partner with QE, SRE/DevOps, and development teams to understand testing pain points and deliver tooling solutions.
• Design, develop, and maintain automated test frameworks for API and backend services using Java and RestAssured • Create and execute automated API test cases to ensure quality, reliability, and coverage • Integrate automated testing into CI/CD pipelines and support release validation processes • Collaborate with developers and stakeholders to define test strategies, quality standards, and acceptance criteria • Analyze test results, identify defects, and drive continuous improvement of QA processes and automation practices
Senior Software Development Engineer in Test, SDET 5
PhreesiaPhreesia empowers patients to take an active role in their health and achieve better outcomes.
• Lead quality for AI-powered experiences. • Design and maintain automated test strategies for Voice AI agents, LLM-powered workflows, and cloud-native applications, ensuring reliability, accuracy, scalability, and performance. • Build automation that scales. • Develop and enhance automation frameworks, integrate testing into CI/CD pipelines, and continuously improve quality processes to deliver fast, reliable feedback. • Validate complex distributed systems. • Perform end-to-end, API, UI, integration, and performance testing across microservices, event-driven architectures, and cloud platforms. • Leverage AI to improve engineering efficiency. • Use AI-assisted tools to accelerate test design, data generation, exploratory testing, coverage analysis, and defect investigation while proactively identifying quality risks. • Partner to deliver production-ready software. • Collaborate with engineers, architects, product managers, and operations teams to troubleshoot issues, monitor application health, participate in code reviews, and ensure high-quality releases. • Raise the quality bar. • Mentor junior SDETs, champion testing best practices, and drive continuous improvements across automation frameworks, tooling, and engineering processes.
Role Description Are you ready to power up your career with Hitachi Energy? We are looking for a passionate High Voltage Test Engineer to join our Transformer Services US engineering team in Earth City, MO. This role is crucial in performing high voltage testing at customer sites, ensuring the reliability and safety of our transformers. With up to 30% travel, you'll have the chance to work in diverse environments and make a significant impact. Join us and be part of a team that values innovation, safety, and integrity. - Set up and operate a mobile high voltage test trailer system. - Perform tests such as applied, induced voltage, PD localization, and temperature rise. - Conduct low voltage testing, including winding resistance, turns ratio, excitation, and power factor. - Analyze electrical testing results and calibrate test equipment. - Communicate effectively with customers and prepare detailed test reports. - Create test plans to support the proposal team. - Provide technical support to field crews and review transformer oil test results. - Uphold core values of safety and integrity in all actions. Qualifications - Bachelor's degree in engineering or related field with 5+ years of experience, or a combination of studies and 10+ years of transformer experience. - Knowledge of industry standards (IEEE, IEC, NEMA). - Proficiency in Microsoft Office 365 and technical report compilation. - Initiative, creativity, and attention to detail. - Collaborative, solutions-oriented approach with strong communication skills. - Work authorization to work for Hitachi Energy in the United States. - Willingness to travel up to 30%. Requirements - Candidate must already have work authorization that would permit them to work for Hitachi Energy in the United States. Benefits - Health Care (medical, dental, vision, etc.) - Financial Wellbeing: Employer sponsored pension - 401(k) Program with generous company match and contribution, Life/AD&D Insurance, disability insurance. - Family Care: Life and Family (Legal, pet, auto, home, identity theft, etc.), special needs support, and adoption assistance. - Work-Life: Enhanced leave programs (FMLA, Military Service Leave, Maternal, Paternal, adoption, vacation, and holiday). - Employee Engagement and Development: Employee Resource Groups (depending on location), tuition reimbursement program, on-demand learning platforms.




