Role Description At Hupo, AI quality is the product - a wrong score, hallucinated feedback, or a broken voice turn is a customer-facing defect, and in a regulated BFSI context, a compliance risk. This role owns and scales the testing of Hupo’s AI pipelines, working closely with QA, Engineering, and Product as the platform grows. It’s a hands-on, high-ownership role in a fast-moving environment - you’ll be a key part of deciding, with evidence, whether each release is safe to ship. What You'll Do - Build and maintain automated testing for AI voice and chat agents, from single conversational turns to full roleplay flows. - Test LLM output quality - correctness, consistency, structured output, language fidelity, prompt regression - using evaluation harnesses (LLM-as-judge, golden datasets, tolerance-based assertions for non-determinism). - Validate RAG pipelines: retrieval relevance, grounding/faithfulness, and answer quality. - Test voice pipelines: STT/TTS accuracy and real-time, low-latency behavior across languages. - Automate across UI/E2E, API, and AI output layers, and build CI/CD from scratch - lint, type-check, tests, evals, coverage gates, deployment checks. - Own release quality: regression strategy, catching breakages early, and clear go/no-go calls. - Cover database and load/performance testing; extend automation to desktop and mobile clients. - Take over and extend existing QA automation, and strengthen shared frameworks, tooling, and QA processes/standards. What Success Looks Like Within your first few months, you'll: - Ramp on Hupo’s existing QA automation suites and take clear ownership of them. - Extend test coverage for LLM and voice-agent behavior, including new evaluation harnesses for non-deterministic output. - Contribute CI/CD improvements - coverage gates, deployment checks - that catch regressions before they reach customers. You’ll also be expected to continuously: - Adapt and expand impact as priorities shift. - Take ownership of new problems as they emerge. Qualifications - 5+ years as a QA Automation Engineer, with a proven track record testing AI systems - not just traditional software. - Builder mindset: established testing frameworks, standards, and CI/CD-integrated automation from scratch. - Hands-on experience testing conversational AI - voice and/or chat agents - on real, shipped projects. - Strong grasp of LLMs, prompt engineering, and RAG - designing tests for nondeterministic output and evaluating retrieval/generation quality. - Understanding of voice pipelines (speech-to-text, text-to-speech) and how to test them in automation. - Hands-on with LLM evaluation/observability tooling - e.g. Langfuse, LangSmith, DeepEval, RAGAS. - Proficient in UI/E2E (e.g. Playwright) and API test automation; experience building CI/CD pipelines from scratch. - Comfort with database testing, load/performance testing, and desktop/mobile test automation. - Solid Git discipline, basic cloud knowledge, and familiarity with the agile sprint lifecycle. - Track record in fast-paced, fast-shipping environments - ramps quickly on unfamiliar systems, pragmatic about process. Nice-to-Haves - TypeScript - for reading the codebase and contributing test/automation code directly. - Voice & real-time: LiveKit / WebRTC; STT/TTS via Azure Speech, Google Cloud Speech, Deepgram, or self-hosted Whisper. - AI/ML infra & eval tooling: GPU inference (vLLM / TGI / Triton), Azure OpenAI, golden datasets, HITL pipelines, Braintrust, Promptfoo. - QA & cloud tooling: Cypress, Postman, multi-cloud (AWS/Azure/GCP), Infrastructure-asCode (Terraform/Pulumi/CloudFormation), Docker/Kubernetes. - Also useful: product analytics familiarity (e.g. PostHog), multilingual/localized testing experience, and production monitoring/error-tracking tools.

Role Description We are looking for an innovative Agentic QA Engineer to define and build the next generation of quality assurance practices for AI-native platforms. This is far more than a traditional QA role—you will combine software testing, automation engineering, and AI system validation to ensure platform reliability, scalability, and intelligent decision-making. You'll be responsible for validating the platform's behavior, including its AI-driven orchestration, routing, and multi-agent workflows. Rather than evaluating customer-built AI agents, your focus will be on ensuring the platform consistently makes reliable, secure, and efficient decisions under real-world conditions. Key Responsibilities - Platform Functional Testing - Design and execute end-to-end functional tests covering agent onboarding, deployment, orchestration, and request routing. - Validate platform behavior across multiple supported agent frameworks and deployment scenarios. - Test failure handling, recovery mechanisms, and edge cases throughout the platform lifecycle. - QA Automation & Reliability - Build and maintain automated regression test suites integrated into CI/CD pipelines. - Develop automated quality gates that prevent deployments when reliability metrics fall below acceptable thresholds. - Perform resilience testing by intentionally introducing failures such as service outages, database interruptions, network failures, and high traffic loads. - Define and monitor Service Level Objectives (SLOs) for platform reliability and availability. - Agentic Testing - Develop testing methodologies for AI-driven platform behavior where deterministic outputs are not possible. - Validate intelligent routing decisions across multiple AI agents. - Test multi-agent orchestration to ensure correct information flow, execution limits, cost controls, and successful task completion. - Verify platform safeguards against infinite agent loops, resource overconsumption, security risks, and cross-tenant data leakage. - Build and maintain a suite of deterministic test agents with predictable behaviors to evaluate platform decision-making accurately. - Quality Engineering - Create adversarial ("red team") testing strategies to uncover weaknesses in AI workflows and orchestration logic. - Convert execution traces into measurable platform health metrics such as routing accuracy, task success rate, execution cost, failure rates, and stuck-loop detection. - Investigate failure patterns and develop clear troubleshooting guides and operational playbooks. - Own quality assurance processes for platform releases and continuously improve testing strategies as the platform evolves. - Contribute toward developing platform testing capabilities that customers can leverage for validating their own AI agents. Qualifications - Strong experience in QA Engineering, SDET, Software Testing, or Site Reliability Engineering. - Proven expertise in building automated testing frameworks and integrating them into CI/CD pipelines. - Hands-on experience with test automation using Python or similar scripting languages. - Strong understanding of software quality principles, distributed systems, and cloud-native architectures. - Familiarity with AI/ML concepts and an understanding of how AI-powered systems behave in production environments. - Excellent analytical and problem-solving skills with an adversarial mindset for identifying edge cases and failure scenarios. - Strong communication skills with the ability to explain technical quality issues to engineering, product, and leadership teams. Preferred Qualifications - Experience working with Large Language Models (LLMs), AI agents, or autonomous AI systems. - Familiarity with agent frameworks such as LangGraph, CrewAI, AutoGen, or similar orchestration platforms. - Knowledge of distributed systems reliability and control plane architectures. - Experience with observability platforms such as OpenTelemetry, Grafana, Datadog, or similar monitoring tools. - Contributions to open-source QA, AI, or developer tooling projects. Benefits - Help define an entirely new discipline at the intersection of Quality Engineering, AI Systems, and Platform Reliability. - Build innovative testing methodologies for intelligent, AI-native platforms. - Work in a highly collaborative environment with significant ownership and technical autonomy. - Make a direct impact on the quality, scalability, and reliability of next-generation AI products. - Enjoy a flexible, remote-friendly work environment with competitive compensation and growth opportunities. Must-Have Skills - QA Automation - Agentic Testing - Test Automation - Python Good-to-Have Skills - Large Language Models (LLMs) - LangGraph - CrewAI - AutoGen - Site Reliability Engineering (SRE) - Distributed Systems - OpenTelemetry - Grafana - Datadog

Role Description We are seeking a Senior Forward Deployed Engineer, Test Automation to join our Solutions Delivery team, to lead the technical design of enterprise test automation solution and lead implementation with a hands-on approach. In this forward-deployed role, you will be the technical and strategic anchor for solution delivery, involving TestPhi™ – Gradera’s AI-First Enterprise test automation platform, leading client-specific implementation, configuration & enhancements. We are looking for a seasoned professional who can strategically design solutions to integrate the platform for customers, lead the technical team, taking full ownership of the end-to-end solution delivery from strategy to production-grade deployment, independently with minimal supervision. Key Responsibilities - Understand client goals for test automation and centralized COE objectives. - Understand Gradera’s TestPhi platform capabilities, vision & be in awareness of the continuously evolving roadmap. - Assess current test automation initiatives, tool stack, processes & methodologies, at client. - Collaborate with client to design the future state for enterprise-grade test-automation framework and operating model, leveraging Gradera’s TestPhi platform. - Implement a pilot using Gradera’s TestPhi platform, configure and enhance. - Lead and scale implementation phase to at least 80% test coverage. - Define blueprint for enterprise test automation foundation, including Test data management, tool integration, CI/CD, process orchestration, quality management, reporting & governance. - Act as a technical beacon for the implementation team, performing hands-on automation tasks, conducting rigorous script reviews and setting the standard for quality engineering. - Monitor and model the maintenance plan for maximum up-time and efficiency of the test automation solution. - Create adoption strategy and intake process, to automate testing for other enterprise applications as part of long-term roadmap. - Contribute to platform development within Gradera, as needed. Qualifications - At least 8 years of professional experience in Software Quality Engineering successfully deploying test automation solutions in Enterprise environments. - At least 2 years of relevant experience with Playwright web test automation, including headless, automated test generation, and Playwright native agents. - At least 2 years of experience working with UiPath, automating tests & processes. - Must-have experience designing & implementing test automation for Enterprise commercial applications, Oracle ERP products or equivalent. - Proven ability to work independently and lead software quality initiatives with minimal supervision. - Experience operating in a Testing COE model, enabling software teams for automated testing, governance, standardization. - Experience defining quality standards & best practices, detailed review of automation tools/technologies. - Expert-level experience in Test data management, configuring for different line of businesses & environments. - Experience resolving test flakiness, increasing test execution success rate. - Experience defining approach to increasing overall test coverage – UI, API and integration tests. - Experience leading & participating different test cycles such as UAT, SIT. - Experience integrating automated tests as part of the DevOps, CI/CD process, with Azure DevOps. - Experience creating quality reports in Grafana, Allure or equivalent technologies. - Experience with mobile app test automation technologies. - Strong experience with BDD, designing solutions that leverage the BDD concepts, Gherkin format, working with business to adopt BDD practices. Additional Qualifications - Industry Experience: Experience in Energy & Utilities domain and understanding of B2B and B2C businesses process in this domain is a significant value-add.

Role Description Our Engineering department is seeking a QA Engineer to join our growing team. This position will work on a small, agile team developing highly scalable solutions that allow our internal and external customers to leverage our business operations systems. Our engineers are actively delivering high-quality code, while continuously improving the development practice, product, and team. - Actively participate in a collaborative development team with the opportunity to influence product direction - Plan, organize, and prioritize test cases and test plans - Serve as an independent reviewer of test plans to deliver reliable, efficient software systems - Set up and manage test environments to simulate real-world scenarios and validate customer configurations before release - Record and communicate software defects with clear, actionable detail to support timely fixes - Establish, maintain, and continuously improve quality management system processes and procedures - Perform computer system setup, software installation and configuration, and peripheral equipment installation and integration - Install new and upgraded hardware and software (e.g. servers, cameras, and other devices) Qualifications - 3+ years of experience supporting commercial software applications - 2+ years of experience with Salesforce CPQ - Associates degree in Computer Science or relevant professional experience - Experience with applications that facilitate customer order to service delivery - Relational SQL skills with the ability to execute DML to support testing activities - Ability to write high-quality functional tests that verify the soundness and performance of delivered features - Adaptability to adapt to changing requirements and frequent customer feedback - Excellent written and verbal communication skills Requirements - Familiarity with Agile principles and software best practices - Experience with Jira as a work management tool - Experience with TestRail as a test management tool - Experience implementing and supporting automated testing of Salesforce configuration and Apex deployments - Experience with serverless application development, deployment and monitoring - Experience testing PHP and node.js based web applications Benefits - Incentive Bonus Plans - Medical, Dental, Vision benefits - 401K with Company Match - 10 Paid Holidays - Generous Paid Time Off Packages - Employee Stock Purchase Plan - Paid Parental & Family Leave - and more! Company Description At Motorola Solutions, we believe that everything starts with our people. We’re a global close-knit community, united by the relentless pursuit to help keep people safer everywhere. We build and connect technologies to help protect people, property and places. Our solutions foster the collaboration that’s critical for safer communities, safer schools, safer hospitals, safer businesses, and ultimately, safer nations. Connect with a career that matters, and help us build a safer future.