Nextech improves the performance of specialty practices with innovative solutions that drive efficiency and fuel growth.
Lead Software Engineer in Test – Agentic AI
Location
United States
Posted
4 days ago
Salary
0
Seniority
Senior
Job Description
Lead Software Engineer in Test – Agentic AI
Nextech Systems
• Design, develop, and maintain scalable automation frameworks and regression test suites across UI, API, backend, and AI-driven applications. • Lead quality engineering initiatives for Agentic AI systems, including LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms. • Validate complex AI agent behaviors such as multi-agent coordination, tool calling, prompt chaining, memory retention, conversational workflows, and fallback/retry mechanisms. • Develop and execute functional, integration, regression, end-to-end, and performance testing strategies across cloud-native and distributed environments. • Collaborate with Software Engineers, Product Managers, and DevOps teams to integrate testing throughout the software development lifecycle. • Build reusable automation solutions to validate APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business processes. • Design and maintain robust test environments, CI/CD integrations, cloud execution platforms, and automation pipelines to improve efficiency and reliability. • Establish and drive best practices related to code quality, maintainability, scalability, dependency injection, observability, and testability. • Lead validation efforts for AI workflow resiliency, distributed system reliability, data consistency, and orchestration performance under failure and edge-case conditions. • Mentor QA engineers and contribute to the development of scalable QA standards, automation strategies, reporting practices, and continuous improvement initiatives.
Job Requirements
- Bachelor’s degree in Computer Science, MIS, Engineering, or a related technical field.
- 8+ years of experience in Software Development or Software Engineer in Test (SDET/SET) roles.
- 5+ years of hands-on experience developing automation frameworks using JavaScript/TypeScript or Java.
- Strong experience with automation tools such as Playwright, Cypress, Postman, K6, or equivalent frameworks.
- 2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.
- Hands-on experience validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes.
- Experience testing AI orchestration frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies.
- Strong expertise in API testing, backend integration testing, UI automation, and end-to-end workflow validation.
- Solid understanding of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles.
- Experience working in cloud-native and containerized environments (Azure, AWS, GCP, Docker, Kubernetes).
- Proven experience integrating automated test suites into CI/CD pipelines and distributed cloud environments.
- Experience validating asynchronous, event-driven, and distributed systems with a focus on resiliency and fault tolerance.
- Strong analytical, debugging, troubleshooting, and problem-solving skills.
- Excellent organizational, communication, and stakeholder collaboration skills in Agile environments.
- Experience with AI observability, telemetry, RAG pipeline validation, vector databases, or prompt evaluation frameworks (preferred).
Benefits
- Generous annual bonus opportunity
- 401(k) with Employer Match
- Flexible Time Off: take time off when you need it without worrying about available hours
- 11 paid holidays
- Your Day Your Way - Celebrate a day of cultural or social significance to you
- Insurance: Choice of Medical, Dental, and Vision plans
- Health Savings Account with employer match
- Flexible Spending Account
- 100% Company-Paid Parental leave (After 6 months with the company)
- 100% Company-Paid Life Insurance and Short/Long Term Disability Insurance
- Nextech Luminary Peer Recognition Program
- Wellness Program including discounts on medical premiums
- Employee Assistance Program with free counseling sessions available
- Corporate Discounts on Retail, Travel, and Entertainment
Related Guides
Related Categories
Related Job Pages
More SDET Jobs
Senior Test Associate Engineer
SonicWallDelivering real-time breach detection and prevention solutions backed by SonicWall Capture Threat Network.
• Perform thorough testing of software applications to identify defects and ensure product quality. • Develop, maintain and execute automation test scripts using Python. • Design, implement and manage automated test frameworks to ensure comprehensive test coverage. • Continuously improve and refactor existing automation scripts and frameworks to enhance efficiency and effectiveness. • Validate and verify networking components, configurations, and protocols as part of the overall software testing process. • Troubleshoot and analyze network issues encountered during testing. • Collaborate with network engineers to ensure seamless integration and performance of networking features. • Work closely with developers to understand application functionality, identify test requirements and resolve defects. • Participate in agile development processes, including sprint planning, stand-ups and retrospectives. • Provide timely and constructive feedback to the development team on software defects and improvements.
Software Automation Test Engineer
InfiterraInfiterra helps IT Distributors and MSPs transform and grow. Our platform automates each step from quote to bill.
• Own the full automated testing coverage across functional, smoke, regression, API, integration, and end-to-end layers - ensuring critical platform flows like ordering, billing, subscriptions, renewals, and vendor integrations are validated reliably before every release. • Own E2E test automation using Selenium and Playwright, and contribute to API and integration-level coverage where it delivers faster, more stable results. • Validate REST APIs thoroughly - payloads, error handling, negative scenarios, and auth flows - so integration failures are caught before they reach production. • Catch quality problems before development starts by driving 3 Amigos sessions, challenging acceptance criteria, and surfacing edge cases during refinement. • Make defects easier to fix — reproduce issues clearly, trace root causes using logs, SQL, and pipeline data, so developers can act without re-investigating. • Support CI/CD pipeline health — monitor automated test execution, investigate failures, eliminate flaky tests, and ensure test results are meaningful enough to base release decisions on. • Raise the quality bar on the automation suite — improve maintainability, close coverage gaps, and leave the framework in better shape than you found it. • Validate data integrity at the source — use SQL and backend checks to confirm that billing, provisioning, and integration flows produce correct outcomes, not just passing UI tests. • Prioritize testing effort by risk — focus coverage where customer impact and defect history and release scope point to the highest likelihood of failure, not where it's easiest to automate. • Own the test data layer — build and maintain fixtures and preconditions so automated and manual tests run reliably without brittle setup dependencies.
• Develop, maintain, and extend automated test suites (UI, API, E2E) using Playwright and TypeScript • Ensure and improve regression coverage for existing and newly delivered features • Maintain test automation framework stability, including refactoring and improving test reliability • Investigate test failures, identify root causes across UI, API, data, and environment layers, and support debugging together with developers • Participate in feature analysis and planning, helping define what should be automated and at what level (UI vs API vs integration)
Senior QA Automation Engineer, AI & LLM Testing
NagarroNagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
• Design, develop, and maintain automated test suites for web applications, APIs, and AI-powered features • Evaluate LLM outputs for response accuracy, relevance, consistency, and hallucination risks • Create prompt tests and quality checks using Promptfoo or similar LLM evaluation frameworks • Build end-to-end UI automation using Playwright, following page object models and maintainable test architecture principles • Develop reusable Python scripts for API testing, automation utilities, and LLM quality evaluation • Create and maintain Postman collections for REST API testing, including environments, assertions, and CI/CD integration • Use GitHub Copilot or similar AI coding assistants to improve test coverage, speed up automation development, and identify quality gaps • Collaborate closely with product managers, developers, and stakeholders to define test strategies, communicate risks, and improve quality practices • Actively contribute to Agile ceremonies, including refinement sessions, sprint planning, retrospectives, and daily collaboration




