QA Automation Lead Department: Technology Location: United States (Remote) FLSA Status: Exempt People Leader: No, potential for future direct reports Travel: <5% Summary Company Summary Accuity partners with hospitals and health systems through a technology-enabled, physician-led model that improves clinical documentation integrity, coding accuracy, reimbursement optimization, and quality outcomes. Job Summary The Quality Assurance (QA) Automation Lead plays a critical role in ensuring the quality, reliability, and scalability of Accuity's AI-driven clinical documentation and revenue cycle solutions. This highly hands-on technical leadership role is responsible for designing and implementing comprehensive test automation frameworks across web applications, APIs, data pipelines, and AI/ML systems. Working closely with data scientists, AI engineers, product teams, and DevOps partners, the QA Automation Lead establishes modern quality engineering and MLOps practices to ensure systems and models are production-ready, compliant, and performant. Responsibilities Automation Strategy and Framework Development - Define and implement enterprise-grade test automation strategies across AI/ML model evaluation, APIs and service layers, data pipelines, and web applications - Design and build scalable, reusable automation frameworks that support continuous delivery and high test coverage - Establish standardized approaches for functional, regression, performance, and data validation testing - Drive adoption of modern testing methodologies, including shift-left testing and test-driven development practices AI and ML Testing and Evaluation - Develop advanced testing frameworks for model validation, including accuracy, precision and recall, drift detection, and reliability monitoring - Implement evaluation approaches for large language models and AI systems, including prompt testing, output validation, and hallucination detection - Build data quality and feature validation pipelines using synthetic and production-like datasets to support robust model testing - Ensure AI systems meet reliability, explainability, and safety standards appropriate for healthcare environments DevOps and MLOps Integration - Partner with DevOps teams to integrate automated testing into CI/CD pipelines and release processes - Contribute to the design and implementation of MLOps frameworks that support continuous integration and deployment of machine learning models - Implement automated validation gates, monitoring, logging, and feedback loops for model promotion and production performance - Ensure alignment between QA automation practices, DevOps workflows, and Azure-based infrastructure Quality Engineering and Delivery Excellence - Establish quality metrics and KPIs, including test coverage, pass rates, defect leakage, and execution time, and build reporting visibility around performance - Lead defect management processes, including triage, root cause analysis, and resolution tracking - Collaborate with Product Owners and Scrum Masters to embed quality practices into Agile workflows - Ensure test coverage aligns with business-critical workflows across clinical documentation and revenue cycle processes Leadership and Mentorship - Serve as a technical leader and mentor within a small QA and engineering team - Establish best practices for automation, code quality, and test design - Conduct code reviews and provide guidance on automation frameworks, tools, and implementation approaches - Influence engineering culture toward a quality-first mindset and continuous improvement Compliance and Healthcare Alignment - Ensure testing frameworks support auditability, traceability, and regulatory compliance requirements - Validate systems against healthcare-specific workflows, including CDI, coding accuracy, and EHR integrations - Incorporate controls for data privacy, security, and model governance into testing and release practices Qualifications Education and Credentials - Bachelor's degree in Computer Science, Engineering, or a related field required Experience - 7 to 10+ years of experience in software quality assurance and automation - Proven experience designing and implementing automation frameworks across web, API, and data systems - Strong programming skills in one or more languages, such as Python, C#, JavaScript, or TypeScript - Experience with test automation tools such as Selenium, Playwright, Cypress, or PyTest - Strong understanding of CI/CD pipelines, DevOps practices, and version control systems - Experience with Agile methodologies and tools such as Jira Preferred Qualifications - Experience testing AI and ML systems, including model validation and data pipeline testing - Experience working in Azure-based environments and integrating with Azure DevOps - Familiarity with MLOps concepts and frameworks - Experience in healthcare technology, including CDI, revenue cycle or coding workflows, and EHR systems and integrations - Knowledge of regulatory and compliance requirements in healthcare systems Core Competencies - Deep technical expertise in automation and quality engineering - Strong problem-solving and analytical skills - Ability to operate hands-on while defining strategy and standards - Excellent cross-functional collaboration with engineering, product, clinical, and data teams - Strong communication skills with the ability to translate technical quality risks into business impact - Leadership and mentorship capabilities Additional Requirements 1) Physical Requirements: The requirements described here are representative of those that must be met by an employee to successfully perform the essential functions of this job with or without reasonable accommodation. Unless otherwise indicated, Accuity positions require interaction with people and technology while either sitting or standing. Employees must be able to communicate via phone, email, etc. and sit for extended periods of time, with or without reasonable accommodations. Physical effort and exposure to physical risk are limited to that of an office role / environment. 2) Position and Employment Statement: While this job description is intended to be an accurate reflection of the job requirements, management reserves the right to modify, add or remove duties from a job and to assign other duties as necessary and at any time. All positions at Accuity Delivery Systems, LLC, are at-will employment, and a position description is not a guarantee of a job or of job responsibilities. While this job description is intended to be an accurate reflection of the job requirements, management reserves the right to modify, add or remove duties from a job and to assign other duties as necessary and at any time. All positions at Accuity are at-will employment, and a position description is not a guarantee of a job or of job responsibilities.