Founded in 1972, the ICW Group is a collection of multi-line property and casualty insurance carriers that provides workers' compensation, earthquake, and perso
QA Engineer - AI and ML
Location
California
Posted
1 day ago
Salary
$95.4K - $160.9K / year
Seniority
Senior
Job Description
QA Engineer - AI and ML
ICW Group
Title: QA Engineer - AI and ML Location: CA-San Diego Job Description: PURPOSE OF JOB The QA Engineer for AI and ML Platforms establishes and evolves quality engineering practices for cloud-native and AI/ML platform solutions, ensuring reliability, scalability, and reproducibility across data, model, and infrastructure layers. The position serves as a key contributor in defining test strategies for distributed systems, data pipelines, and machine learning workflows within AWS and Snowflake environments. ESSENTIAL DUTIES AND RESPONSIBILITIES Executes tasks that span across multiple systems/processes for one or more business projects/areas. - Reviews, analyzes, and evaluates business and IT systems and user needs to determine the best test approach. - Estimates, prioritizes, plans, and coordinates test activities. - Identifies, records, thoroughly documents, and tracks bugs. - Partners with the Development team to perform test via standards based on APIs for communicating between applications. - Tracks quality assurance metrics including reporting on pre-defined KPI, defect densities, and open defects counts. - Influences quality engineering best practices within the scrum team. - Executes and maintains automated regression test suites using open source tools. - Follows standards in accordance with company policy and regulation (MAR, HIPAA, ISO 2700, ISO 27001, etc.). - Recommends process improvement and strengthens the quality standards and implements those improvements on approval. - Audits test deliverables to ensure they meet standards and takes corrective action to mitigate variances. - Leads the QA efforts for large initiatives and collaborates closely with external vendor partners (offshore, nearshore) for work assignments, status calls, working sessions, triage calls etc. Develops detailed, comprehensive, and well-structured test plans and test cases. - Develops, publishes, and implements test strategy, test plans, and test scripts. - Develops and applies testing processes to new and existing products to meet stakeholders needs. - Develops and maintains automated test harnesses and libraries. - Develops and maintains testing standards, procedures, and guidelines to ensure consistent testing procedures. - Creates tests that cover functionality, load, security, etc. at the API and GUI levels. - Identifies risks and develops mitigation strategies. Becomes a subject matter expert for one or more business areas and technical product. - Delivers quality process training to technical staff. - Acts as an internal quality consultant to advise or influence business or technical partners. - Performs quality audits across the various IT functions to ensure that quality standards, procedures, and methodologies are being followed. - Partners with distributed teams, outside vendors, or offshore testing partners. - Coaches and educates others to increase early defect detection rates and decrease defect escape rates. Defines and Implements quality strategies for cloud-native AI and ML platforms. - Partners with Data Scientists and ML Engineers to design and execute testing strategies for AI/ML applications and pipelines, including pipeline validation, data validation, feature and model validation, inference testing, integration and API testing. - Implements testing strategies for infrastructure as code (IaC), including validation of AWS resources provisioned via tools such as Terraform or CloudFormation. - Establishes automated quality gates within CI/CD pipelines to enforce testing standards across build, deploy, and release stages. - Defines observability and quality signals, including logs, metrics, and traces, to proactively detect defects in production systems. Contributes hands-on to debugging and root cause analysis across application, data, and platform layers. - Leads validation of non-functional requirements including performance, scalability, reliability, and resiliency of AI and ML platforms. SUPERVISORY RESPONSIBILITIES This role does not have direct reports but may mentor and/or train less experienced team members. EDUCATION AND EXPERIENCE Bachelor’s degree from four-year college or university IN Computer Science, Information Systems or related field, or equivalent work experience. Minimum 4 years of related experience. Minimum 5 years of IT and/or relevant business experience. CERTIFICATIONS, LICENSES, REGISTRATIONS Certificated Software Test Engineer (CSTE) or International Software Testing Qualification Board (ISTQB) preferred. CP-SAT Selenium Automation Testing certification preferred. Foundation level API certification preferred. KNOWLEDGE AND SKILLS - Ability to anticipate user behavior or risks to systems; ability to work independently; experience querying and using data to enhance testing approach/scope. - Ability to quickly understand and gain domain expertise with complex existing applications and architecture. - Ability to validate data transformations and perform End-to-End validations for ETL/ API enabled integrated systems. - Ability to work effectively with geographically remote and culturally diverse teams. - Experience in analyzing requirements to reduce ambiguity and increase testability; experience developing and reviewing test plans, test cases, and test scripts. - Experience mentoring junior (less experienced) QA team members. - Knowledge of Insurance Industry best practices and business workflows a plus, and Industry standard certification in Quality Assurance is strongly desired. - Must have a customer-service mentality to resolve/avoid conflict and enable a team approach. Ability to speak in front of teams, stakeholders, and management. - Must have ability to learn, analyze, and interpret technical documentation such as software requirements, detailed designs, flow charts and database schemas. - Strong experience with SQL and SQL and database schema designs and experience with building automation frameworks is a must. - Understanding of the software development life cycle processes; understanding of different testing methodologies and their proper application. - Work effectively in both Agile/Scrum and Waterfall (for specific external vendor-led efforts) frameworks. - Expertise with automation feasibility definition and hands-on execution experience; ability to train others on automation practices and tools. - Works in an agile manner and must be flexible to changing priorities. - Experience testing ML frameworks and tools, preferably SageMaker, MLFlow, SnowflakeML. - Experience validating data platforms, ETL/ELT pipelines, or data warehousing solutions such as Snowflake. - Hands-on experience on AI/ML pipeline validation, data validation, model validation, integration and API testing. - Experience implementing automated testing within CI/CD pipelines and working in DevOps-oriented environments. - Experience with testing cloud platforms and cloud resource provisioning (AWS preferred), Infrastructure as Code (IaC), containerization tools like Docker and Kubernetes. PHYSICAL REQUIREMENTS This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines. Occasionally the work environment may be loud. WORK ENVIRONMENT This position operates in an office environment and requires the frequent use of computer, telephone, copier and other standard office equipment. We are currently not offering employment sponsorship for this opportunity. #LI-TM1 #LI-HYBRID The current range for this position is $95,378.98 - $160,850.17 This range is exclusive of fringe benefits and potential bonuses. If hired at ICW Group, your final base salary compensation will be determined by factors unique to each candidate, including experience, education and the location of the role and considers employees performing substantially similar work. WHY JOIN ICW GROUP? - Challenging work and the ability to make a difference - You will have a voice and feel a sense of belonging - We offer a competitive benefits package, with generous medical, dental, and vision plans as well as 401K retirement plans and company match - Bonus potential for all positions - Paid Time Off - Paid holidays throughout the calendar year - Want to continue learning? We’ll support you 100%
Related Guides
Related Categories
Related Job Pages
More QA Engineer Jobs
Quality Engineering Lead
Insight GlobalFounded in 2001, Insight Global (IG) offers enhanced staffing, placement staffing, and temporary-to-permanent staffing services, including long-term and short-t
Job Title: Quality Engineering Lead Location:USA Remote Req # DGO-b09ae312-8ab0-4ad6-9d06-51772da0a78d Pay Rate $57 - $71 (hourly estimate) Job Description: Job Type: 1 year contract About the Role Our client is seeking a highly motivated Quality Engineering Lead to lead and enhance quality assurance efforts across their customer-facing eCommerce platform. This role will initially focus on hands-on manual QA and release regression testing to build a strong understanding of the platform, release cycles, and end-user experience. Over time, this individual will drive the transition toward automated testing using modern frameworks, tools, and AI-assisted workflows. This is a high-impact role with significant visibility across Engineering, Product, UX, and Release Management, with ownership over release validation and influence on production deployment decisions. Key Responsibilities • Lead QA efforts across new features, platform functionality, and production releases • Perform hands-on manual testing for feature validation and regression cycles • Validate critical customer journeys including product discovery, product pages, cart, checkout, and mobile experiences • Own and manage release regression testing processes • Develop and implement QA standards, workflows, and testing strategies • Design and maintain automated test coverage to improve efficiency and scalability • Partner closely with Engineering, Product, TPM, and UX teams during release cycles • Identify, document, and troubleshoot defects with clear technical detail • Act as the quality gate for releases, supporting go/no-go decisions • Drive improvements in release stability, testing efficiency, and overall quality Analytics & Event Tracking Validation This role includes validating analytics and event tracking accuracy across the platform, including: • Customer interaction events and conversion tracking • Analytics instrumentation and DataLayer integrity • Client-side event reliability • Cross-browser and cross-device tracking consistency Experience with tools such as Amplitude, GA4, GTM, Floodlight, and Meta Pixel is highly preferred, along with the ability to identify tracking inconsistencies or silent failures. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. Required Skills & Experience Required Qualifications • 5+ years of QA experience in modern web or eCommerce environments • Strong manual and exploratory testing skills • Experience testing customer-facing applications and release cycles • Ability to independently own and evolve QA processes • Strong communication, organization, and cross-functional collaboration skills Experience working closely with engineering teams in fast-paced environments Nice to Have Skills & Experience Preferred Technical Experience • Automation tools such as Playwright, Cypress, Selenium, or Maestro • API testing, visual regression testing, and cross-browser/device testing • Experience with CI/CD pipelines, GitHub Actions, and test management tools (TestRail, Xray, Zephyr) • Familiarity with modern web technologies (React, Next.js) and headless commerce environments (Shopify) Technical Environment • React / Next.js architecture • Headless eCommerce platform with Shopify backend • Vercel deployments and modern CI/CD workflows AI & Modern QA Tooling Our client is focused on advancing QA through modern and AI-assisted approaches, including: • AI-driven testing and regression automation • Browser automation and intelligent test generation • AI-supported exploratory testing and bug documentation • Developer tools that improve release quality and efficiency Experience with tools such as GitHub Copilot, Cursor, and AI-powered QA solutions is a plus. Ideal Candidate • Detail-oriented and technically curious • Process-driven with strong problem-solving skills • Comfortable balancing hands-on QA with long-term automation strategy • Interested in leveraging AI and modern tools to drive efficiency • Confident making release decisions in a production environment Compensation: $65-70/hr. Exact compensation may vary based on several factors, including skills, experience, and education. Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law. Benefit packages for this role will start on the 1st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.
Role Description We are looking for a Test Engineer to join our engineering team in Bengaluru. In this role, you will design, build, and maintain automated test suites across our platform — spanning UI, API, load, and performance testing. You will work closely with developers and product managers in Agile delivery teams to ensure quality is embedded throughout the development lifecycle, and you will play an active role in shaping testing standards and practices across Send. What You’ll Be Doing as a Test Engineer - UI Test Automation - Design, implement and maintain end-to-end automated test suites for the Send Workbench product using Playwright and TypeScript. - Collaborate with developers and product managers within Agile delivery teams to identify test scenarios and integrate automation into the CI/CD pipeline. - Implement and maintain page object models, fixtures, and reusable test utilities to ensure test suite maintainability and reliability. - Triage and investigate test failures, working closely with engineers to isolate defects and prevent regressions. - API Test Automation - Build and maintain a comprehensive suite of API tests for the Send Platform. - Validate API contract compliance, data integrity, authentication flows, and error handling across all platform services. - Ensure API tests are integrated into deployment pipelines to provide rapid feedback on service quality. - Load & Performance Testing - Design and develop load and performance test scenarios to validate platform behaviour under expected and peak load conditions. - Define and track performance baselines, and work with engineering teams to investigate and resolve performance regressions. - Contribute to the selection and standardisation of load testing tooling across the platform. - AI Product Testing (Desirable) - Design and develop automated tests for AI-powered features within the Send platform, including agentic workflows and AI-assisted underwriting capabilities. - Explore and implement approaches to testing non-deterministic AI outputs, including output validation, prompt regression testing, and behaviour boundary testing. - Partner with AI engineers and product teams to define testability requirements early in the development lifecycle. - Test Automation Group - Contribute to and help shape the Test Automation Group — a cross-functional community of practice that sets testing standards, evaluates tooling, and supports consistent quality practices across Product and Delivery functions. - Champion test automation best practices. - Mentor and support developers and QA colleagues in writing effective automated tests. - Produce and maintain documentation, guidelines, and reusable frameworks to uplift test automation capability across the organisation. Qualifications - Proven experience in test automation, with hands-on expertise in Playwright and TypeScript. - Experience building and maintaining API test suites, with a solid understanding of REST API concepts, authentication, and contract testing. - Familiarity with CI/CD pipelines and integrating automated tests into deployment workflows. - Experience with page object models and building maintainable, scalable test frameworks. - Strong analytical and debugging skills — able to triage test failures and isolate root causes effectively. - Experience with load and performance testing tools and methodologies. - Strong communication skills and the ability to collaborate effectively within Agile delivery teams. - A passion for quality and a proactive approach to sharing knowledge and raising standards across teams. - Experience testing AI-powered features or non-deterministic systems is a nice to have. Benefits - Health Cover: Private medical and dental insurance from day 1, plus access to Spill that provides free confidential mental health support and therapy. - Employees’ Provident Fund (EPF): Employer contributes 12% of your basic salary or INR 1,800 per month, whichever you prefer. - Time Off: 25 days of annual leave, plus public holidays. We also offer volunteering time and a dedicated wellness day. - Enhanced Parental Leave: Includes 12 weeks of fully paid leave for all new parents, along with additional support for birth-giving parents. - Learning and Development: An annual budget via Learnerbly, providing access to books, courses, conferences, and other resources to support your growth. Interview Process - Stage 1 - Introductory Call with our in-house recruiter, Liam (30 minutes): This will cover your background, motivations, and communication skills, along with an opportunity to learn more about Send and the role. - Stage 2 – Technical Interview with Tomas Severo, Test Lead and Raghu Varma Mudunuro, Test Engineer (45 minutes). - Stage 3 – Interview with Sarah Wilkins, Head of Quality and a Lead Engineer (45 minutes). - 30-minute call with a Send Founder.
• Support and improve existing test automation • Identify and troubleshoot defects • Validate new features to ensure system stability, scalability, and compatibility with evolving product functionality • Collaborate with teams to deliver high-quality software solutions • Participate in code reviews and contribute to technical decisions
• Support and improve existing test automation • Identify and troubleshoot defects • Validate new features to ensure system stability, scalability, and compatibility with evolving product functionality • Collaborate with teams to enhance customer service, increase efficiency, and support seamless omnichannel interactions
