Role Description This role at Arbitration Forums is as unique as it is rewarding because of the AF IPAAL Values (Integrity, Passion, Accountability, Achievement, Leadership) and TRI Model (Trust, Respect, Inclusion). The AI Quality Assurance Engineer is responsible for safeguarding the integrity of AI systems by meticulously overseeing the testing process to identify bugs and inconsistencies. This role serves as the last line of defense, ensuring that only the most reliable AI solutions are deployed. This role is responsible for designing, developing, and optimizing test cases for AI solutions, ensuring that models and prompts for generative AI are producing the expected outputs prior to full integration into business processes and applications. The incumbent will partner with stakeholders to drive business value to Arbitration Forums and our members through fully tested AI solutions for deployment through a myriad of implementation methods. Qualifications - Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Science, Linguistics, or a related field. - Minimum of 7 years of experience in AI solution testing, software quality assurance, data governance, data science, or a related role. - Expertise in complex testing frameworks with combined AI and programmatic solutions, including multiple deployment methods. - Auto Insurance claims industry experience preferred. Requirements - Advanced programming knowledge, including mastery of programming languages such as Python and Java. - Deep understanding of machine learning and AI principles, from generative models to predictive advanced algorithms. - Cloud computing knowledge for deploying and managing AI applications on cloud platforms like Microsoft Azure. - Understanding of containerization technologies like Docker and orchestration tools like Kubernetes for scaling AI solutions. - Data management knowledge, including data pre-processing, augmentation, and generation of synthetic data. - Strong understanding of natural language processing concepts and techniques. - Proficiency in programming languages such as Python, R, and familiarity with AI frameworks (e.g., TensorFlow, Selenium, PyTorch). - Working knowledge of testing frameworks, from techniques to automate web applications to validation of machine learning models. - Experience with continuous integration and delivery in the context of DevOps and MLOps. - Working knowledge of cloud services (i.e., MS Azure, AWS, Google Cloud). - Experience with AI tools, such as MS Azure ML, Databricks AI, Snowflake CortexAI, Dataiku. - Strong knowledge of data governance, data security, and compliance practices. - Familiarity with data visualization and reporting tools (e.g., Webfocus, Power BI). - Excellent analytical and problem-solving abilities. - Strong communication and interpersonal skills to collaborate with cross-functional teams. - Ability to lead projects and mentor junior staff. Benefits - This is a fully remote position requiring reliable high-speed internet access and a dedicated workspace. - Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. Additional Duties and Responsibilities - Lead AI systems and model observability efforts to ensure adherence to company policies and enforce governance standards. - Other duties as assigned by manager or project needs.

• Creating and executing comprehensive application testing strategies. • Developing, optimizing and maintaining a suite of test cases, ensuring thorough coverage of functional, regression, integration and performance scenarios. • Collaborating with the development team to manage defects, including monitoring the production environment and participating in documentation, prioritization and resolution. • Innovating and refining testing frameworks, tools, and methodologies to enhance quality and efficiency in the testing process. • Providing guidance during user acceptance testing and providing support to end-users and business leaders • Maintaining awareness of Pega features, industry testing standards and emerging technologies and leading continuous improvement initiatives.

• Thoroughly review treatment plans, progress notes, and other documentation submitted by associate therapists (licensed interns). • Verify that documentation is accurate, complete, and complies with relevant regulatory standards and organizational guidelines. • Monitor and ensure compliance with all applicable laws, regulations, and accreditation standards in the healthcare industry. • Stay up-to-date with industry changes and inform the team in the most effective ways about any updates affecting documentation requirements. • Conduct regular audits to assess the quality of documentation and identify areas for improvement. • Provide constructive feedback to associate therapists and collaborate with them to enhance the quality of their documentation. • Work closely with the leadership team, including clinical supervisors and management, to address any issues related to documentation quality and compliance. • Communicating effectively with providers for documentation coaching and revision purposes prior to co-signature. • Relaying ongoing and urgent matters to CCQAM and other appropriate parties. • Foster a collaborative environment that encourages open communication and continuous improvement. • Maintain accurate records of all documentation reviews, feedback provided, and corrective actions taken. • Generate reports on the overall quality of documentation and compliance levels.

• Analyze new features from Figma prototypes, requirements, and business rules. • Define test strategy based on risk, impact, and data. • Create, review, and validate test scenarios/cases (functional, regression, and exploratory). • Execute manual and automated tests, ensuring traceability and adequate coverage. • Automate regression tests and maintain a stable test suite (maintenance, refactoring, and evolution). • Perform API testing (contracts, critical flows, and regressions). • Plan and execute load/performance tests, analyze results, and record recommendations. • Provide evidence of validations (screenshots, videos, logs, reports) and document results. • Report bugs with quality: reproducible steps, severity/priority, and supporting evidence. • Track tasks in Jira and participate in agile ceremonies (planning, refinement, daily stand-ups, retrospectives). • Create tutorials, manuals, and documentation for users and internal teams. • Use AI tools to support QA activities (e.g., refining scenarios, reviewing acceptance criteria, summarizing evidence, failure analysis), ensuring responsible use and human validation.