• Diseñar, validar y mejorar frameworks de evaluación para agentes de IA. • Implementar suites de pruebas automatizadas y regresión para modelos generativos. • Definir y monitorear métricas de calidad relacionadas con: Relevancia, Fidelidad, Coherencia, Precisión y Hallucinations. • Construir sistemas de evaluación tipo “LLM-as-a-Judge”. • Establecer benchmarks de desempeño para nuevos modelos y agentes existentes. • Validar actualizaciones de prompts, modelos y pipelines RAG. • Colaborar con equipos de IA y desarrollo para definir criterios de aceptación (pass/fail). • Analizar resultados de evaluación y proponer mejoras continuas. • Generar reportes métricos y trazabilidad sobre la calidad de los agentes.

Role Description DocMe360 is seeking a talented Test Engineer to join our dynamic team. The Test Engineer role will provide testing support on a cross-functional team supporting the Department of Veteran Affairs. This role is currently structured as a 1099 contract position, with the potential to transition to a W-2 role in the future. This role is responsible for creating and executing test plans and test scripts for VA software solutions and ensuring alignment with mission objectives and software standards. The Test Engineer must have experience with configuring necessary hardware and operating environments as needed to complete assigned testing, as well as comprehensive technical expertise on IT products, operating systems, software, hardware, systems and networks and specialized environments. Responsibilities - Test Planning & Execution - Work in an Agile framework with developers, end users, and organizations to create test plans and test scripts in a Test Driven Development (TDD) environment - Coordinate with test sites and other team participants to plan test events - Write and assist in the development of test plans and test procedures - Test Environment & Infrastructure - Configure necessary hardware and operating environments as needed to complete assigned testing - Apply technical expertise across IT products, operating systems, software, hardware, systems, networks, and specialized environments to support test execution - Reporting & Issue Management - Create test reports documenting results and findings - Track problems and report on errors that are identified during testing - Manage issues to closure in accordance with identified acceptance criteria - Adhere to the DocMe360 Core Values - Delivery – Deliver all efforts through a culture of honesty, transparency, unwavering integrity, and professional growth - Optimist – Optimize and invest in professional growth, empowerment, and well-being - Collaboration – Collaborate for impact; foster teamwork and shared expertise to achieve success - Motivation – Motivate from within to naturally inspire and energize clients with passion and purpose - Exploration – Explore opportunities for innovative technologies and trends that align with our mission and foster the highest quality of products and services Qualifications - Bachelor's degree (or equivalent experience) - 2+ years of professional experience in software testing or quality assurance - Experience working within Agile development teams alongside developers, end users, and organizational stakeholders - Proven experience writing or assisting in the development of test plans and test procedures Requirements - Experience creating test plans and test scripts in a Test Driven Development (TDD) environment - Comprehensive technical expertise across IT products, operating systems, software, hardware, systems, networks, and specialized environments - Experience configuring hardware and operating environments to support test execution - Experience creating and delivering test reports - Strong analytical and troubleshooting skills with the ability to track and resolve issues against defined acceptance criteria - Ability to coordinate effectively with test sites, team participants, and cross-functional stakeholders - Strong written and verbal communication skills Preferred Qualifications - Experience supporting federal or regulated environments Benefits - Health Care Plan (Medical, Dental, & Vision) - Retirement Plan (401k, IRA) - Paid Time Off (Vacation, Sick & Public Holidays) - Family Leave (Maternity & Paternity) - Training & Development - Work From Home

Role Description DocMe360 is seeking a talented Senior Test Engineer to join our dynamic team. The Senior Test Engineer role will provide testing support on a cross-functional team supporting the Department of Veteran Affairs. This role is currently structured as a 1099 contract position, with the potential to transition to a W-2 role in the future. This role is responsible for creating and executing test plans and test scripts for VA software solutions and ensuring alignment with mission objectives and software standards. The Senior Test Engineer must have experience with configuring necessary hardware and operating environments as needed to complete assigned testing, as well as comprehensive technical expertise on IT products, operating systems, software, hardware, systems and networks, and specialized environments. Responsibilities - Test Planning & Execution - Work in an Agile framework with developers, end users, and organizations to create test plans and test scripts in a Test Driven Development (TDD) environment - Coordinate with test sites and other team participants to plan test events - Write and assist in the development of test plans and test procedures - Test Environment & Infrastructure - Configure necessary hardware and operating environments as needed to complete assigned testing - Apply technical expertise across IT products, operating systems, software, hardware, systems, networks, and specialized environments to support test execution - Reporting & Issue Management - Create test reports documenting results and findings - Track problems and report on errors that are identified during testing - Manage issues to closure in accordance with identified acceptance criteria - Adhere to the DocMe360 Core Values - Delivery – Deliver all efforts through a culture of honesty, transparency, unwavering integrity, and professional growth - Optimist – Optimize and invest in professional growth, empowerment, and well-being - Collaboration – Collaborate for impact; foster teamwork and shared expertise to achieve success - Motivation – Motivate from within to naturally inspire and energize clients with passion and purpose - Exploration – Explore opportunities for innovative technologies and trends that align with our mission and foster the highest quality of products and services Qualifications - Bachelor's degree (or equivalent experience) - 4+ years of professional experience in software testing or quality assurance - Experience working within Agile development teams alongside developers, end users, and organizational stakeholders - Proven experience writing or assisting in the development of test plans and test procedures Requirements - Experience creating test plans and test scripts in a Test Driven Development (TDD) environment - Comprehensive technical expertise across IT products, operating systems, software, hardware, systems, networks, and specialized environments - Experience configuring hardware and operating environments to support test execution - Experience creating and delivering test reports - Strong analytical and troubleshooting skills with the ability to track and resolve issues against defined acceptance criteria - Ability to coordinate effectively with test sites, team participants, and cross-functional stakeholders - Strong written and verbal communication skills Preferred Qualifications - Experience supporting federal or regulated environments Benefits - Health Care Plan (Medical, Dental, & Vision) - Retirement Plan (401k, IRA) - Paid Time Off (Vacation, Sick & Public Holidays) - Family Leave (Maternity & Paternity) - Training & Development - Work From Home

• Ensure the quality of solutions developed by Claranet • Perform testing and validation on products and services • Develop test plans and execute automated and manual tests • Identify bugs and defects • Document and report test results