Role Description - Design verifiers and correctness rubrics for coding tasks to ensure AI agent code functionality. - Enumerate edge cases and build adversarial test cases for comprehensive agent/model evaluation. - Grade agent trajectories and improve test/rubric quality through detailed review. - Work independently and asynchronously to meet deadlines while enhancing AI model performance. - Collaborate with subject matter experts to ensure test consistency and relevance. Qualifications - Must-Have: - 5+ years as an SDET / software test engineer at a real product organization. - Proficiency in writing code and tests using automation frameworks like pytest, Playwright, Cypress. - Experience with CI/CD processes; SDET preferred over manual-only QA. - Clear written communication skills. - Preferred: - Familiarity with AI tools and evaluations. Requirements - Hourly contractor. - Paid weekly via Stripe Connect. Application Process - Upload resume. - AI interview based on your resume. - Submit form. Resources & Support - For details about the interview process and platform information, please check: Mercor Talent Docs . - For any help or support, reach out to: support@mercor.com . - Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

• Collaborate with development teams to design and implement automated test solutions • Perform automated testing for REST APIs and other components • Validate API functionalities using RestAssured and Karate Framework • Integrate testing processes with CI/CD pipelines for seamless deployments • Conduct thorough discussions on testing strategies with the team

Role Description The Market Development Associate (MDEV) works remotely and is primarily responsible for pre-sale initiatives in the customer journey, such as: - Calling campaigns - Data collection - Driving attendance to research and marketing webinars, thought-leadership panels, and targeted virtual events The MDEV will also assist with varied strategic initiatives designed to increase Inside Sales revenue, along with post-sale account management. At times, MDEV may be called on to assist with special projects within the Courseware team, including open sales territory coverage. The MDEV reports to the Manager, Market Development. How you will make an impact: - Accurately convey the zyBooks story and pedagogy as a market-facing sales professional in a number of roles - Work with editorial and marketing teams to launch new products, including market seeding, post-webinar data compilation, and targeted research, as well as creative and analytical brainstorming - Collaborate with the Market Development team to create strategic, actionable, data-driven insights for effective marketing and sales plans within Inside Sales - Identify gaps in market research and data mine as needed - Update Salesforce post-webinar including Contact records, Course records, and Account records - Create post-event tasks for the sales team and communicate intelligence to reps and managers The successful candidate will be a good communicator who is also detail oriented. Someone who is knowledgeable with digital courseware and experienced in higher education sales. Qualifications - BA / BS Degree or equivalent - Higher-education sales experience - Business application software: Google Suite, Microsoft Suite, CRM (Salesforce.com preferred), Zoom, etc. - Solutions-based problem solver, adaptable and strategic - Exceptional written and verbal communication skills - Excellent organizational and planning skills Benefits - Meeting-free Friday afternoons for heads down work and professional development - Opportunities to foster community, learn, and grow through a robust body of employee programming - Commitment to fair, transparent pay and competitive compensation Salary Range 54,300 USD to 74,167 USD

• Translate AI testing strategy into executable test scenarios across LLM outputs, document classification, extraction accuracy, agent workflows, and edge cases • Design adversarial and boundary test inputs to expose hallucination, misclassification, and failure modes • Validate AI outputs for structure, consistency, accuracy, and production readiness against defined performance thresholds • Build reusable Python-based evaluation frameworks, including output validation, hallucination detection, and scoring mechanisms • Develop parameterized test scripts reusable across features, models, and releases • Implement AI-as-Judge frameworks, including prompt design, scoring logic, and calibration of evaluation reliability • Embed evaluation frameworks into CI/CD pipelines to support continuous testing and deployment • Design and operate drift detection frameworks using fixed baseline datasets and scheduled re-evaluation • Establish thresholds to distinguish acceptable variation from performance degradation • Enable release gating by identifying regressions prior to production deployment • Build and maintain ground truth datasets in partnership with subject matter experts • Define standards for classification, extraction accuracy, and acceptable output characteristics • Continually update datasets to reflect evolving business requirements and use cases • Test end-to-end agentic workflows, validating data integrity, error propagation, and fallback behavior • Perform API-level testing of AI pipeline endpoints using Python and Postman/Newman • Validate data persistence and integrity across system layers using SQL • Partner with engineering teams to ensure testability, observability, and system reliability • Define and scale standardized AI evaluation patterns and reusable quality frameworks across VLabs • Contribute to enterprise AI quality standards and reference architectures • Ensure adherence to Responsible AI, data privacy, and governance requirements • Support auditability, traceability, and transparency of AI outputs and evaluation processes • Translate evaluation results into actionable insights for engineering, product, and business stakeholders • Support decision-making on model readiness, release risk, and performance trade-offs • Proactively identify risks, patterns, and systemic issues and escalate appropriately