Requisition ID 2026-4256 Job Summary Aptive seeks a skilled Evaluation Specialist to support the Veterans Health Administration’s (VHA) RISE Implementation initiative by assisting with performance measurement and evaluation activities. This role will focus on supporting the development of evaluation frameworks, data collection strategies, and performance metrics to assess implementation progress and outcomes. The specialist will assist in gathering and analyzing quantitative and qualitative data to support readiness assessments, implementation monitoring, and evaluation of Initial Operating Capability (IOC). Primary Responsibilities In support of implementation and evaluation, the Evaluation Specialist will contribute to tracking key performance indicators (KPIs), developing dashboards and reporting tools, and supporting post-implementation reviews. Responsibilities include assisting in analyzing program performance, identifying trends and areas for improvement, and documenting findings and lessons learned. The specialist will also support continuous improvement efforts by helping translate evaluation results into actionable recommendations and ensuring alignment with program goals and performance expectations. The ideal candidate will have a foundational understanding of program evaluation, data analysis, and performance measurement, along with strong analytical and communication skills. A bachelor’s degree in public policy, public administration, public health, statistics, or a related field is required, with experience or coursework in evaluation methods or healthcare programs preferred. The role requires attention to detail, proficiency in data analysis and visualization tools, and the ability to work collaboratively in a fast-paced environment. Minimum Qualifications - 5 years of experience with measuring program activities. - Bachelor’s Degree in a related field, such as public health, statistics, or business. - Consulting experience supporting large, complex Cabinet-level federal agencies. - Ability to obtain and maintain a public trust clearance. - Legal authorization to work in the U.S.

• Own and evolve the hybrid search pipeline – lexical retrieval, dense vector search, reciprocal rank fusion, and multi-stage reranking on Elasticsearch • Build and train adaptive retrieval models – lightweight classifiers and ranking models that configure search behavior per query, per category, per context (source routing, per-attribute boost prediction, filter mode decisions) • Design and productionize the learning-to-rank system – from feature engineering through model training (LightGBM, ONNX) to production deployment and A/B evaluation • Build the search feedback loop – instrument and integrate behavioral signals (CTR, conversions, add-to-cart) into ranking and retrieval as features for LTR, reward signals for adaptive retrieval, and inputs for search-side personalization • Build the business and ordering layer – separating organic relevance from sponsored/partner placement with quality gates, slot allocation, campaign configuration, and an auction-style approach as the system matures • Own the offline enrichment pipeline – LLM-based product enrichment at scale, data quality monitoring, and index management • Instrument and evaluate everything – bulk evaluation pipelines, per-category metric tracking, regression detection, experiment analysis • Integrate query understanding outputs into retrieval – translating extracted attributes, intents, and constraints into filters, boosts, and retrieval strategy decisions

• Collaborate with the team using Confluence, Discourse, GitLab, and Jira • Perform code reviews and maintain high engineering standards • Work with government leadership to ensure program alignment • Meet with customers to collect requirements and incorporate feedback • Provide regular progress updates to leadership and customers • Conduct multidisciplinary research to support modeling and development goals • Contribute to team vision, direction, and simulation architecture discussions • Design, develop, document, test, and debug space simulation software

• contribute to designing, scaling/building, evaluating, integrating, shipping, and refining reward signals for recommendations by hands-on ML development • promote and role-model best practices of ML systems development, testing, evaluation • lead collaborations and align across PZN to integrate and A/B test mid-term signals in various recommendation systems