• Design and implement comprehensive evaluation frameworks that reflect real-world task success for agentic systems, with a focus on human+AI collaboration outcomes • Build benchmarking pipelines that capture nuanced success indicators including trust calibration, intervention frequency, and agent handoff quality • Lead development of observability tools and instrumentation for analyzing agent behavior in production • Translate complex qualitative and quantitative signals into actionable insights that inform model iteration and product prioritization • Collaborate with researchers, engineers, and product teams to align evaluation methodologies with business and user goals • Own benchmarking infrastructure that enables reproducible, scalable evaluation across AI initiatives • Champion rigorous experimental design and statistical analysis across teams to ensure consistent and meaningful measurement standards

• Design and train advanced ML models (e.g., DNNs, transformers) to power Reddit Ads Ranking models. • Develop and optimize features including embeddings, contextual signals, and cross-session behavior. • Own experimentation strategy: offline metrics, A/B design, readouts, and “next iteration” planning. • Work closely with product, infra, and data teams to drive end-to-end model deployment and performance analysis. • Mentor other MLEs and contribute to modeling best practices across the org. • Shape the long-term modeling vision across one or more domains (conversion, app ads, shopping, brand, etc.).

• Architect and significantly influence ML training/serving systems and tooling that unlock faster iteration and larger models. • Drive adoption and reliability of ML systems and own a portfolio of initiatives across multiple teams. • Improve efficiency: GPU utilization, training runtime, data loading, feature performance, and serving latency. • Write high-quality design docs; run design reviews; set standards for correctness, reliability, and velocity. • Partner cross-org on shared infra sequencing, requirements, and adoption. • Mentor other engineers and contribute to engineering best practices across the org.

• Design, train, and deploy a wide range of AI/ML models for mission-critical applications, ensuring they meet performance and scalability requirements. • Implement and manage automated MLOps pipelines for model monitoring, retraining, and lifecycle management to ensure continuous delivery and reliability. • Optimize model performance, scalability, and resource consumption in production cloud and on-premise environments. • Collaborate with data scientists, software engineers, and systems architects to translate model prototypes into hardened, production-grade solutions. • Uphold software engineering best practices, including robust version control, comprehensive testing, and CI/CD processes. • Stay current with industry trends and best practices in MLOps and operational AI to continuously improve the team's capabilities.