• Define and own what “good” means for search-augmented and agentic AI systems by designing evaluation frameworks that measure real-world quality, reliability, and user-relevant behavior beyond standard benchmarks. • Invent and validate novel evaluation methodologies for non-deterministic systems (LLMs, agents, RAG), including behavioral evals, long-tail and adversarial test sets, and task-specific metrics. • Develop rigorous statistical frameworks for model comparison, regression detection, and uncertainty estimation, ensuring evaluation results are defensible and decision-ready. • Build and maintain scalable evaluation systems—datasets, gold standards, eval harnesses, scoring pipelines, and analysis tooling—that can be reused across products and customers. • Lead customer-facing evaluation research, working directly with enterprise customers to translate domain-specific quality requirements into credible, actionable evals that support product decisions and sales outcomes. • Drive competitive evaluations and internal quality reviews, surfacing meaningful performance differences, trade-offs, and failure modes to inform product strategy and prioritization. • Partner with engineering and product teams to integrate evals into development loops, release gating, and ongoing quality monitoring. • Mentor and set standards for evaluation practice, reviewing eval designs, guiding other scientists, and shaping the long-term evals roadmap as systems become more agentic and complex. • End-to-End Project Leadership: Lead the development of new AI-driven projects, encompassing ideation, prototyping, research, infrastructure design, scalability, monitoring, and evaluation. • Rapid Iteration: Adapt quickly to user feedback and evolving requirements, ensuring continuous improvement in a fast-paced environment.

• Design and fine-tune large-scale VLMs / LLMs — and hybrid architectures — for tasks such as visual reasoning, retrieval, 3D understanding, and embodied interaction. • Build robust, efficient training and evaluation pipelines (data curation, distributed training, mixed precision, scalable fine-tuning). • Conduct in-depth analysis of model performance: ablations, bias / robustness checks, and generalisation studies. • Collaborate across research, engineering, and 3D / graphics teams to bring models from prototype to production. • Publish impactful research and help establish best practices for multimodal model adaptation.

• Rapidly prototype AI-driven use cases using Java-based services, frontend components, and integration with AI/ML tools. • Build POCs leveraging agentic AI frameworks (LangGraph, AutoGen, CrewAI) and test novel interactions with LLMs and external APIs. • Work with SQL/NoSQL databases to model knowledge graphs, embeddings, and AI-generated outputs at scale. • Develop intuitive UIs using React or Angular to test agent-based user workflows. • Partner with architects and ML engineers to evaluate AI frameworks and identify production candidates. • Translate architectural patterns into real-world test cases that help validate platform assumptions.

• Lead and scale high-impact research across product development, brand strategy, and marketing. Independently conduct complex generative and evaluative studies that uncover unmet needs, inform product vision, and de-risk decisions at scale. • Implement advanced AI-native practices and embed AI into research workflows to ensure speed, quality, and impact. Define what it means to work AI-natively across research internally and for our customers. Stay up to date on the latest AI-based research techniques and tools. • Connect the dots, patterns and connections across teams. Ensure insights are shared across teams and inform decisions. Partner with senior leadership to identify and evangelize the highest-impact opportunities for our customers. • Be the in-house expert across all core research methods, including usability testing, ethnographic research, surveys, segmentation, message testing, brand tracking, and concept validation. • Coach designers, marketers, product managers and other employees on conducting research, reviewing their projects and adapting the AI first workflow based on needs • Lead and moderate company-wide Voice of the Customer (VoC) sessions, interviewing real customers in live forums to bring their perspectives directly to cross-functional teams • Manage external research contractors and vendors, including scoping, quality control, and budget alignment • Build and scale a research enablement model, coaching designers, marketers, and product managers to run self-serve research with confidence and consistency. Develop tools, templates, and guidance that scale research capabilities and impact. • Collaborate with cross-functional leadership to build and prioritize a unified, impact-driven research roadmap • Be a skilled communicator to peers to executives. Articulate effective customer stories to drive change and inform strategy of product initiatives to business strategy.