• Design and run experiments to improve agent quality: better tool use, better reasoning, better outputs, using frameworks like DSPy and VLLM • Build and maintain evaluation infrastructure to measure what's working and catch regressions before customers do • Optimize LLM inference: latency, cost, model routing, and quality tradeoffs • Partner with product teams on model selection and performance benchmarking • Work closely with product engineers and PMs to translate customer quality problems into ML hypotheses and solutions • Own models end-to-end: from research and experimentation to production deployment

• Conduct EDA and statistical profiling to identify trends and insights from data. • Perform feature engineering specifically for time-series forecasting. • Extract and transform data from relational databases (RDS, Oracle, PostgreSQL) into analytics-ready formats. • Develop pipelines for data ingestion and processing. • Build classical ML models for time-series forecasting, regression, and capacity/throughput modeling. • Evaluate model performance using metrics such as RMSE, MAE, and MAPE, documenting performance results. • Create insightful data visualizations and dashboards using Amazon QuickSight or equivalent BI tools. • Utilize the Python data stack (pandas, NumPy, scikit-learn, matplotlib/seaborn) for data manipulation and analysis. • Apply SHAP or other model explainability techniques to interpret model outputs. • Work closely with stakeholders to translate business rules into effective feature engineering pipelines. • Engage in milestone-driven, Firm Fixed Price delivery models, ensuring timely project completion.

• Collaborate with other engineering teams and cross functional teams to translate ambiguous business problems into a clear, prioritized ML product roadmap for your team. • Have a 'player-coach' mentality, and contribute hands-on technical expertise while providing strategic direction and mentorship to the team. • Establish a high-performing engineering setup that enables the team to rapidly iterate, experiment, and deploy models, fostering a culture of operational excellence and close partnership with product and business stakeholders. • Drive continuous improvement in the team's processes, tooling, and infrastructure to optimize for velocity, quality, and maintainability. • Actively monitor the ML and AI landscape to ensure the team is continuously up-to-speed on state-of-the-art research, techniques, and tools, promoting their strategic adoption where appropriate. • Recruit, manage, and develop a diverse and highly-skilled team of Machine Learning Engineers, setting clear goals and expectations, and providing timely and actionable feedback. • Define and track key performance indicators (KPIs) to measure the impact and success of the team’s ML products, communicating results clearly to leadership and stakeholders. • Manage project timelines, dependencies, and risks, ensuring timely delivery of high-quality, reliable, and scalable ML solutions.

• Perform rapid diagnosis across model, data, code, infrastructure, and evaluation layers for blocked or unstable efforts. • Identify root causes and define corrective actions required to restore progress. • Communicate findings and resolution plans clearly across research, engineering, and operational teams. • Contribute directly to blocked ML initiatives by implementing fixes across model behavior, data pipelines, and system architecture. • Develop and validate solutions, including debugging, targeted refactoring, and experimental validation. • Ensure that resolved systems are stable, validated, and ready for continued development. • Provide clear handoff artifacts, including working code, documentation, and recommended next steps. • Work across research, infrastructure, platform, evaluation, and integration teams to align on root causes and resolution plans.