Job Closed
This listing is no longer active.
Building the first social AI platform
Machine Learning Engineer, Core Evaluations
Location
Europe
Posted
34 days ago
Salary
0
Seniority
Senior
Job Description
Machine Learning Engineer, Core Evaluations
Cantina
• Designing model evaluation pipelines for models in development and production • Designing user studies for subjective model evaluations. • Converting requirements into measurable metrics. • Designing and developing automated evaluation dashboard to see model performances and compare results. • Training new models to capture new and different evaluation metrics. • Communicating with the model team to help design better models based on the evaluation results. • Communicating with the data team to help decide the type of data necessary to improve model performance. • Communication with the product-manager to make sure product requirements are correctly measured. • Help grow the evaluation team as the founding member. • Lead the evaluation team in the future.
Job Requirements
- Strong experience and intuition for designing metrics that capture model performance.
- Strong experience with designing user studies on Mechanical Turk or similar platforms. .
- Strong experience with model training and fine-tuning for model evaluation.
- Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
- Very strong engineering and programming skills.
- Experience with training ASR, TTS models.
- Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)
Benefits
- Please Note: to ensure that candidates select the most relevant jobs for their skills, we have set up limits to the number of times candidates can apply. The following limits apply to all roles at Cantina:
- Candidates may not apply more than 3 times in any 60 days for any job
- Candidates may not re-apply to any role within 30 days if not presented with an offer (for re-applying to the same job, the limit is 90 days)
Related Guides
Related Job Pages
More Machine Learning Engineer Jobs
• Ship production LLM agents with tool use: orchestration (state, pause/resume, journey/mode routing, failure handling) • MCP or gateway-style tool integration (schemas, auth, idempotency, audit) • Customized evals • Build alongside internal team
• Design, build, and deploy AI agent workflows that enable measurable efficiencies across business functions • Implement orchestration patterns including single-agent and multi-agent systems with appropriate state management • Integrate AI capabilities with existing tools and platforms via APIs, plugins, and function calling • Define and enforce guardrails for agent behavior, capabilities, and permissions in partnership with product and security • Build reusable automation patterns and templates that can be adopted across teams • Work alongside our data platform team to enable AI-driven analytics and reporting tools • Implement monitoring, observability, and feedback loops for deployed AI workflows • Ensure responsible AI practices, including fairness, transparency, and risk mitigation • Evaluate and recommend AI tooling, frameworks, and model providers based on business needs and cost • Stay current on industry trends, emerging AI capabilities, and best practices in automation engineering • Mentor engineers on integrating AI into their development workflows
Engenheiro MLOps, AWS Sênior
3CON Consultoria e SistemasEnabler de inovação e transformação digital, construindo soluções de impacto para os negócios.
• Desenvolver, validar e manter pipelines de MLOps, garantindo a automação de processos e integração com serviços da AWS. • Projetar e manter a arquitetura de sistemas, assegurando alinhamento com os objetivos dos projetos. • Garantir a sustentação e evolução da ferramenta de automação n8n. • Trabalhar em colaboração com cientistas de dados para operacionalizar modelos de machine learning. • Atuar na resolução de chamados relacionados ao time de MLOps. • Propor e desenvolver soluções técnicas (como aplicações web, APIs e agentes baseados em LLMs) para apoiar a área de Ciência de Dados.
Senior Machine Learning Engineer, General AI, ML, Big Data
C-ServTech Talent Solutions | Building High-Impact Teams Worldwide
**What You'll Do** - Define and drive the technical vision for ML solutions across products and platforms - Own the end-to-end software development lifecycle — from design and code reviews through to deployment and operations - Architect high-performance, scalable microservices, including synchronous and asynchronous web services - Build real-time inference pipelines for complex models using Triton, TensorRT, and mixed-precision computing - Mentor engineers, set technical direction, and foster a strong team culture - Champion engineering excellence, system resilience, and continuous operational improvement
