Cantina logo
Cantina

Building the first social AI platform

Machine Learning Engineer, Core Evaluations

Machine Learning EngineerMachine Learning EngineerFull TimeRemoteSeniorTeam 51-200Since founded by Sean ParkerH1B SponsorCompany SiteLinkedIn

Location

United States

Posted

38 days ago

Salary

0

Seniority

Senior

Professional CertificateEnglish

Job Description

Machine Learning Engineer, Core Evaluations

Cantina

• Designing model evaluation pipelines for models in development and production • Designing user studies for subjective model evaluations. • Converting requirements into measurable metrics. • Designing and developing automated evaluation dashboard to see model performances and compare results. • Training new models to capture new and different evaluation metrics. • Communicating with the model team to help design better models based on the evaluation results. • Communicating with the data team to help decide the type of data necessary to improve model performance. • Communication with the product-manager to make sure product requirements are correctly measured. • Help grow the evaluation team as the founding member. • Lead the evaluation team in the future.

Job Requirements

  • Strong experience and intuition for designing metrics that capture model performance.
  • Strong experience with designing user studies on Mechanical Turk or similar platforms.
  • Strong experience with model training and fine-tuning for model evaluation.
  • Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
  • Very strong engineering and programming skills.
  • Experience with training ASR, TTS models.
  • Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)

Related Job Pages

More Machine Learning Engineer Jobs

MIS - Analytics Engineer Virtual• United States Job Description: General Description: Phillips & Cohen Associates Ltd. is an award-winning probate estate and debt settlement industry leader. We are seeking an Analytics Engineer to support Strategy & Analytics function here at PCA and help identify opportunities that drive performance. The ideal candidate should demonstrate strategic curiosity, challenge assumptions and translate analysis into insights. This role combines analytics engineering, data modeling, automation, and applied analytics to transform data into actionable insights, connect data analysis to execution to accelerate growth and competitive advantage. The Analytics Engineer plays critical role in turning data into strategic asset – ensuring the business can rely on accurate, timely insights to drive performance and growth. Responsibilities: Support analytics use cases (segmentation, forecasting, A/B testing). Develop, implement and optimize debt recovery processes, including dialer, scoring and portfolio segmentation. Engineer new data assets and derived datasets when required information is not available in existing source systems. Develop scalable data models and schemas to support portfolio valuation, forecasting, reporting, and operational analytics. Eliminate conflicting metrics and definitions, creating a single source of truth across teams. Build and maintain automated end-to-end analytics workflows, including scheduling, orchestration, alerts, and reporting. Implement data quality rules, validation logic, automated QA checks, and monitoring frameworks to ensure data integrity. Perform exploratory data analysis to identify trends, data gaps, and metric behavior related to portfolio performance. Translate business questions and strategy needs into technical requirements, datasets, and analytical solutions. Create reports, dashboards, and analytical outputs that are usable and interpretable by non-technical stakeholders. Optimize SQL queries and ETL/ELT processes for performance, accuracy, and reliability. Collaborate closely with analytics, engineering, and business partners to align data definitions, metrics, and logic. Document data pipelines, models, transformations, business rules, and data dictionaries to support governance and continuity, ensuring consistent metric definition across the organization. Qualifications: Minimum of 3 years of experience in analytics engineering, data engineering, data analytics, or similar hybrid analytics roles. Advanced SQL skills with hands-on experience building scalable transformations and data models. Proficiency in programming language such as SAS, Python or R for analytics, automation, and modeling. Demonstrated experience designing datasets and data architecture to support advanced analytics and forecasting. Experience with BI and visualization tools such as Power BI, Tableau, or Looker. Strong analytical thinking with the ability to translate complex data into actionable business insights. Ability to work effectively in ambiguous, open-ended problem spaces. Understanding of statistical concepts and predictive modeling techniques. Strong written and verbal communication skills with both technical and non-technical audiences. Experience developing SQL Server jobs, stored procedures, triggers, or similar database objects. Familiarity with Git or version control systems. Background in non-performing debt industry, financial services, portfolio analytics, forecasting, or operations analytics is a plus.

United States
Narvar logo

Senior Software Engineer II, Machine Learning

Narvar

Simplify the everyday lives of consumers.

Full TimeRemoteTeam 201-500Since 2012H1B Sponsor

• Design and deploy machine learning algorithms for use cases spanning e-commerce, consumer trends, markets, logistics, and new products • Work on real-world consumer data for natural language processing, image classification, time series analysis, outlier detection, user modeling, etc • Work with large unstructured data • Be at the intersection of mathematics, machine learning, business, and computer science and impact millions of users through your work • Multiply the effect of your data science and data team members by building frameworks, tools, and methodologies that the entire team use • Provide thought leadership to a team through high quality write-ups, reviews, and a strong vision grounded in practical experience and a wider industry view

Canada
$180K - $230K / year
Fresh Prints logo

ML Engineer – Signal Processing, ASR

Fresh Prints

America's fastest growing collegiate custom apparel company six years in a row.

Full TimeRemoteTeam 11-50Since 2013H1B No Sponsor

• Design, build, and improve ASR, audio, and speech-related ML systems for production. • Develop signal processing pipelines for noisy, compressed, telephony-style, or real-world audio. • Train, fine-tune, evaluate, and deploy models for ASR, audio classification, diarization, redaction, or related tasks. • Own ML workflows end-to-end: data preparation, model training, validation, inference, monitoring, and iteration. • Optimize inference for latency, throughput, cost, and reliability. • Debug model quality issues through data analysis, targeted evaluations, and production monitoring. • Collaborate with product and engineering teams to turn business problems into practical ML solutions.

India
Global InfoTek, Inc. logo

Senior AI/ML Engineer

Global InfoTek, Inc.

Where rising standards meet global demands.

Full TimeRemoteTeam 51-200Since 2001H1B No Sponsor

Role Description GITI is seeking a Senior AI/ML Engineer to support an R&D program focused on passive RF emitter identification and network analysis from real-time sensor data streams. The Senior AI/ML Engineer designs, builds, and validates machine learning models for RF emitter identification, conducts hands-on exploratory data analysis on NDF (Network Description File) sensor datasets, and implements ML data pipelines that operate on constrained tactical edge hardware. Working under the direction of the Principal AI/ML Engineer and program technical lead, the candidate collaborates closely with research scientists and software engineers to translate analytical findings into reproducible, well-documented ML experiments and pipeline components. The role requires strong Python and deep learning skills, comfort with real-world noisy sensor data, and the ability to work in air-gapped Linux environments without cloud infrastructure or GPU acceleration. Responsibilities - Design, build, and validate machine learning models for RF emitter identification — including feature engineering from sensor data, training pipeline development, model evaluation, and iterative refinement based on results. - Conduct hands-on exploratory data analysis on RF sensor datasets using Python and Jupyter notebooks — writing and running analytical code, characterizing feature distributions, identifying data quality issues, and producing documented findings. - Implement and maintain ML data pipelines — ingesting NDF sensor streams, applying rollup and preprocessing logic, constructing training datasets, and ensuring pipeline correctness on constrained edge hardware with no cloud dependency. - Collaborate with the technical lead and Principal AI/ML Engineer to investigate RF sensor data quality, attribution reliability, and feature behavior under contention — writing code to characterize error sources, validate assumptions, and reproduce findings. - Produce clear technical documentation of experiments, model configurations, and results — maintaining reproducibility through disciplined versioning, and contributing to monthly status reports and team knowledge sharing. Qualifications - Bachelor's or Master's (or equivalent) with 5–7 years of hands-on applied experience. Requirements - 5+ years of hands-on applied experience in machine learning, data science, or RF signal processing. - Demonstrated proficiency in Python for ML and data science work — PyTorch or TensorFlow for model development, Pandas/NumPy for data manipulation, and scikit-learn or similar for evaluation and baseline modeling. - Hands-on experience designing, training, and evaluating deep learning models — particularly metric learning, Siamese networks, or other similarity-learning architectures — on real-world, noisy, imbalanced datasets. - Practical experience handling real-world data quality problems — missing values, label noise, class imbalance, systematic bias, and sensor artifacts — and the ability to diagnose and address them without discarding valid data. - Ability to develop and run ML pipelines on Linux-based systems without cloud infrastructure or GPU acceleration — optimizing for CPU-only inference and multi-threaded data processing on resource-constrained x86 hardware. Desired Skills - Familiarity with RF signal characteristics, passive receiver phenomenology, and sensor data interpretation — including awareness of processing artifacts, attribution ambiguities, and measurement limits common in signals intelligence datasets. - Hands-on experience applying machine learning — particularly metric learning, deep learning networks, or similarity-learning architectures — to RF or time-series signal data, including feature engineering, training pipeline development, and model validation. - Exposure to TDMA network protocols or military datalink systems, and interest in learning the signal processing challenges of dense, contested electromagnetic environments. - Familiarity with direction-finding, time-difference-of-arrival (TDOA), or related passive geolocation concepts — understanding of their mathematical foundations and common failure modes is more important than operational experience. - Experience with binary serialization formats (FlatBuffers, Protocol Buffers) and high-throughput sensor data pipelines operating in near-real-time on resource-constrained hardware. - Background in statistical signal processing — error ellipses, bearing estimation uncertainty, feature reliability under noise — with the ability to distinguish statistically significant findings from artifacts of small sample size or improper normalization. Relevant Certifications - Certifications in machine learning, data science, or related technical fields (e.g., TensorFlow Developer Certificate; PyTorch Certified Associate; AWS Certified Machine Learning — Specialty; Microsoft Certified: Azure AI Engineer Associate; Certified Analytics Professional (CAP); etc.). Company Description Global InfoTek Inc. has an award-winning track record of designing, developing, and deploying best-of-breed technologies that address the nation's pressing cyber and advanced technology needs. GITI has rapidly merged pioneering technologies, operational effectiveness, and best business practices for over two decades.

United States