Coalition is the world's first Active Insurance provider designed to help prevent digital risk before it strikes. Founded in 2017, Coalition combines comprehensive insurance coverage and innovative cybersecurity tools to help businesses manage and mitigate potential cyberattacks. Work at Coalition is centered on the joint mission to Protect the Unprotected. We have built a remote-first, highly inclusive culture that welcomes people from diverse backgrounds. We trust each other to take responsibility, share ownership of outcomes, and put in the work together to protect businesses from digital risk. Coalition’s exceptional growth stems from its ability to address real-world problems for organizations of all sizes while remaining true to our founding values of character, humility, responsibility, purpose, authenticity, and inclusion.
Senior Manager, Machine Learning (Data Operations)
Location
United States
Posted
3 days ago
Salary
$134.4K - $200K / year
Seniority
Lead
No structured requirement data.
Job Description
Senior Manager, Machine Learning (Data Operations)
Coalition, Inc.
Role Description Coalition's machine learning models are only as good as the data they're trained on. This role exists to make sure that data is right. You'll own labeling quality and methodology across Coalition — designing annotation tasks, defining quality frameworks, and ensuring every labeled dataset meets the standard required to ship production ML models. You'll manage our labeling platform (Label Studio), work directly with ML and product teams to structure labeling programs, and oversee outsourced labeling vendors to hit quality and throughput targets. This role reports to the Chief Product Officer and sits at the intersection of product, ML, and operations. You won't manage internal labelers — all annotation work is outsourced — but you will be the single point of accountability for whether Coalition's labeled data is accurate, consistent, and fit for purpose. Responsibilities - Labeling quality & methodology: Define annotation guidelines, taxonomies, and edge-case protocols for each labeling program. Establish gold standard datasets, inter-annotator agreement (IAA) targets, and audit sampling processes. Identify and remediate mislabeled data in existing datasets. - Platform & tooling: Serve as the primary user and requirements driver for Label Studio — defining project configuration needs, workflow designs, pre-labeling pipeline requirements, and integration points with ML infrastructure. Partner with the data engineering team that builds and maintains the platform. - Cross-functional partnership: Work with ML engineers, data scientists, and product managers to translate model requirements into well-structured labeling tasks. Challenge teams on task design when labeling instructions are ambiguous or likely to produce unreliable labels. - Vendor management: Source, onboard, and manage external labeling vendors and BPOs in coordination with Coalition's operations team. Set quality SLAs, run calibration sessions, and manage feedback loops to labelers. Hold vendors accountable to accuracy, not just throughput. - Measurement & improvement: Define and track operational metrics — label accuracy, IAA scores, cost per label, turnaround time — and use them to drive continuous improvement. Identify opportunities for active learning, model-assisted labeling, and pre-annotation to reduce cost without sacrificing quality. Qualifications - 5+ years in ML data operations, data labeling, or a related field (ML engineering, data science, or data engineering with heavy labeling exposure) - Deep understanding of annotation quality frameworks: IAA, consensus labeling, gold standard evaluation, error taxonomy, and calibration workflows - Direct experience managing labeling platforms (Label Studio strongly preferred; Scale AI, Labelbox, Prodigy, or similar acceptable) - Track record managing outsourced labeling vendors or BPOs for ML data production - Familiarity with common ML labeling tasks: text classification, NER, document extraction, intent detection - Comfortable working in Python and SQL; bonus if you've built tooling around labeling workflows or quality measurement - Strong opinions on what makes labeled data good or bad, and the willingness to push back when it's bad - Experience in insurance, cybersecurity, or fintech is a plus but not required Compensation Our compensation reflects the cost of labor across several US geographic markets. The US base salary for this position ranges from $134,400/year in our lowest geographic market up to $200,000/year in our highest geographic market. Consistent with applicable laws, an employee's pay within this range is based on a number of factors, which include but are not limited to relevant education, skills, job-related knowledge, qualifications, work experience, credentials, and/or geographic location. Your recruiter can share more on target salary for your location during the interview process. Coalition, Inc. reserves the right to modify this range as needed. Benefits - 100% medical, dental and vision coverage - Flexible PTO policy - Annual home office stipend and WeWork access - Mental & physical health wellness programs (One Medical, Headspace, Wellhub, and more)! - Competitive compensation and opportunity for advancement
Related Guides
Related Categories
Related Job Pages
More Operations Jobs
Head of Operations
Kraken Digital Asset ExchangeWe put the power in your hands to buy, sell, and trade digital currency 🌏
• Oversee the daily operations of the EMI, ensuring all processes run smoothly and efficiently • Develop and implement operational strategies to support Kraken’s EMI growth and scalability • Identify areas for process improvement and implement changes to increase efficiency and effectiveness • Oversee all outsourcing to Kraken group service providers • Ensure that Kraken EMI operations comply with all EMI/PSR requirements and other relevant regulatory frameworks • Establish and maintain robust internal controls to ensure regulatory compliance and operational integrity • Collaborate with the regional compliance officer to prepare for regulatory audits, and to ensure accurate reporting • Identify, assess, and manage operational risks • Develop and implement strategies to mitigate operational risks, ensuring business continuity • Oversee crisis and incident management plans and procedures, ensuring that Kraken EMI is prepared for potential disruptions • Act as PIL's nominated senior executive accountable for outsourcing, in line with the CBI Cross-Industry Guidance on Outsourcing, including oversight of Payward Inc. (PINC) as PIL's material outsourced service provider; • Oversee ongoing monitoring and performance management of outsourced service providers, including PINC, against agreed service levels and risk indicators; • Ensure outsourcing arrangements comply with applicable CBI outsourcing requirements and, where relevant, DORA ICT third-party risk management requirements; • Act as the primary point of contact for outsourcing-related regulatory queries, including notifying the CBI of new or materially changed outsourcing arrangements as required; • Report on outsourcing risk, concentration, and performance to the CEO, Board, and relevant risk/outsourcing committees. • Have overall responsibility for the Fraud function running through PIL, including fraud prevention, detection, and case management outcomes; • Oversee the outsourced Fraud Operations service provided by Payward Inc. (PINC) as PIL's material outsourced service provider, monitoring performance against agreed SLAs/KPIs; • Ensure Fraud risk appetite, controls, and management information align with PIL's Risk Appetite Statement (RAS) and applicable CBI outsourcing/operational resilience requirements; • Escalate material fraud incidents, thematic trends, and control gaps to the CEO, Board, and relevant risk committees on a timely basis. • Oversee Kraken EMI’s technology infrastructure to ensure it meets operational needs and regulatory requirements • Work with technology leadership to ensure the security of Kraken EMIS’s data and systems • Promote technological innovation to enhance Kraken EMI’s operations and user experience • Ensure that Kraken EMI provides excellent, secure customer service and support • Implement mechanisms to gather and act on customer feedback to improve service delivery • Oversee processes for resolving customer complaints and disputes effectively • Ensure that the EMI has operational processes in place that ensure compliance with Consumer Protection Code 2025 • Support the General Manager, E-money and Payments Operations in developing and implementing the EMI’s strategic initiatives • Set operational goals that align with EMI and Kraken overall strategy • Monitor progress towards achieving strategic goals and make adjustments as needed • Collaborate with other senior executives to ensure cohesive and aligned operations; • Build and maintain relationships with key internal and external stakeholders.
• Operational process support & coordination — Partner with the Director to coordinate initiatives across Finance, Procurement, Banking and BPO-delivered processes; prepare and facilitate workshops, capture outputs, and drive follow-up actions to completion. • BPO resource optimisation & SLA support — Support the optimisation of offshore BPO resource utilisation by monitoring workload distribution, routing and workflow efficiency; help build and maintain SLA/KPI reporting for service reviews. • Process documentation, standards & governance — Develop and maintain process maps, workflows, RACIs and SOPs; keep documentation version-controlled and aligned to global standards, and produce clear work instructions and training materials. • ServiceNow framework & service catalogue — Support the development and upkeep of the ServiceNow framework — workflows, categories, ticket types, routing rules and service catalogue structures — ensuring the data model underpins reliable KPI, SLA and reporting. • Stakeholder engagement & communication — Build collaborative relationships across Finance, Procurement, Banking, BPO, IT and the wider business; communicate process updates and changes clearly and prepare materials for governance forums. • IT change request governance — Act as the operational point of contact for change requests (CRs) affecting Finance, Procurement, Banking and BPO processes; coordinate impact assessments and track CRs end-to-end through the governance model.
• Consolidate and ensure the quality of support operation data. • Define, monitor and evolve operational metrics (SLA, TMR, AHT, backlog, aging, throughput, among others). • Create executive and operational dashboards to track performance. • Develop critical analyses of the operation, identifying deviations, bottlenecks and improvement opportunities. • Support leadership in decision-making through data and indicators. • Monitor SLA compliance and the performance of support queues (N1, N2 and N3). • Prepare demand forecasts and operational capacity analyses. • Standardize metrics, definitions and data sources. • Automate reports and data extraction processes. • Manage and evolve Zendesk, ensuring correct tool configuration and reliable information. • Promote a data-driven culture within the support operation.
Director, Vice President – Service Operations
ANP Advanced Network Products, Inc: A Coretelligent CompanyPhiladelphia's Cloud-First Managed IT Service Provider
• Lead the Remote Support and Systems Engineering teams, fostering a culture of accountability, collaboration, and continuous improvement. • Develop and execute the strategic vision for Remote Service Operations, aligning operational priorities with business objectives. • Drive service quality by managing SLA performance, backlog health, utilization, escalations, operational efficiency, and client satisfaction. • Establish performance expectations, coach leaders, and build high-performing technical teams. • Mature ITIL-based Incident, Problem, Change, Request, and Major Incident Management processes. • Standardize operational processes, documentation, quality assurance, and governance to improve consistency and scalability. • Ensure operational readiness for a 24x7x365 support organization through workforce planning, capacity management, and business continuity. • Identify and implement process improvements, automation, and tooling that increase operational efficiency and service quality. • Oversee ServiceNow and supporting ITSM platforms to improve visibility, reporting, and operational governance. • Develop and manage departmental budgets, forecasting, resource planning, and operational financial performance. • Monitor organizational capacity, utilization, and staffing models to support business growth. • Build executive reporting and dashboards that provide meaningful insight into operational performance, service quality, and business trends. • Partner with Service Delivery, Professional Services, Security, Client Success, Sales, Product, and executive leadership to deliver a seamless client experience. • Serve as a trusted advisor to executive leadership, providing operational insights and supporting organizational growth, transformation, and acquisition initiatives.
