Zendesk logo
Zendesk

The intelligent heart of customer experience.

Senior Product Manager, AI Agents Testing

Product ManagerProduct ManagerFull TimeRemoteSeniorTeam 5,001-10,000Since 2007H1B SponsorCompany SiteLinkedIn

Location

Germany

Posted

7 days ago

Salary

0

Seniority

Senior

Bachelor Degree2 yrs expEnglish

Job Description

Senior Product Manager, AI Agents Testing

Zendesk

• Own product strategy and roadmap for AI agent testing — simulation, quality scoring, experimentation, regression detection, and conversation tracing • Ship testing as an integrated experience embedded in the builder and deployment flow • Define how simulation works end-to-end: scenario generation from real conversation patterns, automated pass/fail evaluation, and results that point admins to exactly what broke and where • Build the experimentation layer — A/B testing of agent behavior, staged rollouts with statistical rigor, safe iteration on tone and resolution strategies • Design a pre-publish readiness gate that gives admins a quantified view of risk before every deployment — specific issues, coverage gaps, comparison to current production behavior • Partner with ML, QA, and platform teams on scoring methodology, simulation infrastructure, and tracing architecture • Make all of this usable by non-technical admins — CX managers, bot builders, operations leads who need answers without writing code or filing engineering tickets

Job Requirements

  • Several years of product management experience, with 2+ years building for non-technical users in complex technical domains (QA tooling, no-code platforms, admin consoles, workflow builders) in B2B SaaS
  • Experience shipping AI/ML products where evaluation and reliability were real concerns, not afterthoughts
  • You understand why traditional testing doesn't work for LLM-based systems and have opinions about what does
  • Ability to ship platform capabilities through user-facing product surfaces — you don't just build infrastructure, you make it usable
  • Experience integrating acquired or adjacent products into a unified experience — combining capabilities from different teams, codebases, or organizations into something that feels like one product
  • Track record coordinating across 3+ engineering teams and multiple departments to deliver one coherent product experience

Benefits

  • Health insurance
  • 401(k) matching
  • Flexible work hours
  • Paid time off
  • Remote work options

Related Categories

Related Job Pages

More Product Manager Jobs

AlpacaDB logo

Product Manager – New Assets

AlpacaDB

AlpacaDB, Inc., also known as Alpaca and Alpaca Securities, is an API stock and crypto brokerage platform that enables services to embed investing and developer

Product Manager7 days ago

• Own one core trading sub-surface end-to-end (a new asset class or an ancillary trading product) - drive prioritization, API design, and rollout inside your surface. • Run discovery with partners and prospects against real integration constraints, translate findings into a sequenced backlog for your surface, and push back on requests that don’t fit the platform without over-promising. • Ship to production at the cadence of a high-growth platform - write clear PRDs, align with engineering on API contracts, and produce documentation partners can implement against. • Partner with compliance, legal, and market-structure / execution teams on product decisions that touch licensing, trading rules, best execution, and venue connectivity - comfortable shipping in parallel with them in a regulated environment. • Improve operational efficiency in your surface - exception handling, partner-facing tooling, automation of repetitive ops - and own the metrics. • Track adoption beyond GA - trading volume, order success rate, partner activation, partner-reported blockers - and iterate on docs, SDK ergonomics, and rollout sequencing until your surface is actually being adopted. • Report timely and accurately on product status and results to internal and external stakeholders, including partners.

United States
Full TimeRemoteTeam 10,001+Since 1949H1B Sponsor

• Collaborate with marketing partners to develop annual business and strategic plans. • Execute marketing strategies, planning and coordination for Medtronic ENT’s product lines that build Medtronic’s market penetration in the US for nerve monitoring. • Develop messaging, positioning, and pricing strategies for the product in the US market. • Leverage market research and develop competitive strategies and tactics to grow market share. • Provide input into the long-term platform lifecycle, including analyses to support product line changes: ex .new product development. acquisition or discontinuation. • Prepare forecasts and short-range budgets for product lines. • Gather detailed information, analysis, and investigation to overcome problems and issues faced regarding product marketing. • Lead development of nerve monitoring claims and value proposition. • Develop and maintain strong KOL relationships to enable effective product portfolio strategy. • Provide a central point of contact for US field sales. • Develop marketing communications, collateral, training, presentations, etc. • Develop effective promotional campaigns, sales enablement tools and sales training curriculum. • Consult with Medical Education and Sales Training to develop strategy for the existing procedural areas and any adjacent areas. • Perform duties in compliance with the Medtronic Quality System, environmental, health and safety related site rules, policies, or governmental regulations.

Florida + 1 moreAll locations: Florida | Texas
$133.6K - $200.4K / year
Stitch Fix logo

Product Manager – Product Evaluation, Assortment

Stitch Fix

Changing the way people find what they love.

Product Manager7 days ago
Full TimeRemoteTeam 5,001-10,000Since 2011H1B Sponsor

• Shape how Stitch Fix evaluates and learns about new products from the beginning of their lifecycle. • Apply first-principles thinking to turn ambiguous problems into clear, actionable ideas. • Work on high-impact, highly visible problems that influence inventory investment, product performance, and client experience. • Drive a critical system transformation toward a more unified, scalable merchandising platform. • Partner across merchandising, data science, and engineering to connect strategy, systems, and decision-making. • Help shape how AI changes both what we build and how we build products. • Enjoy diving into technical details across data systems, APIs, and integrations to solve complex problems.

California
$127.1K - $212K / year
Grafana Labs logo

Senior Product Manager

Grafana Labs

Headquartered in New York, New York, Grafana Labs is an IT company specializing in developing accessible, user-friendly, and open-source metric visualization tools. Established in

Product Manager7 days ago

• Transform ideas and feedback into a product vision and roadmap • Empathize with customer problems in order to collect user needs, requirements, and pain points • Curate and properly represent user feedback to Engineering, while providing evidence to support tradeoffs and priorities • Work closely with PMM to enable GTM on product differentiation, including creating and updating presentations, documentation, website content, datasheets, and whitepapers • Use your deep product expertise to make decisions about our go-to-market strategies, pricing, packaging, licensing and tiering

QC + 3 moreAll locations: QC | ON | NU | Canada
$164.5K - $197.4K / year