The intelligent heart of customer experience.
Senior Product Manager, AI Agents Testing
Location
Germany
Posted
7 days ago
Salary
0
Seniority
Senior
Job Description
Senior Product Manager, AI Agents Testing
Zendesk
• Own product strategy and roadmap for AI agent testing — simulation, quality scoring, experimentation, regression detection, and conversation tracing • Ship testing as an integrated experience embedded in the builder and deployment flow • Define how simulation works end-to-end: scenario generation from real conversation patterns, automated pass/fail evaluation, and results that point admins to exactly what broke and where • Build the experimentation layer — A/B testing of agent behavior, staged rollouts with statistical rigor, safe iteration on tone and resolution strategies • Design a pre-publish readiness gate that gives admins a quantified view of risk before every deployment — specific issues, coverage gaps, comparison to current production behavior • Partner with ML, QA, and platform teams on scoring methodology, simulation infrastructure, and tracing architecture • Make all of this usable by non-technical admins — CX managers, bot builders, operations leads who need answers without writing code or filing engineering tickets
Job Requirements
- Several years of product management experience, with 2+ years building for non-technical users in complex technical domains (QA tooling, no-code platforms, admin consoles, workflow builders) in B2B SaaS
- Experience shipping AI/ML products where evaluation and reliability were real concerns, not afterthoughts
- You understand why traditional testing doesn't work for LLM-based systems and have opinions about what does
- Ability to ship platform capabilities through user-facing product surfaces — you don't just build infrastructure, you make it usable
- Experience integrating acquired or adjacent products into a unified experience — combining capabilities from different teams, codebases, or organizations into something that feels like one product
- Track record coordinating across 3+ engineering teams and multiple departments to deliver one coherent product experience
Benefits
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Remote work options
Related Guides
Related Categories
Related Job Pages
More Product Manager Jobs
Product Manager – New Assets
AlpacaDBAlpacaDB, Inc., also known as Alpaca and Alpaca Securities, is an API stock and crypto brokerage platform that enables services to embed investing and developer
• Own one core trading sub-surface end-to-end (a new asset class or an ancillary trading product) - drive prioritization, API design, and rollout inside your surface. • Run discovery with partners and prospects against real integration constraints, translate findings into a sequenced backlog for your surface, and push back on requests that don’t fit the platform without over-promising. • Ship to production at the cadence of a high-growth platform - write clear PRDs, align with engineering on API contracts, and produce documentation partners can implement against. • Partner with compliance, legal, and market-structure / execution teams on product decisions that touch licensing, trading rules, best execution, and venue connectivity - comfortable shipping in parallel with them in a regulated environment. • Improve operational efficiency in your surface - exception handling, partner-facing tooling, automation of repetitive ops - and own the metrics. • Track adoption beyond GA - trading volume, order success rate, partner activation, partner-reported blockers - and iterate on docs, SDK ergonomics, and rollout sequencing until your surface is actually being adopted. • Report timely and accurately on product status and results to internal and external stakeholders, including partners.
• Collaborate with marketing partners to develop annual business and strategic plans. • Execute marketing strategies, planning and coordination for Medtronic ENT’s product lines that build Medtronic’s market penetration in the US for nerve monitoring. • Develop messaging, positioning, and pricing strategies for the product in the US market. • Leverage market research and develop competitive strategies and tactics to grow market share. • Provide input into the long-term platform lifecycle, including analyses to support product line changes: ex .new product development. acquisition or discontinuation. • Prepare forecasts and short-range budgets for product lines. • Gather detailed information, analysis, and investigation to overcome problems and issues faced regarding product marketing. • Lead development of nerve monitoring claims and value proposition. • Develop and maintain strong KOL relationships to enable effective product portfolio strategy. • Provide a central point of contact for US field sales. • Develop marketing communications, collateral, training, presentations, etc. • Develop effective promotional campaigns, sales enablement tools and sales training curriculum. • Consult with Medical Education and Sales Training to develop strategy for the existing procedural areas and any adjacent areas. • Perform duties in compliance with the Medtronic Quality System, environmental, health and safety related site rules, policies, or governmental regulations.
Product Manager – Product Evaluation, Assortment
Stitch FixChanging the way people find what they love.
• Shape how Stitch Fix evaluates and learns about new products from the beginning of their lifecycle. • Apply first-principles thinking to turn ambiguous problems into clear, actionable ideas. • Work on high-impact, highly visible problems that influence inventory investment, product performance, and client experience. • Drive a critical system transformation toward a more unified, scalable merchandising platform. • Partner across merchandising, data science, and engineering to connect strategy, systems, and decision-making. • Help shape how AI changes both what we build and how we build products. • Enjoy diving into technical details across data systems, APIs, and integrations to solve complex problems.
Senior Product Manager
Grafana LabsHeadquartered in New York, New York, Grafana Labs is an IT company specializing in developing accessible, user-friendly, and open-source metric visualization tools. Established in
• Transform ideas and feedback into a product vision and roadmap • Empathize with customer problems in order to collect user needs, requirements, and pain points • Curate and properly represent user feedback to Engineering, while providing evidence to support tradeoffs and priorities • Work closely with PMM to enable GTM on product differentiation, including creating and updating presentations, documentation, website content, datasheets, and whitepapers • Use your deep product expertise to make decisions about our go-to-market strategies, pricing, packaging, licensing and tiering




