Overview As a Principal Product Manager, you will lead the human-in-the-loop evaluation program that drives the quality and experience of M365 Copilot Chat for Business—ensuring M365 Copilot is the best AI assistant for work, grounded in M365 data. You'll build and scale the people, processes, and evaluation pipelines that turn expert human judgment into reliable quality signals that directly shape the product. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities - Define what great looks like for human data—bringing your own knowledge and perspective on the best approaches to collecting, interpreting, and applying human feedback across product decisions. - Lead the human-in-the-loop evaluation program—designing evaluation frameworks, scorecards, and quality benchmarks that measure and continuously raise the bar on M365 Copilot's response quality, helpfulness, and user experience. - Build and manage evaluation workforce operations, including vendor partnerships, annotator onboarding, qualification, training, and continuous performance management. - Partner with data scientists and engineers to scope evaluation needs, define task instructions, calibrate annotators, and ensure evaluation data is reliable and repeatable. Qualifications Required Qualifications: - Bachelor's Degree AND 8+ years experience in product/service/program management or software development - OR equivalent experience. Other Requirements: Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. Preferred Qualifications: - Bachelor's Degree AND 8+ years experience in product/service/program management or software development - OR equivalent experience. - 3+ years of human data or human-in-the-loop evaluation experience. - Experience at an AI research organization or AI data services provider. - 4+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework). - 6+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn). - Experience building and managing workforce programs, including vendor partnerships and annotation operations at scale. - Proficiency in evaluation pipeline design, annotation frameworks, and quality governance. - Track record of translating human feedback and evaluation signals into measurable product impact. #AI, #Humandata Product Management IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.