Job Closed
This listing is no longer active.
AI that handles patient communication perfectly, so healthcare teams can prioritize care.
Conversational Quality Analyst
Location
Philippines
Posted
80 days ago
Salary
$1.2K - $2K / month
Seniority
Senior
Job Description
Conversational Quality Analyst
Hello Patient
• Design and maintain evaluation rules: Define clear, testable pass/fail criteria for Mia’s conversational behavior by translating customer workflows and success metrics into evaluation standards. • Build and manage test suites: Create structured test suites to proactively surface failure modes before launch. • Review simulations and make launch decisions: Review 100+ simulated conversations prior to each bot launch. • Run regression testing: Re-test after model, prompt, workflow, or product updates to detect regressions introduced by changes. • Monitor and correct rule drift: Review dashboards weekly to spot drift or unexpected rule behavior. • Support quality reporting: Collaborate with the Business Analyst to define metrics and requirements for pre-launch quality reports.
Job Requirements
- 3+ years of experience in software QA, test case development, conversation review, trust & safety, or a closely related role
- Exceptional attention to detail and consistency—you catch edge cases others miss
- Strong written communication skills with the ability to clearly explain what’s broken and why it matters
- Strong pattern recognition skills—you identify themes across large volumes of conversations, not just individual failures
- Comfort using modern AI tools (e.g., Cursor) in day-to-day workflows
- Proactive ownership and risk awareness—you flag issues early rather than waiting for deadlines.
- Nice to Have: Experience writing guidelines, rubrics, rules, prompts, or evaluation criteria that define pass/fail outcomes.
- Nice to Have: Background reviewing voice or chat interactions at scale
- Nice to Have: Adversarial testing experience (e.g., fraud detection, cybersecurity, abuse testing, or edge-case analysis)
- Nice to Have: Healthcare or medical practice experience
- Nice to Have: Startup experience building QA or quality processes from scratch.
Benefits
- No benefits (health insurance, PTO, etc.)
Related Guides
Related Categories
Related Job Pages
More Analyst Jobs
Quoting Analyst
BottomlineBottomline is a global leader in business payments and cash management, with over 35 years of experience and moving more than $16 trillion in payments annually.
Why Choose Bottomline? Are you ready to transform the way businesses pay and get paid? Bottomline is a global leader in business payments and cash management, with over 35 years of experience and moving more than $16 trillion in payments annually. We're looking for passionate individuals to join our team and help drive impactful results for our customers. If you're dedicated to delighting customers and promoting growth and innovation - we want you on our team! The Role: As a Quoting Analyst, you will play a key role in analyzing and interpreting customer and commercial data to deliver accurate, timely pricing quotes to Sales, Channel Partners, and prospective end users. This role partners closely with cross‑functional teams to support strategic decision‑making, improve business processes, and drive bookings growth at scale. The ideal candidate is analytical, detail‑oriented, and comfortable managing multiple priorities in a fast‑paced environment. While this position is remote, preference will be given to candidates based on the East Coast to support collaboration across time zones. How you'll contribute: - Analyze large and complex datasets to identify trends, patterns, and insights; summarize key findings for internal stakeholders - Leverage Excel models and analytical tools to interpret data and provide clear, actionable recommendations - Support analysis and modeling for ad hoc projects, trainings, and presentation - Collaborate cross‑functionally to gather, document, and maintain detailed business requirements and processes - Evaluate existing workflows to identify opportunities for improvement; partner with stakeholders to design and implement more efficient processes - Ensure data accuracy, integrity, and reliability across reporting systems - Drive consistency and efficiency in internal reporting by researching best practices and supporting the development of scalable solutions If you have the attributes, skills, and experience listed below, we want to hear from you! - 5+ years of experience in reporting and data analysis, ideally partnering with Sales, customer‑facing teams, and/or Finance within a global technology environment - Hands‑on experience with tools such as Excel, Power Query, Power BI, Salesforce, and AI‑enabled tools - Highly organized, detail‑oriented, and able to manage multiple tasks and projects simultaneously - Flexible and adaptable, with the ability to thrive in a dynamic, evolving environment - Strong problem‑solving skills with the ability to translate data insights into meaningful business action - Proven ability to collaborate cross‑functionally, communicate effectively at all levels of the organization, and lead trainings or presentations - Bachelor’s degree in a relevant field We welcome talent at all career stages and are dedicated to understanding and supporting additional needs. We're proud to be an equal opportunity employer, committed to creating an inclusive and open environment for everyone.
• Monitor and evaluate the quality of customer service and services provided • Track operational and quality metrics, such as SLA, response time, resolution time and recurrence • Conduct audits of tickets, processes and operational workflows • Identify deviations, operational risks and continuous improvement opportunities • Support the definition, development and standardization of quality processes • Prepare operational and executive quality reports and service health reports • Collaborate with technical teams on incident analysis and service improvements • Contribute to increased customer satisfaction, trust and overall customer experience
• Deploy and configure IAM solutions (e.g., Ping Identity). • Participate in meetings with clients and stakeholders to gather requirements. • Create and maintain technical documentation (blueprints, flows, integrations). • Support the squad with technical questions and disseminate best practices. • Participate in agile ceremonies (dailies, planning, reviews, retrospectives). • Integrate legacy and modern systems with identity solutions. • Ensure compliance with security standards (OpenID, SAML, FAPI, Open Banking). • Perform testing, acceptance, and technical validations.
• Participate in IAM system deployments, covering the full cycle (Requirements, Development/Configurations, Testing, User Acceptance, and Support). • Customize integrations between systems (APIs, JSON, XML, Swagger, etc.). • Develop scripts (Java, Python, etc.) for solution customizations. • Create and maintain up-to-date technical documentation (blueprints, operational procedures). • Support tool updates and upgrades. • Monitor the status and progress of support tickets. • Ensure service delivery within contracted SLAs. • Escalate high-impact incidents to the appropriate levels. • Keep clients informed about the status and progress of support activities. • Continuously research new technologies and improvement methodologies.

