Role Description We are looking for a highly skilled Product Analyst with a deep understanding of developing behavior-driven tests with JavaScript and Java languages. As a part of our team, you will play a pivotal role performing complex business analysis and application development related duties that include: - Providing lead direction to other analytical staff - Providing strategic input - Managing delivery that would require coordination across multiple internal and external stakeholders Qualifications - Bachelor's degree in related field - Minimum of 3 years of experience in capturing requirements and experience along all phases of agile software development - Ability to obtain a U.S. Federal Position of Trust clearance designation - Must reside in and be able to perform work in the United States - Must have lived in the United States for 3 of the last 5 years Requirements - Knowledge of Healthcare, Medicare & Medicaid systems & data - Excellent oral and written communication skills - Expertise with Agile tools like JIRA, Confluence, Slack, etc. - Must be able to communicate to management, peers, and customers effectively and professionally - Demonstrate sound judgment/reasoning skills - Experience with manual testing of software products Responsibilities - Develop behavior-driven tests with JavaScript and Java languages - Write automated integration tests, API tests, and/or UI automation tests - Maintain and improve automation architecture design and implementation - Identify automation framework inefficiencies and make recommendations for improvements - Address complex automation issues of the automation team - Mentor other automation team members and train functional testers - Design test strategies, test plans, and test cases from requirements, design documents, and specifications - Record automation test results and report them - Prepare comprehensive reports on testing progress/issues and remedial action - Define, develop, and implement automation best practices and standards and procedures - Execute test scripts during internal system testing - Document test issues and work with the development team to validate/recreate - Perform retesting and regression testing - Oversee tests to ensure that performance and quality products conform to established policy and standards - Investigate problems and develop disposition and corrective actions for recurring discrepancies - Interface with all program staff to ensure requirements are met - Participate in meetings as required Benefits - Top-tier benefits package to invest in your physical, mental, and financial health and wellness - Opportunities to learn new skills, seize new challenges, and advance your career

Role Description We're hiring an AI Product Analyst to help us evaluate, monitor, and improve Voice AI systems operating at scale. You'll work at the intersection of Product, Engineering, Operations, and AI, helping the team understand how AI behaves in the real world, where it creates value, where it fails, and what should be improved next. Why this role: - Work on one of the most visible AI products inside MEDvidi. - Help shape the future of Voice AI in healthcare. - Influence product decisions using real production evidence. - High ownership in a fast-moving, product-driven medical tech company. Responsibilities: - Analyze production behavior of Voice AI Agents and conversational AI workflows. - Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations. - Work directly with call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights. - Design evaluation frameworks for measuring AI quality, reliability, and business impact. - Build monitoring systems and dashboards for AI performance visibility. - Measure the impact of AI systems on operational efficiency, business outcomes, and patient experience. - Detect quality shifts, anomalies, and emerging failure modes in production environments. - Partner closely with Product, Engineering, QA, and Operations teams on AI initiatives. - Support rapid decision-making in fast-changing and ambiguous environments. This role is not for you if: - You prefer working only with structured datasets and predefined metrics. - You are uncomfortable analyzing raw conversations, transcripts, and AI-generated outputs. - You expect clearly defined answers rather than investigating ambiguous problems. - You are not interested in understanding how AI systems fail in production. Qualifications - 3+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics roles within complex production systems. - Strong SQL skills. - Python for analytical investigations and working with large-scale text datasets. - Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes. - Experience building monitoring systems, analytical frameworks, or operational dashboards. - Strong communication skills and ability to translate analysis into product decisions. - Ability to work effectively in ambiguous and fast-changing environments. - Fluent Russian; English level B2 or higher. - Ability to work CET-overlapping hours. Requirements - Experience with conversational AI or AI agents. - Experience evaluating LLM outputs. - Experience with AI quality metrics and evaluation methodologies. - Experience with prompt iteration and AI evaluation frameworks. - Experience working with ML pipelines or AI-powered products. Benefits - High-impact environment: A chance to influence how AI is used in real patient interactions. - Growth & compensation: Competitive compensation and clear growth opportunities. - Remote-first: Fully remote, long-term B2B collaboration. - Health & wellness: Health insurance after the probation period, plus sports & wellness compensation. - Career development: Career growth, including personalized English lessons via Preply. - Time off: 19 working days of vacation per year, 3 additional wellness days, and paid sick leave for the first 5 working days. - Culture: Thoughtful gifts for important life events.

• Translate complex business needs into clear requirements, user stories, and scalable solutions • Collaborate with Product Owners and tech teams to shape AI-driven product use cases • Analyze user needs and drive data-driven product improvements (incl. A/B testing) • Own end-to-end delivery: planning, timelines, risks, and execution • Lead agile ceremonies and ensure on-time, high-quality delivery • Support development of AI solutions, frameworks, and innovative use cases • Act as a bridge between business and engineering to ensure alignment and clarity • Drive stakeholder communication, remove blockers, and enable continuous improvement

• Work closely with product feature teams to understand their roadmap and translate initiative goals into data KPIs, targets, and measurement plans. • Extract and integrate data from various sources to support analytical efforts, coordinating with data product, engineering, data science, and other analytics teams as needed. • Lead A/B test experiments, validate test data, monitor progress, perform post-test analysis, and provide recommendations for feature release decisions, collaborating with the experimentation platform team as necessary. • Convert business questions into data analytics tasks, conduct thorough analyses to segment and explore data, identify key drivers, and generate actionable insights to inform business decisions. • Collaborate with data scientists on research and innovation, applying standardized processes and enhancing analytics methodologies as they become available. Contribute to the development and use of digital tools, reporting visualization, and analytical storytelling. • Work with the reporting team to define and create automated and self-serve dashboards, providing relevant information efficiently to optimize business operations and track key initiatives.