Peach Pilot — Principal QA Engineer
Location
United States
Posted
43 days ago
Salary
0
Seniority
Lead
No structured requirement data.
Job Description
Peach Pilot — Principal QA Engineer
Hive Financial Systems
Peach Pilot — Principal QA Engineer (AI Systems & Platform) Remote — US Based | Full-Time | Early Engineering Team The Mission: Trust Has to Be Earned — Every Release 95% of enterprise AI pilots fail — not because the technology is broken, but because users don't trust it. At Peach Pilot, we are building an enterprise AI operating system where trust is the product. That means every feature we ship must work exactly as the user expects, every time. One broken interaction at the wrong moment can undo months of adoption. You are the last line of defense before our platform reaches a CFO's desk. We are a funded startup co-founded by Mario Montag (ex-McKinsey, Predikto founder — acquired by Fortune 50) and JP James (Georgia Tech alum, US patent holder in AI/ML, Senior Fellow at the National War College, four-time Atlanta 500 honoree). The Role This is a player-coach hire. You will build and own the QA function at Peach Pilot — writing test code, designing eval pipelines, and setting the quality bar — while also standing up and growing a QA team as the company scales. We are not looking for someone who manages spreadsheets and delegates everything. We are looking for someone who can do the work, knows what good looks like, and builds a team around that standard. The Challenge: QA for AI is a Different Problem Traditional QA assumes deterministic outputs. LLMs don't give you that. You will be building a quality function from scratch in an environment where: - Multi-model routing (Claude, GPT-4o, Grok, Gemini) means the same input can produce different outputs depending on which model handled it. - Agent orchestration and governance agents must maintain a structurally separate audit trail — any drift between execution and governance is a critical failure. - The file ingestion pipeline (Word, Excel, PowerPoint, PDF) must survive edge cases that enterprise clients will find within the first week of deployment. - Your users are CEOs and operations leaders who have never used a terminal. A confusing error state isn't a minor bug — it kills adoption. This is not a ticket-closing role. This is a quality architecture and team leadership role. What You Will Own & Build Build the QA Foundation (First 90 Days) - Establish the testing framework from zero: unit, integration, end-to-end, and LLM-specific evaluation pipelines. - Define quality standards, test coverage requirements, and documentation practices in partnership with the Lead Engineer. - Audit the existing platform and identify the highest-risk surfaces before the next major customer deployment. - Define the team structure you will need — onshore vs. offshore mix, roles, and a hiring roadmap — and begin executing against it. Build and Lead the QA Team - Recruit, hire, and onboard QA engineers as the team grows, setting clear expectations, working standards, and a bar for technical excellence from day one. - Mentor junior and mid-level QA engineers — building their ability to own test domains independently rather than creating dependency on you. - Act as the quality culture carrier across the full engineering team — QA is not a department, it is everyone's responsibility, and you will make that real. - Report directly to the Lead Engineer and participate in product planning to ensure quality is designed in, not bolted on. AI & Agent Testing - Design evaluation frameworks for non-deterministic LLM outputs — including prompt regression testing, model drift detection, and output quality scoring across Claude, GPT-4o, Grok, and Gemini. - Build automated test suites for the agent orchestration layer, including governance agent audit trail integrity and human-override behavior. - Validate the Enterprise Knowledge Graph (Neo4j + vector search) for data accuracy, retrieval quality, and failure modes under real enterprise data conditions. Platform & Integration Testing - Own end-to-end testing of the file ingestion pipeline across document types (Word, Excel, PowerPoint, PDF) including encryption, formatting edge cases, and audit trail continuity. - Validate streaming response handling, latency thresholds, and graceful degradation when a model is unavailable or slow. - Test multi-model routing logic to confirm cost-optimized task allocation behaves correctly across LLM providers. UX Quality & FDE Support - Partner with the Full-Stack Engineer to define and test trust-layer UX standards — onboarding flows, progressive disclosure, uncertainty states, and real-time document viewers. - Build reusable test playbooks for Forward Deployed Engineers to use in new customer deployments and agent configurations. - Act as the internal advocate for the non-technical enterprise user — if a CEO would be confused by it, it doesn't ship. Who You Are - The Player-Coach: You have 7+ years of QA engineering experience, with at least 3 years in a lead or senior role where you both wrote test code and managed or mentored other engineers. You do not delegate the hard problems — you model how to solve them. - Team Builder: You have experience leading and growing QA teams. You know how to hire for technical excellence, set up engineers to own domains independently, and build accountability without micromanagement. - AI-Native Tester: You have hands-on experience testing LLM-powered applications — you understand prompt sensitivity, output variance, and how to build eval pipelines that catch regressions across model updates. - Automation-First: You write test code. Python is your primary tool. You have built and maintained CI/CD-integrated test suites and you don't wait for someone to file a bug to find one. - Integration Expert: You are comfortable testing complex API chains, async/streaming responses, and multi-service workflows. Document processing pipelines and knowledge graph outputs don't intimidate you. - 0-to-1 Mindset: You have built a QA function from the ground up in an early-stage environment. You know when to move fast and when to go deep, and you can make that call without being told. - Enterprise Empathy: You understand that your end users are not developers. You test for confusion and trust failure, not just broken functionality. The Stack You'll Test Against - AI/LLM: Anthropic Claude, OpenAI GPT-4o, xAI Grok, Gemini, OpenClaw - Frontend: React/Next.js, TypeScript, Tailwind CSS - Backend: Python, Node.js/TypeScript (FastAPI/Express) - Data & Graph: Neo4j, Snowflake, Azure Cosmos DB, Azure AI Search - Infrastructure: Azure (Functions, Key Vault), CI/CD pipelines - Visualization: Plotly, D3, Recharts, Mermaid Even Better If - You have experience with LLM evaluation frameworks (e.g., LangSmith, PromptFlow, or custom eval pipelines). - You have tested agent frameworks such as LangChain or CrewAI. - You have a background in enterprise software or regulated industries where audit trail integrity is non-negotiable. - You have worked alongside Forward Deployed or solutions engineering teams and understand field deployment risk. Why This is Different - You are building the QA function and the team — not inheriting either. Your decisions will define how this company ships software for the next five years. - You will work directly with the founding engineering team. Your findings shape the roadmap, not a backlog queue. - Real enterprise data, real deployments, real consequences. No toy environments. - Meaningful equity as a founding-team engineering hire. Compensation & Benefits - Base Salary: Broad range (Commensurate with experience) - Equity: Meaningful founding-team equity package - Benefits: Comprehensive medical, dental, and vision; 401(k); flexible PTO - Location: Fully Remote — US Based The Clincher Tell us about a quality failure — one you caught before it shipped, or one that got through. What did you build or change after it, and how did you make sure your team could catch the next one without you?
Related Guides
Related Categories
Related Job Pages
More QA Automation Engineer Jobs
About Us Tutor Me Education is a EdTech company specializing in services for school districts. We move fast, and we're hiring our first dedicated automation engineer to multiply what our small team can do. The Role You will own automation and AI across the company. Your job is to find manual, repetitive, or slow work and eliminate it — with scripts, AI agents, and connected workflows. You'll report directly to the founder, pick your own stack, and ship weekly. This role is not about building customer-facing platforms or production web apps. It's about making our internal operation run like a machine. What You'll Build AI Agents & LLM Workflows - Chatbots and AI assistants for tutor recruiting screening, parent FAQs, and internal knowledge Q&A - LLM pipelines that extract data from school district documents, IEPs, and contracts - Automated generation of district-ready impact reports, proposals, and outreach emails - RAG systems over our internal docs so the team can query company knowledge Automation & Integrations - Connect our CRM, scheduling platform, Google Workspace, email, and comms tools - Automated tutor onboarding, document collection, and compliance tracking - Data syncing between systems — kill the copy-paste work - Alerting and monitoring workflows (Slack/email triggers from key events) Internal Tooling (lightweight only) - Google Apps Script, Airtable automations, simple internal dashboards - No production web apps, no customer-facing platforms — if you love building full SaaS products, this isn't the role What We Offer - 80K–120K PHP/month based on experience - 10 days PTO + holidays - Performance reviews every 6 months with raise eligibility - Direct access to the founder — you'll see your work change how the company runs
Spécialiste - Normes, pratiques et conformité en assurance des entreprises
Promutuel AssurancePromutuel Assurance est un leader de l’assurance de dommages au Québec.
Une carrière accessible et valorisante Vous êtes en quête de nouveauté? Promutuel Assurance vous offre d’évoluer au cœur d’un milieu de travail tourné vers l’humain, où règne un esprit d’équipe hors du commun. Voici l’occasion de développer vos compétences au sein d’une communauté de spécialistes de partout au Québec, qui s’affairent à façonner l’avenir de notre industrie. À propos du poste Ce poste clé au cœur de la conformité et de l’excellence des pratiques en assurance des entreprises est offert en mode 100 % télétravail, avec un rôle conseil à fort impact. Avoir une maîtrise avancée de la langue anglaise, autant à l’oral qu’à l’écrit, afin d’assister et de servir quotidiennement, notre clientèle anglophone présente sur le territoire du Nouveau-Brunswick et répondre aux objectifs de croissance du Groupe Promutuel au moyen d’interactions complexes liées notamment aux assurances ou aux secteurs d’activités du Groupe. Responsabilités - Évaluer la qualité des pratiques au moyen de différents types d’audits afin d'en assurer la conformité aux normes en vigueur, aux procédures internes et aux orientations de l’entreprise. - Mesurer la qualité de notre rôle-conseil ainsi que l’expérience vécue par nos membres-assurés et nos partenaires d’affaires à l'aide de divers outils et indicateurs. - Identifier et recommander des stratégies d’amélioration des procédures dans une optique de valeur ajoutée et d'efficacité opérationnelle, pour nos membres-assurés et nos partenaires d’affaires. - Contribuer à l’élaboration des contenus de formation et à leur diffusion et accompagner les ressources dans le développement de leurs compétences. - Assumer un rôle-conseil en matière de conformité réglementaire - Traiter les plaintes et assumer toute autre responsabilité connexe au sein de l’équipe. Avez-vous ce profil-LÀ? - Détenir un DEC ou un AEC en assurance de dommages - Posséder un permis valide de l’AMF ou un titre de PAA ou FPAA (un atout) - Détenir une accréditation de formateur de la ChAD ou toute autre formation pertinente (un atout) - Cumuler au moins dix (10) années d’expérience en analyse de risques en assurance des particuliers ou des entreprises - Maîtriser la suite Microsoft Office - Démontrer une excellente capacité d’analyse et un bon esprit de synthèse - Faire preuve d’autonomie, de proactivité et de rigueur professionnelle au quotidien - S’investir pleinement dans ses mandats, avec une ouverture naturelle au changement et à l’évolution des pratiques - Savoir vulgariser des concepts complexes afin d’influencer, conseiller et mobiliser efficacement Une panoplie d’avantages ici et LÀ - Rémunération avec échelles évolutives - Plusieurs ressources pour vous supporter, vous et vos proches - Assurances collectives pour vous et votre famille - Service de télémédecine pour répondre rapidement aux préoccupations de vous et vos proches - Un Programme d’aide aux Employés et leur famille pour toute sorte de support - Régime de retraite à prestations déterminées avec participation de l’employeur - Rabais sur vos assurance auto et habitation - Activités sociales organisées par l’employeur - Bureaux modernes, adaptés aux différents modes de travail - Certification Bien au travail Niveau 1 Un employeur qui priorise votre bien-être, vous êtes rendu(e) LÀ! - Soyez vous-même au sein d’équipes qui ont le bien-être, le respect et l’inclusion à cœur - Intégrez une équipe tissée serrée qui prône la solidarité - Venez bonifier notre savoir collectif grâce à vos expériences diversifiées - Évoluez dans une entreprise engagée à faire une différence durable et à adopter des pratiques responsables - Faites valoir vos idées et vos opinions - Contribuez à offrir une expérience distinctive à nos membres-assurés - Joignez une organisation qui vous accompagnera dans votre dépassement professionnel - Œuvrez dans un milieu de travail, classé au 3e rang du top 60 des leaders en expérience membre selon la plateforme Officevibe de Workleap Soyez vous-même chez nous Chez Promutuel Assurance, nous célébrons la diversité de notre monde et nous croyons que c'est cette unicité qui fait notre richesse. Toute personne, peu importe son profil, est encouragée à postuler chez nous. Offrir un milieu de travail inclusif, humain et bienveillant demeure notre priorité. Si vous avez des besoins spécifiques liés à votre candidature, n'hésitez pas à communiquer avec nous.
¡Prepárate para vivir una nueva etapa con Periferia IT Group! 🌟Si tienes experiencia como QA Automation Engineer y quieres generar un impacto en el mundo tecnológico, esta es tu oportunidad para unirte a nuestro equipo. Trabajarás con más de 1,000 profesionales en una multinacional colombiana líder en el sector TI, con fuerte presencia en América Latina y Estados Unidos. ¡Impulsa tu carrera con nosotros y haz crecer tu potencial! 🚀📈💼💻 ¿Te gustaría llevar tu carrera al siguiente nivel con Periferia IT Group? - 👥 Únete a nuestra cultura Periferia enfocada al servicio y la innovación, donde cada idea cuenta para superar las expectativas de nuestros clientes. - 🚀 Sumérgete en un entorno laboral dinámico que fomenta el aprendizaje continuo y el desarrollo personal. - 💡 Participa en proyectos desafiantes donde podrás aplicar tus conocimientos y habilidades para generar un impacto real. - 🤝 Disfruta de un equilibrio entre tu vida personal y laboral, mientras promovemos el bienestar integral de todos nuestros colaboradores. Para destacar en tu postulación debes: - 🎓 Ser PROFESIONAL en Ingeniería de Sistemas, Ingeniería Industrial o carreras afines. 💻 Contar con experiencia superior a 4 años en desarrollo de proyectos de automatización de pruebas (QA Automation). 📊 Dominar en un nivel ALTO herramientas de automatización como Selenium, Playwright o Cucumber, así como pruebas de APIs (Postman, REST Assured, Karate, SoapUI) y bases de datos. 🌱 Tener la capacidad de adaptarse a nuevas tecnologías y enfoques innovadores para maximizar su impacto. Tus responsabilidades serán: - 📈 Diseñar, desarrollar y ejecutar frameworks de automatización escalables para pruebas E2E en web, APIs y móviles. 🧠 Definir estrategias de prueba alineadas a metodologías ágiles, garantizando calidad, eficiencia y cobertura de los productos. 💼 Optimizar continuamente los procesos de automatización, integrando pruebas en pipelines y asegurando la calidad del software entregado. Lo que encontrarás al unirte al mundo de Periferia IT Group: - 🏠 Trabajo remoto en Bogotá. - 📜 Contrato a término indefinido - 💰 Salario a convenir según experiencia y habilidades. - 🎁 Beneficios a la carta para tu bienestar y estilo de vida. ¡Da el paso y únete hoy a nuestro equipo Periferia IT! 🎉
We are supporting a leading financial services organization on a large-scale Contact Centre transformation initiative, with a focus on Amazon Connect and IVR modernization. Reporting to the QA Manager, this role is responsible for ensuring the reliability, performance, and accuracy of IVR workflows, call routing logic, and contact centre applications through a mix of manual and automated testing. Employment Type: 6-month contract (extension possible) Location: Toronto, ON - Remote What You'll Do: - Design, develop, and execute test strategies and test cases for IVR and contact centre applications. - Perform end-to-end testing of call flows, routing logic, and customer journeys within Amazon Connect. - Build and maintain automation frameworks for API, backend, and IVR testing. - Validate integrations between Amazon Connect and downstream systems (CRM, APIs, databases). - Conduct functional, regression, and integration testing across contact centre platforms. - Analyze call logs, system behavior, and test results to identify defects and root causes. - Collaborate with developers, product owners, and business stakeholders to ensure quality delivery. - Support release cycles, including test planning, execution, and defect tracking. Requirements - 5+ years of experience in QA / Quality Engineering, with a focus on test automation. - Hands-on experience in IVR / Contact Centre testing for Amazon Connect. - Experience with test automation tools (e.g., Selenium, API testing tools). - Solid understanding of call flows, routing logic, and IVR systems. - Experience with performance or load testing for contact centre systems. - Knowledge of CI/CD pipelines and test automation integration. - Experience working in Agile / Scrum environments. - Financial Services or regulated industry experience. ------------------------ At Talencore, we stand against racism and discriminatory practices. We stand against violence. We proudly support and partner with organizations that lift people up so everyone can succeed and thrive, and honour our candidates' experiences, perspectives and unique identity. We are committed to continuing to make diversity, equity, and inclusion part of everything we do – from how we work with our talented candidates to how we support our clients in building their diverse workforce.
