AI Quality Analyst

Location

USA Timezones

Posted

36 days ago

Salary

0

Seniority

Mid Level

No structured requirement data.

Job Description

AI Quality Analyst

Texas Sports Academy Main

Role Description We run an AI-powered SMS bot and an AI-graded call training system for a private school network. Both systems have extensive test suites (400+ test files) and a RAG-based knowledge base that drives bot behavior. We need someone to own the full quality cycle: - Review production conversations to find bad bot responses - Create test files (JSON) that reproduce the issue - Fix the knowledge base content (Markdown files, WRONG/RIGHT examples) - Run test suites to verify fixes and catch regressions - Maintain and expand the knowledge base as our program evolves This is NOT a software engineering role. The infrastructure is built. You're operating it — editing JSON, writing Markdown, running CLI scripts, reading test reports. Qualifications - Native-level English fluency (non-negotiable) - Comfortable editing JSON and Markdown in an IDE - Can run commands in a terminal without hand-holding - Extreme attention to detail - Ability to learn a complex domain quickly (education, state government programs, etc) Requirements - JSON test files (you'll write and edit these daily) - Markdown knowledge base documents - Terminal commands (copy-paste and run scripts) - Git (commit, push, basic branching) - VS Code or similar editor Nice-to-Haves - Experience writing test cases or QA documentation - Experience with chatbot QA, conversational AI testing, or LLM evaluation Hours & Ramp-Up - 30 hrs/week to start, ramping up as needed. US time zones required. - The first 1-2 weeks will be focused on learning our domain — how our school works, how the state voucher program works, compliance rules, etc. We have extensive internal documentation.

Related Categories

Related Job Pages

More Analyst Jobs

BuiltIn Integration Sandbox logo

Senior Financial Analyst

BuiltIn Integration Sandbox

We deliver the most advanced and flexible learning experience for certification, credentialing, test prep, continuing education, and training. Our cloud-based learning platform helps training organizations, associations, and the extended enterprise deliver a highly engaging and effective learning experience for individuals looking to advance their careers. We incorporate the latest in learner-centered technology, including personalization, gamification, data science, usability, and omni-channel delivery. We’re committed to helping people learn better, and that starts with our own people. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Analyst36 days ago
Full TimeRemoteTeam 201-500

Role Description This position is remote (along with all of our BenchPrep positions). BenchPrep is looking for a hardworking, ambitious Staff/Senior Accountant to support its growing operations. This position will report to the Controller. Within the Accounting function at BenchPrep, you'll play a pivotal role in our finance team. Your focus will be on maintaining accurate financial records, preparing reports, analyzing data, and ensuring regulatory compliance. You'll contribute to budgeting, forecasting, audits, and process improvements, supporting our financial health and strategic decision-making. The ideal candidate has a strong background in accounting, financial analysis, and reporting, along with excellent analytical skills, attention to detail, and the ability to communicate financial information effectively across the organization. Proactivity, problem-solving abilities, and a commitment to continuous improvement are also essential. - Manage accounting function including account reconciliations, journal entries, AR, AP, and Billing - Analysis and review of company expenditures, confirming proper allocation of expenses to the various departments, and reporting to these departments as necessary - Review transactions to ensure accounting treatment is in accordance with GAAP and company policies and procedures - Analyze billing transactions to ensure proper allocation of revenue recognition policies - Review general ledger accounts and making all necessary journal entries including posting monthly, quarterly, and yearly accruals - Assist in building financial statements including balance sheet, income statement, and changes in financial position, as well as budget variance analysis - Assist in the design and preparation of budgets for review by management - Prepare year-end schedules for the annual tax returns - Look for ways to improve upon accounting processes and procedures Qualifications - Bachelor’s degree in accounting; CPA preferred - 3-5 years (Staff Accountant), or 5-8 (Sr Accountant) years in accounting or auditing; public accounting experience preferred - Team player with a positive attitude towards contributing and learning - Thorough knowledge and experience in the areas of financial statement preparation, GAAP, and revenue recognition - Strong verbal/written communication skills - Excellent analytical and problem solving skills - Ability to work well independently and in a team environment - Self starter, quick learner, and ability to multitask on a regular basis - Detail-oriented with excellent organizational skills and deadline driven - Capacity to thrive in a fast-paced environment - Strong sense of ownership on issues and following through to completion - Advanced proficiency with Microsoft Excel - Experience working in accounting systems (NetSuite, Intacct, Quickbooks, etc.) strongly preferred Benefits - Medical and ancillary benefits - Potential for yearly bonus - Stipends for work-from-home expenses - Flexible paid time off (our version of unlimited PTO) - Annual professional development stipend dollars - Competitive base salary Company Description We deliver the most advanced and flexible learning experience for certification, credentialing, test prep, continuing education, and training. Our cloud-based learning platform helps training organizations, associations and the extended enterprise deliver a highly engaging and effective learning experience for individuals looking to advance their careers. We incorporate the latest in learner-centered technology, including personalization, gamification, data science, usability and omni-channel delivery. We’re sitting in a pivotal time in the BenchPrep history. The number of learners on our cloud-based learning platform has reached nearly 11 million as of 2024. We’re committed to helping people learn better, and that starts with our own people. We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We are looking for high performing and motivated professionals who are excited about the chance to leverage technology to impact the lives of millions of learners. Join us.

United States
$65K - $85K / year
Job Closed
Analyst36 days ago
Full TimeRemoteTeam 1,001-5,000H1B Sponsor

• Managing the daily operations of the T&E program at Blackboard • Approving expense reports • Managing global card programs both domestically and internationally • Reconciling T&E charges in our expense travel tool, Concur • Managing travel for the company and working closely with the Senior Travel & Expense Analyst.

Colombia
Job Closed
ília logo

Junior Personnel Department Analyst - Affirmative for People with Disabilities (PCD)

ília

Somos especialistas em tecnologia, dados e design, impulsionando a transformação digital de grandes players do mercado há mais de 10 anos, nos setores financeiro, seguros e mobilidade. Com mais de 450 profissionais, estamos presentes no Brasil e Europa, atendendo aos mercados da América Latina, Europa e América do Norte, desenvolvendo produtos digitais de alta qualidade e com foco em resultados de negócios. Certificada pelo 7° ano consecutivo como Great Place to work aqui na ília acreditamos que pessoas mudam o mundo, e investimos nelas. Nossas awesome deliveries são feitas de pessoas para pessoas, afinal awesome people make awesome deliveries!

Analyst36 days ago
Full TimeRemoteTeam 501-1,000

Role Description - Processar o fechamento e conferência da folha de pagamento; - Atuar com eSocial, FGTS Digital, DCTFWeb e demais obrigações legais; - Conduzir rotinas de rescisão, férias, admissão; - Realizar manutenção do cadastro de funcionários e controle de ponto; - Acompanhar e gerenciar processos de benefícios, como vale-transporte, vale-alimentação, plano de saúde, entre outros benefícios; - Atendimento aos colaboradores para esclarecimento de dúvidas relacionadas à folha, benefícios, férias, rescisões e outros temas de DP; Qualifications - Experiência prévia em Departamento Pessoal e sistemas de folha; - Vivência prática com eSocial, DCTFWeb e FGTS Digital; - Vivência prática em Google Sheets e Excel. Requirements - Conhecimento em acordos e convenções coletivas; - Formação/Cursando superior em Administração, Recursos Humanos, ou áreas correlatas. Benefits - Nossa contratação é CLT- 40h semanais com jornada flexível, sendo executada de forma remota. - Para Saúde e bem-estar: - Plano de Saúde e Odontológico SulAmérica extensivo a dependentes; - Vale Alimentação/Refeição em cartão flexível Caju benefícios; - Seguro de Vida; - Auxílio Home-Office em cartão flexível Caju benefícios; - Wellhub (Gympass); - Sesc – extensivo a dependentes, com acesso aos serviços em todo o Brasil; - TotalPass. - Para o seu Desenvolvimento: - ília University: Universidade Corporativa com mais de 20 mil cursos disponíveis para desenvolvimento pessoal e profissional; - Language Academy: Escola de idiomas para ílians; - í-talks e Chapter: Momentos e cerimônias em que o time compartilha práticas, estudos, projetos e ideias. - Benefícios não convencionais: - Plano de Saúde PET - Guapeco; - Onhappy – benefício de viagem a lazer, com liberdade para você viajar com quem quiser; - BYOD - Alugamos o seu notebook pessoal te pagando um valor mensal para que você o use; - Seu Niver, seu bolo! - Seu Networking Vale Prêmio - Programa de premiação a indicação de novos funcionários.

Brazil
Job Closed
Full TimeRemoteTeam 1,001-5,000H1B No Sponsor

• Support annual planning process for volume and investment. • Support in developing distribution strategy, goals, and tracking performance. • Identify opportunities for plan optimization and provide recommendations to the Leadership Team. • Support end-to-end management of the planning process to ensure all key decision milestones are met. • Track, analyze, and report on variances to commercial volume and investment plans. • Develop insights into monthly forecasts with input from cross-functional teams. • Develop planning learnings that result in optimization of the planning process. • Build strong relationships with commercial leadership teams using a consultative approach. • Addition ad hoc analysis and duties as assigned.

New York
$87.6K - $131.4K / year