Volga Partners logo
Volga Partners

Smart and Reliable Technology Solutions

Claims Auditing Subject Matter Expert, AI Evaluation Specialist – On-Call

Artificial IntelligenceArtificial IntelligenceContractRemoteSeniorTeam 1,001-5,000Since 2020H1B SponsorCompany SiteLinkedIn

Location

United States

Posted

1 day ago

Salary

$35 - $94.2K / hour

Seniority

Senior

Bachelor Degree4 yrs expEnglish

Job Description

Claims Auditing Subject Matter Expert, AI Evaluation Specialist – On-Call

Volga Partners

• Create and review finance, audit, insurance, compliance, and legal-related prompts, scenarios, and analytical tasks. • Evaluate AI-generated responses for factual accuracy, reasoning quality, and business relevance. • Apply your professional expertise to improve AI model performance. • Review analyses, assumptions, calculations, documentation, and recommendations. • Identify inaccuracies, inconsistencies, logical gaps, and opportunities for improvement. • Provide structured written feedback and quality assessments. • Collaborate with project teams to maintain high-quality deliverables. • Meet established productivity, quality, and turnaround expectations.

Job Requirements

  • Bachelor's degree in Finance, Accounting, Economics, Business, Mathematics, Statistics, Law, Insurance, Risk Management, or a related field.
  • Minimum 4 years of professional experience within one or more of the listed finance, audit, insurance, compliance, legal, or risk disciplines.
  • Strong analytical and critical-thinking skills.
  • Excellent written English communication skills.
  • Ability to evaluate complex business, financial, audit, compliance, insurance, or legal concepts and clearly explain your reasoning.
  • Reliable internet connection.
  • Ability to work independently in a remote environment.

Related Job Pages

More Artificial Intelligence Jobs

ContractRemoteTeam 1,001-5,000Since 2020H1B Sponsor

• Create and review finance-related prompts, scenarios, and analytical tasks. • Evaluate AI-generated responses for financial accuracy, reasoning quality, and business relevance. • Apply subject matter expertise to improve model performance and financial understanding. • Review financial analyses, assumptions, calculations, and recommendations. • Identify inaccuracies, inconsistencies, and areas for improvement. • Provide structured feedback and quality assessments. • Collaborate with project teams to maintain high-quality deliverables. • Meet established project timelines and productivity goals.

India
$5 - $9 / hour
Keywords Studios logo

Language AI Solutions Specialist

Keywords Studios

International technical and creative services provider to the global video games industry and beyond.

Full TimeRemoteTeam 10,001+Since 1998H1B No Sponsor

• As a Language AI Solutions Specialist, you will bridge the gap between traditional localization and cutting-edge AI. • You will be responsible for the end-to-end lifecycle of translation systems—from legacy Neural MT to modern LLM-based solutions. • Your mission is to architect, tune, and deploy high-performance language models that drive global scalability while maintaining uncompromising linguistic quality. • Hybrid Engine Deployment: Design, test, and certify Neural MT systems (KantanMT) and fine-tuned LLMs tailored to client-specific domains. • Performance Analytics: Deliver comprehensive quality reports, including Predictive TER scores, MQM-DQF/COMET/G-Eval metrics, and Human-in-the-Loop (HITL) evaluations. • Production Integration: Working closely with internal stakeholders (Development, Production, etc.) to oversee the seamless transition of validated models into live production environments. • AI Model Tuning & Optimization: Execute supervised fine-tuning (SFT) on LLMs to align output with Client/Project specific style, voice, and domain-specific terminology. • RAG Corpus Management: Architect and maintain high-quality corpora for Retrieval-Augmented Generation, ensuring that vector databases are populated with clean, relevant, and deduplicated data. • Data Engineering: Perform advanced data analysis, cleansing, and data generation to prepare clean datasets for model training and tuning. • Output Verification: Utilize native-level linguistic expertise to perform deep-dive audits of NMT and LLM outputs, identifying nuanced errors such as hallucinations, cultural insensitivity, or tone inconsistency that automated metrics might miss. • Linguistic Asset Engineering: Manage and optimize Translation Memories (TMs) and Termbases to serve as the foundational "ground truth" for AI training. • Quality Control: Implement and evolve Machine Translation Post-Editing (MTPE) workflows, integrating AI-assisted quality estimation (QE) to reduce manual overhead. • Analytics & Productivity Insights: Gather and analyze post-deployment data, focusing on Edit Distance, time-to-market reduction, and cost-per-word efficiency. • Stakeholder Reporting: Translate complex technical data into actionable insights for Program Managers and clients, recommending rebuilds or retraining cycles where necessary. • Operations & Troubleshooting: Serve as the escalation point for engine failures, API latency issues, or unexpected model "hallucinations.". • Process Documentation: Create "Living Documentation" for evolving AI workflows to ensure team-wide alignment and scalability.

South Korea
Full TimeRemoteTeam 1,001-5,000Since 2020H1B Sponsor

Role Description Volga Partners is seeking experienced claims auditing professionals to join a specialized AI initiative supporting the development, training, and evaluation of next-generation financial and insurance reasoning models. This opportunity allows you to apply your expertise in a unique and innovative way while working remotely and earning competitive project-based compensation. This is an on-call, project-based contract opportunity supporting ongoing AI training and evaluation initiatives. Candidates selected for this opportunity will become part of Volga Partners' on-call network of subject matter experts. - Create and review finance-related prompts, scenarios, and analytical tasks. - Evaluate AI-generated responses for financial accuracy, reasoning quality, and business relevance. - Apply subject matter expertise to improve model performance and financial understanding. - Review financial analyses, assumptions, calculations, and recommendations. - Identify inaccuracies, inconsistencies, and areas for improvement. - Provide structured feedback and quality assessments. - Collaborate with project teams to maintain high-quality deliverables. - Meet established project timelines and productivity goals. Qualifications - Bachelor's degree in Finance, Accounting, Economics, Business, Mathematics, Statistics, Law, Insurance, Risk Management, or a related field. - Minimum 4 years of professional experience within one or more of the listed finance, audit, insurance, compliance, legal, or risk disciplines. - Strong analytical and critical-thinking skills. - Excellent written English communication skills. - Ability to evaluate complex business, financial, audit, compliance, insurance, or legal concepts and clearly explain your reasoning. - Reliable internet connection. - Ability to work independently in a remote environment. Requirements - CFA Charterholder or CFA Candidate (Preferred) - ACCA, ACA, CMA, MBA Finance, M.Com, or similar credentials such as CIA, CISA or JD (Preferred) - Experience in financial modeling, valuation, forecasting, budgeting, investment analysis, or quantitative research (Preferred) - Experience reviewing technical content, research, or analytical reports (Preferred) - Exposure to AI, machine learning, data annotation, or model evaluation projects is a plus (Preferred) Benefits - $5.00–$9.00 USD per hour (depending on qualifications and project) - Fully Remote - Flexible Schedule - On-call assignments - 0–20 hours per week, depending on project availability - Temporary Contract Opportunity - Opportunities to participate in future AI evaluation initiatives Application Requirements - Updated Resume/CV - Primary Area of Expertise - Secondary Area of Expertise (if applicable) - Total Years of Relevant Experience - Current Location - Availability (Hours Per Day) - Personal Email Address

India + 1 moreAll locations: India | Pakistan
$5 - $9 / hour
Keywords Studios logo

Language AI Solutions Specialist

Keywords Studios

International technical and creative services provider to the global video games industry and beyond.

Full TimeRemoteTeam 10,001+Since 1998H1B No Sponsor

Role Description As a Language AI Solutions Specialist, you will bridge the gap between traditional localization and cutting-edge AI. You will be responsible for the end-to-end lifecycle of translation systems—from legacy Neural MT to modern LLM-based solutions. Your mission is to architect, tune, and deploy high-performance language models that drive global scalability while maintaining uncompromising linguistic quality. Key Deliverables - Hybrid Engine Deployment: Design, test, and certify Neural MT systems (KantanMT) and fine-tuned LLMs tailored to client-specific domains. - Performance Analytics: Deliver comprehensive quality reports, including Predictive TER scores, MQM-DQF/COMET/G-Eval metrics, and Human-in-the-Loop (HITL) evaluations. - Production Integration: Working closely with internal stakeholders (Development, Production, etc.) to oversee the seamless transition of validated models into live production environments. Core Responsibilities - AI Model Tuning & Optimization - Fine-Tuning & Hyperparameters: Execute supervised fine-tuning (SFT) on LLMs to align output with Client/Project specific style, voice, and domain-specific terminology. - RAG Corpus Management: Architect and maintain high-quality corpora for Retrieval-Augmented Generation, ensuring that vector databases are populated with clean, relevant, and deduplicated data. - Data Engineering: Perform advanced data analysis, cleansing, and data generation to prepare clean datasets for model training and tuning. - Output Verification: Utilize native-level linguistic expertise to perform deep-dive audits of NMT and LLM outputs, identifying nuanced errors such as hallucinations, cultural insensitivity, or tone inconsistency that automated metrics might miss. - Linguistic Asset Engineering - Modernization: Manage and optimize Translation Memories (TMs) and Termbases to serve as the foundational "ground truth" for AI training. - Quality Control: Implement and evolve Machine Translation Post-Editing (MTPE) workflows, integrating AI-assisted quality estimation (QE) to reduce manual overhead. - Analytics & Productivity Insights - Impact Metrics: Gather and analyze post-deployment data, focusing on Edit Distance, time-to-market reduction, and cost-per-word efficiency. - Stakeholder Reporting: Translate complex technical data into actionable insights for Program Managers and clients, recommending rebuilds or retraining cycles where necessary. - Operations & Troubleshooting - Technical Support: Serve as the escalation point for engine failures, API latency issues, or unexpected model "hallucinations." - Process Documentation: Create "Living Documentation" for evolving AI workflows to ensure team-wide alignment and scalability. Qualifications - Fluent English and native level of any core Asian languages (Chinese, Japanese or Korean) - 3-4 years of experience in the localization industry, working as a Project Manager, Translator, Post-Editor or QA tester - Machine Translation Post Editing: Solid understanding of MTPE, proven track record in correcting NMT or LLM outputs for fluent, accurate and brand-consistent target text. - Degree in Translation, Translation Technology or relevant experience. - Aptitude for analytics: Ability to understand and work with string-matching and edit metrics (e.g., explaining Edit Distance in a post editing context, understanding MQM-DQF frameworks etc.) Preferred Skills - Solid understanding of CAT tools and CMS/TMS systems in the localization industry - AI/LLM applications - knowledge of prompt engineering, RAG corpus management and familiarity with Vector Spaces - Basic knowledge of Python and Regex When You Apply - Please submit your resume in English. Resumes in any other language will not be reviewed. - Due to the high volume of applications, only shortlisted candidates will be contacted. - As this is a remote position, benefits may vary depending on the employing entity and the candidate’s country/state of residence. - Keywords Studios is committed to fair and ethical hiring practices. We expect candidates to conduct themselves with honesty and to participate in all applications and interviews independently, presenting their own knowledge and experience. Candidates should refrain from using AI assistance or third-party tools or services that could influence, capture, or otherwise interfere with the authenticity of the assessment process. - By providing your information in this application, you understand that we will collect and process your information in accordance with our Applicant Privacy Notice. Role Information - Studio: Keywords Studios - Location: Asia Pacific, Mainland China, Japan, Korea, Singapore, Taiwan - Area of Work: QA Testing Services, Localization - Service: Globalize - Employment Type: Full Time - Working Pattern: Remote

Asia Pacific