Role Description Are you an expert at navigating the complex architecture of Large Language Models? Welo Data is seeking a highly technical Senior Prompt Engineer based in Japan to lead the end-to-end migration of template workflows into high-performance LLM autoraters. This is a specialized role for a technical architect who understands that "perfecting a prompt" is a rigorous engineering discipline. You will leverage advanced APG/APO tools and manual refinement to ensure our automated systems meet—and exceed—human accuracy baselines in both German and English contexts. - The Mission: Automated Quality at Scale - Architectural Migration: Take full ownership of the end-to-end technical migration of templates to LLM autoraters. - Optimization Leadership: Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and logic deadlocks. - Metrics-Driven Excellence: Continuously measure quality against "gold data" baselines, tracking precision, recall, and F1 scores to justify launch readiness. - Edge-Case Engineering: Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools cannot solve. Qualifications - Bachelor’s, Master’s, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field. - 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning. - High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance. - Fast learner capable of mastering proprietary internal tools and "Goose API" style interfaces with minimal oversight. Requirements - Native fluency in Japanese and professional fluency in English. - Part-Time (Set your own hours within project milestones). - 100% Remote (Must be currently based in Japan). - Freelance / Independent Contractor. Preferred Technical Skills - Familiarity with shadowbot monitoring and disagreement tracking between human and LLM ratings. - Hands-on experience with Chain-of-Thought (CoT) prompting and APO systems. - Deep linguistic expertise, including a strong understanding of semantics and formal logic. - Proven ability to draft high-level Launch Certification Documentation. Company Description Join our team at Welo Data and embark on a journey of growth and innovation.

Role Description Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey. Position: Language Model Evaluator Type: Full-time or Part-time Contract Work Compensation: $23/hour Location: Geography restricted to Egypt, Saudi Arabia, UAE, USA Role Responsibilities - Evaluate LLM-generated responses on their ability to effectively answer user queries. - Conduct fact-checking using trusted public sources and external tools. - Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies. - Assess reasoning quality, clarity, tone, and completeness of responses. - Ensure model responses align with expected conversational behavior and system guidelines. - Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines. Qualifications - Must-Have: - Bachelor’s degree - Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in Arabic - Significant experience using large language models (LLMs) - Excellent writing skills - Strong attention to detail - Adaptable and comfortable moving across topics, domains, and customer requirements - Background or experience in domains requiring structured analytical thinking - Excellent college-level mathematics skills - Preferred: - Prior experience with RLHF, model evaluation, or data annotation work - Experience writing or editing high-quality written content - Experience comparing multiple outputs and making fine-grained qualitative judgments - Familiarity with evaluation rubrics, benchmarks, or quality scoring systems Application Process - Upload resume - AI interview based on your resume - Submit form Resources & Support - For details about the interview process and platform information, please check: Interview Process Details - For any help or support, reach out to: support@mercor.com PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Role Description Are you an expert at navigating the complex logic of Large Language Models? Welo Data is seeking a technical Prompt Engineer to lead the end-to-end migration of template workflows into high-performance LLM autoraters. This isn't just about writing prompts—it’s about engineering a technical bridge. You will use advanced APG/APO tools and manual refinement to ensure our automated systems meet (and exceed) human accuracy baselines. - The Mission: Architecting the Future of Rating - Technical Migration: Take ownership of the workflow for transitioning templates to LLM autoraters. - Optimization Leadership: Run and supervise Automated Prompt Optimization (APO) tools, identifying where logic plateaus and providing the manual "spark" to overcome deadlocks. - Metrics-Driven Accuracy: Continuously measure quality against gold data, calculating critical performance metrics like precision, recall, and $F_1$ scores. - Edge-Case Engineering: Solve complex scenarios by designing manual prompts that handle anti-patterns and broken logic in legacy architectures. Qualifications - 2+ years of experience as a Prompt Engineer. - BS, MS, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field. - Fast learner capable of mastering proprietary internal tools and interfaces (like the Goose API) with minimal supervision. - Strong ability to identify error patterns and use SQL or data analytics tools to analyze model performance. Requirements - Familiarity with shadowbot disagreement tracking between humans and LLMs. - Hands-on experience with Chain-of-Thought (CoT) and few-shot learning. - Proven ability to draft high-level Launch Certification Documentation. Benefits - Part-Time (Flexible hours within project milestones) - 100% Remote (Must be based in the United States) - Freelance / Independent Contractor Company Description Join our team at Welo Data and embark on a journey of growth and innovation.

Role Description Are you an expert at navigating the complex architecture of Large Language Models? Welo Data is seeking a Senior Prompt Engineer to lead the technical migration of template workflows into high-performance LLM autoraters. This role is designed for a technical specialist who understands that "perfecting a prompt" is a rigorous engineering discipline. You will leverage advanced APG/APO tools and manual refinement to ensure our automated systems meet—and exceed—human crowd baselines in accuracy and nuance. - The Mission: Automated Quality at Scale - Architectural Migration: Take full ownership of the end-to-end technical migration of templates to LLM autoraters. - Optimization Leadership: Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and deadlocks. - Metrics-Driven Excellence: Continuously measure quality against "gold data" baselines, tracking precision, recall, and $F_1$ scores to justify launch readiness. - Edge-Case Engineering: Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools can't solve. Qualifications - Bachelor’s, Master’s, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field. - 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning. - High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance. - Fast learner capable of mastering proprietary internal tools and "Goose API" style interfaces with minimal oversight. Requirements - Familiarity with shadowbot monitoring and disagreement tracking. - Experience in AI model evaluation and software engineering. - Deep understanding of semantics, logic, and Chain-of-Thought (CoT) prompting. - Proven ability to draft high-level Launch Certification Documentation. Recruitment & Onboarding - Technical Review: Submit your CV and portfolio of LLM optimization work. - Prompt Assessment: Demonstrate your ability to navigate complex template clusters and APO deadlocks. - Tooling Deep-Dive: Get access to our internal technical suite and migration workflows. - Launch: Begin your part-time engagement and lead the shift to LLM-driven autorating.