Job Closed
This listing is no longer active.
Innodata, with over 35 years of expertise, is a trusted leader in data solutions and AI innovation. The company specializes in training and deploying generative
AI/ML Research Engineer, LLM Post-Training & Evaluation
Location
New Jersey
Posted
103 days ago
Salary
0
Seniority
Mid Level
Job Description
AI/ML Research Engineer, LLM Post-Training & Evaluation
Innodata
• Lead or co-lead technically complex ML engineering projects from initial customer discussions through implementation and delivery • Design, build, and improve LLM training and post-training pipelines, including data ingestion, preprocessing, fine-tuning, evaluation, and experiment tracking • Implement and optimize evaluation systems for LLMs and multimodal models, including offline benchmarks and task-specific test harnesses • Integrate human-in-the-loop and AI-augmented evaluation signals into model development workflows • Build robust infrastructure and tooling for reproducible experimentation, metrics logging, and regression monitoring • Diagnose model behavior and pipeline failures, including data issues, training instability, metric inconsistencies, and evaluation drift • Collaborate with Language Data Scientists and Applied Research Scientists to translate evaluation frameworks into executable systems • Work closely with customer technical stakeholders to understand goals, constraints, and success criteria; propose and implement technically sound solutions • Contribute to internal research and platform development, including benchmark frameworks, evaluation tooling, and post-training workflow improvements • Contribute to best practices and standards for LLM training, evaluation, and quality assurance across projects • Mentor junior engineers and contribute to technical design reviews, documentation, and engineering rigor across the team
Job Requirements
- BS/MS/PhD in Computer Science, Machine Learning, AI, Applied Mathematics, or related quantitative technical field (MS/PhD preferred)
- 2-3 years of relevant industry or research engineering experience in ML/AI systems
- Hands-on experience with LLM training / fine-tuning / post-training, including at least one of: supervised fine-tuning (SFT) preference optimization (e.g., DPO or related methods) RLHF / RLAIF-style workflows task- or domain-adaptation of foundation models
- Strong programming skills in Python and experience building production-quality ML code
- Experience with modern ML frameworks (e.g., PyTorch, JAX, TensorFlow) and model libraries/tooling (e.g., Hugging Face ecosystem, vLLM, distributed training stacks)
- Experience designing and implementing evaluation pipelines for LLM/ML systems, including metrics computation, dataset handling, and experiment comparisons
- Strong understanding of data pipelines and ML systems engineering, including reproducibility, observability, and debugging
- Experience with large-scale distributed ML systems and performance optimization for training/evaluation workloads (GPU/accelerator environments preferred)
- Experience with large-scale data processing and workflow orchestration in support of model training/evaluation
- Ability to collaborate directly with technical stakeholders including research scientists, ML engineers, data engineers, and customer technical leads
- Strong written and verbal communication skills, including the ability to explain complex technical tradeoffs to both technical and non-technical audiences.
Benefits
- Health insurance
- Retirement plans
Related Guides
Related Categories
Related Job Pages
More Research Engineer Jobs
Freelance Expert Consultant (AI Training & Evaluation)
MindriftApply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
We are building the next generation of AI agents capable of navigating the complex consulting tasks. To achieve this, we are looking for consultants with 5+ years of experience at McKinsey & Company, Boston Consulting Group, or Bain & Company. The role is not client-facing and does not involve stakeholder management. Instead, the consultant will use their direct experience from real consulting projects to: - Recreate realistic consulting project environments - Design structured problem-solving tasks - Define what high-quality outputs should look like - Evaluate AI-generated outputs against consulting standards Why an MBB Background Matters Practical knowledge of how consulting projects actually work in detail is required. Specifically: - How ambiguous client questions are translated into structured workplans - How hypotheses are formed and tested - How data gaps are handled in real projects - What “good” looks like in slide decks, models, and executive summaries - How trade-offs and recommendations are developed under uncertainty The AI must be trained on environments that reflect real consulting complexity. Only someone who has worked on multiple MBB projects can reliably recreate this level of realism. While we value all analytical backgrounds, this role specifically requires first-hand MBB project experience. What You’ll Do Design realistic consulting environments - Create detailed project scenarios (industry context, financials, constraints, incomplete information) - Develop structured tasks such as market sizing, due diligence, cost reduction, growth strategy, or operational diagnosis Define what “good” looks like - Set clear evaluation criteria for high-quality consulting outputs - Distinguish between superficial answers and consulting-grade reasoning Evaluate AI outputs - Review analyses and identify logical errors, weak assumptions, and structural gaps - Provide precise, actionable feedback to improve system performance Continuously improve training quality - Refine scenarios, edge cases, and evaluation frameworks - Increase realism based on real project experience Skills & Requirements - 5+ years at McKinsey, BCG, or Bain - Strong structured problem-solving skills - Ability to translate vague problems into clear analytical steps - High attention to logical consistency - Independent working style Basic technical familiarity required: - Reading structured formats (e.g., JSON or YAML) - Basic Git/GitHub usage - Clear written English (B2+) What This Role Is Not - No client interaction - No stakeholder management - No sales or business development - No team leadership responsibilities This is a focused analytical role for consultants who want to apply their problem-solving skills to advanced AI systems.
Freelance Expert Consultant (AI Training & Evaluation)
MindriftApply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
We are building the next generation of AI agents capable of navigating the complex consulting tasks. To achieve this, we are looking for consultants with 5+ years of experience at McKinsey & Company, Boston Consulting Group, or Bain & Company. The role is not client-facing and does not involve stakeholder management. Instead, the consultant will use their direct experience from real consulting projects to: - Recreate realistic consulting project environments - Design structured problem-solving tasks - Define what high-quality outputs should look like - Evaluate AI-generated outputs against consulting standards Why an MBB Background Matters Practical knowledge of how consulting projects actually work in detail is required. Specifically: - How ambiguous client questions are translated into structured workplans - How hypotheses are formed and tested - How data gaps are handled in real projects - What “good” looks like in slide decks, models, and executive summaries - How trade-offs and recommendations are developed under uncertainty The AI must be trained on environments that reflect real consulting complexity. Only someone who has worked on multiple MBB projects can reliably recreate this level of realism. While we value all analytical backgrounds, this role specifically requires first-hand MBB project experience. What You’ll Do Design realistic consulting environments - Create detailed project scenarios (industry context, financials, constraints, incomplete information) - Develop structured tasks such as market sizing, due diligence, cost reduction, growth strategy, or operational diagnosis Define what “good” looks like - Set clear evaluation criteria for high-quality consulting outputs - Distinguish between superficial answers and consulting-grade reasoning Evaluate AI outputs - Review analyses and identify logical errors, weak assumptions, and structural gaps - Provide precise, actionable feedback to improve system performance Continuously improve training quality - Refine scenarios, edge cases, and evaluation frameworks - Increase realism based on real project experience Skills & Requirements - 5+ years at McKinsey, BCG, or Bain - Strong structured problem-solving skills - Ability to translate vague problems into clear analytical steps - High attention to logical consistency - Independent working style Basic technical familiarity required: - Reading structured formats (e.g., JSON or YAML) - Basic Git/GitHub usage - Clear written English (B2+) What This Role Is Not - No client interaction - No stakeholder management - No sales or business development - No team leadership responsibilities This is a focused analytical role for consultants who want to apply their problem-solving skills to advanced AI systems.
Freelance Expert Consultant (AI Training & Evaluation)
MindriftApply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
We are building the next generation of AI agents capable of navigating the complex consulting tasks. To achieve this, we are looking for consultants with 5+ years of experience at McKinsey & Company, Boston Consulting Group, or Bain & Company. The role is not client-facing and does not involve stakeholder management. Instead, the consultant will use their direct experience from real consulting projects to: - Recreate realistic consulting project environments - Design structured problem-solving tasks - Define what high-quality outputs should look like - Evaluate AI-generated outputs against consulting standards Why an MBB Background Matters Practical knowledge of how consulting projects actually work in detail is required. Specifically: - How ambiguous client questions are translated into structured workplans - How hypotheses are formed and tested - How data gaps are handled in real projects - What “good” looks like in slide decks, models, and executive summaries - How trade-offs and recommendations are developed under uncertainty The AI must be trained on environments that reflect real consulting complexity. Only someone who has worked on multiple MBB projects can reliably recreate this level of realism. While we value all analytical backgrounds, this role specifically requires first-hand MBB project experience. What You’ll Do Design realistic consulting environments - Create detailed project scenarios (industry context, financials, constraints, incomplete information) - Develop structured tasks such as market sizing, due diligence, cost reduction, growth strategy, or operational diagnosis Define what “good” looks like - Set clear evaluation criteria for high-quality consulting outputs - Distinguish between superficial answers and consulting-grade reasoning Evaluate AI outputs - Review analyses and identify logical errors, weak assumptions, and structural gaps - Provide precise, actionable feedback to improve system performance Continuously improve training quality - Refine scenarios, edge cases, and evaluation frameworks - Increase realism based on real project experience Skills & Requirements - 5+ years at McKinsey, BCG, or Bain - Strong structured problem-solving skills - Ability to translate vague problems into clear analytical steps - High attention to logical consistency - Independent working style Basic technical familiarity required: - Reading structured formats (e.g., JSON or YAML) - Basic Git/GitHub usage - Clear written English (B2+) What This Role Is Not - No client interaction - No stakeholder management - No sales or business development - No team leadership responsibilities This is a focused analytical role for consultants who want to apply their problem-solving skills to advanced AI systems.
Freelance Expert Consultant (AI Training & Evaluation)
MindriftApply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
We are building the next generation of AI agents capable of navigating the complex consulting tasks. To achieve this, we are looking for consultants with 5+ years of experience at McKinsey & Company, Boston Consulting Group, or Bain & Company. The role is not client-facing and does not involve stakeholder management. Instead, the consultant will use their direct experience from real consulting projects to: - Recreate realistic consulting project environments - Design structured problem-solving tasks - Define what high-quality outputs should look like - Evaluate AI-generated outputs against consulting standards Why an MBB Background Matters Practical knowledge of how consulting projects actually work in detail is required. Specifically: - How ambiguous client questions are translated into structured workplans - How hypotheses are formed and tested - How data gaps are handled in real projects - What “good” looks like in slide decks, models, and executive summaries - How trade-offs and recommendations are developed under uncertainty The AI must be trained on environments that reflect real consulting complexity. Only someone who has worked on multiple MBB projects can reliably recreate this level of realism. While we value all analytical backgrounds, this role specifically requires first-hand MBB project experience. What You’ll Do Design realistic consulting environments - Create detailed project scenarios (industry context, financials, constraints, incomplete information) - Develop structured tasks such as market sizing, due diligence, cost reduction, growth strategy, or operational diagnosis Define what “good” looks like - Set clear evaluation criteria for high-quality consulting outputs - Distinguish between superficial answers and consulting-grade reasoning Evaluate AI outputs - Review analyses and identify logical errors, weak assumptions, and structural gaps - Provide precise, actionable feedback to improve system performance Continuously improve training quality - Refine scenarios, edge cases, and evaluation frameworks - Increase realism based on real project experience Skills & Requirements - 5+ years at McKinsey, BCG, or Bain - Strong structured problem-solving skills - Ability to translate vague problems into clear analytical steps - High attention to logical consistency - Independent working style Basic technical familiarity required: - Reading structured formats (e.g., JSON or YAML) - Basic Git/GitHub usage - Clear written English (B2+) What This Role Is Not - No client interaction - No stakeholder management - No sales or business development - No team leadership responsibilities This is a focused analytical role for consultants who want to apply their problem-solving skills to advanced AI systems.
