Role Description Innodata is expanding its GenAI research capability to advance state-of-the-art evaluation and post-training methods for LLM and multimodal systems. As an Applied Research Scientist, LLM Evaluation & Post-Training, you will lead research and experimentation on how evaluation design, measurement strategies, and feedback signals influence model improvement. This role is ideal for a technically rigorous researcher who is deeply fluent in modern LLM evaluation and post-training, and who can turn research insight into practical methods for customer solutions and internal platform innovation. You will work across human-in-the-loop and AI-augmented workflows, partnering with Language Data Scientists and AI/ML Research Engineers to design and validate evaluation frameworks that drive measurable model gains. The ideal candidate combines strong experimental and statistical judgment with hands-on technical ability and can engage as a peer with research and engineering stakeholders at leading AI companies. What You’ll Own - Define the next generation of evaluation-driven model improvement workflows. - Study how different evaluation approaches (human, automated, hybrid) shape model selection and post-training outcomes. - Design experiments that produce credible, actionable conclusions. - Design benchmark datasets, develop evaluation taxonomies and protocols, define metrics and scoring methodologies, analyze failure modes, and test how changes in evaluation setup affect downstream fine-tuning results. - Support customer engagements by bringing scientific rigor to evaluation strategy, methodology review, and technical recommendations. - Define and execute a research agenda focused on LLM evaluation and post-training, especially evaluation-driven model improvement. - Design rigorous experiments to study how evaluation methodologies impact fine-tuning and post-training outcomes. - Develop and validate evaluation frameworks for LLM and multimodal systems, including: - benchmark/task design - scoring methods - judge/model-assisted evaluation - human evaluation protocols - robustness/stress testing - Lead research on advanced evaluation domains, including long-context, cross-modal, and dynamic multi-turn evaluations. - Study the effectiveness and limitations of existing evaluation techniques, and propose improved methodologies with clear validity and scalability tradeoffs. - Analyze model behavior and failure patterns; generate actionable recommendations for model improvement and evaluation redesign. - Collaborate with AI/ML Research Engineers to translate research methods into scalable evaluation and post-training pipelines. - Collaborate with Language Data Scientists to integrate human-in-the-loop and synthetic data/evaluation strategies into research programs. - Engage with customer technical stakeholders to understand evaluation goals, review methodologies, and provide expert recommendations. - Contribute to internal benchmark datasets, evaluation frameworks, and reusable research assets. - Produce high-quality technical documentation, internal research reports, and client-facing materials explaining methods, results, assumptions, and limitations. - Contribute to thought leadership and best practices in LLM evaluation, post-training, and GenAI quality measurement. Qualifications - MS/PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or a related quantitative scientific field (PhD strongly preferred). - 5+ years of relevant experience in applied research / research science in ML/AI, with substantial work in LLMs or foundation models. - Demonstrated experience with LLM evaluation, benchmarking, alignment, post-training, or model quality research. - Strong foundation in experimental design, statistical analysis, and scientific reasoning for ML systems. - Strong coding skills in Python for research experimentation and analysis (e.g., data processing, evaluation pipelines, statistical analysis, visualization). - Experience working with modern ML tooling/frameworks (e.g., PyTorch, Hugging Face, JAX/TensorFlow as applicable) sufficient to design and execute model/evaluation experiments. - Ability to evaluate and compare human and automated evaluation methods, including tradeoffs in cost, reliability, validity, and scalability. - Experience designing evaluation studies and protocols that are reproducible across datasets, model versions, and evaluation runs. - Ability to collaborate directly with technical stakeholders including research scientists, ML engineers, data scientists, and customer technical counterparts. - Strong communication skills and ability to present nuanced technical conclusions, assumptions, and limitations clearly. Requirements - The expected salary range for this position is $175,000 – $225,000 USD per year, based on experience, skills, and qualifications. Company Description Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

• Manage large, complex research and evaluation projects focused on community safety, violence prevention, and related systems-change efforts. • Anticipate project risks and develop mitigation strategies to ensure timely, high quality deliverables. • Ensure compliance with ethical standards, IRB requirements, and data governance policies. • Lead and contribute to proposals and capture efforts for federal, state, local, foundation, and other client opportunities related to community safety. • Develop competitive proposal strategies, including conceptual design, methodological plans, staffing, and budgets. • Identify new business opportunities and cultivate partnerships that advance the organization’s mission. • Serve as a senior methodological leader, advising teams on rigorous quantitative and/or qualitative approaches. • Lead the review, analysis, synthesis, and visualization of data using appropriate practices and techniques. • Ensure data integrity, reproducibility, and strong documentation across all analytic workflows. • Translate complex research findings—both orally and in writing—into actionable insights tailored to policymakers, practitioners, and community partners. • Manage client and stakeholder relationships with professionalism and awareness of the broader policy and practice landscape. • Cultivate external relationships with researchers, practitioners, community partners, and other leaders working across the community safety and prevention landscape. • Manage and mentor staff to build technical capacity in community safety, prevention science, research design, and client service. • Provide thought partnership and coaching to mid level researchers on research design, analysis, and dissemination. • Model inclusive leadership practices and foster a collaborative team environment.

• Formulates and leads/co-leads novel projects with team or enables matrix collaboration on project/technology solutions • Generates innovative ideas within own team and/or project team/functional community • Establishes target dates and priorities to enable data-driven advancements in project teams • Responsible for overseeing the progress of the study • Ensures compliance with appropriate GLP regulations and all relevant international regulatory guidelines

Lead support for computational social science research by providing expert guidance on methodologies, conducting outreach to identify needs, and designing instructional materials and workshops for diverse audiences.