Enabling a high-quality and viable healthcare system
Senior Generative AI Scientist II – Model Risk & Validation
Location
United States
Posted
3 days ago
Salary
$145K - $170K / year
Seniority
Senior
Job Description
Senior Generative AI Scientist II – Model Risk & Validation
Cotiviti
• delivering solutions that help our clients identify payment integrity issues, reduce the cost of healthcare processes, or improve the quality of healthcare outcomes. • conduct independent model validation of existing models for benchmarking, assessment, and gauging effectiveness. • determine aspects of model drift and related data drift for the purpose of model risk management (MRM). • apply deep expertise with AI/ML/GenAI model development, including hands-on experience with model building and model evaluation. • benchmark and potentially rebuild existing models as needed using updated data, and potentially newer, more modern and effective algorithms. • actively drive improvements in model monitoring activities, including methods for model registration, model metadata management, and conceptualizing approaches for related tools and techniques. • complete all responsibilities as outlined in the annual performance review and/or goal setting. • complete all special projects and other duties as assigned.
Job Requirements
- Graduate Degree in a quantitative discipline such as Computer Science/Engineering, Statistics, Operations Research covering Advanced Statistics, Machine learning and AI.
- Experience with the latest techniques in natural language processing including transformers, fine-tuning LLMs, measuring/benchmarking and deploying LLMs with tools such as HuggingFace, Langchain, LLAMA/Mistral and OpenAI, vector databases.
- 5+ years of hands-on data science/AI experience, using typical machine learning and data science tools including pandas, scikit-learn, keras, nltk, and TensorFlow/PyTorch, GPU.
- General understanding of Responsible AI (RAI), including explainability (XAI), AI NIST RMF, and related AI risk management frameworks.
- Experience and understanding evaluating models for bias and fairness, with aptitude for detecting bias in the model design and data, as well as using metrics such as SHAP and LIME.
- Understanding appropriate model metrics and techniques for managing, evaluating and monitoring GenAI models and LLMs
- Experience building production-grade machine learning deployments on AWS, Azure, or GCP.
- Experience working with Apache Spark™ and large-scale distributed datasets.
- Experience communicating technical concepts to non-technical and technical audiences is a plus.
- Passion for collaboration, learn it all mindset and driving value with AI.
Benefits
- medical, dental, vision, disability, and life insurance coverage
- 401(k) savings plans
- paid family leave
- 9 paid holidays per year
- 17-27 days of Paid Time Off (PTO) per year, depending on specific level and length of service with Cotiviti
Related Guides
Related Job Pages
More AI Research Scientist Jobs
Role Description You advance the frontier of what's detectable and measurable in AI systems, then hand that work directly to engineers to productise. Research that ships, not research that sits. What You Own - Research and IP development - Expand the jailbreak and adversarial attack library beyond public methods. - Build bias, fairness, and hallucination eval methodologies that hold up to regulators. - Develop interpretability techniques that explain why a model failed (not just that it failed). - Maintain a live radar on AI safety literature. - Productisation bridge - Translate findings into engineering specs the red team and MLOps engineers can build from without hand-holding. - Define ground truth for eval harnesses. - Own the methodology documentation behind customer compliance reports. - External credibility - Publish papers, technical posts, benchmark releases. - Represent Disseqt at conferences and regulator briefings. - Contribute to open benchmarks where it builds community trust. Qualifications - Strong ML foundations at architecture level, not just API level. - Hands-on evaluation, bias, or AI safety experience. - Code that runs in production, not just notebooks. - Clear technical writing for two audiences (compliance officer and research peer). - Genuine interest in AI governance as a problem space. Requirements - Published work in AI safety, fairness, interpretability, or adversarial ML. - LLM red teaming experience. - Familiarity with EU AI Act, NIST AI RMF, ISO 42001. - Prior startup experience where research had to ship. Probably Not for You If - You want to work on benchmark maximisation for foundation model capabilities. - You need a large team around you. - You're uncomfortable with your research being used commercially and scrutinised by auditors. Stack - Python - PyTorch - HuggingFace - Open-weight and API-accessed frontier models. - Fluent enough in the engineering environment to hand off cleanly — not a full-stack role, but you don't stop at a notebook. Benefits - Protected research time that doesn't get eaten by sprints. - Publishing support — your research output belongs to you as much as Disseqt. - Direct access to live enterprise AI risk problems, not synthetic toy datasets. - Competitive salary, meaningful equity, early-stage upside.
• Contribute to sophisticated research in large language models (LLMs), natural language processing (NLP), and multimodal deep learning • Help build, implement, and evaluate machine learning models and algorithms • Participate in data collection, preprocessing, and analysis • Assist in developing and fine-tuning AI models for various applications • Create experiments, compare algorithms, tune hyperparameters, and document results • Share findings with the team, contribute to reports and presentations, and help onboard new team members
• Translate machine learning research into practical, domain-specific solutions • Own applied research and experimentation across computer vision, LLMs, and AI workflows • Drive the design and execution of experiments end to end • Help shape technical direction by identifying high-impact opportunities • Make pragmatic trade-off calls across model accuracy, interpretability, latency, and cost • Share experience with research teammates to encourage rigor in a collaborative environment
• Conduct end-to-end research and engineering on vision-language models, covering training, evaluation, and optimization across the full model development lifecycle. • Design and implement post-training pipelines including supervised fine-tuning, knowledge distillation, and reinforcement learning from human feedback. • Develop and maintain high-quality multimodal datasets, including data curation, filtering, and balancing for domain-specific tasks. • Drive model efficiency and deployability, adapting models for resource-constrained environments using compression and optimization techniques. • Design and implement evaluation frameworks and benchmarks to measure model performance, robustness, and real-world task success. • Build and scale training workflows across distributed GPU infrastructure. • Identify and resolve bottlenecks in training pipelines to achieve state-of-the-art model quality on target benchmarks. • Contribute to and leverage open-source ecosystems including models, datasets, and tooling to accelerate development. • Stay current with the latest research in multimodal learning and vision-language systems, translating relevant findings into practical improvements. • Publish research findings in top-tier AI conferences and journals where applicable.



