Preference Model

Role Description We're looking for PhD or Master's students, and gifted undergrads to spend an internship with us working on building RL training environments for large language models. This role blends research and engineering. It will require you to both develop novel approaches and realize them in code. Your work will include: - Designing and implementing RL environments - Conducting experiments and evaluations - Delivering your work into production training runs - Collaborating with other researchers and engineers Qualifications - You're an undergrad or PhD student in CS, ML, math, physics, or a related field. - You write real code, not just research prototypes. - You read ML papers for fun in your free time. Requirements - Strong Python skills - Familiarity with how LLMs work, what they're good at, and where they fall short - Ability to work independently, take feedback, and iterate fast - You may be a good fit if one of the following applies: - You understand transformer internals and have worked with training or inference code - You've written CUDA kernels or worked with low-level GPU programming - You have a research area you know deeply (publications, public code, or strong coursework) - You read broadly across ML and can connect ideas from different subfields - You've built interactive environments, simulations, or complex software systems Benefits - Paid Internship with opportunity to return full time based on performance - Ownership and autonomy in a fast moving startup environment - Opportunity to work with top machine learning engineers - Competitive cash and equity compensation (>90th percentile) - Lunch provided everyday onsite - Weekly snack orders Company Description Preference Model is building automated ML research engineering. Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions. Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.

• Design and build MLE environments • Teach LLMs better reasoning and advanced concepts from modern ML

Location: San Francisco preferred, remote considered Duration: 10-12 weeks, Summer 2026 Compensation: Paid internship About Us Preference Model is building the next generation of training data to power the future of AI. Today's models are powerful but fail to reach their potential across diverse use cases because so many of the tasks that we want to use these models are out of distribution. Preference Model creates RL environments where models encounter research and engineering problems, iterate, and learn from realistic feedback loops. Our founding team has previous experience on Anthropic’s data team building data infrastructure, tokenizers, and datasets behind the Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential. We are backed by A16Z. About the Role We're looking for PhD students and gifted undergrads to spend the summer building RL training environments for large language models. What you'll do - Design and build RL environments that test LLM reasoning on ML, systems, and research problems - Write clean, production-grade Python (not notebooks) - Work with Docker, build reproducible environments, debug when things break - Translate ML papers and concepts into concrete training tasks Who we're looking for You're an undergrad or PhD student in CS, ML, math, physics, or a related field. You write real code, not just research prototypes. You read ML papers for fun in your free time. Must have: - Strong Python skills - Familiarity with how LLMs work, what they're good at, and where they fall short - Ability to work independently, take feedback, and iterate fast Any of these would make you stand out: - You understand transformer internals and have worked with training or inference code - You've written CUDA kernels or worked with low-level GPU programming - You have a research area you know deeply (publications, public code, or strong coursework) - You read broadly across ML and can connect ideas from different subfields - You've built interactive environments, simulations, or complex software systems How to apply Send your resume and a short note (2-3 sentences is fine) about what area of ML you're most interested in and why. Links to code, papers, or projects are more useful than a long cover letter.