Role Description You'll work across the lab on agent capability evals, benchmark design, LLM-as-judge systems, failure analysis, and the infrastructure that ties it together. This is a high-growth, high-ownership role on a small team, and you'll ship evaluation infrastructure that researchers depend on from day one. Responsibilities - Run the full eval pipeline end to end and reproduce known results during onboarding, pairing with a senior engineer on your first task. - Build a judge calibration protocol: sample human-labeled decisions, measure agreement (κ, per-class P/R), identify drift zones, and document it so anyone can re-run it. - Extend an existing benchmark (GAIA, τ-Bench, SWE-bench slice, etc.) with new tasks targeting known capability gaps, including the prompt, environment, rubric, automated grader, and QA. - Run failure analysis on model outputs: categorize failure modes, quantify prevalence, and write up findings with recommendations for training data, judge prompts, or benchmark changes. - Own a recurring eval workflow (weekly regression suite, judge drift dashboard, red-team evaluation for a new capability) and ship tooling researchers actually use. Qualifications - 3+ years in software engineering, ML engineering, data science, or a research-adjacent role, with concrete evaluation experience from coursework, an internship, a side project, open source work, or a job. - Experience with at least one LLM evaluation framework (Harbor, Nemo Evaluator, etc.), with real opinions on what it does well and where it falls short. - Hands-on experience with LLMs: prompting, few-shot design, and ideally fine-tuning or RAG; regular use of coding agents. - Solid Python. You write clean, tested, version-controlled code that a colleague could run without you babysitting it. - Comfort with Git, CI/CD basics, Docker, and the Linux command line (SSH, tmux, debugging a remote job). - Understanding of basic eval statistics: why accuracy misleads on imbalanced judges, what Cohen's κ measures, how to think about confidence intervals on a metric. - At least 3 of the following: - You can explain why LLM-as-judge needs calibration. - You've done failure analysis and can tell model bugs apart from prompt, grader, or retrieval issues. - You know at least two agent benchmarks (GAIA, AgentBench, τ-Bench, MINT, SWE-bench, WebShop, ALFWorld) and a limitation of each. - You've designed or extended an eval dataset with happy paths, edge cases, and adversarial examples. - You've thought about non-determinism in eval, how you sample, how many runs, how you report variance. - You communicate clearly to both researchers and engineers, in the right language for each. - You're comfortable with ambiguity, can turn a half-formed request into a plan, and know when to ask for help. Preferred - RLVR / RLHF pipeline experience. - Training data curation experience. - Distributed eval orchestration experience. - Benchmark design from scratch. - Red teaming and adversarial eval experience. - Familiarity with psychometrics or measurement theory.

• Develops, tests, deploys, and maintains software for recommendation engines • Collaborates in agile processes to create valuable user stories • Mentors junior engineers on software development practices • Leads system design initiatives and drives best practices in scalable architecture • Integrates AI and machine learning capabilities into the product recommendation journey

Role Description We are looking for a Machine Learning Engineer to join our engineering team to help us manage our diverse and growing set of initiatives. This position is full-time and 100% remote. - Experiment with emerging technologies and contribute to building new models and systems - Implement prototypes of the algorithms and models you design in Python - Focus on delivering solutions to production (this is not a research-only role) - Design training and evaluation protocols - Set up monitoring for performance metrics and overall system behaviour including alerts for any anomaly detected - Partner closely with the platform engineering team to streamline and optimize MLOps workflows - Use Kanban to manage multiple releases per week - Maintain high code quality through code reviews and automated tests Here are a few indicators that you're the right person: - You enjoy a fun, creative, and engaging working atmosphere free of brilliant jerks - You want to be part of a small team inside a large company with massive opportunity for growth - You enjoy collaboration with other teams including product, biz dev, and our in-house QA team - You eagerly dig into complex engineering problems Qualifications - You have hands-on experience implementing production machine learning systems at scale - Expertise with Python ML libraries like TensorFlow, PyTorch, Scikit-Learn etc. - Familiarity with ML tools like MLFlow, Ray Serving - Familiarity with building data pipelines including SQL and manipulating large structured or unstructured datasets for analysis - Familiarity with AWS and Google Cloud big data products Requirements - We welcome diversity and non-traditional paths into all of our roles. - We believe in hiring the right person as opposed to the right combination of keywords. Benefits - Verve provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws.

Role Description As a Sr. ML Engineer focused on Reinforcement Learning, you will design, implement, and optimize RL algorithms that enable intelligent agents to operate in dynamic, unstructured environments. This role involves working closely with cross-functional teams to design, test, and deploy innovative solutions that improve the performance and capabilities of our robotic systems. This role can be located in our Columbus, Ohio Headquarters or Remote. - Design, implement, and evaluate RL algorithms for robotic control, motion planning, and adaptive behaviors in dynamic, unstructured environments. - Develop and integrate RL policies with robot control systems, ensuring compatibility with hardware constraints and real-time requirements. - Collaborate with perception teams to fuse RL with vision, depth, and sensor data for robust decision-making. - Build and maintain sim-to-real pipelines, including domain randomization and transfer learning techniques. - Conduct experiments on physical robots, including designing safety protocols and monitoring for unexpected behaviors. - Leverage simulation environments (Isaac Gym, Gazebo, MuJoCo, PyBullet) for large-scale training before real-world validation. - Continuously improve model efficiency to operate within compute and latency constraints on embedded robotic systems. Qualifications - Master’s or PhD in Computer Science, Robotics, Machine Learning, or related field, or equivalent practical experience. - Experience developing and deploying reinforcement learning algorithms on real-world systems. - Proficiency in Python and deep learning frameworks such as PyTorch or TensorFlow. - Experience with simulation environments (e.g., MuJoCo, Isaac Gym). - Solid understanding of probability, statistics, and optimization. - Experience with training and deploying ML models in production systems. Benefits - Daily free lunch to keep you fueled and connected with the team. - Flexible PTO so you can take the time you need, when you need it. - Comprehensive medical, dental, and vision coverage. - 6 weeks fully paid parental leave, plus an additional 6–8 weeks for birthing parents (12–14 weeks total). - 401(k) retirement plan through Empower. - Generous employee referral bonuses—help us grow our team! Company Description At Path Robotics we love coming to work to solve interesting and tough challenges but also because our ideas are welcomed and valued. We encourage unique thinking and are dedicated to creating a diverse and inclusive environment. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. If you require a reasonable accommodation to participate in the application process or any part of the hiring process, please contact HR@path-robotics.com. We are committed to providing equal access and will work with qualified individuals to ensure a fair and accessible hiring experience. We will respond to your request within 48 hours.