Title: Senior Manager, Intelligence Systems Lab Location: Bothell United States Job Description: Working with Us Challenging. Meaningful. Life-changing. Those aren't words that are usually associated with a job. But working at Bristol Myers Squibb is anything but usual. Here, uniquely interesting work happens every day, in every department. From optimizing a production line to the latest breakthroughs in cell therapy, this is work that transforms the lives of patients, and the careers of those who do it. You'll get the chance to grow and thrive through opportunities uncommon in scale and scope, alongside high-achieving teams. Take your career farther than you thought possible. Bristol Myers Squibb recognizes the importance of balance and flexibility in our work environment. We offer a wide variety of competitive benefits, services and programs that provide our employees with the resources to pursue their goals, both at work and in their personal lives. Read more: careers.bms.com/working-with-us. Description: In support of enterprise AI strategy, the Senior Manager of Intelligence Systems Lab conducts research into the principles of intelligence and develops agentic AI systems grounded in rigorous evaluation science. This hands-on technical role focuses on understanding and implementing reasoning, planning, memory, and generalization capabilities for AI agents, while building benchmarking methodologies to assess agent performance. The position supports cross-functional stakeholders in Commercial, Manufacturing, Clinical Development, and Research functions. Key Responsibilities and Major Duties: - Research and Framework Development: - Conduct research on reasoning, planning, memory, and generalization architectures for AI agents and multi-agent systems. - Develop and test frameworks for agent cognition, including abstraction, goal decomposition, and adaptive planning. - Contribute to the lab's understanding of foundational principles underlying intelligent behavior in artificial systems. - Stay current with academic literature and industry advances in AI reasoning, cognitive architectures, and agent design. - Benchmarking and Evaluation Science: - Design and execute benchmarking experiments to evaluate agent reasoning, reliability, and generalization capabilities. - Develop evaluation methodologies for assessing AI agent performance across diverse tasks and domains. - Build test harnesses, evaluation datasets, and measurement frameworks for multi-agent systems. - Apply rigorous experimental design and statistical analysis to benchmark results. - Agentic AI Development: - Serve as a hands-on AI engineer, designing, developing, and deploying advanced agentic solutions. - Build multi-agent AI systems using frameworks such as Strands, LangGraph, DSPy, and similar agentic architectures. - Develop RAG (Retrieval Augmented Generation) applications, knowledge graphs, and conversational AI solutions. - Implement autonomous workflows that demonstrate advances in reasoning and planning capabilities. - Rapid Prototyping and Innovation: - Turn research concepts into working prototypes that demonstrate advances in agent intelligence. - Conduct proof-of-concept development to assess feasibility of new reasoning and planning approaches. - Seek opportunities to translate research findings into production-ready capabilities. - Contribute to documenting research findings, experimental results, and technical methodologies. - Cross-Functional Collaboration: - Work cross-functionally with analytics and business stakeholders to pilot AI solutions grounded in research insights. - Translate complex AI research concepts into business language to drive understanding and adoption. - Partner with the broader Applied Intelligence organization to advance agentic AI capabilities. - Engage with Commercial, GPS/Manufacturing, Clinical Development, and Research teams in presenting research-informed solutions. - Mentorship and Best Practices: - Contribute to implementing best practices in agent evaluation, benchmarking science, and research methodology. - Provide guidance to junior team members on AI/ML techniques, research methods, and development approaches. - Support knowledge sharing through internal presentations, documentation, and collaborative research. Qualifications/Degree/Certification/Licensure: - Educational Background: BA/BS required (quantitative area of study such as Computer Science, Data Science, Statistics, Mathematics, Cognitive Science, or Engineering preferred). MS or other graduate degree preferred. - Technical Proficiency: Proficiency in Python (R or Julia), with hands-on experience with ML frameworks (Scikit-Learn, TensorFlow, PyTorch, etc.) Experience with agentic AI frameworks (Strands, LangGraph, AutoGen, DSPy, CrewAI, or similar). Experience with LLMs, prompt engineering, and RAG architectures. Familiarity with benchmarking and evaluation methodologies for AI systems. Proficiency in SQL and experience with cloud-based environments (AWS, Azure, GCP). Experience with Git and software version control. Experience with experimental design and statistical analysis preferred - Communication Skills: Excellent communication and presentation skills, with a proven ability to explain complex research and analyses to both technical and non-technical stakeholders. Experience presenting at conferences or publishing research preferred. - Experience: A minimum of 3 years of hands-on experience in data science, machine learning, AI development, or AI research. Relevant advanced degrees contribute to required years of experience. Experience with pharmaceutical or life sciences industry preferred. - Cross-Functional Collaboration: Proven experience in working on complex research or development projects with multi-functional team members. #LI-Hybrid #AIEngineering If you come across a role that intrigues you but doesn't perfectly line up with your resume, we encourage you to apply anyway. You could be one step away from work that will transform your life and career. Compensation Overview: Bothell - WA - US: $147,080 - $178,221 Princeton - NJ - US: $133,710 - $162,019 San Diego - CA - US: $147,080 - $178,221 Seattle - WA: $147,080 - $178,221 The starting compensation range(s) for this role are listed above for a full-time employee (FTE) basis. Additional incentive cash and stock opportunities (based on eligibility) may be available. The starting pay rate takes into account characteristics of the job, such as required skills, where the job is performed, the employee's work schedule, job-related knowledge, and experience. Final, individual compensation will be decided based on demonstrated experience. Eligibility for specific benefits listed on our careers site may vary based on the job and location. For more on benefits, please visit https://careers.bms.com/life-at-bms/. Benefit offerings are subject to the terms and conditions of the applicable plans in effect at the time and may require enrollment. Our benefits include: - Health Coverage: Medical, pharmacy, dental, and vision care. - Wellbeing Support: Programs such as BMS Well-Being Account, BMS Living Life Better, and Employee Assistance Programs (EAP). - Financial Well-being and Protection: 401(k) plan, short- and long-term disability, life insurance, accident insurance, supplemental health insurance, business travel protection, personal liability protection, identity theft benefit, legal support, and survivor support. Work-life benefits include: Paid Time Off - US Exempt Employees: flexible time off (unlimited, with manager approval, 11 paid national holidays (not applicable to employees in Phoenix, AZ, Puerto Rico or Rayzebio employees) - Phoenix, AZ, Puerto Rico and Rayzebio Exempt, Non-Exempt, Hourly Employees: 160 hours annual paid vacation for new hires with manager approval, 11 national holidays, and 3 optional holidays Based on eligibility*, additional time off for employees may include unlimited paid sick time, up to 2 paid volunteer days per year, summer hours flexibility, leaves of absence for medical, personal, parental, caregiver, bereavement, and military needs and an annual Global Shutdown between Christmas and New Years Day. All global employees full and part-time who are actively employed at and paid directly by BMS at the end of the calendar year are eligible to take advantage of the Global Shutdown. - Eligibility Disclosure: The summer hours program is for United States (U.S.) office-based employees due to the unique nature of their work. Summer hours are generally not available for field sales and manufacturing operations and may also be limited for the capability centers. Employees in remote-by-design or lab-based roles may be eligible for summer hours, depending on the nature of their work, and should discuss eligibility with their manager. Employees covered under a collective bargaining agreement should consult that document to determine if they are eligible. Contractors, leased workers and other service providers are not eligible to participate in the program. Uniquely Interesting Work, Life-changing Careers With a single vision as inspiring as "Transforming patients' lives through science ", every BMS employee plays an integral role in work that goes far beyond ordinary. Each of us is empowered to apply our individual talents and unique perspectives in a supportive culture, promoting global participation in clinical trials, while our shared values of passion, innovation, urgency, accountability, inclusion and integrity bring out the highest potential of each of our colleagues. On-site Protocol BMS has an occupancy structure that determines where an employee is required to conduct their work. This structure includes site-essential, site-by-design, field-based and remote-by-design jobs. The occupancy type that you are assigned is determined by the nature and responsibilities of your role: Site-essential roles require 100% of shifts onsite at your assigned facility. Site-by-design roles may be eligible for a hybrid work model with at least 50% onsite at your assigned facility. For these roles, onsite presence is considered an essential job function and is critical to collaboration, innovation, productivity, and a positive Company culture. For field-based and remote-by-design roles the ability to physically travel to visit customers, patients or business partners and to attend meetings on behalf of BMS as directed is an essential job function. Supporting People with Disabilities BMS is dedicated to ensuring that people with disabilities can excel through a transparent recruitment process, reasonable workplace accommodations/adjustments and ongoing support in their roles. Applicants can request a reasonable workplace accommodation/adjustment prior to accepting a job offer. If you require reasonable accommodations/adjustments in completing this application, or in any part of the recruitment process, direct your inquiries to adastaffingsupport@bms.com. Visit careers.bms.com/eeo-accessibility to access our complete Equal Employment Opportunity statement. Candidate Rights BMS will consider for employment qualified applicants with arrest and conviction records, pursuant to applicable laws in your area. If you live in or expect to work from Los Angeles County if hired for this position, please visit this page for important additional information: https://careers.bms.com/california-residents/ Data Protection We will never request payments, financial information, or social security numbers during our application or recruitment process. Learn more about protecting yourself at https://careers.bms.com/fraud-protection. Any data processed in connection with role applications will be treated in accordance with applicable data privacy policies and regulations. If you believe that the job posting is missing information required by local law or incorrect in any way, please contact BMS at TAEnablement@bms.com. Please provide the Job Title and Requisition number so we can review. Communications related to your application should not be sent to this email and you will not receive a response. Inquiries related to the status of your application should be directed to Chat with Ripley. R1599046 : Senior Manager, Intelligence Systems Lab