As the AI platform for business transformation, we're putting AI to work across organizations — freeing people for work that matters. Making old tech work with new tech. Reaching across departments, from the front office to the back office and every office in between. Our ambition? To become the AI defining enterprise software company of the 21st century (or "AI DESCO21C," as we like to call it). With more than 8,400+ customers, we serve approximately 90% of the Fortune 500®, and we're proud to be a Fortune 100 Best Companies to Work For® and World's Most Admired Companies™. Explore your future career with us, visit www.careers.servicenow.com From Fortune. ©2026 Fortune Media IP Limited. All rights reserved. Used under license.
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Location
California
Posted
3 days ago
Salary
$201.3K - $352.3K / year
Seniority
Senior
Job Description
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
ServiceNow
Company Description It all started when engineer Fred Luddy wrote code that automated a tedious task for his coworker, Phyllis. She cried tears of joy. That moment inspired Fred to build a company that could do that for everyone-freeing people from busywork so they could focus on meaningful work. Today, ServiceNow is the AI control tower for business reinvention. Our ServiceNow AI platform brings together any AI, any data, and any workflow- helping 85% of the Fortune 500® work smarter, faster, and better. We're building an AI-native culture where technology and talent are unstoppable together. And we're just getting started. Join us to put AI to work for people. Job Description The Advanced Technology Group (ATG) at ServiceNow is a customer-focused innovation group building intelligent software and smart user experiences using existing and latest advanced technologies to enable end-to-end, industry-leading work experiences for customers. We are a group of researchers, applied scientists, engineers, and product managers with a dual mission. We build and evolve the AI platform, and partner with teams to build products and end-to-end AI-powered work experiences. In equal measure, we lay the foundations, research, experiment, and de-risk AI technologies that unlock new work experiences in the future. Job Description We are seeking an exceptional, data-driven Senior Engineering Manager, Agentic & GenAI Benchmarking and Evaluations to establish and lead AI evaluation practices for both ServiceNow and our customers. As ServiceNow shifts enterprise workflows from simple generation to complex, autonomous agents, ensuring system reliability, safety, and accuracy is paramount. In this role, you will lead a specialized team of AI evaluation engineers and data scientists. Your team will build the infrastructure, rigorous validation frameworks, and benchmarks that quantify the performance of Now Assist agentic workflows across multi-step orchestration, tool-calling, and enterprise-grounded reasoning. You will bridge the gap between frontier AI research and hard production metrics, directly impacting the trust and adoption of autonomous workflows for millions of enterprise users. What You Get To Do In This Role - Build the Evaluation Infrastructure: Design, own, and scale automated testing and evaluation harnesses (unit evals, integration evals, and production drift monitors) to measure agent quality and eliminate regressions. - Define Enterprise AI Benchmarks: Create standard, repeatable evaluation frameworks tailored to complex business workflows-assessing multi-agent orchestration, intent routing, multi-step planning loops, and long-term memory accuracy. - Validate Grounding & RAG Pipelines: Partner with search and data fabric teams to systematically evaluate Retrieval-Augmented Generation (RAG) pipelines, hybrid search, and semantic re-ranking systems. - Model Selection Optimization: Rigorously benchmark frontier LLMs (e.g., OpenAI, Anthropic, Google, and proprietary ServiceNow models) to evaluate trade-offs across execution capabilities, latency, context-window efficiency, and inference costs. - Lead a High-Performing Team: Recruit, mentor, and foster an AI-native engineering team, driving engineering best practices, prompt-infrastructure stability, and production-grade rigor. - Cross-Functional Leadership: Collaborate with Core Product, Machine Learning Platforms, and Engineering leads to translate baseline performance statistics into actionable product improvements and model fine-tuning targets. Qualifications To be successful in this role you have: - 8+ years of professional software engineering or machine learning experience, including 3+ years managing or technically leading high-performing AI/ML teams. - Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving. This may include using AI-powered tools, automating workflows, analyzing AI-driven insights, or exploring AI's potential impact on the function or industry. - Strong foundational knowledge of frontier AI SDKs and deep experience deploying or testing agentic/probabilistic software architectures (multi-agent orchestration, tool execution, and probabilistic feature deployment). - Demonstrated experience implementing rigorous AI metrics (e.g., ROUGE, BLEU, G-Eval, LLM-as-a-judge patterns, and custom deterministic evaluation code) at an enterprise scale. - Proficiency in Python and familiarity with data analytics infrastructures (SQL, Pandas, NumPy) alongside standard MLOps tracking platforms. - Experience with complex knowledge infrastructure, SaaS platform architectures, or relational datasets (e.g., Knowledge Graphs, CMDBs). - Ability to translate deeply technical evaluation data into executive-level risk assessments, ROI summaries, and strategic roadmap recommendations. - Bachelor's or higher degree in Computer Science, Data Science, Machine Learning, or a highly quantitative field (Master's or Ph.D. is a plus). For positions in this location, we offer a base pay of $201,300 - $352,300, plus equity (when applicable), variable/incentive compensation and benefits. Sales positions generally offer a competitive On Target Earnings (OTE) incentive compensation structure. Please note that the base pay shown is a guideline, and individual total compensation will vary based on factors such as qualifications, skill level, competencies, and work location. We also offer health plans, including flexible spending accounts, a 401(k) Plan with company match, ESPP, matching donations, a flexible time away plan and family leave programs. Compensation is based on the geographic location in which the role is located and is subject to change based on work location. Additional Information Work Personas We approach our distributed world of work with flexibility and trust. Work personas (flexible, remote, or required in office) are categories that are assigned to ServiceNow employees depending on the nature of their work and their assigned work location. Learn more here . To determine eligibility for a work persona, ServiceNow may confirm the distance between your primary residence and the closest ServiceNow office using a third-party service. Equal Opportunity Employer ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, national origin, age, disability, gender identity, veteran status, or any other category protected by law. In addition, all qualified applicants with arrest or conviction records will be considered for employment in accordance with legal requirements. Accommodations We strive to create an accessible and inclusive experience for all candidates. If you require a reasonable accommodation to complete any part of the application process, or are unable to use this online application and need an alternative method to apply, please contact globaltalentss@servicenow.com for assistance. Export Control Regulations For positions requiring access to controlled technology subject to export control regulations, including the U.S. Export Administration Regulations (EAR), ServiceNow may be required to obtain export control approval from government authorities for certain individuals. All employment is contingent upon ServiceNow obtaining any export license or other approval that may be required by relevant export control authorities. From Fortune. ©2026 Fortune Media IP Limited. All rights reserved. Used under license.
Related Guides
Related Categories
Related Job Pages
More Engineering Manager Jobs
• Be responsible for selecting, leading & developing the team • Visit his/her clients and colleagues systematically, monitors specific customers/ key partners and significant projects of his/her area of responsibility. • Be responsible for complying to the departmental procedures of operation. • Be responsible for the company’s expenses, sales, profits and turnover within his/ her area of responsibility. • Develop an in depth understanding of the market conditions and issues, including the competitors, potential clients and markets’ trends stemming from both primary and secondary information sources. • Prior experience from a sales job position or from a job position that is relative with the customer services. • Experience from a similar position and partners’ management. • Experience and knowledge of the company's operating procedures and especially of the sales department.
Senior Engineering Manager, Systems
Noda AINoda AI is an all-in-one commercial real estate management platform dedicated to optimizing how modern buildings operate through smarter, more efficient, and su
Title: Senior Engineering Manager, Systems Location: Austin Department: Engineering Job Description: Location: Austin, TX (Hybrid on-site, with approximately 20% typical travel, and up to 40% during field exercises and major integration events) Reports to: Head of Engineering (interim Systems Engineering lead) Clearance Requirement: U.S. Citizen with the ability to obtain a security clearance About NODA NODA is a veteran-owned, venture-backed technology company that is transforming how unmanned systems collaborate in complex, mission-critical environments. We are developing next-generation solutions that enable the autonomous orchestration of heterogeneous unmanned systems across air, sea, land, and space with vital applications in the defense, intelligence, and commercial sectors. NODA is a systems integration company at its core. We exist to make heterogeneous unmanned systems work together, and Systems Engineering is where that promise either holds or breaks. As NODA scales across products, partners, and deployment environments, keeping the system coherent in practice, not just on paper, is what separates a platform that works in the field from one that only works in a demo. Joining NODA means working on meaningful technology that pushes the boundaries of autonomy alongside a team that thrives on innovation, rapid iteration, and collaboration. The Role We are seeking a Systems Engineering Manager to build and run the team responsible for NODA's system-level integrity as we scale. This is a high-ownership, still-technical manager role. You will personally profile systems, debug the cross-domain issues no one else can cleanly own, and build a team that catches what other teams miss, whether the gap was left deliberately or by accident. Your work will focus on the seams: the places where one unowned handoff, one gap left behind by another team, or one integration nobody traced end to end can break the orchestration. You will think vertically enough to root-cause a single hard failure all the way down, and horizontally enough to see how a fix in one domain breaks another. Success in this role requires the ability to grow the bench from where it is today into the multi-pod team NODA needs, while defining how Systems Engineering operates along the way. Key Responsibilities - Hire, level, and develop engineers across software, networking, RF, and hardware-adjacent debugging to build the bench NODA needs as headcount scales - Coach cross-layer debugging as a team capability, teaching engineers when to go deep in one domain, when to go wide across several, and when to delegate - Personally profile systems and root-cause the issues too ambiguous or too cross-cutting for anyone else to cleanly own - Own System Health for your scope, including performance and stability KPIs, MTTR on cross-system issues, degradation detection, and the tooling and telemetry that makes it possible - Own crash-to-root-cause analysis and the verification evidence behind release go/no-go calls - Treat security and CMMC-derived constraints as first-class design inputs, not a parallel compliance track - Catch the gaps other teams leave, deliberately or inadvertently, before they surface as integration failures or field incidents - Translate field, customer, and BD requirements into constraints your team can build against - Represent Systems Engineering at cross-team design sessions and the Architecture Review Board, and escalate system integrity and delivery risk per the Systems Engineering charter Required Qualifications - 8+ years of engineering experience, with substantial hands-on time debugging distributed, real-time, or multi-domain systems in production - 2+ years managing engineers, ideally across more than one technical domain - A real track record as an expert debugger, the person teams bring the hardest, most ambiguous cross-system issues to, regardless of layer - Able to think both ways: horizontal breadth across software, RF, networking, firmware, and hardware, and vertical depth to root-cause a single hard failure all the way down when breadth alone won't solve it - Experience owning system health, performance monitoring, or release-readiness processes - Working knowledge of networking fundamentals and real-time or low-latency middleware - Comfortable with hardware bench diagnostics (multimeter, oscilloscope) and reading firmware-level behavior well enough to know when to loop in a specialist - Working understanding of DoD security and compliance frameworks (e.g. CMMC) as they apply to system design, or demonstrated ability to ramp fast - U.S. Citizenship with the ability to obtain a security clearance Preferred Qualifications - Experience scaling a cross-domain engineering team 2x+ in under a year - Experience in defense, aerospace, or other mission-critical or regulated environments - Direct CMMC Level 2 (or similar) experience in a systems engineering context - Experience with embedded and edge-constrained systems (embedded Linux, RTOS, bare-metal) - Familiarity with SIL and HIL pipelines and sim-to-field validation - Experience authoring or overseeing ICDs, requirements traceability, or MBSE artifacts Skills & Attributes - Systems thinker comfortable operating across software, networking, RF, firmware, and hardware boundaries - Strong debugger and root-cause analyst, with the discipline to trace hard failures across layers rather than stopping at the first plausible symptom - Effective team-builder and coach, capable of teaching cross-domain judgment as a repeatable team capability - Detail-oriented approach to system reliability, security, and release readiness in mission-critical applications - Collaborative mindset with the ability to work effectively across engineering, autonomy, and integration teams - Bias for action, with the judgment to escalate cross-team gaps with clear options rather than just problems - Mission driven, with a personal commitment to supporting the nation's defense What We Offer - Hybrid work environment - Competitive pay, including equity - Flexible time off - Generous PTO policy - Federal holidays - Generous health, dental, and vision benefits - Free OneMedical membership - 401(k) Growth Path at NODA - L5 – Technical Lead, Systems Engineering: hands-on across software, networking, RF, and hardware boundaries; informally mentors engineers; builds the cross-domain credibility this role is built on - L6 – Engineering Manager, Systems Engineering: manages a pod; hands-on in debugging and profiling day-to-day; executes System Health with architect guidance - L7 – Senior Engineering Manager, Systems Engineering: manages the full implementation bench; owns System Health outcomes independently; sets technical direction multiple pods execute against; represents the team at the Architecture Review Board We are an Equal Opportunity Employer and welcome applicants from all backgrounds. All qualified individuals will receive consideration for employment regardless of race, age, color, religion, sex, national origin, disability, or protected veteran status.
• Lead complex, large-scale security programs that span multiple functions, run on multi-year horizons, and shape business direction • Lead the execution strategy and delivery plans for resilient CI/CD, software supply chain security, cloud security initiatives, and AI-enabled security services • Synthesize program status and complex risks into clear narratives to inform executive decisions • Champion a customer-first approach for the engineering teams you serve • Strengthen program management practices and standards to ensure consistent, high-quality delivery
• responsible for leading and mentoring a team of engineers • driving the development of networking for the Cilium project, security and observability solutions • growing and mentoring software engineers and technical leaders • delivering continuous feedback and performance reviews • tracking project progress and helping engineers grow their project management skills • preserving the engineering culture as the team scales


