Job Closed
This listing is no longer active.
Protect Today. Predict Tomorrow.
Principal Agentic Engineer
Location
United States
Posted
74 days ago
Salary
0
Seniority
Lead
Job Description
Principal Agentic Engineer
ZeroFox
ZeroFox protects organizations from external threats across the public attack surface, and we're building agentic AI into the core of how that work gets done. You'll define and build the production agentic systems that power ZeroFox's product: the architectures, tooling, and practices that make agents manage state across steps, handle failures without losing context, and know when to escalate to a human. This is deep agent systems work. You'll be solving the hardest problems in multi-step orchestration, reliability, and evaluation for non-deterministic systems operating on adversarial data. What you'll do - Design and implement production agent architectures: state management, error handling, retry logic, graceful degradation, human-in-the-loop escalation - Build evaluation and testing frameworks for non-deterministic agent workflows: offline tests, synthetic data generation, regression checks, and post-deploy monitoring - Implement orchestration patterns: multi-agent coordination, tool-calling chains, memory management, context window optimization - Define deployment and governance practices: agent versioning, rollback, behavioral telemetry, anomaly detection - Instrument agents with tracing, logging, and observability that make production behavior debuggable - Establish architectural standards and best practices for agentic development through design reviews, pairing, and mentorship
Job Requirements
- Required qualifications
- 7+ years of software engineering experience building production systems, with hands-on experience designing and deploying LLM-based agents
- Deep knowledge of agent reliability patterns: state management, error boundaries, escalation logic, context management, tool-calling failure modes
- Experience with agent orchestration frameworks and understanding of tradeoffs between existing frameworks and building custom
- Strong backend engineering fundamentals: testing, monitoring, deployment, debugging, performance optimization
- Hands-on experience with retrieval-augmented systems and understanding of how retrieval quality affects agent behavior
- Experience building systems that handle adversarial or noisy input (cybersecurity, fraud detection, content moderation, or similar domains)
- Familiarity with cloud-based AI deployment, including observability, reliability, and cost considerations
- What success looks like
- Agents ship to production with defined reliability contracts and clear operational ownership
- Agent failures are detected and handled systematically through telemetry and evaluation pipelines, not customer reports
- The team has reusable patterns for state management, error handling, and escalation that new agent projects build on rather than reinvent
- Preferred qualifications
- Experience defining agent governance practices (versioning, behavioral telemetry, rollback) in a production environment
- Background in cybersecurity or adjacent domain with high-noise, adversarial data
- Experience with tool-use architectures and integration patterns for connecting agents to external systems
- Track record of raising engineering capability across a team through mentoring, pairing, or design review
Benefits
- Competitive compensation
- Community-driven culture with employee events
- Regular catered lunches for in-office work; snacks, drinks available daily
- Generous time off
- Comprehensive health benefits & 401(k) plan
- Fun, modern workspace
- Respectful and nourishing work environment, where every opinion is heard and everyone is encouraged to be an active part of the organizational culture
- About us
- ZeroFox is on a mission to make the internet safer for all. Innovation is at our core – we are relentless in the pursuit of finding new ways to disrupt external cyber threats on the surface, deep, and dark web. ZeroFox offers the only unified cybersecurity platform combining advanced AI analytics, digital risk and privacy protection, full-spectrum threat intelligence, and a robust portfolio of breach, incident and takedown response capabilities to protect customers from growing threats across the external attack surface. It’s a great time to join us in the Fox Den – with fresh private equity funding, expanding investments in AI, a people-first culture, and centers of excellence around the world, we’re growing like never before. If you’re looking for a mission-oriented, customer-focused, collaborative team and ready to take the fight to the adversary, apply to join us in the Den today.
- Equal Opportunity
- We aim to build a team that represents a variety of backgrounds, perspectives, and skills. We embrace inclusion and ensure equal employment opportunity without discrimination or harassment based on race, color, religion, sex (including pregnancy, childbirth, or related medical conditions), sexual orientation, gender identity or expression, age, disability, national origin, marital or domestic/civil partnership status, genetic information, citizenship status, military or veteran status, or any other personal characteristic.
Related Guides
Related Categories
Related Job Pages
More Engineer Jobs
• Research and develop innovative new solutions that will help to shape the future direction of Broadcom • Work within a scrum team in an agile development environment, with an opportunity to gain exposure to many of the key agile roles • Participate in design and code reviews • Participate in test plan reviews and develop automated test scripts to deliver high quality software solutions
Transmission Engineer
Camus EnergyTake control of a changing grid. Camus is high-performance, low-complexity grid management for a zero-carbon future.
• Perform modeling and technical analyses to evaluate proposed data center/large load sites for flexible interconnections, such as: • Time series power flow simulations • Contingency analyses • Production cost model simulations • Configuring probabilistic scenario analyses • Inform and shape the product roadmap for flexible interconnection planning and operations • In collaboration with Camus’ Customer Success team, engage with data center and utility customers to evaluate and implement flexible interconnections. • Work with our utility customers/partners to evaluate and understand utility data sets so that they can be integrated into the Camus platform. • Collaborate with Camus’ Customer Success and Engineering teams to integrate utility-specific transmission models, GIS data, and time series data into the Camus platform. • Stay informed on the regulatory and policy landscape around large load interconnection processes for the relevant utility and/or ISO/RTO regions of focus • Inform development of the operational model supporting flexible interconnections in utility and ISO/RTO operations • Document product capabilities and requirements that support the analysis and adoption of flexible interconnections in the transmission system
• Work closely with circuit designers to optimize the floor plan; • Perform daily layout editing, DRC/LVS tasks to ensure the quality of the layout; • Employ efficient design methodologies to reduce the number of iterations and promptly deliver the product.
Lead Human Factors Engineer
Blue Water ThinkingBlue Water Thinking is a VA CVE-certified Service Disabled Veteran Owned Small Business (SDVOSB).
• Lead and manage resources and studies across multiple scrum teams and projects. • Execute and coordinate activities needed to develop plans, define specifications, capture objectives, and design strategies. • Perform comprehensive risk assessments, evaluate alternatives, and assess targeted increment objectives. • Suggest redesigns or minimize negative impacts where products or processes are essential, considering all user impacts. • Communicate human factors issues clearly to engineering teams and end users, both formally and informally. • Maintain a strategy to support the teams’ increment objectives with a focus on operational excellence. • Evaluate products and processes for compliance with human factors engineering (HFE) standards and guidelines. • Document findings, strategies, and recommendations in clear, actionable reports. • Apply SAFe’s Lean Agile principles to foster high-performing teams and continuous improvement. • Support iterative design and development cycles to maximize value delivery.
