Job Closed
This listing is no longer active.
We are the first public safety operating system empowering over 2500 cities to eliminate crime.
Staff AI Systems Engineer
Location
United States
Posted
62 days ago
Salary
$200K - $225K / year
Seniority
Lead
Job Description
Staff AI Systems Engineer
Flock Safety
• Help support our emerging product, Night Shift, an AI research assistant that amplifies the impact of investigators by automating the tedious, repetitive steps involved in working a case. • Work closely with partners in Engineering (Backend, Frontend, and Design) in a fast-paced environment. • Be one of the earliest technical contributors to our system architecture for agentic AI, and will own our AI evaluation framework. • Achieve measurably faster, more accurate leads for every officer and every shift. • Immerse yourself in the current system design and agent/tooling landscape. Understand the core customer use cases and data flows. • Support the team by shipping a few quick wins (e.g., refining tool APIs, prompt engineering, fixing bugs). • Stand up the foundational eval and observability scaffolding (datasets, metrics, KPIs, reporting). • Propose a technical architecture and implementation plan for an agent evaluation framework. • Deliver the MVP evaluation harness to produce initial metrics, enable debugging and perform regression testing. • Take on a system feature that offers demonstrated improvement against your MVP evaluation suite. • Productionize the evaluation and observability platform and make it the source of truth for quality and safety.
Job Requirements
- Familiarity with Agentic Systems: Hands-on experience with LLM agents including:
- LLM API use (e.g. LangChain/LangGraph, vLLM, OpenAI/Gemini/Anthropic APIs)
- Agent Design: tool use (e.g. via MCP), retrieval, memory, grounding/attribution for claims, and guardrails.
- Architectural patterns: planning and hand-off for multi-agent systems, context management.
- RAG: vector/hybrid search (e.g. pgvector, turbopuffer, rerankers, etc.)
- ML Platform expertise: 5+ years building and shipping ML systems to production; experience in the following areas:
- Backend Python and JS familiarity required; Typescript/Golang familiarity welcome
- Web services (e.g. Express/FastAPI, REST, SSE, JWTs)
- Cloud Infrastructure (e.g. AWS, Terraform, VPC, Networking)
- Backend databases/stores (e.g. Postgres, Redis)
- Observability (e.g. Prometheus, Grafana, OpenTelemetry, LangSmith/Langfuse)
- [Preferred] Durable execution (e.g. Temporal, Hatchet)
- [Preferred] OLAP (e.g. ClickHouse, Bigquery)
- [Preferred] ML Inference (e.g. PyTorch, TensorRT, NVIDIA Triton), ideally in multimodal domains (text/image/video)
- [Preferred] Compute orchestration (e.g. Kubernetes, Prefect, Ray)
- Experience with LLM Evaluations at scale: You’ve built offline/online eval harnesses and are familiar with the methodologies and metrics to measure:
- Search, retrieval, and recommendation performance
- Safety & robustness (security, compliance, red-teaming, regression testing)
- Cost, performance and latency trade-offs
- [Preferred] Agentic task success, trajectory quality, preference learning (e.g. SFT, DPO, RLHF, LLM-as-judge)
Benefits
- Flexible PTO: We offer non-accrual PTO, plus 11 company holidays.
- Fully-paid health benefits plan for employees: including Medical, Dental, and Vision and an HSA match.
- Family Leave: All employees receive 12 weeks of 100% paid parental leave. Birthing parents are eligible for an additional 6-8 weeks of physical recovery time.
- Fertility & Family Benefits: We have partnered with Maven, a complete digital health benefit for starting and raising a family. Flock will provide a $50,000-lifetime maximum benefit related to eligible adoption, surrogacy, or fertility expenses.
- Spring Health: Spring Health offers a variety of mental health benefits, including therapy, coaching, medication management, and digital tools, all tailored to each individual's needs.
- Caregiver Support: We have partnered with Cariloop to provide our employees with caregiver support.
- Carta Tax Advisor: Employees receive 1:1 sessions with Equity Tax Advisors who can address individual grants, model tax scenarios, and answer general questions.
- ERGs: We offer four ERGs today - Women of Flock, Flock Proud, LEOs and Melanin Motion.
- WFH Stipend: $150 per month to cover the costs of working from home.
- Productivity Stipend: $300 per year for Audible, Calm, Masterclass, Duolingo, etc.
- Home Office Stipend: A one-time $750 to help you create your dream office.
Related Guides
Related Categories
Related Job Pages
More Systems Engineer Jobs
• Design, implement, and maintain automated systems for employee onboarding, access management, and offboarding that scale globally • Build and maintain RBAC and birthright access policies • Deploy and manage secure device management solutions for our globally distributed team • Utilize infrastructure-as-code to standardize configuration management • Automate routine IT workflows and manage the IT ticketing process • Partner with Security to establish and monitor compliance controls across endpoints • Evaluate and recommend new IT tools and SaaS platforms that improve security, scalability, and employee experience • Create clear documentation and educational resources
• Provide IT support and day-to-day maintenance of systems for all users • Manage servers (Windows & Ubuntu), Active Directory, virtual machines, backups, and IT documentation • Support Microsoft 365, Windows operating systems, and network devices • Assist with software installation, network management, and IT security initiatives • Develop IT solutions using languages such as PHP, Python, C#.net core, Laravel, and maintain databases • Generate reports and maintain BI dashboards from company IT systems • Provide security appliance support and day-to-day maintenance of security appliances deployed on client sites • Provide onsite Security engineering task support to clients
• Handle tickets and perform in-depth technical analysis • Resolve complex incidents and identify and address root causes • Document permanent solutions and technical standards to reduce recurrence • Create and maintain detailed documentation (architecture, workflows, dependencies) • Support CMDB processes and configuration governance • Define observability standards (logs, metrics, alerts) for all systems • Ensure compliance with Brazil's LGPD and internal policies (access control, data segregation) • Support audits and respond to critical vulnerabilities • Define strategies for high availability, backup, and recovery • Validate disaster recovery tests and contingency plans for critical systems
Systems Engineer – SME
General DynamicsGeneral Dynamics is a global aerospace and defense company offering products designed to provide safety and security to people around the world. In the past, General Dynamics has p
• Provide senior-level technical leadership and architectural oversight for MMIS modernization initiatives • Lead architecture governance activities, including establishing and coordinating Architecture Review Board processes • Direct major infrastructure modernization efforts, including hardware and operating system migrations • Oversee consolidation of multi-domain environments into a single enterprise domain to support MMIS platform operations • Provide architectural guidance for cloud adoption and migration of MMIS applications to cloud hosting environments • Support development of Medicaid enterprise architecture frameworks and modularity strategies • Guide cross-functional teams in the development of high-quality design documentation and adherence to application and business design best practices • Develop strategies for development and testing environments, supporting system readiness and quality assurance processes • Provide mentorship and training to technical and business team members on MMIS architecture and batch development standards
