Grafana Labs supports organizations’ monitoring, visualization and observability goals. 950,000+ active installations
Senior AI Engineer
Location
United States
Posted
3 days ago
Salary
$154.4K - $185.3K / year
Seniority
Senior
Job Description
Senior AI Engineer
Grafana Labs
• Own end-to-end development of multi-agent AI systems, from architecture and implementation through testing, deployment, and ongoing operation • Build modular, composable agentic systems using orchestration frameworks (LangChain, CrewAI, Anthropic MCP, or similar) that operate 24/7 across teams • Develop reusable agentic skills that agents invoke across interfaces (Slack, dashboards, internal apps, CLIs) • Implement observability and feedback loops including logging, performance metrics, prompt iteration, model evaluation, and cost management • Establish governance and compliance standards for AI workflows including access controls, audit trails, PII handling, and human-in-the-loop escalation paths • Build MCP servers, APIs, CLIs, and microservices connecting AI models to business systems (BigQuery, Slack, CRMs, email, calendars, analytics tools) • Architect data flows for retrieval-augmented generation (RAG), connecting LLMs to internal knowledge bases, customer data, and real-time business context • Build serverless or containerized services (GCP Cloud Functions, Cloud Run) that scale with usage and integrate with Grafana's cloud infrastructure • Partner with RevOps, Demand Generation, Regional Marketing, and SDR teams to scope high-impact automation problems, identify bottlenecks, and build solutions with measurable business outcomes • Design and deploy workflows using orchestration tools (n8n, Workato, or custom platforms) with CI/CD, testing, and production reliability standards • Build systems designed for self-service with documentation, playbooks, and enablement materials that let partner teams operate independently.
Job Requirements
- 8+ years of software engineering experience with depth in backend development, systems integration, or data/analytics engineering
- 2+ years hands-on experience applying LLMs/AI to production workflows, not just prototypes
- Strong proficiency in Python and JavaScript/Node.js with Git-based workflows, code review practices, and testing discipline
- Hands-on experience with LLM frameworks and patterns including prompt engineering, RAG, function calling/tool use, structured output parsing, and evaluation
- Experience building and operating multi-agent systems at scale including agent decomposition, orchestration patterns (sequential chains, router/dispatcher, parallel fan-out), state management, and production monitoring
- You diagnose business problems before writing code. You think in workflows and outcomes, not just functions.
- Deep familiarity with Google Cloud Platform, BigQuery, and serverless/containerized services (Cloud Functions, Cloud Run)
- Understanding of LLM failure modes and production mitigations including confidence thresholds, fallback logic, human escalation, and cost/latency management
- Proven ability to identify high-leverage problems, push back on low-impact requests, and deliver end-to-end with minimal direction
- Fluent with AI-assisted development tools (GitHub Copilot, Cursor, Claude Code). You use AI to build AI systems
- Clear technical communicator who can explain complex systems in simple terms to both engineers and business stakeholders.
Benefits
- 100% Remote, Global Culture
- Scaling Organization
- Transparent Communication
- Innovation-Driven
- Open Source Roots
- Empowered Teams
- Career Growth Pathways
- Approachable Leadership
- Passionate People
- In-Person onboarding
- Balance is Key - 30 days annual leave including Grafana Shutdown Days
Related Guides
Related Job Pages
More AI Engineer Jobs
AI Engineer / AI Developer – SME Projects
COREEN GmbHStrategische Unternehmensentwicklung & KI-Transformation für den Mittelstand
• Development of AI prototypes, MVPs and production-ready applications • Building RAG systems, chatbot and assistant solutions based on company data • Integration of AI models, APIs, data sources, document stores and vector databases • Development of automations and AI workflows, e.g., using Python, APIs, n8n, Make or agent frameworks • Technical evaluation of use cases regarding feasibility, effort, data requirements, security and operations • Close collaboration with consultants, domain experts and client teams • Documentation of architecture, assumptions, limitations, risks and operational requirements • Contributing to the development of reusable components, templates and technical standards
• You decide when, where, and how you work on projects and tasks. • We match you with work that fits your skills. • Gain more experience and develop various skills. • Have access to various opportunities across different industries and companies. • Be a part of the team with full-time collaboration.
• Use AI coding assistants such as OpenAI Codex, Claude Code, Cursor, GitHub Copilot, or equivalent tools as part of daily development workflows. • Apply AI-assisted approaches to software design, implementation, testing, debugging, refactoring, and documentation. • Develop and maintain reusable prompts, workflows, skills, and agent instructions that improve engineering effectiveness and consistency. • Use agentic development techniques to accelerate feature delivery while maintaining engineering quality standards. • Contribute to the evolution of AI-enabled engineering practices across teams and projects. • Design, build, and maintain modern web applications across frontend and backend systems. • Develop scalable APIs, services, and user interfaces using modern engineering practices. • Contribute to architectural decisions and system design discussions. • Ensure software solutions meet performance, security, reliability, and maintainability requirements. • Collaborate with multidisciplinary teams to deliver high-quality customer experiences. • Review and validate AI-generated code to ensure production readiness. • Implement automated testing and quality assurance practices throughout the SDLC. • Participate in code reviews and contribute to engineering standards and best practices. • Design and implement mechanisms that increase trust in AI-generated outputs, including automated validation, testing strategies, specification conformance checks, evaluation frameworks, and feedback loops. • Establish guardrails and verification workflows that ensure agent-generated code and artifacts meet functional, security, performance, and maintainability requirements. • Evaluate the effectiveness, reliability, and quality of AI-assisted development workflows and continuously improve them. • Contribute to the development of reliable engineering practices for agentic software development. • Work closely with Product Owners, Designers, QA Engineers, and other Software Engineers. • Support the adoption of AI-enabled engineering practices across teams. • Share knowledge, workflows, and best practices related to AI-assisted software development. • Mentor other engineers on effective and responsible use of AI throughout the software development lifecycle.
Senior Manager, AI Engineering
HackerOnePeace of mind from security's greatest minds. #TogetherWeHitHarder
• Own two of the most strategically important product surfaces in our platform: AI-Native Intelligence Dashboard and Payments & Monetization • Lead the team of senior engineers, set technical direction, and ship AI-powered capabilities that customers can see, feel, and measure • Multiply your impact by leading, mentoring, and developing Engineering Managers and senior engineers • Own the outcome for execution across multiple AI engineering teams • Drive an AI First approach to engineering • Practice Change Agility by guiding teams through evolving AI technologies • Apply First Principles Problem Solving to complex AI and product challenges • Leverage Data-Driven Decision Making to assess system performance • Collaborate cross-functionally with Product, Design, Data, Security Research, and other Engineering leaders




