Senior AI Engineer
Location
United States
Posted
1 day ago
Salary
$190K - $250K / year
Seniority
Senior
Job Description
Senior AI Engineer
Atria Institute
• Architect and ship production AI systems end-to-end (orchestration, retrieval, inference, evaluation, and monitoring) for clinical decision support, document understanding, and agentic medical companions. • Design agentic workflows with custom skills and harnesses, using frontier LLMs and self-hosted models where appropriate, and resisting the urge to reach for an agent when a function call would do. • Lead development of structured information extraction pipelines from clinical documents, including automated verification systems and human-in-the-loop gates, where necessary. • Build robust evaluation infrastructure (golden datasets, LLM-as-judge, regression tests, online A/B evaluation) that gates every model and prompt change. Accuracy, safety, cost, and latency are first-class metrics. • Engineer for latency and cost: streaming, caching, prompt compression, model routing, and inference-cost budgets. • Own production health: trace- and span-level observability, prompt versioning, drift detection, cost monitoring, replay-from-production for debugging, PHI-safe logging throughout, and the general art of finding out something is broken before a clinician does. • Operationalize models trained by the AI Scientist team: serving, evaluation in production, rollback paths, and the feedback loops that turn real usage into training and eval data, rather than into a Slack thread nobody reads. • Partner with clinicians, the data engineering team, and our research collaborators to translate clinical requirements into specifications and deliverable systems. • Educate and mentor PMs and engineers across Atria on applied AI best practices, from prompt and evaluation design to choosing the right model for the job (which, surprisingly often, is not the largest one). • Set technical direction for AI engineering at Atria: architecture decisions, build-vs-buy calls, and the evaluation and observability standards the team actually works to. • Raise the bar on engineering practices, code review, observability, and incident response. Quietly, persistently, and with grace. • Drive vendor and partner technical due diligence for AI/ML vendors: BAA scope, PHI handling, sub-processor obligations, and IP terms. • Mentor other AI engineers through code review, design docs, and architecture decisions, with the patient conviction that good systems are made twice: once badly, then properly.
Job Requirements
- A bias to ship. You would rather have a rough thing in production tomorrow that you can measure and improve than a beautiful thing in a design doc next quarter.
- A scrappy streak. You can pick up an unfamiliar tool, framework, or clinical concept on a Wednesday and have something working with it by Friday.
- A serious drive to keep getting better. You read other people's code, papers, and post-mortems. You treat being wrong as cheap information rather than an event.
- Hands-on experience building and operating LLM-powered or production ML systems: 4+ years.
- Strong Python skills and deep familiarity with at least one production backend framework (FastAPI, Flask, etc.).
- Hands-on experience with LLM orchestration, function/tool calling, and retrieval system design: chunking, hybrid search, reranking, and the rest of the unfashionable middle bit that makes RAG actually work.
- Working knowledge of a cloud data warehouse (Snowflake strongly preferred) and modern data tooling such as dbt, Dagster, or Airflow.
- A track record of designing evaluation systems for LLM applications — including the ones that caught regressions before users did, and the ones that didn't.
- Familiarity with safety considerations for LLM applications: guardrails, adversarial testing, and the failure modes that tend to introduce themselves at 4pm on a Friday.
- Practical understanding of HIPAA, BAAs, PHI handling, and the everyday realities of working in a regulated environment without losing the will to live.
- Strong written communication: design docs, technical RFCs, and documentation that someone else can actually follow six months later.
Benefits
- Excellent health and wellness benefits, fully covered by Atria, effective date of hire
- OneMedical membership for employees & dependents, giving access to 24/7 virtual care
- Fertility & family planning
- Company-covered preventive health screenings through partner hospitals (calcium score)
- Fitness Perks, including Wellhub +
- 401k contributions and 4% match starting after 6 months
- Flexible Time Off
- Continuing medical education (CME) and CEU support for professional licensure
Related Guides
Related Job Pages
More AI Engineer Jobs
Senior Full Stack Developer – Tech Lead, AI Tools
StormXShop at your favorite stores and earn free Bitcoin or Ethereum. StormX is the easiest and safest way to start earning!
• Provide technical leadership for the AI Tools development unit, defining coding standards, architecture, quality, and security • Conduct code reviews, pair programming, and mentor mid-level and junior developers to raise the team's technical level • Participate in product planning with stakeholders, translating business requirements into viable, prioritized technical solutions • Define and evolve the unit's technical roadmap, including decisions on stack, infrastructure, integrations, and technical debt • Establish and track engineering metrics (lead time, quality, performance, availability) and SLOs for the products under your responsibility • Design and develop full stack applications using Python (FastAPI/Flask) on the backend and React/TypeScript on the frontend • Architect and build robust, scalable, secure, and well-documented RESTful APIs (OpenAPI/Swagger) • Model and optimize relational databases, especially PostgreSQL, including schema design, complex queries, indexes, and performance tuning • Make architecture decisions focused on microservices, event-driven architecture, and horizontal scalability • Build modern, accessible, high-performance interfaces using Vite, Zod, Tailwind, and Mantine, with state management via Zustand, Context API or Recoil
Role Description As a Senior AI Engineer at Atria, you will own the design, development, and production deployment of the LLM-powered systems that put AI directly in front of clinicians and members: - Agentic medical companions - Clinical decision support - Document understanding pipelines that turn unstructured clinical data into usable signal This is a hands-on, high-ownership role for an engineer who wants to do work with direct patient impact. What you'll do - Architect and ship production AI systems end-to-end (orchestration, retrieval, inference, evaluation, and monitoring) for clinical decision support, document understanding, and agentic medical companions. - Design agentic workflows with custom skills and harnesses, using frontier LLMs and self-hosted models where appropriate. - Lead development of structured information extraction pipelines from clinical documents, including automated verification systems and human-in-the-loop gates, where necessary. - Build robust evaluation infrastructure (golden datasets, LLM-as-judge, regression tests, online A/B evaluation) that gates every model and prompt change. - Engineer for latency and cost: streaming, caching, prompt compression, model routing, and inference-cost budgets. - Own production health: trace- and span-level observability, prompt versioning, drift detection, cost monitoring, replay-from-production for debugging, PHI-safe logging throughout. - Operationalize models trained by the AI Scientist team: serving, evaluation in production, rollback paths, and feedback loops that turn real usage into training and eval data. - Partner with clinicians, the data engineering team, and research collaborators to translate clinical requirements into specifications and deliverable systems. - Educate and mentor PMs and engineers across Atria on applied AI best practices. Leadership and Influence - Set technical direction for AI engineering at Atria: architecture decisions, build-vs-buy calls, and evaluation and observability standards. - Raise the bar on engineering practices, code review, observability, and incident response. - Drive vendor and partner technical due diligence for AI/ML vendors: BAA scope, PHI handling, sub-processor obligations, and IP terms. - Mentor other AI engineers through code review, design docs, and architecture decisions. - A bias to ship: prefer a rough thing in production tomorrow over a beautiful thing in a design doc next quarter. - A scrappy streak: pick up an unfamiliar tool, framework, or clinical concept quickly. - A serious drive to keep getting better: read others' code, papers, and post-mortems. Qualifications - Hands-on experience building and operating LLM-powered or production ML systems: 4+ years. - Strong Python skills and deep familiarity with at least one production backend framework (FastAPI, Flask, etc.). - Hands-on experience with LLM orchestration, function/tool calling, and retrieval system design. - Working knowledge of a cloud data warehouse (Snowflake strongly preferred) and modern data tooling such as dbt, Dagster, or Airflow. - A track record of designing evaluation systems for LLM applications. - Familiarity with safety considerations for LLM applications. - Practical understanding of HIPAA, BAAs, PHI handling, and working in a regulated environment. - Strong written communication: design docs, technical RFCs, and documentation. Nice to have - Experience in healthcare, clinical informatics, or other regulated domains. - Familiarity with medical ontologies (LOINC, SNOMED CT, ICD-10, RxNorm) and clinical data standards (FHIR, HL7). - Experience with self-hosted model serving and inference optimization. - Multimodal experience: vision, structured data, or time-series data alongside text. Salary and Benefits - Salary range: $190,000 - $250,000 + performance-based bonus - Excellent health and wellness benefits, fully covered by Atria, effective date of hire - OneMedical membership for employees & dependents, giving access to 24/7 virtual care - Fertility & family planning - Company-covered preventive health screenings through partner hospitals - Fitness Perks, including Wellhub + - 401k contributions and 4% match starting after 6 months - Flexible Time Off - Continuing medical education (CME) and CEU support for professional licensure
Role Description We are looking for a Senior AI Engineer to join our team and design, develop, and deploy innovative AI-powered solutions that solve complex business challenges. In this role, you will work on the development and integration of modern Artificial Intelligence applications, leveraging Large Language Models (LLMs), cloud technologies, and scalable software engineering practices. You will collaborate with cross-functional teams to deliver secure, reliable, and production-ready AI solutions. - Design, develop, and deploy AI-powered applications and services - Build and integrate solutions using Large Language Models (LLMs) - Develop and optimize AI models and intelligent workflows - Integrate AI capabilities into enterprise applications and business processes - Develop and maintain REST APIs supporting AI services - Collaborate with software engineers, data engineers, and business stakeholders to define AI solutions - Monitor, maintain, and improve the performance of AI applications - Contribute to technical architecture, documentation, and engineering best practices - Stay up to date with emerging AI technologies and industry trends Qualifications - Bachelor's degree in Computer Science, Artificial Intelligence, Software Engineering, Data Science, or a related field - 5+ years of experience in Software Engineering, AI Engineering, or Machine Learning - Strong programming experience in Python - Experience working with Large Language Models (LLMs) - Experience with OpenAI, Azure OpenAI, Anthropic Claude, or similar AI platforms - Experience building AI-powered applications and integrating AI services - Experience developing REST APIs - Experience with SQL and/or NoSQL databases - Experience with cloud platforms such as Azure, AWS, or Google Cloud - Experience with Git and CI/CD pipelines - Strong analytical and problem-solving skills - Excellent written and verbal communication skills in English Requirements - Experience with Retrieval-Augmented Generation (RAG) - Experience with LangChain, Semantic Kernel, or similar AI frameworks - Experience with Docker and Kubernetes - Experience with vector databases - Experience with AI agents and intelligent automation solutions Benefits - Work on cutting-edge AI and Generative AI projects - Collaborate with experienced engineering and technology teams - Build innovative solutions using modern AI technologies - Contribute to impactful and technically challenging projects - Develop your expertise in one of the fastest-growing areas of software engineering
• Build and operationalize end-to-end agentic AI solutions, like RAG pipelines, multi-agent workflows, and human-in-the-loop steps, for enterprise clients • Translate solution designs and business requirements into clean, well-tested production-grade code aligned with phData methodologies, standards, and best practices. • Build production-grade agents emphasizing reliability, scalability, security, observability, and evaluation and guardrail strategies for safe, accurate performance before and after deployment. • Ensure engagements are delivered on time, within scope, and with measurable business value for clients. • Contribute hands-on to reusable agentic accelerators and reference implementations aligned with the Intelligence Platform that can be deployed across multiple clients. • Help establish engineering patterns, tooling, and ways of working as part of the zero-to-one buildout of the India delivery center. • Collaborate with cross-functional partners, including forward-deployed engineers, data engineers, platform/DevOps engineers, and business stakeholders to deliver successful client engagements. • Ensure high-quality deliverables through code reviews, technical documentation, automated testing, and adherence to security and governance standards. • Partner with practice and account leaders to identify opportunities to expand engagements, improve delivery patterns, and standardize AI deployment approaches. • Contribute to internal initiatives such as IP development, accelerators, reference architectures, templates, playbooks, and training for colleagues on best practices for AI engineering. • Represent phData with professionalism in all interactions, communicating clearly with both technical and non-technical stakeholders.


