• Build and ship AI features end-to-end (model → system → user experience) • Design and iterate on prompts, tools, memory, and agent workflows • Turn raw model outputs into structured, reliable, and predictable behaviors • Debug issues across the full stack (model, orchestration, infra, UX) • Optimize for latency, cost, and production reliability • Develop lightweight evaluation frameworks to measure real-world performance • Work closely with product and engineering to translate ambiguous problems into working systems

• Build and ship AI features end-to-end (model → system → user experience) • Design and iterate on prompts, tools, memory, and agent workflows • Turn raw model outputs into structured, reliable, and predictable behaviors • Debug issues across the full stack (model, orchestration, infra, UX) • Optimize for latency, cost, and production reliability • Develop lightweight evaluation frameworks to measure real-world performance • Work closely with product and engineering to translate ambiguous problems into working systems

• Build AI-powered infrastructure that gives VC and PE clients a genuine edge — proprietary systems that extract signals, automate decisions, and compound in value over time. • Agentic AI Systems: Multi-step LLM workflows, RAG pipelines, and agent orchestration systems — owned from architecture to production. • Full-Stack AI Applications: Client-facing web applications with AI embedded throughout — Python/FastAPI backends, React frontends, integrated with LLM providers (OpenAI, Anthropic, Gemini). • Data Platform Engineering: Scalable pipelines and cloud infrastructure (AWS/GCP) that underpin AI features — vector databases, data ingestion layers, API integrations. • Technical Discovery & Client Engagement: Translating business needs into AI-first technical proposals. • AI Quality & Internal Standards: Guardrails, automated testing, and observability for AI systems.

AI Engineer III locations Remote time type Full time job requisition id R12173 Job Summary: The AI Engineer III is the lead builder of the "Engine Room," responsible for the reliability, scalability, and observability of the Pantheon AI Mesh and Azure AI Foundry stack. This role designs and maintains the CI/CD pipelines for models and agents, ensuring that AI solutions are deployed securely, monitored for drift, and can be rolled back instantly if issues arise. Essential Functions: - Architect and maintain the LLMOps/GenAIOps toolchain, including model registries, prompt version control, and reproducible training pipelines. - Implement and manage the Azure AI Foundry environment, configuring model routers, quota management, and private endpoints for secure inferencing. - Develop comprehensive observability dashboards to track model latency, token costs, hallucination rates, and drift. - Automate "Policy-as-API" controls within the orchestration layer to enforce governance guardrails (e.g., PII filtering) at runtime. - Collaborate with the Platform SRE team to ensure high availability and disaster recovery for mission-critical clinical agents. - Manage the "Model Registry," ensuring all deployed models have associated version history, performance metrics, and rollback targets. - Configure and maintain "Vector Databases" and RAG pipelines, optimizing retrieval performance and index freshness. - Implement "Prompt Filtering" and content moderation gateways to prevent jailbreaks and enforce safety standards at the infrastructure level. - Develop "Blue/Green" or "Canary" deployment strategies for AI agents to safely test new model versions in production. - Manage the "API Gateway" for all AI services, ensuring authentication, rate limiting, and usage logging are enforced. - Optimize "GPU/CPU Orchestration" to control compute costs while maintaining performance SLAs for high-volume inference. - Build automated "Drift Detection" alerts that trigger retraining or human review when model performance degrades below a set threshold. - Perform any other job related duties as requested. Education and Experience: - Bachelor's degree in Computer Science, Engineering, or related technical field required - Equivalent years of relevant work experience may be accepted in lieu of required education - Five (5) years of IT engineering experience, with at least three (3) years specialized in DevOps, MLOps, or Cloud Infrastructure required - Experience with Azure AI Services (Azure OpenAI, AI Search, Azure ML) and container orchestration (Kubernetes/AKS) required - Experience building and maintaining CI/CD pipelines for machine learning models or complex software applications required Competencies, Knowledge and Skills: - Mastery of Python and scripting languages for automation and infrastructure-as-code (Terraform, Bicep, ARM templates) - Deep understanding of LLMOps principles: Prompt versioning, model registry management, and evaluation pipelines (e.g., MLflow, Prompt Flow) - Proficiency in Azure Networking and Security, including Private Endpoints, VNET integration, and API Management (APIM) configuration - Knowledge of Vector Databases and RAG (Retrieval Augmented Generation) infrastructure requirements - Strong observability skills, utilizing tools like Azure Monitor or App Insights to track token usage, latency, and drift Licensure and Certification: - Microsoft Certified: Azure AI Engineer Associate or Azure DevOps Engineer Expert preferred - CKA (Certified Kubernetes Administrator) preferred Working Conditions: - General office environment; may be required to sit or stand for extended periods of time - Travel is not typically required Compensation Range: $94,100.00 - $164,800.00 CareSource takes into consideration a combination of a candidate’s education, training, and experience as well as the position’s scope and complexity, the discretion and latitude required for the role, and other external and internal data when establishing a salary level. In addition to base compensation, you may qualify for a bonus tied to company and individual performance. We are highly invested in every employee’s total well-being and offer a substantial and comprehensive total rewards package. Compensation Type (hourly/salary): Salary Organization Level Competencies - Fostering a Collaborative Workplace Culture - Cultivate Partnerships - Develop Self and Others - Drive Execution - Influence Others - Pursue Personal Excellence - Understand the Business This job description is not all inclusive. CareSource reserves the right to amend this job description at any time. CareSource is an Equal Opportunity Employer. We are dedicated to fostering an environment of belonging that welcomes and supports individuals of all backgrounds. #LI-GM1