• Build and maintain evaluation frameworks for LLMs and generative AI systems tailored to public safety and intelligence use cases. • Design guardrails and alignment strategies to minimize bias, toxicity, hallucinations, and other ethical risks in production workflows. • Partner with AI engineers and data scientists to define online and offline evaluation metrics (e.g., model drifts, data drifts, factual accuracy, consistency, safety, interpretability). • Implement continuous evaluation pipelines for AI models, integrated into CI/CD and production monitoring systems. • Collaborate with stakeholders to stress test models against edge cases, adversarial prompts, and sensitive data scenarios. • Research and integrate third-party evaluation frameworks and solutions; adapt them to our regulated, high-stakes environment. • Work with product and customer-facing teams to ensure explainability, transparency, and auditability of AI outputs. • Provide technical leadership in responsible AI practices, influencing standards across the organization. • Contribute to DevOps/MLOps workflows for deployment, monitoring, and scaling of AI evaluation and guardrail systems (experience with Kubernetes is a plus). • Document best practices and findings, and share knowledge across teams to foster a culture of responsible AI innovation.

• Design, plan, and build all aspects of our products frontend, UI, data pipelines, and backend components. • Take leadership and pursue the best, state-of-the-art solutions, within the dynamic requirements and timelines. • Take E2E ownership of all aspects of the development cycle. • Focus on continuous growth and improvement, in every aspect (personal, products, processes, tools, skills, etc.). • Mentor engineers to develop new skills and grow professionally.

• Provide technical leadership on a new team prototyping and experimenting with new AI features • Productionize and ship AI integrations into Modern Health’s core product • Collaborate with cross-functional teams to deliver product features on time • Stay up-to-date with the latest AI technologies and trends

• Execute full software development life cycle, with a DevOps practice adopting CI/CD • Write well-designed, testable code. We currently use NodeJs, TypeScript, Python, and looking to use Golang for new backend services • Integrate software components into a fully functional software system • Troubleshoot, debug and upgrade existing systems • Deploy programs and evaluate user feedback • Comply with project plans and industry standards • Ensure software is updated with latest features