• Build and maintain automated evaluation pipelines for AI model quality. • Implement task-specific benchmarks and test suites. • Design quality dashboards tracking accuracy, regression, and safety metrics. • Implement automated regression testing for every model iteration. • Build comparison frameworks for side-by-side evaluation of model variants. • Analyze evaluation results to identify failure modes and report to the ML team. • Maintain evaluation datasets: versioning, quality validation, coverage analysis. • Support A/B testing infrastructure for production model validation.

Ready to join a team that’s all in? At Imprivata, we deliver unified access and security management programs that eliminate friction, empowering healthcare and mission-critical organizations to work smarter, faster, and more securely. We believe work can be more than a job or task—it’s a collective spirit; the type that emboldens creativity, embraces challenge, and fosters excitement. We are constantly raising the bar on what’s possible, owning the outcome of our triumphs and trials, staying nimble amidst change, and cultivating an environment where we win together. Here, your ideas matter, your differences are celebrated, and your work drives real results—for your career, your teammates, and our customers. When you join Imprivata, you embark on a shared journey of ambition and growth. We’re committed to building an inclusive workplace where everyone feels valued and supported. If you’re looking for a place to match your passion with purpose—and where every day you can make an impact—you’ll find it here. We are seeking a Senior Engineering Excellence & AI Enablement Director. This is a hybrid opportunity based out of our Waltham, MA office. Job Summary The Senior Engineering Excellence and AI Enablement Director will leads the systems, workflows, and enablement mechanisms that help Engineering deliver software with greater speed, quality, and predictability. In the near term, the role will focus primarily on scaling the effective use of AI-assisted development tools and practices across Engineering. Additionally, this role is responsible for improving how teams plan, build, test, document, release, and support software, combining workflow design, tooling strategy, adoption, and performance measurement into a cohesive operating model. In the near term, the role will focus primarily on scaling the effective use of AI-assisted development tools and practices across Engineering. Duties and Responsibilities Improve the Engineering Operating Model - Improve workflows across planning, development, code review, testing, documentation, release readiness, and post-release follow-through. - Reduce friction in handoffs, clarify decision rights, and strengthen execution consistency across teams. - Ensure processes are practical, lightweight, and aligned to the needs of software delivery. Coordinate Tooling Strategy and Standardization - Evaluate and standardize the tools that support engineering productivity, code quality, test automation, documentation, and release execution. - Identify high-value opportunities to apply AI to software development, testing, documentation, and engineering support workflows. - Define approved usage patterns, guardrails, and operating standards for new tools in partnership with Platform/DevOps/SRE, Security, and IT. Facilitate Enablement and Adoption - Build training, playbooks, examples, and communities of practice that help teams adopt new tools and workflows effectively. For example: - AI coding assistants in full PDLC (Code Assistance and Agentic Code Generation) - AI automated code review for Architecture, Quality, Security, Maintainability, Performance, etc. - AI test generation for Unit, Component, Integration, System, E2E, and Acceptance testing - AI generated documentation for internal and external stakeholders - AI Agents for escalation, bug squashing, debugging - Set clear expectations for engineering managers and technical leaders. - Track adoption, remove barriers to sustained usage, and reinforce consistent execution across teams. Measure Performance and Champion Continuous Improvement - Establish a clear scorecard for engineering operations, program progress, and productivity. - Provide regular reporting to the eStaff and Engineering leadership on progress, risks, and impact. - Use data to prioritize investments and continuous improvement efforts. Partner Across the Organization - Work closely with Product Management to improve planning and delivery interfaces across R&D. - Partner with Finance on business cases and budgets for productivity initiatives and tooling investments. - Represent Engineering in cross-functional productivity and AI initiatives and ensure Engineering priorities are reflected in broader standards and decisions. - Other duties as assigned and required. Required Qualifications - 10+ years of experience in software engineering, engineering operations, developer productivity, platform engineering, or related leadership roles - Experience improving software delivery systems through process, tooling, and organizational change - Strong understanding of modern engineering practices, including CI/CD, code review, testing, release management, reliability, and developer experience - Track record of leading cross-functional initiatives and driving adoption at scale - Strong communication and influencing skills, with the ability to work effectively across engineering and executive stakeholders Desired Qualifications - Experience introducing AI-assisted development tools or other modern productivity technologies into engineering organizations - Experience in regulated or security-sensitive SaaS environments - Experience building or scaling internal developer platforms, tooling programs, or productivity functions This position offers a total compensation range of $309,000.00 to $352,000.00 (inclusive of base salary and variable compensation, such as bonuses and incentives). In addition, more information about Imprivata’s benefit offerings can be found here. This range represents the high and low end of Imprivata’s compensation range for this position. Actual compensation will vary and may be above or below the range based on various factors, such as a candidate’s location, skills, experience, and qualifications. At Imprivata, we have a top-notch work environment, developmental opportunities, a competitive total rewards package, and the desire to have fun. If you have the skills and qualifications as we have described above, we want to hear from you! Imprivata provides equal employment opportunities, regardless of race, religion, age, sex, national origin, disability status, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. #LI-Remote #LI-ML1

• Develop machine learning models related to equipment telemetry • Deploy ML/DL Models into cloud-based access for internal use • Design data pipelines to ingest, clean, and analyze sensor, operational, and service data • Continuously refine models based on real-world equipment performance • Design and build supportive AI tools for the service, operations, and engineering team • Translate engineering formulas, design rules, and constraints into reliable, user-friendly digital tools • Collaborate with engineers to ensure accuracy, usability, and maintainability • Leverage Agentic AI to operationalize the AI project development • Assist the AI/ML team in developing the AI digital twin • Provide knowledge and understanding of ML behaviors relative to AI digital twins • Support additional data and digital tool initiatives as required by the business • Evaluate new opportunities where advanced analytics can improve products, services, or operations • Contribute to long-term data strategy at ThermalWorks • Lead a team of junior developers in additional projects.

Role Description We are seeking a highly skilled and experienced Senior AI/ML Engineer to join our growing data and machine learning organization. In this role, you will design, build, and scale intelligent systems that power our product, operations, and analytics. You will work closely with data engineers, product managers, platform engineers, and business stakeholders to develop production-grade machine learning models and AI-driven solutions on top of our Databricks Lakehouse platform. A successful candidate is both an innovative ML practitioner and a strong hands-on engineer who can take projects from concept to production. You are comfortable navigating ambiguity, working with incomplete data, leading technical discussions, and implementing systems that are robust, observable, and maintainable. You thrive in collaborative environments and enjoy building scalable ML foundations that accelerate development across teams. What You’ll Do - Model Development & Applied Machine Learning - Build, train, evaluate, and deploy machine learning models for prediction, classification, NLP, anomaly detection, and generative AI use cases. - Apply modern ML techniques, experimentation frameworks, and statistical best practices to ensure model accuracy, fairness, and reliability. - Develop LLM-driven applications, prompt engineering strategies, and retrieval-augmented generation (RAG) systems when applicable. - Data Engineering & Feature Development - Design and implement scalable features using Delta Lake, Spark, and Databricks Feature Store. - Partner with data engineering teams to understand data availability, quality, lineage, and ingestion patterns. - Build automated, reproducible pipelines that support training, validation, and model refresh cycles. - MLOps & Productionization - Own end-to-end ML lifecycle using Databricks workflows, MLflow, feature stores, and model registries. - Develop CI/CD and automated model deployment pipelines that ensure performance and reliability. - Implement monitoring for drift, model degradation, data quality, and performance regressions. - AI Systems Architecture - Design modular, scalable ML architectures that integrate with APIs, data warehouses, microservices, and downstream applications. - Evaluate when to apply classical ML, deep learning, or LLM-driven approaches based on business constraints. - Experimentation & Evaluation - Develop A/B tests, offline/online evaluation frameworks, and statistical validation strategies. - Analyze model results with clarity and communicate insights to technical and non-technical partners. - Cross-Functional Collaboration - Work closely with product, engineering, and business teams to identify ML opportunities, refine requirements, and deliver measurable outcomes. - Participate in architecture reviews, technical planning sessions, and roadmap discussions. - Document work in a way that is scalable and easy for future engineers to adopt. - Continuous Learning - Stay up to date on emerging ML frameworks, LLM advancements, Databricks capabilities, and scalable architecture patterns. - Explore new tools, libraries, and platforms that can enhance model performance or development efficiency. Qualifications - 7+ years of experience in machine learning, applied AI, or similar engineering roles. - Strong expertise building ML models with Python, Spark, Databricks, and MLflow. - Deep knowledge of modern ML techniques: supervised/unsupervised models, deep learning, transformers, embeddings, vector stores, and LLM-based systems. - Solid understanding of software engineering principles: version control, testing, CI/CD, observability, and modular architecture. - Experience deploying ML models to production with reliable pipelines and monitoring. - Strong ability to explain technical concepts to non-technical stakeholders. - Experience working in agile product environments. - Proficiency with SQL and working with large-scale distributed datasets. Nice to Have - Experience with Databricks Model Serving, Unity Catalog, Feature Store, and Delta Live Tables. - Experience building LLM-powered applications, RAG systems, fine-tuning, or model distillation. - Familiarity with cloud infrastructure (AWS and Azure), Kubernetes, and container orchestration. - Background in statistics, computer science, machine learning engineering, or related fields. - Strong interest in building foundational ML platforms, tools, and frameworks for internal teams. - Experience with real-time ML systems, streaming data, or event-driven architectures. Benefits - Competitive Compensation & Total Rewards Incentives - Comprehensive Healthcare Coverage - Multiple 401(k) Savings Plan Options - Auto Enrollment in Employer-Directed Retirement Account Feature (100% employer-funded!) - Generous Paid Time Off – Including 12 Paid Holidays, Volunteer Time Off and Paid Family Leave - Disability, Life, and Long Term Care Insurance - Tuition Reimbursement, Student Loan Repayment and Training & Certification Support - Wellness support including gym membership reimbursement and Employee Assistance Program resources (work/life support, financial & legal management) - Caregiver and Mental Health Support Services