Canvas Medical logo
Canvas Medical

Accelerating Everyday Medicine

Applied AI Software Engineer

Software EngineerSoftware EngineerOtherRemoteSeniorTeam 11-50Since 2015H1B No SponsorCompany SiteLinkedIn

Location

California

Posted

103 days ago

Salary

$300K - $400K / year

Seniority

Senior

Bachelor Degree9 yrs expEnglishGeminiOpenAI APIPythonSQL

Job Description

Applied AI Software Engineer

Canvas Medical

Canvas Medical is the electronic medical records (EMR) and payments development platform for healthcare. We build modern, elegant front- and back-end tooling to enable new ways for developers and clinicians to collaborate to solve healthcare’s toughest challenges. Canvas is institutionally backed by some of the greatest technology investors in the world (funded notable health tech companies such as GoodRx, Oscar Health, and Hims & Hers Health). The Role We’re hiring an Applied AI Software Engineer to lead evaluations for agents in development and the post-deployment fleet of agents operating in Canvas to automate work for our customers. You will help develop agents in Canvas using state of the art foundation model inference and fine-tuning APIs along with our server-side SDK. The server-side SDK provides extensive tools and virtually all the context necessary for excellent agent performance. You’ll be responsible for designing and running rigorous evaluation experiments that measure performance, safety, and reliability across a wide variety of clinical, operational, and financial use cases. This role is ideal for someone with deep experience evaluating LLM-based agents at scale. You’ll create high-fidelity unit evals and end-to-end evaluations, define expert-determined ground truth outcomes, and manage iterations across model variants, prompts, tool use, and context window configurations. Your work will directly inform model selection, fine-tuning, and go/no-go decisions for AI features used in production settings. You’ll collaborate with product, ML engineering, and clinical informatics teams to ensure that Canvas's AI agents are not only capable, but trustworthy and robust under real-world healthcare constraints. You will also work with technical product marketers and developer advocates to help our broader developer community and the broader market understand the uniquely differentiated value of agents in Canvas. Who You Are What You’ll Do Design and execute large-scale evaluation plans for LLM-based agents performing clinical documentation, scheduling, billing, communications, and general workflow automation tasks. Build end-to-end test harnesses that validate model behavior under different configurations (prompt templates, context sources, tool availability, etc.). Partner with clinicians to define accurate expected outcomes (gold standard) for performance comparisons in domains of clinical consequence, and partner with other subject matter experts in other non-clinical domains. Run and replicate experiments across multiple models, parameters, and interaction types to determine optimal configurations. Deploy and maintain ongoing sampling for post-deployment governance of agent fleets. Analyze results and summarize tradeoffs in clarity for product and engineering stakeholders, as well as for technical stakeholders among our customers and the broader market. Take ownership over internal eval tooling and infrastructure, ensuring speed, rigor, and reproducibility. Identify and recommend candidates for reinforcement fine-tuning or retrieval augmentation based on gaps identified in evals. What Success Looks Like at 90 Days An expanded set of robust evaluation suites exists for all major AI features currently in development and in production. We have well-defined correctness criteria for each workflow and a reliable source of expert-determined outcome objects. Product and engineering teams have integrated your evaluation tools into their daily workflows. Evaluation results are clearly documented and reproducible, enabling trust in the performance trajectory. Your have effectively engaged your marketing counterparts to translate your work into key messages to the market and to Canvas customers. Qualifications 5+ years of experience in applied machine learning or AI engineering, with a focus on evaluation and benchmarking. Proficiency with foundation model APIs and experience orchestrating complex agent behaviors via prompts or tools. Experience designing and running high-throughput evaluation pipelines, ideally including human-in-the-loop or expert-labeled benchmarks. Superlative Python engineering skills and familiarity with experiment management tools and data engineering toolsets in general including, yes, SQL and database management. Familiarity with clinical or healthcare data is a strong plus. Experience with reinforcement fine-tuning, model monitoring, or RLHF is a plus. Research shows that women and other minority groups might avoid applying if they don’t meet 100% of the qualifications. We encourage you to apply even if you don’t meet everything listed in the job posting. Canvas Medical provides equal employment opportunities to all employees and applicants for employment without regard to race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Job Requirements

  • You have extensive hands-on experience evaluating LLM-based systems, including multi-agent architectures and prompt-based pipelines.
  • You are deeply familiar with foundation model APIs (OpenAI, Claude, Gemini, etc.) and how to systematically benchmark agent performance using those models in applied settings.
  • You care about correctness and reproducibility and have built or contributed to frameworks for automated evals, annotation pipelines, and experiment tracking.
  • You bring structure to ambiguity and know how to define “correctness” in complex, nuanced domains.
  • You are comfortable collaborating across engineering, product, and clinical subject matter experts.
  • You are not afraid of complexity and are energized by the rigor required in healthcare deployments.

Related Job Pages

More Software Engineer Jobs

OtherRemoteTeam 33Since 2019

Unqork Configurator Location: Remote (LATAM preferred) Employment Type: Full-time / Contract About the Role We’re looking for an Unqork Configurator to help us design, build, and optimize applications on the Unqork platform. You’ll translate business needs into scalable, no-code solutions, ensuring functionality, usability, and performance align with project goals. This role blends technical configuration, problem-solving, and collaboration with stakeholders. What You’ll Do Configure applications on the Unqork platform using its no-code components. Work closely with business analysts, product owners, and engineers to understand requirements. Translate requirements into workflows, integrations, and UI/UX within Unqork. Create and maintain application modules, forms, rules, and data models. Configure APIs and integrations between Unqork and external systems. Perform testing, debugging, and quality assurance to ensure smooth deployments. Document configurations, workflows, and best practices. Support go-live activities and post-launch troubleshooting. What We’re Looking For Hands-on experience with the Unqork platform (configuration, workflows, data modeling, APIs). Strong understanding of no-code/low-code development concepts . Experience in customer experience, fintech, insurance, healthcare, or enterprise applications is a plus. Ability to work collaboratively in an agile environment with cross-functional teams. Strong analytical and problem-solving skills. Excellent communication skills in English (written and verbal). Nice to Have Unqork certification(s). Familiarity with REST APIs, JSON, and integration concepts. Previous experience with CX, BPM, or workflow-driven applications. Knowledge of data security and compliance in regulated industries. About Builders: Builders leverages highly skilled distributed engineering teams with Latin America’s top software talent. Connecting top technology startups, mid-size companies and large enterprises across North America with superior tech talent needed to innovate, compete and excel in their field of business.

United States
Job Closed
TetraScience logo

Software Engineer III - Data Applications

TetraScience

TetraScience is a cloud-native technology company that develops software and hardware solutions for monitoring and managing research experiments, as well as clo

Software Engineer103 days ago

Who We Are TetraScience is the Scientific Data and AI company. We are catalyzing the Scientific AI revolution by designing and industrializing AI-native scientific data sets, which we bring to life in a growing suite of next gen lab data management solutions, scientific use cases, and AI-enabled outcomes. TetraScience is the category leader in this vital new market, generating more revenue than all other companies in the aggregate. In the last year alone, the world’s dominant players in compute, cloud, data, and AI infrastructure have converged on TetraScience as the de facto standard, entering into co-innovation and go-to-market partnerships: Latest News and Announcements | TetraScience Newsroom In connection with your candidacy, you will be asked to carefully review the Tetra Way letter, authored directly by Patrick Grady, our co-founder and CEO. This letter is designed to assist you in better understanding whether TetraScience is the right fit for you from a values and ethos perspective. It is impossible to overstate the importance of this document and you are encouraged to take it literally and reflect on whether you are aligned with our unique approach to company and team building. If you join us, you will be expected to embody its contents each day. What You Will Do Be a member of the Tetra engineering team building infrastructure to support scientific analysis software Self-start and make concrete progress in the face of ambiguity or conflicting requirements Address the resiliency, scale, and high availability requirements of these tools Deliver a high-quality product following the agile software development methodology Partner with the product management team to take the vision and ideas and turn them into reality Be comfortable working with a geographically dispersed team, in various time zones Learn, grow, and be challenged. You will speak up and represent your position amongst peers and leadership while remaining resilient and open to constructive feedback. Requirements What You Have Done 5+ Years of full stack development experience Proficient with Node.js, Typescript, and associated technologies, OR Python and associated technologies Proficient with Databases and SQL Proficient with cloud infrastructure providers like AWS, Azure, or GCP Familiar with container technologies like Docker Experience writing maintainable unit tests, and automated integration tests Good application debugging skills Strong communication skills, including technical writing Bachelors or Masters degree in Computer Science, or in a relevant scientific field Familiarity with distributed systems for large-scale data processing is a plus Familiarity with Streamlit, Plotly Dash, etc for data visualization is a plus Experience in Life Sciences or scientific data is a big plus! Benefits 100% employer-paid benefits for all eligible employees and immediate family members Unlimited paid time off (PTO) 401K Flexible working arrangements - Remote work Company paid Life Insurance, LTD/STD A culture of continuous improvement where you can grow your career and get coaching No visa sponsorship is available for this position #LIRemote

United States
Job Closed

Software Development Engineer II (Lifecycle Engineering)

iRhythm Technologies

Founded in 2006, iRhythm Technologies fuses healthcare with technology to positively impact people's lives. The company's platform, ZIO by iRhythm, enables phys

Software Engineer103 days ago

Career-defining. Life-changing. At iRhythm, you’ll have the opportunity to grow your skills and your career while impacting the lives of people around the world. iRhythm is shaping a future where everyone, everywhere can access the best possible cardiac health solutions. Every day, we collaborate, create, and constantly reimagine what’s possible. We think big and move fast, driven by our commitment to put patients first and improve lives. We need builders like you. Curious and innovative problem solvers looking for the chance to meaningfully shape the future of cardiac health, our company, and your career About This Role: iRhythm’s Product Development team is seeking a highly skilled Software Development Engineer to join our Lifecycle (Sustaining) Engineering team. This role focuses on supporting existing products through their lifecycle, ensuring quality, compliance, and continuous improvement while maintaining business continuity. In this role, you will own and execute product tooling/fixturing software (C#) changes that keep our released medical device products safe, compliant, manufacturable, and reliable. You’ll work closely with our product development team of embedded firmware and C# software engineers and cross-functionally with hardware, quality, and operations to resolve field and factory issues, manage obsolescence, improve yield, and maintain regulatory compliance. This opportunity offers substantial career growth potential while positively impacting lives through design and technology. Responsibilities Design and develop C# software updates to manufacturing test systems (fixture software, functional test scripts, ICT/boundary scan integration) and drive improvements to yield, throughput, and reliability. Own software changes across released manufacturing test fixtures including bug fixes, minor feature enhancements, refactors, and obsolescence-driven updates working within established design controls and change management processes. Define and enforce cybersecurity best practices throughout the manufacturing software development lifecycle (SDLC), including code reviews, and vulnerability assessments. Familiarity with NIST SP 800-series security controls for software design, including the implementation of Multi-Factor Authentication (MFA), identity management, and cryptographic best practices. Contribute to secure product maintenance by assessing and implementing vulnerability patches, managing third‑party dependencies current in line with evolving guidance. Maintain configuration management (requirements, code, test assets, DHF/DMR updates, SBOMs), ensuring complete documentation and traceability through release. Minimum Qualifications BS in Computer Engineering, Computer Science, or related field 2+ years’ experience in software development (C#) . Strong programming skills in C# or a similar modern object-oriented programming language. Strong understanding of FDA QSR, ISO 13485, IEC 60601, ISO 14971 and IEC 62304 standards. Familiarity with software QA processes in regulated environments (e.g., medical device, FAA, SEC, DoD, Title 21 CFR Part 11) is preferred. Experience with Python or other scripting languages for automation or integration tasks is a plus. Cybersecurity for medical devices experience (SBOM management, vulnerability assessment, third party component governance) is a plus. Location: On-site (Cypress, CA – Orange County) Location: Orange County Actual compensation may vary depending on job-related factors including knowledge, skills, experience, and work location. Estimated Pay Range $88,000.00 - $110,000.00 As a part of our core values, we ensure an inclusive workforce. We welcome and celebrate people of all backgrounds, experiences, skills, and perspectives. iRhythm Technologies, Inc. is an Equal Opportunity Employer. We will consider for employment all qualified applicants with arrest and conviction records in accordance with all applicable laws. iRhythm provides reasonable accommodations for qualified individuals with disabilities in job application procedures, including those who may have any difficulty using our online system. If you need such an accommodation, you may contact us at taops@irhythmtech.com About iRhythm Technologies iRhythm is a leading digital healthcare company that creates trusted solutions that detect, predict, and prevent disease. Combining wearable biosensors and cloud-based data analytics with powerful proprietary algorithms, iRhythm distills data from millions of heartbeats into clinically actionable information. Through a relentless focus on patient care, iRhythm’s vision is to deliver better data, better insights, and better health for all. Make iRhythm your path forward. Zio, the heart monitor that changed the game. There have been instances where individuals not associated with iRhythm have impersonated iRhythm employees pretending to be involved in the iRhythm recruiting process, or created postings for positions that do not exist. Please note that all open positions will always be shown here on the iRhythm Careers page, and all communications regarding the application, interview and hiring process will come from a @irhythmtech.com email address. Please check any communications to be sure they come directly from @irhythmtech.com email address. If you believe you have been the victim of an imposter or want to confirm that the person you are communicating with is legitimate, please contact taops@irhythmtech.com. Written offers of employment will be extended in a formal offer letter from an @irhythmtech.com email address ONLY . For more information, see https://www.ftc.gov/business-guidance/blog/2023/01/taking-ploy-out-employment-scams and https://www.ic3.gov/Media/Y2020/PSA200121

California + 1 moreAll locations: California | Canada
Job Closed
OtherRemoteTeam 1-10H1B No Sponsor

Join Afero and change the world, a gazillion connections at a time! Our vision is to make all the world's devices really smart and truly secure, through innovation and scale. Come make it happen. Afero is the leading PaaS for IoT manufacturers and brands, enabling connected devices to be both smart and secure. Afero built its foundation on security, data privacy, simplicity and ease of use. Today, Afero technology powers millions of devices across more than 1,000 SKUs from 50+ manufacturers, including well known brands like Defiant, Hampton Bay and EcoSmart. Afero has more than 123 patents, one of the largest IoT patent portfolios globally, and was named an 2024 IoT Breakthrough Winner for IoT Security Platform of the Year . Afero is driven by a team of world-class executives and veterans in mobile, hardware, security, consumer and cloud, leveraging experience from Google, Apple, Microsoft, Amazon, Netflix, Disney, Roku and Danger. Afero provides a fantastic opportunity to explore and to expand your expertise while driving product innovation into exciting new ecosystems. To learn more, visit www.afero.io . Why You’ll Want to Work With Us For a small team, Afero engineers collectively do a little bit of everything—from cloud applications and infrastructure to mobile development on multiple platforms, firmware on a wide range of devices, to board-level hardware design and implementation of secure wireless devices. Everyone here can follow their curiosity to broader horizons. If you enjoy deep dives into assorted technology stacks to understand end-to-end system workflows, can adapt past experiences to new solutions, and can communicate ideas clearly among diverse audiences, Afero is a fantastic opportunity to explore and to expand your expertise. What We’re Looking For At Afero, everyone on the team contributes a distinct perspective. We are looking for problem solvers with great skills and experience. Afero provides tight integration among mobile apps, cloud and hardware for our systems. As we scale and grow, we are looking for engineers with passion to build best in class software. Key Responsibilities Design, develop, and maintain cloud-based services and applications for IoT solutions. Collaborate with hardware and software engineers to integrate IoT devices with cloud platforms. Implement scalable and secure cloud architectures that handle large volumes of data and real-time processing. Optimize cloud services for performance, reliability, and cost-efficiency. Develop and maintain APIs for device communication and data exchange. Ensure data security and privacy by implementing best practices and compliance standards. Monitor, troubleshoot, and resolve issues related to cloud services and infrastructure. Mentor and guide junior engineers, fostering a culture of continuous learning and improvement. Stay updated with the latest trends and technologies in cloud computing and IoT. Some on call required Qualifications

California
$225K - $305K / year
Job Closed