Building foundational AI for speech transcription and understanding.
Embedded AI Engineer, On-Device Models
Location
United States
Posted
2 days ago
Salary
$219.3K - $274.1K / year
Seniority
Senior
Job Description
Embedded AI Engineer, On-Device Models
Deepgram
• Take Deepgram's Speech and Conversational models and get them running on embedded and low-power consumer hardware — defining the architecture for on-device, real-time inference across a diverse range of processors and accelerators. • Optimize models for constrained targets through quantization, pruning, distillation, operator fusion, and architecture-specific compilation to meet strict latency, memory, power, and thermal budgets. • Write and optimize performance-critical runtime code (C, C++, and/or Rust) for embedded environments, including bare-metal and real-time operating systems such as FreeRTOS and Zephyr. • Integrate with industry-standard edge inference runtimes and vendor NPU/DSP toolchains, mapping model graphs efficiently onto on-device accelerators and CPU/GPU/NPU heterogeneity. • Build the on-device runtime plumbing: model packaging, deployment pipelines, over-the-air update mechanisms, and lightweight telemetry for devices operating with limited or intermittent connectivity. • Establish repeatable benchmarking and validation across target hardware — measuring latency, accuracy, power consumption, memory footprint, and resource utilization — and catch regressions before they ship. • Partner with silicon and device vendors on SDK integration and performance tuning, getting our models to run efficiently on new chipsets and reference platforms. • Collaborate with Research and Engine teams to influence model architectures toward edge-friendly designs from the start, reducing the optimization burden at deployment time.
Job Requirements
- Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices.
- Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments.
- Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation.
- Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains.
- A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance.
- Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development.
- Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly.
Benefits
- Offers Equity
- Offers Bonus
- 10% Annual Bonus
Related Guides
Related Categories
Related Job Pages
More Systems Engineer Jobs
• redefine and ship large, foundational improvements to Mercury’s design language • partner with Brand and other craft leaders to bring expressive moments into the product • set the craft bar for the design org • actively coach product designers on their craft • partner with design systems designers and engineers • represent design systems and visual craft in conversations with Brand, Engineering, and design leadership
Senior Software Engineer, Core Systems
PanthalassaWe are a renewable energy and ocean technology company committed to rapidly developing and deploying technologies that will ensure a sustainable future for Earth by unlocking the vast energy potential of its oceans.
Role Description Our core technology is the Node, a device that produces energy in the ocean’s harshest conditions for years without human maintenance or intervention. As a Senior Software Engineer on the Core Systems team, you will contribute to the design, development, and optimization of the onboard data aggregation and telemetry systems. - Work with our Core team and partner with Shore-side software engineers to build reliable data pipelines, low-latency telemetry paths, and storage systems that function in constrained, unpredictable marine environments. - Collaborate closely with networking and security engineers to ensure onboard systems communicate reliably and remain free of critical vulnerabilities. - Shape how data is captured, moved, and understood onboard, enabling resilient autonomy and breakthrough operational insight for the rest of the company. - Enhance the quality and reliability of our onboard software by expanding unit testing frameworks, automated integration tests, continuous deployment pipelines, and code reviews and documentation. Qualifications - 7+ years of experience developing software. - 3+ years of experience with data-intensive or mission-critical systems. - Proficiency in a systems programming language (Rust, C++, or similar). - Proficiency with version control (we use git). - Competency with CI/CD pipelines for automated tests and deployment. - Experience building or maintaining network-facing software – handling connection lifecycles, protocol implementation, or communication reliability in production systems. - Understanding of fundamentals in computer networking (L2–L4), transport protocols, congestion control, and distributed systems. - Experience designing systems for constrained, high-latency, intermittent, and/or expensive links (e.g., satellite, maritime, remote ops). - Familiarity with security principles for networked systems (authentication/authorization, encryption, key management). - Ability to work in a fast-paced and self-starting atmosphere. - Ability to communicate thoughtfully and empathetically with people from many different educational and professional backgrounds. - Motivation and curiosity to learn a wide variety of concepts across many disciplines, not just software engineering! Requirements - Build and maintain the network-facing software layers that enable Node to communicate reliably with shore. - Design egress telemetry pipelines that use bandwidth efficiently, weighing tradeoffs between data fidelity, timeliness, and link constraints. - Collaborate on APIs and schemas for command, control, and telemetry exchanges between subsystems, ensuring reliability is built in from the start. - Partner with network engineers to ensure a cohesive strategy for how bandwidth is divided and prioritized among competing data streams. - Ensure that software degrades gracefully during connectivity loss or link degradation. - Work with network and security engineers to review onboard software for vulnerabilities and harden network-facing services. - Analyze system performance and implement optimizations for latency, throughput, and resource efficiency. - Extend and improve existing unit testing frameworks, integration tests, and CI/CD pipelines. - Strengthen code review, documentation, and general software engineering practices within the team. - Participate in design reviews, architecture discussions, and cross-team technical collaborations. Benefits - Cash compensation of $175,000–$230,000. - Equity in the company. - Flexible paid time off. - Health insurance (the company pays 100% of gold level PPO plan for full-time employees, their partners, and dependents). - Dental insurance (the company pays 33% for full-time employees and 100% for their partners and dependents). - Vision insurance (the company pays 100% for full-time employees, their partners, and dependents). - Disability insurance (the company pays 100% for a policy to provide long-term financial support if you become disabled). - Ability to contribute to tax-advantaged accounts, including 401(k), health FSA, and dependent care FSA. - Relocation assistance to facilitate your move to Portland (if needed). Company Description We are a renewable energy and ocean technology company committed to rapidly developing and deploying technologies that will ensure a sustainable future for Earth by unlocking the vast energy potential of its oceans.
• Lead the drafting, editing, and continuous improvement of general equipment documentation, specifically comprehensive Installation, Operation, and Maintenance (IOM) manuals for advanced thermal management infrastructure. • Develop, establish, and optimize structured documentation workflows, including the creation, review, and management of standardized engineering submittal packages for customer approval and contractor use. • Develop and refine precise Sequence of Operations (SOO) documents. • Analyze heat transfer and energy conversion components—including advanced heat exchanger designs, refrigeration systems, and heat pumps—to accurately describe their functionality in user-facing support documents. • Independently develop, review, and edit comprehensive project and customer reports. Maintain clear visibility over timelines, deliverables, and project milestones using collaborative project management platforms. • Work closely with the R&D and product development teams to extract technical specifications and utilize in-house documentation tools to support continuous product optimization.
• Serve as a primary analyst and subject matter expert for HR systems, with a strong focus on UKG as the primary HRIS and Greenhouse as the applicant tracking system • Own day-to-day HR systems administration, including employee changes, data maintenance, workflow support, troubleshooting, configuration updates, system permissions, routine maintenance, and testing within established governance • Support reporting, data integrity, and audit activities by building and validating recurring and ad hoc reports, maintaining deep UKG BI reporting knowledge, and conducting audits and reconciliations across HR systems and connected processes • Support file feeds, integrations, bulk uploads, and recurring system activities tied to annual compensation cycles and other people programs • Help administer and improve FreshService as the HR Operations ticketing system, while providing timely and thoughtful support to employees and internal stakeholders on system troubleshooting and process questions • Support administration and reporting for compliance and learning processes, including annual anti-harassment training in Docebo, while expanding systems support over time to additional platforms such as Quantum • Develop and maintain SOPs, process guides, troubleshooting steps, FAQs, and knowledge base content so processes are documented, scalable, and not dependent on one individual • Identify and implement process, system, and automation improvements that reduce manual work, improve the employee experience, and help the HR Operations team scale • Lead or support the team’s adoption of AI by identifying practical use cases, creating agents or automations, and embedding scalable solutions into team workflows • Partner closely with HR Operations Specialists and Managers, as well as cross-functional teams including IT, Payroll, Finance, Legal, HRBPs, Total Rewards, and Talent Acquisition, to resolve issues, improve workflows, and support scalable execution • Bring forward new ideas, manage competing priorities and projects with clear milestones, monitor system releases and best practices, and support ad hoc reporting, audits, and operational initiatives based on team priorities


