Building foundational AI for speech transcription and understanding.
Senior Technical Program Manager – AI Tooling & Systems
Location
United States
Posted
16 hours ago
Salary
$152K - $190K / year
Seniority
Senior
Job Description
Senior Technical Program Manager – AI Tooling & Systems
Deepgram
• Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring • Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms) • Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines • Drive cost and latency optimization for real-time inference workloads at scale • Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure) • Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows • Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems
Job Requirements
- 5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent)
- Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer
- Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring)
- Proven ability to translate ML/research requirements into robust, scalable infrastructure
- Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost)
- Excellent communication with both technical and non-technical stakeholders
- Familiarity with high-growth or startup environments
- It Would Be Great If You Had
- Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar)
- Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies)
- Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar)
- Background in feature stores, vector databases, or real-time ML systems
- Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure
- Experience with multi-region model serving or edge deployment
- Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML)
Benefits
- Offers Equity
- Offers Bonus
- 10% Annual Bonus
Related Guides
Related Categories
Related Job Pages
More Engineering Manager Jobs
Assistant- Associate Director of Development, College of Engineering
Penn State UniversityPenn State is a leading state university with 24 campuses throughout the state of Pennsylvania. The institution offers its student body a broad range of academi
Title: Assistant/Associate Director of Development, College of Engineering Location: United States Job Description: Full time job requisition id REQ_0000075772 APPLICATION INSTRUCTIONS: - CURRENT PENN STATE EMPLOYEE (faculty, staff, technical service, or student), please login to Workday to complete theinternal application process. Please do not apply here, apply internally through Workday. - CURRENT PENN STATE STUDENT (not employed previously at the university) and seeking employment with Penn State, please login to Workday to complete the student application process. Please do not apply here, apply internally through Workday. - If you are NOT a current employee or student, please click “Apply” and complete the application process for external applicants. Approval of remote and hybrid work is not guaranteed regardless of work location. For additional information on remote work at Penn State, see Notice to Out of State Applicants. POSITION SPECIFICS Penn State is currently seeking a frontline fundraiser in the role of Assistant/Associate Director of Development to engage alumni and friends as partners in supporting the College of Engineering, one of the nation’s leading educators of world-class engineers and leading producers of world-class research. As a member of the Division of Development and Alumni Relations (DDAR), you can play an integral role in Penn State’s future and contribute to one of the most successful fundraising and alumni relations operations in the country. Reporting to the Senior Director of Major Gifts in the College of Engineering and working closely with the college’s leadership, as well as other partners and central development offices, this Assistant/Associate Director of Development will: - Identify and cultivate a prospect pool of graduates and other key constituencies; - Articulate Penn State’s needs, ambitions, and values through excellent informal and formal communications, including written proposals; - Plan and execute solicitation strategies leading to major gifts; - Travel extensively throughout the region and across the country to pursue these strategies and build enduring relationships, tracked to a fixed and appropriately aggressive set of fundraising metrics. The successful candidate will have: - A track record of success in securing major gifts and meeting fundraising goals; - Exemplary interpersonal and communication skills required to build relationships with sophisticated and high-net worth audiences; - The ability to create and maintain an inclusive and respectful environment for all students and colleagues; - A proven ability to self-motivate and work both independently and as part of a team; - A commitment to professional development, learning, and being mentored. DDAR is supportive of flexible work arrangements when aligned with the ability to meet the needs of the unit and the essential duties of the position. Questions related to flexible work arrangements should be directed to the hiring manager during the interview process. MINIMUM EDUCATION, WORK EXPERIENCE & REQUIRED CERTIFICATIONS If filled as Major Gift Officer - Intermediate Professional, this position requires: Bachelor's Degree 1+ years of relevant experience; or an equivalent combination of education and experience accepted Required Certifications: None If filled as Major Gift Officer - Professional, this position requires: Bachelor's Degree No prior relevant work experience required; previous relevant work experience accepted in lieu of education. Required Certifications: None The College of Engineering Philanthropy will have a special impact in the College of Engineering, whose more than 100,000 living alumni include leaders, innovators, and entrepreneurs in every field of engineering. Its undergraduate program is the second-largest in the country, and, along with the Engineering graduate program, it is ranked among the top twenty-five of its kind nationwide by U.S. News and World Report. Across twelve departments and schools and more than thirty research centers, the College of Engineering’s world-class faculty is in the top ten for number of grants and publications, and it is committed to interdisciplinary research and hands-on education that prepares students for success in a rapidly changing world: engr.psu.edu Penn State’s Division of Development and Alumni Relations If you believe in the power of higher education—and philanthropy—to shape the public good, you’ll excel right here in Penn State’s Division of Development and Alumni Relations (DDAR). Our organization includes more than 500 professionals engaging a community of more than 700,000 alumni who believe in the power of giving back. Across our interdisciplinary teams of fundraisers, alumni relations professionals, communicators, event planners, financial experts, and more, there is a place for you to make a difference in the lives of students and faculty while taking your own career to new heights. Learn more about us at raise.psu.edu and explore the success of our most recent campaign at greaterpennstate.psu.edu. Building a Career and a Life at Penn State Across all campuses and an online World Campus, our 100,000 students and 17,000 faculty and staff know the real measure of success goes beyond the classroom—it’s the positive impact made on communities across the globe. This position is based at the University Park campus, located in State College, Pennsylvania. State College is ranked among the lowest-stress and safest small cities in the country, with excellent public schools, beautiful parks and other natural assets, and a broad range of cultural and athletic events and venues: statecollegepa.us and statecollege.com BACKGROUND CHECKS/CLEARANCES Employment with the University will require successful completion of background check(s) in accordance with University policies. This position requires that you operate a motor vehicle as a part of your job duties. A valid driver’s license and successful completion of a motor vehicle records check will be required in addition to standard background checks. Penn State does not sponsor or take over sponsorship of a staff employment Visa. Applicants must be authorized to work in the U.S. SALARY & BENEFITS The salary range for this position, including all possible grades, is $59,580.00 - $108,444.00. Salary Structure - Information on Penn State's salary structure Penn State provides a competitive benefits package for full-time employees designed to support both personal and professional well-being. In addition to comprehensive medical, dental, and vision coverage, employees enjoy robust retirement plans and substantial paid time off which includes holidays, vacation and sick time. One of the standout benefits is the generous 75% tuition discount, available to employees as well as eligible spouses and children. For more detailed information, please visit our Benefits Page. CAMPUS SECURITY CRIME STATISTICS Pursuant to the Jeanne Clery Disclosure of Campus Security Policy and Campus Crime Statistics Act and the Pennsylvania Act of 1988, Penn State publishes a combined Annual Security and Annual Fire Safety Report (ASR). The ASR includes crime statistics and institutional policies concerning campus security, such as those concerning alcohol and drug use, crime prevention, the reporting of crimes, sexual assault, and other matters. The ASR is available for review here. EEO IS THE LAW Penn State is an equal opportunity employer and is committed to providing employment opportunities to all qualified applicants without regard to race, color, religion, age, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. If you are unable to use our online application process due to an impairment or disability. Penn State is committed to and accountable for advancing equity, respect, and belonging. We embrace individual uniqueness, as well as a culture of belonging that supports equity initiatives, leverages the educational and institutional benefits of inclusion in society, and provides opportunities for engagement intended to help all members of the community thrive. We value belonging as a core strength and an essential element of the university’s teaching, research, and service mission. Federal Contractors Labor Law Poster PA State Labor Law Poster Penn State Policies Copyright Information Hotlines
• Lead a team of 6–8 engineers across the SSH client, web application, and backend services — running 1:1s, performance management, mentorship, career growth, and day-to-day delivery. • Own technical architecture across the stack you're responsible for — frontend systems, terminal/SSH client patterns, backend service design, API contracts, performance, and the long-term shape of the codebase. • Drive engineering process. Define how the team plans, ships, and learns. Introduce lightweight rituals where they help (sprint cadence, code review, on-call, incident response) and resist process that slows the team down. • Partner with Product on roadmap, tech debt prioritization, and translating customer needs into engineering work. Help sequence what the team builds and make the tradeoffs explicit. • Stay technical. Review architecture proposals, unblock engineers, challenge technical tradeoffs, step in during incidents, and contribute code when it's the highest-leverage thing you can do (~20% of your time, give or take). • Hire and build the team. Source, interview, sell, and onboard engineers. Help define rubrics and shape the culture as we scale. • Communicate across the org. Translate technical risk and tradeoffs to founders, product, design, and customer-facing teams. Operate autonomously in a fast-moving environment.
• Prendre la responsabilité de l'équipe APIs & Performance • Pousser l'équipe vers l'automatisation et l'utilisation de l'IA • Installer une culture d'information proactive
Head of AI Engineering – Enablement
TebraWe empower independent practices to bring modernized care to patients everywhere.
• Lead AI strategy and use case discovery for organizational efficiency • Perform deep-dive assessments and build high-impact workflows • Own the full lifecycle from rapid prototyping to production-grade deployment • Define approved tools and data-handling rules for AI • Partner with functions to identify high-value use cases




