Stack AV logo
Stack AV

Revolutionizing the Transportation of Goods

Senior Software Engineer, Machine Learning Inference Platform

Machine Learning EngineerMachine Learning EngineerFull TimeRemoteSeniorTeam 51-200H1B No SponsorCompany SiteLinkedIn

Location

Pennsylvania

Posted

3 days ago

Salary

0

Seniority

Senior

Bachelor Degree4 yrs expEnglishDistributed SystemsgRPCPythonPyTorchRustGo

Job Description

Senior Software Engineer, Machine Learning Inference Platform

Stack AV

• Own technical design and delivery of subsystems in a high-throughput, low-latency inference platform capable of handling multi-tenant, enterprise-grade inference workloads. • Develop robust API layers (gRPC, WebSockets, REST, etc.) and developer SDKs that abstract complex distributed inference orchestration into seamless, reliable token streams. • Build and harden a multi-tenant control plane to enable accurate metering, rate limiting, quotas, tenant isolation and noisy-neighbor fairness across the platform. • Optimize inference performance across the entire system stack, including the model engine layer. • Build observability and SLOs to gain insights into system economics, cache-hit rates, GPU utilization and cost accounting per model and per tenant. • Partner with product and infrastructure teams on model onboarding, capacity planning, external API contracts and customer adoption. • Decompose ambiguous work, drive issues to closure, and raise the engineering bar through code quality, reviews, testing, and mentoring.

Job Requirements

  • Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  • Experience: 4+ years of experience building and operating backend distributed systems end to end.
  • Strong Data & ML systems fundamentals: data-intensive distributed systems, concurrency, networking and performance profiling.
  • Hands-on experience with large-scale inference services on GPUs, including KV caches, prefill/decode stages and throughput/latency trade-offs.
  • Direct experience with inference engines (TensorRT, vLLM, etc) or serving frameworks (Dynamo, Triton or equivalent).
  • Technical Skills:
  • Strong programming skills in C++, Go, Rust or Python.
  • Familiarity with deep learning frameworks (PyTorch, etc.) as well as model parallelism.
  • Familiarity with GPU computing primitives such as CUDA, NCCL, NVLink, and hardware-specific optimizations.
  • Practical understanding of high-performance networking architectures, including InfiniBand, RoCE, and low-latency cluster communication.
  • Problem-Solving: Strong analytical and problem-solving skills.
  • Autonomous vehicles (AV) experience is a bonus.

Benefits

  • Health insurance
  • Professional development opportunities

Related Job Pages

More Machine Learning Engineer Jobs

Docker, Inc logo

ML Engineer

Docker, Inc

Docker helps developers bring their ideas to life by conquering the complexity of app development.

Full TimeRemoteTeam 51-200H1B No Sponsor

• Design, train, evaluate, and ship ML systems that power governance and security capabilities, starting with problems like prompt injection detection, behavioral anomaly detection, trust scoring, and policy recommendations. • Build the supporting infrastructure: data pipelines, feature stores, model serving, evaluation harnesses, and the feedback loops that make iteration fast. • Make pragmatic build-vs-buy calls. Use frontier models, off-the-shelf tooling, and managed services to move quickly; invest in custom systems where they create durable advantage. • Set technical direction for the team's ML work. Own the architecture, evaluation methodology, model lifecycle, and the bar for shipping. • Help recruit, mentor, and shape the team as it grows. • This role may require participation in a 24/7 on-call rotation for the Agentic Platform; carry genuine pager responsibility for the services you build and operate

California
$138.5K - $225.5K / year
Greenlight Planet logo

Machine Learning Engineer

Greenlight Planet

Powering access to brighter lives in Africa, Asia, and beyond

Full TimeRemoteTeam 1,001-5,000Since 2009H1B No Sponsor

• Research, design, train, and productionize deep learning models for speech/audio (ASR, speaker diarization, audio classification, TTS, noise suppression) and computer vision (detection, segmentation, classification, video understanding) use cases. • Architect training pipelines capable of handling large-scale datasets, manage data preprocessing, augmentation, and versioning workflows. • Select and adapt state-of-the-art architectures (Transformers, CNNs, RNNs, diffusion models, etc.) and fine-tune or distill pre-trained models for production constraints. • Optimize models for inference - quantization, pruning, knowledge distillation - targeting latency, throughput, and memory budgets on CPU/GPU/edge hardware. • Work with software engineers to integrate models into production systems, design APIs and microservices for model serving. • Define and track evaluation benchmarks, monitor model performance in production, and drive continuous improvement cycles. • Stay current with research literature, evaluate and implement relevant SOTA techniques, contribute to internal technical forums.

India
Google logo

Senior Security Engineer, Artificial Intelligence - Machine Learning

Google

Since its founding in 1998, Google has grown well beyond the search engine launched by Larry Page and Sergey Brin in a university dorm room. It's now one of the

Title: Senior Security Engineer, AI/ML, National Security, Public Sector Location: Remote Maryland United States Job Description: In accordance with Washington state law, we are highlighting our comprehensive benefits package, which is available to all eligible US based employees. Benefits for this role include: - Health, dental, vision, life, disability insurance - Retirement Benefits: 401(k) with company match - Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment - Sick Time: 40 hours/year (increased to 69 hours/year for Seattle) including 5 discretionary sick days per instance - Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks - Baby Bonding Leave: 18 weeks - Holidays: 13 paid days per year Note: Google's hybrid workplace includes remote and in-office roles. By applying to this position you will have an opportunity to share your preferred working location from the following: Remote location(s): Maryland, USA. Minimum qualifications: - Bachelor's degree in Computer Science, Data Science, Artificial Intelligence, or a related technical field or equivalent practical experience. - 5 years of experience in AI/ML development, AI infrastructure engineering, or software development. - 5 years of experience with containerization (Docker) and orchestration (Kubernetes). - 5 years of experience with Python and with libraries like PyTorch, TensorFlow, or Hugging Face Transformers. - Ability to travel up to 25% of the time as needed. - Must possess an active Top Secret/SCI security clearance with current polygraph. Preferred qualifications: - 5 years of experience in AI/ML research or software development. - Experience with LLM deployment frameworks such as vLLM, NVIDIA Triton, or Ollama and agent development. - Knowledge of open worldwide application security project (OWASP) for LLMs or similar security frameworks. - Familiarity with cloud-native AI services (e.g., cloud computing platform, Google Vertex AI). - Track record of deploying AI models on air-gapped or on-premises high-performance computing (HPC) systems. About the job Our Security team works to create and maintain the safest operating environment for Google's users and developers. Security Engineers work with network equipment and actively monitor our systems for attacks and intrusions. In this role, you will also work with software engineers to proactively identify and fix security flaws and vulnerabilities. In this role, you will help us build the most resilient AI infrastructure in the world. This role is designed for a technical expert in Artificial Intelligence and Machine Learning, with a primary interest in how those systems can be defended against adversarial manipulation. You will be responsible for the security configuration of AI deployments, from local on-prem GPU clusters to cloud-native environments. You will understand the nuances of LLMs, neural networks, and containerized ML pipelines, and will apply that knowledge to the frontier of security. You will have an understanding of how Large Language Models (LLMs) work under the hood and to develop the next generation of automated defenses and adversarial testing frameworks. Google Public Sector brings the magic of Google to the mission of government and education with solutions purpose-built for enterprises. We focus on helping United States public sector institutions accelerate their digital transformations, and we continue to make significant investments and grow our team to meet the complex needs of local, state and federal government and educational institutions.Individual pay is determined by factors including job-related skills, experience, and relevant education or training. US: $174000 - $253000 (USD) + 15% bonus target + bonus + equity + benefits Learn more about benefits at Google. Responsibilities - Architect and manage LLM deployments across on-premises (NVIDIA/AMD) and cloud (cloud computing platform, Google Cloud platform (GCP) environments. Audit multi-agent orchestration, agent construction, and vector databases to map data flows and enforce privilege boundaries. - Use Docker and Kubernetes to orchestrate scalable inference and training environments, optimizing Graphics Processing Unit (GPU) utilization and resource isolation. - Protect model weights, secure data ingestion, and harden inference endpoints across the Machine Learning operations (MLOps) lifecycle. - Investigate and mitigate AI-specific threats (e.g., prompt injection, jailbreaking, data poisoning). Map testing findings to MITRE ATLAS, OWASP for LLMs, and STRIDE models. - Bridge local high-compute clusters and cloud AI services while maintaining a consistent security posture.

Maryland
$174K - $253K / year

Director, Artificial Intelligence - Machine Learning Applied Science

UnitedHealth Group

UnitedHealth Group is a healthcare and well-being company that’s dedicated to improving the health outcomes of millions around the world. We are comprised of

Title: Director, AI/ML Applied Science - Remote Location: Minnetonka United States Job Description: Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. Optum AI is UnitedHealth Group's enterprise AI team. We are AI/ML scientists and engineers with deep expertise in AI/ML engineering for health care. We develop AI/ML solutions for the highest impact opportunities across UnitedHealth Group businesses including UnitedHealthcare, Optum Financial, Optum Health, Optum Insight, and Optum Rx. In addition to transforming the health care journey through responsible AI/ML innovation, our charter also includes developing and supporting an enterprise AI/ML development platform. Optum AI team members: - Have impact at scale: We have the data and resources to make an impact at scale. When our solutions are deployed, they have the potential to make health care system work better for everyone. - Do ground-breaking work: Many of our current projects involve cutting edge ML, NLP and LLM techniques. Generative AI methods for working with structured and unstructured health care data are continuously being developed and improved. We are working in one of the most important frontiers of AI/ML research and development. - Partner with world-class experts on innovative solutions: Our team members are developing novel AI/ML solutions to business challenges. In some cases, this includes the opportunity to file patents and publish papers about the methods we develop. We also collaborate with AI/ML researchers at some of the world's top universities. You'll enjoy the flexibility to work remotely * from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week. Primary Responsibilities: - Develop novel machine learning models for prediction of health outcomes and improved care, engagement models, and other applications from both structured and unstructured data - Develop and execute the vision for engineering platforms, and reusable code that will be used by thousands of other engineers for automation and improved efficiency of AI/ML development - Stay up to date with latest technologies and literature recommending solutions to improve business goals - Mentor data scientists, AI/ML scientists, and engineers to ensure the delivery of the AI/ML projects, and provide guidance on how to best use specific tools or technologies to achieve the desired results - Collaborate with cross-functional teams to determine applicability of AI to business problems - Closely collaborate with the Responsible Use of AI (RUAI) team to ensure that the delivered solutions are compliant with the company policies and standards - Write both internal and external documentation of the novel algorithms, including patents and publications in machine learning and other scientific conferences and journals as well and/or infrastructure design and deployment procedures - Evangelize the benefits of the AI models and platforms built in our team to drive adoption for employees and functional roles across the UHG enterprise including technology, operations, marketing, legal, compliance, people team, internal infrastructure, and more You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in. Required Qualifications: - Masters degree in Computer Science, Math, Statistics, or a related field and 10+ years experience OR a PhD in Computer Science or a related field and 5+ years experience - 3+ years experience focused on AI/ML/NLP solutions delivery - 5+ years of experience experience leading and implementing AI/ML projects from ideation to delivery, evaluation, and maintenance in production - Hands-on experience implementing AI/ML and/or NLP solutions - Proven experience in Python and AI/ML frameworks and tools (e.g., MLFlow, TensorFLow, PyTorch) - Experience developing and deploying data pipelines, machine learning models, or applications on cloud platforms (e.g., Azure, AWS, Databricks, AzureML) - Experience with Gen AI solution pipelines (e.g., RAG) and Large Language Modeling and Transformer Architectures (e.g., BERT, GPT, etc.) - Proven presentation experience with the ability to explain complex AI/ML concept and results to technical and non-technical audiences - Proven understanding of mathematical foundations of machine learning, including statistics, linear algebra, and computer science Preferred Qualifications: - PhD in Computer Science, Mathematics, Statistics, or a related discipline - Experience in healthcare (AI) - Experience working with cross-functional and distributed teams in a global and diverse environment - Experience in establishing AI/ML best practices, standards, and ethics - Working knowledge of Software Development tools and practices including DevOps and CI/CD tools (e.g., Git, Jenkins, Docker, Kubernetes, etc.) - Security and vulnerability management (package scans, remediation) - Familiarity with data versioning tools (Delta Lake, DVC, LakeFS, etc.) - Experience with model observability tools for insights into the behavior, performance, and health of your deployed ML models (tracking, alerting, compliance monitoring, etc.) - All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $134,600 to $230,800 annually based on full-time employment. We comply with all minimum wage laws as applicable. Application Deadline: This will be posted for a minimum of 2 business days or until a sufficient candidate pool has been collected. Job posting may come down early due to volume of applicants. At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission. #optumaiscience UnitedHealth Group is an Equal Employment Opportunity employer under applicable law and qualified applicants will receive consideration for employment without regard to race, national origin, religion, age, color, sex, sexual orientation, gender identity, disability, or protected veteran status, or any other characteristic protected by local, state, or federal laws, rules, or regulations. UnitedHealth Group is a drug - free workplace. Candidates are required to pass a drug test before beginning employment.

Minnesota + 1 moreAll locations: Minnesota | District Of Columbia
$134.6K - $230.8K / year