LLM Inference Deployment Engineer

Location

United States

Posted

11 days ago

Salary

$180K - $240K / year

Seniority

Senior

Job Description

LLM Inference Deployment Engineer

EnCharge AI

• Deploy and optimize LLMs (GPT, LLaMA, Mistral, Falcon, etc.) post-training from libraries like HuggingFace • Utilize inference runtimes such as ONNX Runtime, vLLM for efficient execution. • Optimize batching, caching, and tensor parallelism to improve LLM scalability in real-time applications. • Develop and maintain high-performance inference pipelines using Docker, Kubernetes, and other inference servers.

Job Requirements

  • Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or related field.
  • Experience in LLM inference deployment, model optimization, and runtime engineering.
  • Strong expertise in LLM inference frameworks (PyTorch, ONNX Runtime, vLLM, TensorRT-LLM, DeepSpeed).
  • In-depth knowledge of the Python programming language for model integration and performance tuning.
  • Strong understanding of high-level model representations and experience implementing framework-level optimizations for Generative AI use cases
  • Experience with containerized AI deployments (Docker, Kubernetes, Triton Inference Server, TensorFlow Serving, TorchServe).
  • Strong knowledge of LLM memory optimization strategies for long-context applications.
  • Experience with real-time LLM applications (chatbots, code generation, retrieval-augmented generation).

Related Categories

Related Job Pages

More DevOps Engineer Jobs

Full TimeRemoteTeam 501-1,000H1B Sponsor

• Collaborate with and support our creative, tight-knit development team • Design, deploy, and operate Loadsmart's critical systems while balancing reliability, cost, and agility • Play a key role in driving reliability projects with engineering teams • Utilize your intuitive problem-solving skills and contagious positive attitude to tackle challenging and exciting issues, inspiring those around you • Collect metrics and understand their business impact, encouraging the team to do the same • Perform troubleshooting and root-cause analysis of system operation issues • Be accountable for the platform's Service Level Agreements and Objectives • Provide infrastructure support during off-hours as needed • Take ownership of software infrastructure projects • Seek, give, and receive constructive feedback through code and specification reviews.

Brazil
Truelogic Software logo

Senior DevOps Engineer – Financial

Truelogic Software

Premium boutique software development company that helps brands with big ideas to make a difference in people’s lives.

DevOps Engineer11 days ago
Full TimeRemoteTeam 501-1,000Since 2004H1B No Sponsor

• Design, build, and maintain cloud environments within Microsoft Azure using best practices for scalability, reliability, and cost efficiency. • Implement and manage IaC using Terraform to automate resource provisioning and environment configuration. • Deploy and manage Azure resources including Azure Kubernetes Service (AKS), App Services, Function Apps, Azure API Management, Virtual Networks, Load Balancers, Storage Accounts, and Azure SQL Server / Azure SQL Databases. • Build and maintain CI/CD pipelines using Azure DevOps for application deployments, infrastructure, and automated testing. • Collaborate with development teams using C#, .NET, Visual Studio, and Angular to optimize the build, test, and release processes. • Integrate pipelines with container registries, package management, and automated approval workflows. • Develop container strategies using Docker and manage workloads running on Kubernetes (AKS). • Implement best practices for scaling, monitoring, logging, and securing Kubernetes clusters. • Monitor system performance, troubleshoot issues, and implement improvements to enhance reliability and uptime. • Manage Linux-based infrastructure, ensuring proper configuration, patching, and hardening. • Work with SQL Server databases to support deployments, migrations, and performance optimization. • Implement secure configurations for Azure resources and Kubernetes clusters while enforcing governance policies, identity and access controls, and environment standards. • Ensure compliance with organizational and industry security requirements.

Dominican Republic
Truelogic Software logo

Senior DevOps Engineer – Financial

Truelogic Software

Premium boutique software development company that helps brands with big ideas to make a difference in people’s lives.

DevOps Engineer11 days ago
Full TimeRemoteTeam 501-1,000Since 2004H1B No Sponsor

• Design, build, and maintain cloud environments within Microsoft Azure using best practices for scalability, reliability, and cost efficiency. • Implement and manage IaC using Terraform to automate resource provisioning and environment configuration. • Deploy and manage Azure resources including Azure Kubernetes Service (AKS), App Services, Function Apps, Azure API Management, Virtual Networks, Load Balancers, Storage Accounts, and Azure SQL Server / Azure SQL Databases. • Build and maintain CI/CD pipelines using Azure DevOps for application deployments, infrastructure, and automated testing. • Collaborate with development teams using C#, .NET, Visual Studio, and Angular to optimize the build, test, and release processes. • Integrate pipelines with container registries, package management, and automated approval workflows. • Develop container strategies using Docker and manage workloads running on Kubernetes (AKS). • Implement best practices for scaling, monitoring, logging, and securing Kubernetes clusters. • Monitor system performance, troubleshoot issues, and implement improvements to enhance reliability and uptime. • Manage Linux-based infrastructure, ensuring proper configuration, patching, and hardening. • Work with SQL Server databases to support deployments, migrations, and performance optimization. • Implement secure configurations for Azure resources and Kubernetes clusters while enforcing governance policies, identity and access controls, and environment standards. • Ensure compliance with organizational and industry security requirements.

Colombia

BigData DevOps Engineer

Experian

We're unlocking the power of data to help create a better tomorrow.

DevOps Engineer11 days ago
Full TimeRemoteTeam 10,001+Since 1996H1B Sponsor

Role Description As an important aide to both the IT Infrastructure and Development teams, you will help support existing systems 24x7 and be responsible for administering current Big Data environments. You will manage BigData Group environments and will work with teammates to maintain working solutions for our big data tech stack. To support product development following the product roadmap for product maintenance and enhancement such that the quality of software deliverables maintains excellent customer relationships and increases the customer base. You will report to the Engineering Manager, Data Platforms. If you have the skills, we would love to talk to you! What you'll do: - Implement, support, and administer Hadoop and AWS EMR environments. - Maintain Terraform and IaC for provisioning and managing AWS resources. - Provide CI/CD solutions. - Automate deployment and operations of Big Data technologies using DevOps tools. - Onboard Hadoop users and manage Kerberos, HDFS, Hive, HBase, and Yarn access. - Implement security best practices across HBase, HDFS, Kafka, Hive, and related components. - Tune Hadoop clusters, MapReduce, Spark workloads, and optimize EMR for performance and cost. - Monitor group health, logs, storage, and perform proactive capacity management. - Collaborate with infrastructure, network, DB, application, and BI teams to ensure platform reliability. - Support upgrades and patching for EMR, HBase, Spark, and other data platforms. - Troubleshoot system issues and analyze CPU, memory, OS, storage, and network performance. - Deploy and manage Hadoop clusters, configure HA, manage nodes, schedule jobs, and perform backups. - Support integrations across cloud/on-prem networks and data platforms. - Configure and maintain tools like Sentry, Spark, Kafka, Oozie, Solr, MongoDB, DocumentDB, and ELK. - Integrate AD/LDAP with Cloudera, manage Sentry policies. Qualifications - Typically requires a bachelor's degree (in Computer Science or related field) or equivalent. - 3+ years of Hadoop infrastructure administration – AWS EMR or Cloudera environments. - 2+ years of Linux (Redhat) system administration (Optional). - Cloud Platforms IaaS/PaaS – Cloud solutions: AWS, Azure, VMWare. - Gen AI / Agentic AI knowledge is a plus. - Automation skills – Ansible and Terraform. - Kerberos administration skills. - Experience coding languages SQL, Python. - Good experience in CI/CD tools such as Jenkins, Puppet, and Shell scripting. - Must have knowledge of DevOps tools. - Working knowledge of YARN, HBase, Hive, Spark, and Kafka. - Experience working with geographically distributed teams. - Bachelors or master's degree in Computer Science or equivalent experience. - Knowledge of our strategy and use of back-office applications. - To multi-lingual and multicultural environment, additional language skills are a bonus. - Receptive to change. - With our users at all levels. Benefits - Medical, life and dental insurance. - Asociacion Solidarista. - International Share Save Plan. - Flex Work/Work from home. - Paid time off. - Annual Performance Bonus. - Education Reimbursement. - Family Bonding. - Bereavement Leave. - Referral Program. - And more. This is a fully remote job opportunity. #LI-Remote Employee Status: Regular Role Type: Hybrid Department: Information Technology & Systems Schedule: Full Time

Worldwide