DigitalOcean logo
DigitalOcean

The cloud ☁️ of choice for developers, startups, and growing digital businesses around the world.

Senior Software Engineer I - AI Inference Data Plane

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 2011H1B SponsorCompany SiteLinkedIn

Location

United States

Posted

4 days ago

Salary

$139.2K - $174K / year

Seniority

Senior

Job Description

Senior Software Engineer I - AI Inference Data Plane

DigitalOcean

Role Description DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. We are seeking a Senior Engineer 2 to join our AI Inference Data Plane team. In this role, you will be a key technical leader responsible for designing, developing, and delivering high-scale, resilient data plane services that power our "Inference as a Service" offering. - Technical Leadership: Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models. - System Design: Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards. - Performance Optimization: Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing. - Collaboration: Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs. - Distributed Serving at Scale: Build on Kubernetes-native distributed inference frameworks like llm-d (or alternatives such as NVIDIA Dynamo, Ray Serve) to deliver prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models. - Flow Control & Load Balancing: Solve the distributed-systems problems unique to LLM serving — inference-aware load balancing on queue depth, cache locality, and predicted latency; flow control and fairness across tenants; autoscaling inference pools; and moving gigabytes of KV-cache between prefill and decode instances with negligible overhead. - Open Source Contributions: Contribute upstream to llm-d, vLLM, and the inference gateway ecosystem, and represent DigitalOcean in these communities. - Mentorship: Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement. - Operational Excellence: Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health. Qualifications - Hands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or TensorRT. - Familiarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray Serve. - Hands-on experience with vLLM or alternatives (SGLang, TensorRT-LLM, TGI, Modular MAX), including internals like continuous batching, paged attention, and prefix caching. - Understanding of why cluster-scale serving is hard: KV-cache locality is partitioned across workers, naive round-robin routing destroys cache hit rates and tail latency, and disaggregated prefill/decode requires fast cross-pod KV transfer (e.g., NIXL). - Merged contributions to vLLM, llm-d, SGLang, or similar projects strongly preferred. - Knowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization). - Expert-level proficiency in GoLang or Python and familiarity with gRPC. - Proven experience shipping customer-facing software products and running critical services in a high-scale environment similar to DigitalOcean. - Experience integrating and building with open-source software. Requirements - Compensation Range: $139,200 - $174,000 - This is a remote role. - JR: 2026-7624 - #LI-Remote Benefits - Competitive array of benefits to support employee well-being. - Reimbursement for relevant conferences, training, and education. - Access to LinkedIn Learning's 10,000+ courses for continued growth and development. - Equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.

Related Categories

Related Job Pages

More Data Engineer Jobs

Sparq logo

Senior Data Engineer

Sparq

In the age of AI, differentiation isn’t in what you build - it’s in the problems you choose to solve and the outcomes they unlock. That’s why we help leaders cut through the noise, focus on what matters most, and solve it right the first time. By fusing problem-first thinking, deep technical craftsmanship, and fast, flawless delivery, we de-risk transformation and deliver solutions that stick, scale, and prove their worth. With a deep bench of end-to-end technologists, architects, and engineers, no challenge is too complex and no solution is half-built. At Sparq, your mission is our mission. We’re modular by design - meeting you where you are and accelerating you toward where you’re meant to be. We don’t just guide - we climb with you. Embedded alongside your teams, we chart the course, build with precision, and navigate complexity until your outcomes are achieved. Built to solve, not just to build.

Data Engineer4 days ago

Role Description Why you will enjoy Mondays again: - Opportunity to work alongside a talented, diverse team in a collaborative and creative environment - Ongoing investment in your growth through training, mentorship, and industry certifications - Hands-on exposure to cutting-edge technologies across multiple industries - Recognition programs and employee awards within an inclusive culture that celebrates great work, including our AI Innovators Program - Remote or hybrid work options - Competitive salary + bonus opportunities - Robust benefits package, matching 401(k) plan, and substantial PTO - Tuition reimbursement A Day in the Life: - Build and migrate large-scale batch data pipelines from legacy data platforms into Databricks. - Develop and optimize production-grade solutions using Python, Spark, and advanced SQL. - Support parallel legacy and cloud data environments throughout the migration and validation process. - Serve as a subject matter expert while remaining hands-on with day-to-day development and troubleshooting. - Actively use AI tools, including Claude through our Anthropic partnership, to streamline tasks, improve the quality of your work, and share best practices with teammates while continuously seeking new ways to integrate AI into your everyday workflows. - Lead technical design conversations and guide implementation within an established architecture and project framework. - Collaborate with developers, architects, and business stakeholders to communicate technical decisions, risks, and progress. - Improve pipeline reliability and performance through testing, monitoring, tuning, and production support. - Use GitHub Copilot when helpful to support development productivity. - Participate in a rotating on-call schedule approximately every four to six weeks. Qualifications - Recent hands-on experience building production batch solutions in Databricks, ideally within the past 12–18 months. - Strong development experience with Python and Apache Spark. - Advanced SQL skills, including complex query development and performance tuning. - Consultative approach and problem-solving skills to successfully align digital solutions with long-term business goals of the client. - Experience delivering large-scale migrations from legacy data platforms to cloud-based environments, preferably involving Teradata or a comparable platform. - Experience supporting legacy and cloud systems operating in parallel during a migration. - The ability to lead design and technical-direction discussions while remaining actively involved in coding and implementation. - Hands-on experience using AI tools to enhance daily work, or a strong desire to do so, including a willingness to experiment, learn, and champion AI adoption within your role. - Strong communication skills across technical teams, architects, business partners, and leadership. - Availability to participate in an on-call rotation. - Experience with Azure data services—such as Data Factory, Synapse Analytics, Event Hubs, Delta Lake, Cosmos DB, and Azure DevOps—is preferred. - Exposure to Big Data technologies, NoSQL databases, Git, Jenkins, CI/CD practices, and Agile delivery methodologies is a plus. Equal Employment Opportunity Policy Sparq is proud to offer equal employment opportunity without regard to age, color, disability, gender, gender identity, genetic information, marital status, military status, national origin, race, religion, sexual orientation, veteran status, or any other legally protected characteristic. We are committed to providing equal employment opportunities and believe in an inclusive workplace. If you require reasonable accommodations to participate in the job application or interview process, please let us know by contacting recruiting@teamsparq.com . #LI-REMOTE

United States
v4c.ai logo

Data Engineer

v4c.ai

We Unify. We Elevate. We Foresee

Data Engineer4 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor

• Design, develop, and maintain large-scale data systems • Develop and implement ETL processes using various tools and technologies • Collaborate with cross-functional teams to design and implement data models • Work with big data tools like Hadoop, Spark, PySpark, and Kafka • Develop scalable and efficient data pipelines • Troubleshoot data-related issues and optimize data systems • Transition and upskill into Databricks & AI/ML projects

India
PwC logo

Data Engineer - Senior Associate

PwC

Build what’s next — with tech that matters PwC provides professional services across Audit and Assurance, Advisory and Tax — powered by a global network of over 370,000 people in 149 countries. You may know us for our business expertise, but technology is core to how we help clients move faster, build trust and deliver meaningful outcomes. As a technologist, you’ll work on agile teams with experienced engineers and product thinkers — using AI, cloud, cybersecurity and more to design scalable, real-world solutions. You’ll keep learning, stay challenged and be part of a network where your growth is built in — and your work drives what’s next.

Data Engineer4 days ago
Full TimeRemoteTeam 10,001+Since 1998H1B Sponsor

The Opportunity As a Data Engineer - Senior Associate, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions. As a Senior Associate, you will build meaningful client connections and learn how to manage and inspire others. You will navigate increasingly complex situations, grow your personal brand, and deepen your technical skills. You are expected to anticipate the needs of your teams and clients, and to deliver quality work. Embracing increased ambiguity, you will be comfortable when the path forward isn't clear, using these moments as opportunities to grow. In this role within our Technology Consulting practice, you will leverage advanced technologies and techniques to design and develop robust data solutions for clients. You will transform raw data into actionable insights, enabling informed decision-making and driving business growth. By using a broad range of tools, methodologies, and techniques, you will generate new ideas and solve problems, contributing to the overall strategy and objectives of your projects. This position offers a chance to develop a deeper understanding of the business context and how it is evolving. Responsibilities - Designing and implementing data infrastructure and systems to facilitate efficient data processing and analysis - Developing and maintaining data pipelines, integration, and transformation solutions to support client needs - Utilizing Amazon Web Services (AWS) and Azure Data Factory to enhance data engineering capabilities - Applying data architecture development and database management skills to optimize data solutions - Leveraging Apache Airflow and Apache Hadoop for scalable data processing and workflow management - Building and managing data lakes and warehouses to support large-scale data storage and retrieval - Confirming data quality and validation through rigorous testing and performance tuning - Collaborating with clients to understand their data requirements and deliver actionable insights - Utilizing Databricks Unified Data Analytics Platform for advanced data analytics and visualization - Implementing data security best practices to protect sensitive information and maintain compliance - Applying dimensional modeling and directed acyclic graphs (DAGs) for efficient data organization and processing - Supporting the development of data strategies to drive business growth and informed decision-making What You Must Have - At least a Bachelor's degree - At least 2 years of experience What Sets You Apart - Preference for at least one of the following fields of study: Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, Mathematical Statistics - Demonstrating proficiency in data engineering platforms like Databricks - Utilizing cloud platforms such as AWS and Microsoft Azure - Excelling in data architecture development and data modeling - Implementing data pipeline and data integration strategies - Navigating complex data environments with Apache Hadoop and Airflow - Applying critical thinking to solve data-related challenges The salary range for this position is: $77,000 - $202,000. Actual compensation within the range will be dependent upon the individual's skills, experience, qualifications and location, and applicable employment laws. All hired individuals are eligible for an annual discretionary bonus. PwC offers a wide range of benefits, including medical, dental, vision, 401k, holiday pay, vacation, personal and family sick leave, and more. To view our benefits at a glance, please visit the following link: https://pwc.to/benefits-at-a-glance As PwC is an equal opportunity employer, all qualified applicants will receive consideration for employment at PwC without regard to race; color; religion; national origin; sex (including pregnancy, sexual orientation, and gender identity); age; disability; genetic information (including family medical history); veteran, marital, or citizenship status; or, any other status protected by law. PwC does not intend to hire experienced or entry level job seekers who will need, now or in the future, PwC sponsorship through the H-1B lottery, except as set forth within the following policy: https://pwc.to/H-1B-Lottery-Policy. Learn more about how we work: https://pwc.to/how-we-work For only those qualified applicants that are impacted by the Los Angeles County Fair Chance Ordinance for Employers, the Los Angeles' Fair Chance Initiative for Hiring Ordinance, the San Francisco Fair Chance Ordinance, San Diego County Fair Chance Ordinance, and the California Fair Chance Act, where applicable, arrest or conviction records will be considered for Employment in accordance with these laws. At PwC, we recognize that conviction records may have a direct, adverse, and negative relationship to responsibilities such as accessing sensitive company or customer information, handling proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted workplace for all. Applications will be accepted until the position is filled or the posting is removed, unless otherwise set forth on the following webpage. Please visit this link for information about anticipated application deadlines: https://pwc.to/us-application-deadlines #LI-Hybrid #BI-Hybrid

United States
$77K - $202K / year
Job Closed
PwC logo

Data Engineer - Manager

PwC

Build what’s next — with tech that matters PwC provides professional services across Audit and Assurance, Advisory and Tax — powered by a global network of over 370,000 people in 149 countries. You may know us for our business expertise, but technology is core to how we help clients move faster, build trust and deliver meaningful outcomes. As a technologist, you’ll work on agile teams with experienced engineers and product thinkers — using AI, cloud, cybersecurity and more to design scalable, real-world solutions. You’ll keep learning, stay challenged and be part of a network where your growth is built in — and your work drives what’s next.

Data Engineer4 days ago
Full TimeRemoteTeam 10,001+Since 1998H1B Sponsor

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth. In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions. Enhancing your leadership style, you motivate, develop and inspire others to deliver quality. You are responsible for coaching, leveraging team member's unique strengths, and managing performance to deliver on client expectations. With your growing knowledge of how business works, you play an important role in identifying opportunities that contribute to the success of our Firm. You are expected to lead with integrity and authenticity, articulating our purpose and values in a meaningful way. You embrace technology and innovation to enhance your delivery and encourage others to do the same. Examples of the skills, knowledge, and experiences you need to lead and deliver value at this level include but are not limited to: Analyse and identify the linkages and interactions between the component parts of an entire system. Take ownership of projects, ensuring their successful planning, budgeting, execution, and completion. Partner with team leadership to ensure collective ownership of quality, timelines, and deliverables. Develop skills outside your comfort zone, and encourage others to do the same. Effectively mentor others. Use the review of work as an opportunity to deepen the expertise of team members. Address conflicts or issues, engaging in difficult conversations with clients, team members and other stakeholders, escalating where appropriate. Uphold and reinforce professional and technical standards (e.g. refer to specific PwC tax and audit guidance), the Firm's code of conduct, and independence requirements. As part of the Data and Analytics Engineering team you can design and implement thorough data architecture strategies that meet current and future business needs. As a Manager you can lead the development of data models, support compliance with data governance policies, and collaborate with business stakeholders to translate data requirements into technical solutions. You can also build and enhance ETL/ELT pipelines, manage data warehouses and data lakes, and implement data security practices. Responsibilities - Design and implement thorough data architecture strategies - Lead the development of data models - Achieve compliance with data governance policies - Collaborate with business stakeholders to translate data requirements - Build and enhance ETL/ELT pipelines - Manage data warehouses and data lakes - Implement data security leading practices - Foster a culture of data-driven decision making What You Must Have - Bachelor's Degree in Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, or Mathematical Statistics - 5 years of experience What Sets You Apart - Certification in Cloud Platforms [e.g., AWS Solutions Architect, AWS Data Engineer, Google Professional Cloud Architect, GCP Data Engineer Microsoft Azure Solutions Architect, Azure Data Engineer Associate, or Snowflake Core, Snowflake Databricks Data Engineer Associate] is a plus - Designing and implementing thorough data architecture strategies that meet the current and future business needs - Developing and documenting data models, data flow diagrams, and data architecture guidelines - Verifying data architecture is compliant with data governance and data security policies - Collaborating with business stakeholders to understand their data requirements and translate them into technical solutions - Evaluating and recommending new data technologies and tools to enhance data architecture - Building, maintaining, and improving ETL/ELT pipelines for data ingestion, processing, and storage across batch and real-time data processing - Building, maintaining, and improving Data Quality rules leveraging DQ tools and/or other ETL/ELT tools - Developing and deploying scalable data storage solutions using AWS, Azure and GCP services such as S3, Amazon RDS, DynamoDB, Azure Data Lake Storage, Azure Cosmos DB, Azure SQL DB, GCP Cloud Storage etc. - Implementing data integration solutions using AWS Glue, AWS Lambda, Azure Data Factory, Azure Functions, GCP Functions, GCP Dataproc, Dataflow and other relevant services - Designing and managing data warehouses and data lakes, verifying data is organized and accessible - Monitoring and troubleshooting data pipelines, data warehouses and workflows to verify data quality, system reliability, performance and cost management - Implementing IAM roles and policies to manage access and permissions within AWS, Azure, GCP - Use AWS CloudFormation, Azure Resource Manager templates, Terraform for infrastructure as code (IaC) deployments - Use AWS, Azure and GCP DevOps services to build and deploy DevOps pipelines - Implementing data security practices using AWS, Azure, GCP, Snowflake or Databricks - Improving Cloud resources for cost, performance, and scalability - Proficiency in SQL and experience with relational databases - Proficient in programming languages such as Python, Java, or Scala - Familiarity with big data technologies like Hadoop, Spark, or Kafka is a plus - Experience with machine learning and data science workflows is a plus - Knowledge of data governance and data security practices - Demonstrating analytical, problem-solving, and communication skills - Having the ability to work independently and as part of a team in a fast-paced environment - Applying modern, cloud-based technology skills, ability to research emerging trends, analyst publications, and adoption of modern technologies in solution architectures - Collaborating and contributing as a team member: understanding personal and team roles, contributing to a positive working environment by building proven relationships with team members, proactively seeking guidance, clarification and feedback - Prioritizing and handling multiple tasks, researching and analyzing pertinent client, industry and technical matters, utilizing problem-solving skills, and communicating effectively in written and verbal formats to various audiences (including various levels of management and external clients) in a professional business environment - Coaching and collaborating with associates who assist with this work, including providing coaching, feedback and guidance on work performance The salary range for this position is: $99,000 - $232,000. Actual compensation within the range will be dependent upon the individual's skills, experience, qualifications and location, and applicable employment laws. All hired individuals are eligible for an annual discretionary bonus. PwC offers a wide range of benefits, including medical, dental, vision, 401k, holiday pay, vacation, personal and family sick leave, and more. To view our benefits at a glance, please visit the following link: https://pwc.to/benefits-at-a-glance As PwC is an equal opportunity employer, all qualified applicants will receive consideration for employment at PwC without regard to race; color; religion; national origin; sex (including pregnancy, sexual orientation, and gender identity); age; disability; genetic information (including family medical history); veteran, marital, or citizenship status; or, any other status protected by law. PwC does not intend to hire experienced or entry level job seekers who will need, now or in the future, PwC sponsorship through the H-1B lottery, except as set forth within the following policy: https://pwc.to/H-1B-Lottery-Policy. Learn more about how we work: https://pwc.to/how-we-work For only those qualified applicants that are impacted by the Los Angeles County Fair Chance Ordinance for Employers, the Los Angeles' Fair Chance Initiative for Hiring Ordinance, the San Francisco Fair Chance Ordinance, San Diego County Fair Chance Ordinance, and the California Fair Chance Act, where applicable, arrest or conviction records will be considered for Employment in accordance with these laws. At PwC, we recognize that conviction records may have a direct, adverse, and negative relationship to responsibilities such as accessing sensitive company or customer information, handling proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted workplace for all. Applications will be accepted until the position is filled or the posting is removed, unless otherwise set forth on the following webpage. Please visit this link for information about anticipated application deadlines: https://pwc.to/us-application-deadlines #LI-Hybrid #BI-Hybrid

United States
$99K - $232K / year
Job Closed