Job Closed

This listing is no longer active.

Datavail logo
Datavail

We help clients turn data into decisions no matter where it lives-in apps, on-prem, in a hybrid model, or in the cloud.

Data Engineer, Databricks

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 2007H1B SponsorCompany SiteLinkedIn

Location

United States

Posted

63 days ago

Salary

0

Seniority

Senior

Job Description

Data Engineer, Databricks

Datavail

• Lead and contribute to end-to-end Databricks implementations for clients, including data migration, Lakehouse architecture, and pipeline development • Gather technical requirements, design solutions, and present recommendations to client stakeholders (technical and business) • Build scalable ETL/ELT pipelines using PySpark, Delta Lake, Delta Live Tables (DLT), and Databricks Workflows • Design and implement Databricks Genie • Design and implement semantic layers • Use Databricks AI features to accelerate development, debugging, and code optimization • Design and implement Lakebase architectures for operational and analytical workloads, including transactional data use cases • Develop solutions using SDLC best practices, including modular code design, testing, and documentation • Use Git based version control with proper branching strategies • Implement CI/CD pipelines for Databricks asset • Implement data quality checks, validations, and expectations within workflows • Design and implement Unity Catalog governance, security, and lineage solutions • Optimize Databricks workloads for performance, cost, and reliability (Photon, cluster policies, Liquid Clustering, Auto Loader, etc.) • Integrate Databricks with client ecosystems (Azure, AWS, GCP, Snowflake, Kafka, legacy systems, etc.) • Support client workshops, proof-of-concepts (POCs), and knowledge transfer sessions • Deliver projects following consulting methodologies while meeting quality, timeline, and budget expectations • Document architectures, runbooks, and best practices for client use • Participate in solutioning activities (scoping, estimation, technical demos) as needed

Job Requirements

  • 3 -5 years of hands-on Databricks experience (or strong Spark experience with significant recent Databricks work)
  • Proven experience delivering Databricks projects in a consulting or professional services environment (preferred) or equivalent client-facing project delivery
  • Strong proficiency in PySpark, Spark SQL, Python, and SQL
  • Deep experience with Delta Lake, Unity Catalog, Delta Live Tables, and Databricks Jobs
  • Hands-on experience with Git version control, pull requests, code reviews, and collaborative development workflows
  • Cloud platform experience (Azure Databricks, AWS, or GCP - at least one)
  • Excellent client-facing and communication skills - able to explain complex concepts to both technical and non-technical audiences
  • Solid understanding of data governance, security, and Lakehouse best practices
  • Bachelor's degree in Computer Science, Engineering, or related field (or equivalent experience)

Benefits

  • As a Databricks Data Engineer, you will work directly with clients across multiple industries to design, implement, and optimize Databricks-based data solutions
  • You will be a key member of our Professional Services delivery teams, delivering high-quality projects on time and within scope while building strong client relationships
  • This is a client-facing role that combines hands-on technical delivery with consulting best practices
  • Collaborate with client data teams to ensure successful adoption and handover of solutions

Related Categories

Related Job Pages

More Data Engineer Jobs

DAS42 logo

Data Engineer Consultant

DAS42

Faster Insights. Better Decisions.

Data Engineer63 days ago
Full TimeRemoteTeam 51-200Since 2013H1B No Sponsor

• Guide clients on optimizing their data environment to work most effectively and efficiently for them. Clarify management objectives through data solutions. • Develop system engineering, integrations, and architectures based on client needs. • Implement and provide advice on data warehouse solutions, ETL pipelines, and business intelligence reporting tools. • Develop a data model around stated use cases to capture client’s KPIs and data transformations. Validation and testing of data models. • Teach technical data modeling concepts to a variety of audiences, including developers, data architects, business users, and IT professionals. • Support, maintain, and document clients’ data environments. • Work within a project management framework to meet objectives, understand scope, and impact of your work across an organization. • Work with the Sales and Marketing teams to develop Thought Leadership and support our sellers by talking to clients about DAS42’s capabilities.

United States
$95K - $125K / year
Job Closed
Milliman logo

Data Engineer – Entry Level

Milliman

Solutions for a world at risk™

Data Engineer63 days ago
Full TimeRemoteTeam 1,001-5,000Since 1947H1B Sponsor

• Build Pipelines: Design and maintain scalable data pipelines to ingest and enrich healthcare data using Databricks and Spark. • Optimize Data Workflows: Improve data intake processes and optimize SparkSQL/Python workloads for performance, scalability, and cost efficiency. • Design Data Models: Partner with senior engineers to develop data marts and semantic models that power analytics products and reporting. • Ensure Quality: Monitor pipeline health, troubleshoot failures, and implement data validation and quality controls. • Learn & Grow: Expand your knowledge of the full data lifecycle, cloud infrastructure (Azure/AWS), and healthcare data standards.

Arizona + 19 moreAll locations: Arizona | California | Connecticut | Florida | Idaho | Illinois | Maine | New Jersey | New York | Oregon | Maryland | Massachusetts | Missouri | Pennsylvania | Texas | Utah | Vermont | Virginia | Washington | Wisconsin
$71.7K - $131.9K / year
Job Closed

Lead Data Engineer – Platform

Valtech

A pioneer in the fields of digital and technology, Valtech is a global business-transformation agency that was founded in 1993 to deliver "innovation with a pur

Data Engineer63 days ago

• Design, build, and maintain scalable data engineering frameworks and platform utilities used across engineering teams • Develop reusable patterns, templates, and abstractions to standardise and accelerate delivery • Define and evolve platform architecture decisions, ensuring scalability, maintainability and consistency • Design and implement CI/CD pipelines and automation frameworks to improve engineering velocity • Define and enforce engineering standards for testing, code quality, deployment and documentation • Identify and eliminate manual or repetitive processes through automation and tooling improvements • Integrate AI-assisted development tools into engineering workflows to improve productivity • Develop and maintain AI engineering assets such as coding guidelines, prompt frameworks and reusable agent configurations • Lead the development and operational support of core data transformation frameworks (including dbt Core at enterprise scale) • Investigate and resolve framework-level issues, including deployment failures, dependency conflicts and production incidents • Support onboarding and enablement of engineering teams adopting platform tooling • Act as the main technical point of contact for platform and framework-related queries • Partner with engineering teams to identify pain points and translate them into platform improvements • Ensure platform tooling meets security, compliance and operational requirements • Conduct and support code and design reviews across platform components • Monitor platform health, performance and adoption, iterating based on feedback and metrics • Contribute to documentation, developer guides and enablement materials to improve usability and adoption

Poland
Job Closed
Providence logo

Data Engineer I

Providence

At Providence, our strength lies in Our Promise of “Know me, care for me, ease my way.” Working at our family of organizations means that regardless of your role, we’ll walk alongside you in your career, supporting you so you can support others. We provide best-in-class benefits and foster an inclusive workplace where diversity is valued, and everyone is essential, heard, and respected. Together, our 120,000 caregivers serve in over 50 hospitals, over 1,000 clinics and a full range of health and social services across Alaska, California, Montana, New Mexico, Oregon, Texas and Washington. As a comprehensive health care organization, we are serving more people, advancing best practices and continuing our more than 100-year tradition of serving the poor and vulnerable.

Data Engineer64 days ago
Full TimeRemoteTeam 10,001+Since 1856H1B Sponsor

• Designs and builds modern data-centric software applications to support clinical and operational processes • Builds data pipelines and transformations, data enrichment processes, provisioning layers, and user interfaces • Works closely with the Product, Platform, and Architecture teams to deliver on joint efforts

Washington
$45K - $70K / year
Job Closed