Data Engineer – Databricks, Lakehouse, Power BI

Data EngineerData EngineerFull TimeRemoteLeadTeam 5,001-10,000H1B SponsorCompany SiteLinkedIn

Location

India

Posted

8 days ago

Salary

0

Seniority

Lead

Bachelor Degree8 yrs expEnglishAzureERPPythonSparkSQL

Job Description

Data Engineer – Databricks, Lakehouse, Power BI

Hillenbrand

• Design, develop, and maintain scalable data pipelines using Databricks and Delta Lake • Build and manage transformations across bronze, silver, and gold layers • Optimize processing for performance, reliability, and cost • Integrate data from ERP, CRM, APIs, and other enterprise systems • Develop gold-layer datasets aligned to standardized business definitions • Translate business requirements into reusable data models • Ensure consistency of core metrics across reporting • Align Databricks outputs with Power BI semantic models • Implement automated data quality checks and validation rules • Build testable, production-ready pipelines • Support impact analysis using lineage tools • Participate in CI/CD and deployment processes • Monitor and optimize pipeline performance • Troubleshoot issues across environments • Ensure data consistency between dev, test, and prod • Support high-volume data workloads • Use AI-assisted tools for development (e.g., Copilot, Databricks Agents) (Preferred) • Explore AI agents for testing, lineage analysis, and optimization (Preferred) • Partner with BI and business teams • Support governance and cataloging efforts • Document data models and pipelines

Job Requirements

  • Bachelor’s degree in relevant field
  • 8+ years data engineering experience
  • Strong Databricks, Spark, Delta Lake experience
  • SQL and Python proficiency
  • Experience with Power BI or similar tools
  • Azure Data Factory, Synapse, or Data Lake experience (Preferred)
  • DevOps and CI/CD experience (Preferred)
  • AI agents or AI-assisted development exposure (Preferred)
  • Experience in large, complex enterprise environments (Preferred)

Benefits

  • Health insurance
  • 401(k) matching
  • Flexible work hours
  • Paid time off

Related Categories

Related Job Pages

More Data Engineer Jobs

Endava logo

Data Architect

Endava

Technology is our how. And people are our why.

Data Engineer8 days ago
Full TimeRemoteTeam 10,001+Since 2000H1B No Sponsor

• Lead architecture and design activities for data, analytics, and AI initiatives • Define end-to-end data platform architectures, including ingestion, storage, transformation, governance, and consumption layers • Develop target-state architectures, migration strategies, and modernization roadmaps • Conduct architecture assessments and provide recommendations aligned to business objectives • Collaborate with engineering teams to ensure successful implementation of architectural designs • Establish architecture standards, best practices, and reusable patterns • Design and implement modern cloud-based data architectures • Lead architecture efforts involving Snowflake, Databricks, and major cloud platforms (AWS, Azure, and/or Google Cloud) • Architect solutions leveraging data lakes, lakehouses, data warehouses, streaming platforms, and AI/ML services • Serve as a trusted advisor to client stakeholders, including technical leaders, architects, and business executives • Facilitate architecture workshops, discovery sessions, and strategy discussions • Translate business requirements into actionable technical architectures and implementation plans • Present architectural recommendations and executive-level findings to client audiences • Support sales pursuits through solution design, proposal development, and client presentations • Mentor architects and engineers across the organization

Texas
Job Closed
Quantrics Enterprises Inc. logo

Senior Data Engineering Developer

Quantrics Enterprises Inc.

At Quantrics, we believe in the power of the individual to create a better future - for our customers

Data Engineer8 days ago
Full TimeRemoteTeam 5,001-10,000Since 2016H1B No Sponsor

• Build and optimize real-time streaming pipelines to ensure efficient routing of voice and chat session data into enterprise data platforms • Integrate generative AI solutions with enterprise knowledge bases to ensure accurate and context-aware conversational agent performance • Maintain and configure low-latency databases to ensure secure and reliable storage of session states and operational metadata • Design system architectures and technical blueprints to ensure scalable, resilient, and high-performance solutions • Delegate and manage engineering tasks to ensure effective team productivity and capability development • Develop and execute testing processes to identify system issues and ensure quality and stability in deployed solutions • Collaborate with stakeholders to align requirements, manage expectations, and ensure successful project delivery • Oversee automated PII redaction workflows across voice and text streams to ensure full compliance with data privacy requirements prior to storage.

Philippines

Role Description - Aufbau, Betrieb und Weiterentwicklung einer skalierbaren Datenplattform (Warehouse / Lake / Lakehouse) - Entwicklung und Wartung robuster ETL/ELT-Pipelines zur Integration unterschiedlichster Datenquellen - Sicherstellung von Datenqualität, Performance, Skalierbarkeit und Kostenoptimierung - Implementierung von Monitoring-, Governance- und Sicherheitskonzepten (inkl. DSGVO) - Enge Zusammenarbeit mit Analytics-, Product- und Engineering-Teams - Automatisierung von Datenprozessen sowie kontinuierliche Verbesserung bestehender Architekturen - Aktive Mitgestaltung der Datenstrategie und Evaluation neuer Technologien Qualifications - Mehrjährige Erfahrung im Data Engineering oder in einer vergleichbaren Rolle - Sehr gute Programmierkenntnisse in Python und SQL sowie saubere Engineering-Standards (Tests, CI/CD) - Fundiertes Verständnis moderner Datenarchitekturen und Datenmodellierung - Praxiserfahrung mit Cloud-Plattformen (AWS, GCP oder Azure) und gängigen Data-Tools (z. B. Airflow, dbt, Spark, Snowflake) - Hohes Qualitäts- und Ownership-Mindset für produktive Datenpipelines - Fähigkeit, technische Lösungen verständlich zu kommunizieren und Business-Anforderungen zu übersetzen - Teamplayer mit Interesse an Mentoring und kontinuierlicher Weiterentwicklung - Verhandlungssichere Englischkenntnisse (erforderlich); Deutschkenntnisse sind ein Plus Benefits - Attraktive, marktgerechte Vergütung - Flexibles Arbeitsmodell: Remote-first mit optionaler Präsenz im Büro - 30 Urlaubstage - Strukturiertes Onboarding mit persönlicher Begleitung durch das Recruiting-Team - Transparente Kommunikation durch wöchentliche 1:1s und regelmäßige All-Hands-Meetings - Zugang zu Corporate Benefits - Langfristige Perspektive in einer wachsenden Unternehmensgruppe

Germany + 1 moreAll locations: Germany | Austria
Dispensed Global logo

Staff Data Engineer

Dispensed Global

Dispensed: Your alternative therapy journey to wellness starts here.

Data Engineer9 days ago
Full TimeRemoteTeam 51-200Since 2021H1B No Sponsor

Role Description This is a Staff-level role in title and in scope, but it's not a role where you'll be directing traffic from a distance. You'll be the only Data Engineer in our ANZ team (for now), which means you're setting the architectural direction and doing the hands-on work. Warehouse infrastructure, ETL and reverse ETL, pipeline builds, fact table design, data dictionary governance, that's all you. The organisational influence is real, but so is the technical execution. The decisions you make here will be ones that engineers, analysts, and clinical teams depend on for a long time. So will the standards you set. This isn't a role where you'll be handed a finished data function to run, you'll be shaping what it becomes, and building a lot of it yourself. What You'll Be Doing: - Defining the architectural patterns, modelling standards, and pipeline conventions that teams across the organisation build on and building them yourself. - Owning the warehouse infrastructure, ETL and reverse ETL processes, authoritative fact tables, and data dictionary governance end-to-end. - Leading the most strategically significant data initiatives in your area and dealing with the blockers, unclear priorities, dependency gaps and quality failures that get in the way. - Working closely with senior stakeholders in product, clinical, and technology to make sure data is part of the conversation early, not bolted on after. - Owning data quality across your domain, auditing gaps, surfacing inconsistencies across markets, and driving the remediation work. - Developing the analysts and junior engineers around you, including the harder conversations about growth and direction. - Helping senior leadership understand what the data function can and can't reliably deliver, so investment decisions are grounded in reality. Qualifications - 10+ years in data engineering, with experience setting technical direction and still being the person who builds it. - Deep hands-on expertise with BigQuery, dbt or equivalent, and AWS. - Comfortable being the most senior data engineering voice in the room while still being deeply in the weeds day-to-day. - A track record of making architectural decisions others have depended on. - Experience working with senior stakeholders and bringing a commercial perspective to data decisions. - Someone who takes developing people as seriously as the technical work. Requirements - Nice to have: Healthcare or regulated industry experience. - Experience scaling a data function through a growth period. - Data observability, data contracts, or governance at scale. Benefits - Work From Anywhere. 🌍 - A competitive salary and awesome benefits package. 💰 - A supportive and positive work environment. 🌟 - Opportunities to grow and develop your career. 📈 - Opportunity to transform lives through alternative medicine. 💡

Worldwide