Job Closed

This listing is no longer active.

opinov8 logo
opinov8

Globally recognized digital and engineering solutions partner.

Senior Data Engineer, Databricks

Data EngineerData EngineerFull TimeRemoteSeniorTeam 201-500Since 2017H1B No SponsorCompany SiteLinkedIn

Location

Portugal

Posted

73 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer, Databricks

opinov8

• Design, develop, and maintain scalable data pipelines using Microsoft Azure and Databricks (Spark). • Build and optimize ETL/ELT workflows in Databricks, leveraging PySpark and SQL for large-scale data processing. • Integrate and land various data sources—including internal, licensed third-party, and public datasets—into the global data lake. • Develop and maintain Delta Lake architectures to ensure reliable, high-performance data storage and processing. • Collaborate with business analysts, data scientists, and engineering teams to define data requirements and deliver high-quality data products. • Use Python and SQL to automate data ingestion, transformation, and validation processes. • Monitor, troubleshoot, and optimize Databricks jobs and clusters for performance, scalability, and cost efficiency. • Implement data quality, governance, and security best practices across the platform. • Participate in documentation, code reviews, and knowledge sharing across a global team.

Job Requirements

  • 5+ years of professional experience in data engineering or similar roles.
  • Strong hands-on experience with Databricks, including Apache Spark (PySpark/SQL) and Delta Lake.
  • Proven experience building scalable data pipelines in Azure (ADF, Synapse, Blob Storage, Azure SQL, Functions, etc.).
  • Proficiency in Python and SQL for data engineering and transformation tasks.
  • Experience working with large-scale, distributed data processing systems.
  • Solid understanding of data modelling, data warehousing, and modern data lake architectures.
  • Experience integrating multiple data sources, including structured and unstructured data.
  • Strong problem-solving, documentation, and collaboration skills.

Benefits

  • Digital-First Approach: Great talent knows no borders! You can work from wherever you are — we hire and collaborate with professionals worldwide.
  • Remote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere.
  • Exciting Projects: Dive into impactful projects across industries that challenge and spark creativity.
  • Boost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience.
  • Join the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.

Related Categories

Related Job Pages

More Data Engineer Jobs

Full TimeRemoteTeam 51-200Since 2010H1B Sponsor

• Own the vision and technical strategy for data engineering and analytics engineering at Philo, setting the direction for how data is ingested, modeled, and delivered across the organization • Own the team’s roadmap, balancing strategic long-term investments with near-term stakeholder needs • Lead and grow the Data Engineering team, including hiring, career development, performance management, and supporting the team in generally thriving • Lead the design and implementation of Philo's data platform architecture, including setting technical standards and best practices for data engineering and analytics engineering work • Collaborate with cross-functional teams to understand data platform needs and translate business requirements into technical solutions • Build and maintain stakeholder trust in data and reporting — ensuring that business teams have reliable, well-documented, and well-understood data products • Design scalable data pipelines and infrastructure; drive technical decisions with a focus on long-term sustainability • Implement data governance, security, and quality control processes as needed • Ultimately, deliver on the alchemy of transforming raw, messy application data into trusted business insight through scalable dimensional modeling and a metric layer that faithfully reflects product and customer behavior.

California + 2 moreAll locations: California | New York | Massachusetts
$200K - $275K / year

Data Specialist, Data Engineer – BI, Data Lake

Nexer Enterprise Applications

We are the tech company of the future. A promising future.

Data Engineer73 days ago
Full TimeRemoteTeam 1,001-5,000Since 2007H1B No Sponsor

• Assess the current data and BI architecture • Organize and standardize datasets within the Data Lake • Define standardized data models • Implement data governance best practices • Build data ingestion and transformation pipelines • Structure analytical layers (raw, curated, analytics) • Improve BI query performance and quality • Work with business teams to ensure consistency of metrics • Document data structures and models used • Support the evolution of the company's data architecture

Brazil

Role Description We are seeking a true Data Modeler who specializes in Kimball dimensional modeling to drive the architecture for our client’s platform. This is a hands-on modeling role (80% modeling, 20% ETL/Transformation) where you will be the resident Kimball expert. You will not be responsible for production ingestion or building DBT pipelines; instead, you will architect star schemas, write the complex SQL transformations to validate the business logic in Databricks, and hand off the validated logic to the ETL engineering team. - Architect and design star schemas utilizing the Kimball methodology. - Work directly in Databricks/Spark notebooks to explore data, mock up tables, and write advanced SQL to transform source data into dimensional models. - Act as a mentor and coach to internal team members transitioning to dimensional modeling. - Partner with the central data engineering team, providing them with validated SQL and table sketches to deploy into production via DBT. Qualifications - Expert-level mastery of dimensional modeling (Kimball methodology, Star Schemas). - Strong hands-on SQL skills specifically for data transformation. - Experience in a Databricks/Spark environment (Highly preferred). Requirements - Python experience. - Familiarity with SAP HANA source systems. - Experience in the manufacturing domain. - Applicants must be authorized to work for any employer in the U.S. Company Description

United States
Job Closed
Neon logo

Senior Data Engineer

Neon

Boas-vindas ao nosso Neonverso!

Data Engineer73 days ago
OtherRemoteTeam 1,001-5,000Since 2016H1B Sponsor

• Design, build, and maintain scalable, robust, and automatable data pipelines to support ingestion and processing across structured, unstructured, and media data types (including audio, text, and video). • Partner with product, data science, and business stakeholders to translate business goals and customer needs into engineering requirements — starting with the desired outcome and working backwards into concrete, scalable solutions. • Improve data quality, observability, lineage, and reliability across our modular processing pipeline architecture. • Build and maintain core data infrastructure components (e.g., ingestion APIs, metadata services, observability tooling) that support parallel processing, microservices, and config-driven workflows. • Help evolve our approach to handling time-series and media content at scale, optimizing for throughput, cost efficiency, and real-world performance. • Collaborate across teams to validate production data flows, support onboarding of new datasets, and troubleshoot issues quickly and effectively.

United States
$160K - $220K / year
Job Closed