Job Closed
This listing is no longer active.
Globally recognized digital and engineering solutions partner.
Senior Data Engineer, Databricks
Location
Portugal
Posted
73 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer, Databricks
opinov8
• Design, develop, and maintain scalable data pipelines using Microsoft Azure and Databricks (Spark). • Build and optimize ETL/ELT workflows in Databricks, leveraging PySpark and SQL for large-scale data processing. • Integrate and land various data sources—including internal, licensed third-party, and public datasets—into the global data lake. • Develop and maintain Delta Lake architectures to ensure reliable, high-performance data storage and processing. • Collaborate with business analysts, data scientists, and engineering teams to define data requirements and deliver high-quality data products. • Use Python and SQL to automate data ingestion, transformation, and validation processes. • Monitor, troubleshoot, and optimize Databricks jobs and clusters for performance, scalability, and cost efficiency. • Implement data quality, governance, and security best practices across the platform. • Participate in documentation, code reviews, and knowledge sharing across a global team.
Job Requirements
- 5+ years of professional experience in data engineering or similar roles.
- Strong hands-on experience with Databricks, including Apache Spark (PySpark/SQL) and Delta Lake.
- Proven experience building scalable data pipelines in Azure (ADF, Synapse, Blob Storage, Azure SQL, Functions, etc.).
- Proficiency in Python and SQL for data engineering and transformation tasks.
- Experience working with large-scale, distributed data processing systems.
- Solid understanding of data modelling, data warehousing, and modern data lake architectures.
- Experience integrating multiple data sources, including structured and unstructured data.
- Strong problem-solving, documentation, and collaboration skills.
Benefits
- Digital-First Approach: Great talent knows no borders! You can work from wherever you are — we hire and collaborate with professionals worldwide.
- Remote Work Model: Balance your professional and personal life with our flexible working conditions, empowering you to deliver your best from anywhere.
- Exciting Projects: Dive into impactful projects across industries that challenge and spark creativity.
- Boost Your Expertise: Grow your career with continuous learning, development opportunities, and hands-on experience.
- Join the Best Team Ever: Collaborate with our diverse and cross-cultural team of passionate technologists and creative thinkers.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Own the vision and technical strategy for data engineering and analytics engineering at Philo, setting the direction for how data is ingested, modeled, and delivered across the organization • Own the team’s roadmap, balancing strategic long-term investments with near-term stakeholder needs • Lead and grow the Data Engineering team, including hiring, career development, performance management, and supporting the team in generally thriving • Lead the design and implementation of Philo's data platform architecture, including setting technical standards and best practices for data engineering and analytics engineering work • Collaborate with cross-functional teams to understand data platform needs and translate business requirements into technical solutions • Build and maintain stakeholder trust in data and reporting — ensuring that business teams have reliable, well-documented, and well-understood data products • Design scalable data pipelines and infrastructure; drive technical decisions with a focus on long-term sustainability • Implement data governance, security, and quality control processes as needed • Ultimately, deliver on the alchemy of transforming raw, messy application data into trusted business insight through scalable dimensional modeling and a metric layer that faithfully reflects product and customer behavior.
Data Specialist, Data Engineer – BI, Data Lake
Nexer Enterprise ApplicationsWe are the tech company of the future. A promising future.
• Assess the current data and BI architecture • Organize and standardize datasets within the Data Lake • Define standardized data models • Implement data governance best practices • Build data ingestion and transformation pipelines • Structure analytical layers (raw, curated, analytics) • Improve BI query performance and quality • Work with business teams to ensure consistency of metrics • Document data structures and models used • Support the evolution of the company's data architecture
Role Description We are seeking a true Data Modeler who specializes in Kimball dimensional modeling to drive the architecture for our client’s platform. This is a hands-on modeling role (80% modeling, 20% ETL/Transformation) where you will be the resident Kimball expert. You will not be responsible for production ingestion or building DBT pipelines; instead, you will architect star schemas, write the complex SQL transformations to validate the business logic in Databricks, and hand off the validated logic to the ETL engineering team. - Architect and design star schemas utilizing the Kimball methodology. - Work directly in Databricks/Spark notebooks to explore data, mock up tables, and write advanced SQL to transform source data into dimensional models. - Act as a mentor and coach to internal team members transitioning to dimensional modeling. - Partner with the central data engineering team, providing them with validated SQL and table sketches to deploy into production via DBT. Qualifications - Expert-level mastery of dimensional modeling (Kimball methodology, Star Schemas). - Strong hands-on SQL skills specifically for data transformation. - Experience in a Databricks/Spark environment (Highly preferred). Requirements - Python experience. - Familiarity with SAP HANA source systems. - Experience in the manufacturing domain. - Applicants must be authorized to work for any employer in the U.S. Company Description
• Design, build, and maintain scalable, robust, and automatable data pipelines to support ingestion and processing across structured, unstructured, and media data types (including audio, text, and video). • Partner with product, data science, and business stakeholders to translate business goals and customer needs into engineering requirements — starting with the desired outcome and working backwards into concrete, scalable solutions. • Improve data quality, observability, lineage, and reliability across our modular processing pipeline architecture. • Build and maintain core data infrastructure components (e.g., ingestion APIs, metadata services, observability tooling) that support parallel processing, microservices, and config-driven workflows. • Help evolve our approach to handling time-series and media content at scale, optimizing for throughput, cost efficiency, and real-world performance. • Collaborate across teams to validate production data flows, support onboarding of new datasets, and troubleshoot issues quickly and effectively.


