#MakeDataMatter #HumanizingTheFuture
Senior Data Engineer, Databricks
Location
Portugal
Posted
1 day ago
Salary
€500 - €600 / day
Seniority
Senior
Job Description
Senior Data Engineer, Databricks
Keyrus
• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Python and SQL. • Build and optimise batch and near real-time data processing solutions on the Microsoft Azure ecosystem. • Develop and maintain Delta Lake-based data architectures. • Develop robust and scalable data products and modern data platform components supporting strategic business initiatives. • Contribute to CI/CD implementation and automation for data solutions. • Apply engineering best practices around source control, testing, deployment, and monitoring. • Support Infrastructure-as-Code and environment standardisation initiatives. • Ensure high standards of data quality, observability, monitoring, security, and operational reliability. • Collaborate with Product Managers, Architects, Data Engineers, and business stakeholders to deliver high-quality outcomes. • Participate in technical design discussions and contribute to solution architecture decisions. • Support platform governance, cost optimisation, technical documentation, and knowledge sharing initiatives.
Job Requirements
- 5+ years of experience in Data Engineering
- 3+ years of hands-on experience with Databricks
- Proven experience building and maintaining modern cloud data platforms
- Experience working within Agile delivery environments
- Proven ability to work effectively in distributed and international teams
- Professional proficiency in English.
- Strong hands-on expertise with: Databricks, PySpark, Delta Lake, Databricks SQL, Workflow orchestration, Performance tuning
- Strong proficiency in: Python, SQL
- Experience with: Microsoft Azure Data Platform services, Azure DevOps, Git-based development, CI/CD implementation, Data modelling, Data warehousing concepts
- Good understanding of: Data Governance, Security and access management, Monitoring and observability.
- Ability to design and deliver scalable, secure, and cost-efficient data solutions.
Benefits
- Meal allowance: €10.20/day
- Flexible benefits plan
- Private medical insurance
- 22 days of annual leave, increasing every 3 years (up to 25 days)
- Continuous learning via KLX – Keyrus Learning Experience
- A collaborative, international, and human-centred work environment
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Staff Fullstack Engineer - Data Products
GitLabBuild software faster. The One DevOps Platform enables your entire org to collaborate around your code. We're hiring.
GitLab is the intelligent orchestration platform for DevSecOps. GitLab enables organizations to increase developer productivity, improve operational efficiency, reduce security and compliance risk, and accelerate digital transformation. More than 50 million registered users and more than 50% of the Fortune 100* trust GitLab to ship better, more secure software faster. The same principles built into our products are reflected in how our team works: we embrace AI as a core productivity multiplier, with all team members expected to incorporate AI into their daily workflows to drive efficiency, innovation, and impact. GitLab is where careers accelerate, innovation flourishes, and every voice is valued. Our high-performance culture is driven by our values and continuous knowledge exchange, enabling our team members to reach their full potential while collaborating with industry leaders to solve complex problems. Co-create the future with us as we build technology that transforms how the world develops software. *Fortune 500® is a registered trademark of Fortune Media IP Limited, used under license. Claim based on GitLab data. Fortune 100 refers to the top 20% ranked companies in the 2025 Fortune 500 list, published in June 2025. Fortune and Fortune Media IP Limited are not affiliated with, and do not endorse products or services of GitLab. Staff Fullstack Engineer - Data ProductsAn overview of this roleAs a Staff Fullstack Engineer - Data Products, you will help build GitLab's in-product insights on top of the Data Insights Platform and the GitLab Knowledge Graph. You will be the technical anchor across reporting dashboards, inbound graph ingestion, and outbound data delivery, helping turn software delivery data into reliable and actionable intelligence for customers and internal teams. This is a strong fit if you want to work in a 0 to 1 space where you can shape what gets built, make early architectural choices, and guide work from specification through production health. In this role, you will provide technical guidance across your domain, partner closely with Product, Design, and backend teams, and own outcomes across GitLab.com, Dedicated, and Self-Managed environments as we build out this area. What you’ll do - Architect how GitLab and third-party data is ingested, modeled, and synchronized into the Knowledge Graph as a near-real-time graph of the development ecosystem. - Coordinate integration with external systems such as Jira, observability tools, Zendesk, and ServiceNow to add business context to data products. - Design the Data Marketplace so customers can consume GitLab data through Snowflake, Databricks, and BigQuery without engineering handoffs. - Publish GitLab observability data as OpenTelemetry and support the APIs used for data access. - Define operational readiness for the systems your team ships across GitLab.com, Dedicated, and Self-Managed deployments. - Translate ambiguous product problems into practical technical plans and iterative roadmaps in partnership with Product, Design, and the graph backend team. - Resolve cross-team coordination with Graph Backend, AI Platform, and Infrastructure to keep delivery moving. - Mentor senior and intermediate engineers through design reviews, pairing, and code review. What you’ll bring - Experience owning product or platform systems end to end across multiple engineering and product teams at the staff level. - Strong fullstack technical skills with proficiency in Go, Ruby, and Node. - Depth in at least one area such as reporting and metrics platforms, graph data systems, or data access and warehouse integrations including Snowflake, Databricks, or BigQuery. - Hands-on experience building production data ingestion, systems integrations, or data access capabilities. - Experience operating multi-tenant systems across both SaaS and self-managed environments. - Strong judgment in ambiguity, with the ability to make tradeoffs, create clarity, and move early-stage work forward. - Clear written communication skills, including architecture proposals and design records that help distributed teams make progress asynchronously. - A practical and proactive approach to collaboration, with transferable experience welcomed from adjacent platform, data, or product engineering backgrounds. About the teamThe Data Products team is building the foundation for how GitLab turns software delivery data into reliable, interoperable, and actionable intelligence for customers and internal teams. Our scope includes Software Engineering Intelligence, dashboard and reporting frameworks, external data access and output, and internal and external data ingestion across areas such as DORA metrics, value stream reporting, AI impact, software delivery trends, security and quality metrics, custom reporting, data access APIs, OpenTelemetry-based output, and ingestion into the Knowledge Graph and Data Insights Platform. We work in a high-ambiguity, high-impact space, and we collaborate closely across product and platform groups using GitLab's asynchronous, distributed way of working. How GitLab Supports Full-Time Employees - Benefits to support your health, finances, and well-being - Flexible Paid Time Off - Team Member Resource Groups - Equity Compensation & Employee Stock Purchase Plan - Growth and Development Fund - Parental Leave Please note that we welcome interest from candidates with varying levels of experience; many successful candidates do not meet every single requirement. Additionally, studies have shown that people from underrepresented groups are less likely to apply to a job unless they meet every single qualification. If you're excited about this role, please apply and allow our recruiters to assess your application. Country Hiring Guidelines: GitLab hires new team members in countries around the world. All of our roles are remote, however some roles may carry specific location-based eligibility requirements. Our Talent Acquisition team can help answer any questions about location after starting the recruiting process. Privacy Policy: Please review our Recruitment Privacy Policy. Your privacy is important to us. GitLab is proud to be an equal opportunity workplace and is an affirmative action employer. GitLab’s policies and practices relating to recruitment, employment, career development and advancement, promotion, and retirement are based solely on merit, regardless of race, color, religion, ancestry, sex (including pregnancy, lactation, sexual orientation, gender identity, or gender expression), national origin, age, citizenship, marital status, mental or physical disability, genetic information (including family medical history), discharge status from the military, protected veteran status (which includes disabled veterans, recently separated veterans, active duty wartime or campaign badge veterans, and Armed Forces service medal veterans), or any other basis protected by law. GitLab will not tolerate discrimination or harassment based on any of these characteristics. See also GitLab’s EEO Policy and EEO is the Law. If you have a disability or special need that requires accommodation, please let us know during the recruiting process.
Role Description You will be part of a multi-disciplinary technology team, working closely with our customers (business, software vendors, and partners). Within the team, you will be responsible for a suite of data processes and will participate in all aspects from design through to testing and implementation. You will be surrounded by data professionals that strive for excellence and data best practices to realize business value. As a Data Engineer, you will work in the data engineering team to build and maintain data pipelines to ingest data into the warehouse and support integration into other systems. You will apply your knowledge of good data engineering practices and standards, and data technologies to deliver target state design and implementation. The role is challenging, and you must be adept at problem-solving and able to respond to changing priorities and rapidly evolving requirements that may have a direct impact on services to users. This role would suit a professional who is keen to grow their career in a busy team that values cognitive diversity and diversity of lived experience. Duties & Responsibilities - Maintain, support, and monitor existing production SSIS packages and SQL Queries and Stored Procedures, CICD pipelines to ensure all data loads on the data warehouse meet data quality standards and business SLA requirements. - Build, maintain, support, and monitor Synapse data engineering pipelines on the data platform if required. - Participate and contribute to data architecture design, data modelling, gathering and analysis of data requirements; understand, document, communicate, and build appropriate solutions. - Participate, design, build, deliver, and document data-related projects with various environment-specific data analytics technologies. - Promote data engineering best practices with CICD pipelines and automation. - Collaborate and work closely with team members and contribute significantly to building a high-performing, collaborative, transparent, and result-driven data engineering team. - Support the Data Engineering Practice Team Manager with best fit-to-purpose data engineering solutions, quality engineering artifacts, and high standard documentation. - Follow Data Ethics standards to protect personal information and meet our customers’, partners’, and community’s expectations. Qualifications - 8+ years demonstrable experience in design, build, and support of data engineering pipelines in data warehousing, data ingestion, cleansing, manipulation, modelling, and reporting. - Experience in ETL using Microsoft technologies. - Strong experience in writing MS SQL server queries, stored procedures, and SSIS Packages. Experience with SSRS would be an advantage. - Experience of manipulating semi-structured data (XML, JSON). - Strong knowledge and extensive experience in working in an Agile framework with CI/CD using modern DevOps / Data Ops integrated processes with YAML pipelines. - Bachelor’s degree in computer/data science technical or related field is a must. Post-graduate is highly regarded. - Knowledge of Azure Synapse data engineering pipelines, PySpark notebooks, data platform lake house architecture, and Azure SQL ODS storage is desirable.
Intermediate Fullstack Engineer – Data Products
GitLabBuild software faster. The One DevOps Platform enables your entire org to collaborate around your code. We're hiring.
• Develop well-scoped features, with support from Senior and Staff Engineers, for how GitLab and third-party data is ingested, modeled, and synced into the knowledge graph, meeting team goals for data freshness, sync reliability, and graph coverage. • Build parts of integrations with external systems for business context, including Jira, observability tools, Zendesk, and ServiceNow, increasing indexed business context while meeting team goals for data completeness and freshness in the graph. • Build features for the Software Engineering Intelligence dashboards, including DORA, value stream reporting, GitLab Duo, and software development lifecycle metrics rolled up across organizational structures, meeting team goals for reporting accuracy, performance, and coverage. • Contribute to design discussions and technical direction for your features, with guidance from Senior and Staff Engineers, helping deliver secure, reliable, and performant solutions on schedule. • Raise blockers and risks early, and work with Senior Engineers to unblock your work so milestones stay on track and delivery risk is reduced. • Participate in design review and code review, and apply feedback to improve code quality, maintainability, and delivery velocity. • Build a deep understanding of the system, including the data model and the platform the team builds on, so you can ship changes more independently and resolve issues faster.
Role Description We are looking for a Corporate Data Architect who will help organize and standardize how the organization understands and uses data. Your main task will be to design and develop data architecture at the level of the entire organization so that data is consistent, well defined, and ready to be used in analytics, AI, and system integrations. You will act as a bridge between the worlds of business, data, and technology, connecting strategy, standards, and practice. - Design semantic, logical, and conceptual data models based on business domains and organizational processes. - Define and standardize the corporate data model as part of the Data Governance initiative. - Create and maintain reference architecture (logical and technical reference architectures) and design patterns for data domains. - Co-create the strategy and roadmap for data architecture development that supports the vision of the Data Governance program and organizational goals. - Collaborate with Data Governance, Data Management, Analytics, AI, and Data Engineering teams to integrate models with the data catalog and metadata management tools. - Support technical teams in implementing models in data layers. - Participate in designing data integration, flow, and quality processes (data lineage, data contracts, data quality). - Identify and certify authoritative data sources within organizational domains. - Monitor the compliance of data solutions with Data Governance policies and standards as well as architectural recommendations. - Support the resolution of data conflicts and issues. - Create architectural documentation, metamodels, and inter-domain relationship diagrams. - Consult on and develop good practices in the area of modeling, metadata, and information architecture. - Collaborate on defining the vision, priorities, and directions for the development of data architecture in the organization. Qualifications - Experience in data modeling (conceptual, logical, physical), preferably in a large organization or within complex data ecosystems. - Practical knowledge of tools and methodologies such as ER, UML, IDEF1X, Data Vault, 3NF, Kimball/Inmon. - Ability to work with data catalogs and metadata management tools (e.g. DataHub, Collibra, Alation, Atlan). - Knowledge of SQL and relational data models. - Experience working with database systems and data architecture in a cloud environment (AWS). - Ability to work with engineering and analytics teams to translate business needs into data models. - Understanding of data security, availability, and control principles. - Good command of English (reading documentation, community discussions). Requirements - Experience in Data Governance, Data Quality, Master Data, and Metadata Management projects. - Knowledge of concepts and technologies related to the semantic data layer: ontologies, RDF, OWL, GraphQL, dbt Semantic Layer, Semantic Kernel. - Knowledge of Python for automation, model validation, and API integrations. - Knowledge of event-driven data architecture. - Experience in creating and maintaining architectural documentation. Benefits - Flexible employment and remote work. - International projects with leading global clients. - International business trips. - Non-corporate atmosphere. - Language classes. - Internal & external training. - Private healthcare and insurance. - Multisport card. - Well-being initiatives.


