Elevate e-commerce - and yourself - on a daily basis. Develop the e-commerce of the future with us.
Senior Data Engineer
Location
Germany
Posted
43 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Kaufland e-commerce
Role Description Your tasks – this is what awaits you in detail: - You design, develop and maintain our streaming and CDC backbone (Apache Kafka, Debezium, Schema Registry, BigQuery Sinks) and our self-built workflow management platform Navarch, running on Kubernetes. - You drive architectural initiatives across the platform end-to-end — from idea to production — spanning streaming, orchestration, the warehouse and the observability layer that sits across them. - You operate and evolve our BigQuery multi-project setup, shaping access management, partitioning, slot strategy and cost-aware design patterns. - You shape how the platform supports domain analysts who ship their own pipelines — clear ownership boundaries, quality safeguards, and less hand-off friction between producers and the platform team. - You build and maintain observability across the data platform (Datadog, Slack-native alerting, cost dashboards) and you take care of our CI/CD pipelines on our self-hosted GitLab instance. - You shape how we manage BigQuery and Kafka cost at platform scale — FinOps is a live practice on this team, not a one-off. - You bring a senior perspective into architectural decisions across the platform — design reviews, merge-request reviews, and the multi-quarter initiatives the team runs. Qualifications - You have several years of senior-level data-engineering experience, demonstrated by independent ownership of production systems — running on-call, shipping cross-component changes, and influencing architecture without process scaffolding around you. - You have strong problem-solving skills with a demonstrated ability to troubleshoot complex, cross-component issues (schema evolution, broker lag, query performance, cost anomalies). - You have professional experience in software engineering using Python and you have a working command of SQL on top of cloud data warehouses — partitioning, clustering, query cost, idempotency. - Strong experience with object-oriented programming and software design patterns, applied to data engineering applications. - You have hands-on experience with at least four of the following technologies: - Message brokers, ideally Apache Kafka (broker operations, schema registry, KRaft). - Change data capture, ideally Debezium. - Orchestration tools (in-house, Airflow, Dagster or similar). - Public Cloud Providers, ideally Google Cloud Platform. - Container orchestration, ideally Kubernetes (Helm, operators, KEDA/VPA). - Infrastructure as Code, ideally Terraform and/or Ansible. - Data modeling and transformation, ideally dbt or comparable. - You can articulate trade-offs in your past decisions — batch vs. stream, build vs. buy, latency vs. cost, isolation vs. coupling — with concrete numbers from your own work. - You thrive in a Kanban-style, low-process environment where seniors organise their own work and rotate triage weekly. - You are fluent in English (C1) and enjoy working in a dynamic, international environment. Benefits - Create your own work-life balance: You have the flexibility to choose between working remotely (within Germany) or from one of our locations in Cologne, Darmstadt, Düsseldorf, or in Berlin! - Do you want to move to Germany? No problem – we offer you an attractive relocation package to give you a smooth start. - Urban Sports Club and RSG Group Fitness Studios: Get top deals for fitness, swimming, yoga and more. - Mental Well-Being: We support you on your well-being journey with special offerings such as Instahelp, the digital platform for online psychological counseling. - Vacation & Sabbatical: Enjoy 30 days of vacation per year and the opportunity to take a sabbatical once you have been part of the team for a certain period! - Option for Pluxee restaurant vouchers: Buy Pluxee vouchers through us and benefit from tax-advantaged meal allowances! - ‘Deutschlandticket’: We subsidize your train season ticket for more mobility. - Employee Discount: You will receive a monthly coupon for Kaufland.de. - Free choice of operating system: MacOS or Ubuntu Linux, it's up to you. - Boost your growth: Benefit from our online language learning programs, diverse in-house training, and our automated 360-degree feedback. We cover the costs for relevant conferences, training opportunities, and approved team workshops to strengthen personal interactions.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Diseñar e implementar soluciones analíticas y de datos en Microsoft Azure • Desarrollar pipelines y notebooks (PySpark, Spark SQL, SparkR, Python, T-SQL) para la integración, transformación y explotación de datos • Valoración de experiencia en Fabric y con gobernanza y catalogación de datos con Microsoft Purview
• Senior Data Architect serves as an experienced individual contributor within a team, with the expectation that you will further develop your leadership, guidance, and mentoring skills • With minimal oversight from leadership, you will be responsible for supporting the goal of meeting scope, schedule, and delivery requirements • A Senior Data Architect impacts the long-term goals of the program, while contributing to the development of the program's data architecture strategy • You may serve as the discipline's primary lead when working with stakeholders and utilize strong influential skills to drive improvements in data architecture processes and practices • Lead complex legacy data migrations end-to-end, including schema mapping, data profiling, phased migration sequencing, parallel-run validation, and cutover with automated rollback procedures • Design and maintain cloud-native data pipelines using event-driven infrastructure (SNS/SQS, Lambda/Step Functions) to support BI and analytics workflows • Architect data analytics repositories that are production-ready, monitored, and fully documented so agency staff and external tools can operate without contractor dependency • Manage digital artifact and content management lifecycles including upload, storage, review, and archiving across AWS and Azure/M365 environments • Drive data governance across programs, including lineage documentation, dictionary standardization, and glossary development • Align data architecture and standardize access patterns to make data available to BI tools, apps, and web platforms • Translate business and agency requirements into functional BI portals and dashboards • Present architectural decisions clearly to mixed audiences, from engineers to product owners to federal agency stakeholders • Mentor junior analysts and engineers and participate in technical interviews and candidate evaluations • May lead small, less critical, or temporary structures and projects
Role Description We are looking for a Tech Lead DataOps to join a large-scale, international program focused on building and scaling the Group’s Enterprise Cloud Data Warehouse. The platform centralizes global data across major business domains (Finance, Customers, Operations, etc.) using modern cloud data architectures and Agile/Scrum methodologies. In this role, you will lead the transversal platform and reliability efforts, driving industrialization, automation, CI/CD pipelines, and overall platform stability in a fully remote, multicultural environment. Main Responsibilities - Technical Leadership & DataOps Governance - Define and evolve technical standards, ensuring the reliability, scalability, and performance of data workflows. - Lead industrialization and automation initiatives across development and deployment processes, supporting multiple squads on DataOps best practices. - Contribute to technical roadmap definitions, architecture decisions, and continuous ecosystem improvements. - Automation & Platform Engineering - Design, maintain, and evolve internal development and deployment tooling around dbt , Airflow , and Snowflake . - Develop and optimize internal CLI tools for automated dbt model generation, YAML testing, DAG creation, and deployment automation. - Contribute to the integration of AI/LLM capabilities into development and DataOps workflows to reduce manual operations. - CI/CD & Deployment Engineering - Design, implement, and maintain secure and automated multi-environment Data CI/CD pipelines. - Ensure deployment quality during release cycles, collaborating with project squads and supporting release governance. - Orchestration, Reliability & Operations - Supervise, optimize, and design Apache Airflow orchestration workflows and execution DAGs. - Implement monitoring, alerting, and observability capabilities to maximize platform stability and operational efficiency. - Contribute to incident resolution and root cause analysis. Qualifications - Methodology: Strong knowledge and hands-on experience with Data Vault 2.0 methodology applied to enterprise data platforms. - Core Tech Stack: Advanced proficiency in Python (focused on data platforms/automation) and SQL query optimization. - Data Ecosystem & Orchestration: Solid hands-on experience with dbt and Apache Airflow (including DAG design and workflow optimization). - CI/CD & Cloud Infrastructure: Good knowledge of DevOps practices using Git/GitLab , Jenkins , Docker , Kubernetes , and Snowflake (or equivalent MPP platforms). - Leadership & Soft Skills: Strong transversal technical leadership, an autonomous/proactive mindset, and excellent communication skills to collaborate with multicultural, distributed teams. - Languages: Professional proficiency in both English and French is required. Nice to have skills - Experience with specific data modeling and automation tools like AutomateDV and DBSchema . - Practical exposure to modern observability and data platform reliability practices. - Experience or strong interest in integrating AI/LLM tooling (such as GitHub Copilot) into DataOps development and deployment workflows. Contract Type Long term contract
Senior Technical Project Manager – Snowflake, Data Migration
MiratechHelping Visionaries Change the World
• Lead end-to-end delivery of large-scale, scope-based data platforms and cloud migration projects, from initiation through closure. • Provide technical oversight and act as a bridge between engineering teams and client stakeholders, ensuring alignment on scope, timelines, and quality. • Oversee technical prerequisites and delivery readiness, including environment access, data ingestion setup, CDC analysis, SLAs, and validation processes. • Drive stakeholder communication, risk management, issue resolution, and milestone-based sign-offs. • Lead and support cross-functional engineering teams through clear prioritization, performance management, and delivery governance. • Own project financials, including forecasting, cost control, invoicing approvals, and scope change management. • Prepare and present regular project status reports, highlighting progress, risks, dependencies, and key metrics. • Support pre-sales and growth initiatives by contributing to estimations, delivery strategies, and follow-on phase planning. • Partner with recruiting teams to support hiring and onboarding for specialized technical roles.




