Senior Data Engineer (m/w/d)
Location
Germany + 2 moreAll locations: Germany | Italy | Spain
Posted
47 days ago
Salary
€60K - €80K / year
Seniority
Senior
Job Description
Senior Data Engineer (m/w/d)
QuoIntelligence
The Opportunity You'll own the full data stack for cybersecurity intelligence at QuoIntelligence, building and evolving the pipelines that power our Mercury platform and analyst workflows. This is a role where your architectural decisions have real, visible impact, and where you'll collaborate closely with a small, expert team of two engineers on the data side. Our platform runs on DigitalOcean with a custom-built infrastructure: ZMQ message queues, Docker containers, hand-rolled deployment pipelines, and in-house-maintained Python libraries. It was built before today's managed services existed, and it works because engineers who genuinely understood distributed systems built it from the ground up. The first major project is leading our migration to AWS, transitioning from this custom infrastructure to managed services, without disrupting production. It's the kind of challenge that requires both depth and adaptability: keeping what exists healthy while architecting what comes next. If you thrive in that kind of dual mandate, this role was designed for you. You'll Thrive Here If... You enjoy working across the full stack rather than specializing deeply in one layer. You're energised by inheriting a well-built yet unconventional system and improving it. You're comfortable with ambiguity and find it motivating rather than frustrating. And you want your work to matter not just to a ticket queue, but to the analysts and products that rely on it every day. This is probably not the right fit if your data engineering experience is entirely on managed platforms like Databricks, Snowflake, or BigQuery, or if you're looking for a fully provisioned cloud environment from day one. The AWS migration is the destination you'll help us get there. One more thing: this is an AI-native company. Our products run on AI. We expect engineering to run on it, too. If your relationship with AI stops at asking ChatGPT to explain error messages, this isn't the right fit. What You'll Do - Build and design data pipelines for ingestion, processing, and modeling of cybersecurity intelligence that feeds Mercury and analyst workflows - Partner with Threat Intelligence Engineers on data access patterns, tool integration, and evolving data sources as priorities shift - Shape the team's technical direction: with two engineers, your judgment carries weight - Keep the existing DigitalOcean platform running: manage containers, handle library updates, and debug custom services with confidence - Lead the AWS migration: plan and execute the transition to managed services, with measurable benchmarks at every phase AI-First in Data Engineering AI is an operating principle here. We use AI tools as a core part of how we work. On a lean team maintaining hand-built infrastructure, they're what let us move at the pace of a much larger engineering org. AI as an operating system. You use Copilot, Claude, Cursor (or equivalents) daily to navigate Go services you didn't write, debug custom ZMQ queues, and generate Docker configurations as part of your daily workflow. Evaluate, Integrate, Repeat. Not every AI-generated suggestion belongs in a pipeline processing cybersecurity intelligence data. You test tools against QI's actual problems (migrating custom services to AWS, parsing threat feed data, maintaining manually versioned Python libraries) and drop what doesn't hold up in production. Define success first. Every pipeline feeding Mercury has defined criteria before it ships: ingestion throughput, data freshness, and error rates. We run the AWS migration on measurable benchmarks at each phase, not just "it works on staging." AI accelerates the testing loop, but the loop needs a target. What You'll Bring Must-haves: - Python as your primary language for data engineering work - A solid foundation in pipeline design, you can reason about data from ingestion through modeling - Docker fluency: deploying, debugging, and managing containerized services is routine for you - Experience building custom solutions when no off-the-shelf solution fits, writing the infrastructure yourself, not just assembling managed services - Cloud platform experience (AWS preferred, GCP, or Azure also relevant): enough to architect the target state of a managed-services migration - Microservices architecture or distributed systems: you've designed or maintained service-oriented systems - Working familiarity with Go: you'll read, debug, and modify Go code in some production services. Side projects and CLI tools count; we expect a ramp-up period You'll also need professional proficiency in English (the team works across countries). Nice-to-Haves: - Redis or ZMQ experience for message queuing - Legacy system maintenance: keeping aging infrastructure healthy while building its replacement - Cybersecurity or threat intelligence background Recruitment Process We aim to be as transparent as possible throughout the process and will share updates with you whenever we have progress on our end. To manage your expectations transparently, we have structured the recruitment process as follows: 1. Recruiter Screen 2. Take-Home Assignment 3. Technical Interview 4. Culture Add Interview 5. Offer & Background Check We welcome applications regardless of gender, nationality, ethnic origin, religion, disability, age, or sexual identity. Diversity is key to producing high-quality intelligence.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Clinical Data Coordinator - Bangalore, India ICON plc is a world-leading healthcare intelligence and clinical research organization. We’re proud to foster an inclusive environment driving innovation and excellence, and we welcome you to join us on our mission to shape the future of clinical development We are currently seeking a Clinical Data Coordinator II to join our diverse and dynamic team. As a Clinical Data Coordinator II at ICON, you will play a pivotal role in designing and analyzing clinical trials, interpreting complex medical data, and contributing to the advancement of inNvative treatments and therapies. What you will be doing - Assist Data Management Study Lead in maintenance of eCRF, Data Validation Specifications, and Study Specific Procedures. - Manage clinical and third-party data reconciliation based on edit specifications and data review plans. - Tracks and communicates status of study and task metrics to clinical data scientist, project team, and functional management. - Address data related questions and recommend potential solutions. - Identify root cause to systematically resolve data issues. Your profile - Bachelor's degree in a relevant field, such as Life Sciences or Healthcare. - Intermediate kNwledge of clinical data management within the pharmaceutical or biotechNlogy industry. - Familiarity with data management software and systems (e.g., Medidata, Oracle RDC, or similar). - Strong attention to detail and the ability to work effectively in a fast-paced environment. - Excellent communication skills and the ability to collaborate with cross-functional teams. - KNwledge of regulatory guidelines and standards (e.g., ICH-GCP) is a plus. What ICON can offer you: Our success depends on the quality of our people. That’s why we’ve made it a priority to build a diverse culture that rewards high performance and nurtures talent. In addition to your competitive salary, ICON offers a range of additional benefits. Our benefits are designed to be competitive within each country and are focused on well-being and work life balance opportunities for you and your family. Our benefits examples include: - Various annual leave entitlements - A range of health insurance offerings to suit you and your family’s needs. - Competitive retirement planning offerings to maximize savings and plan with confidence for the years ahead. - Global Employee Assistance Programme, TELUS Health, offering 24-hour access to a global network of over 80,000 independent specialised professionals who are there to support you and your family’s well-being. - Life assurance - Flexible country-specific optional benefits, including childcare vouchers, bike purchase schemes, discounted gym memberships, subsidised travel passes, health assessments, among others. Visit our careers site to read more about the benefits ICON offers. At ICON, inclusion & belonging are fundamental to our culture and values. We’re dedicated to providing an inclusive and accessible environment for all candidates. ICON is committed to providing a workplace free of discrimination and harassment. All qualified applicants will receive equal consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. If, because of a medical condition or disability, you need a reasonable accommodation for any part of the application process, or in order to perform the essential functions of a position, please let us know or submit a request here. Interested in the role, but unsure if you meet all of the requirements? We would encourage you to apply regardless – there’s every chance you’re exactly what we’re looking for here at ICON whether it is for this or other roles. Are you a current ICON Employee? Please click here to apply
• Design and deliver high-quality, scalable data products that provide a unified source of truth across a complex ecosystem of 60+ global sub-brands • Build and optimize robust ETL/ELT pipelines to ingest and transform massive volumes of structured and unstructured data • Lead the design of secure, cost-efficient data architectures • Implement rigorous data governance frameworks, including lineage tracking, metadata management, and quality controls • Act as a strategic 'sparring partner' for Analytics, ML, and AI teams • Manage and tune data platforms for peak performance • Mentor junior members and foster a culture of technical excellence
• Build and maintain high-throughput data pipelines for batch and streaming workloads. • Co-design data models and validate them with backend teams during the kickoff phase. • Ensure sustainable architecture and data integrity and educate other stakeholders throughout Yubo on those challenges. • Work closely with ML engineers to provide clean, ML-ready datasets. • Ensure data quality, availability, and order of entry at volumes of 300k+ db entries per second, and engineer ways to scale that even further. • Document pipelines, transformations, and architecture clearly for future devs and your future self (you’ll thank yourself later)
Senior Data Engineer
JUMOYou will be based in South Africa or Uganda. We operate a remote first working approach where working remotely is our default way of working. We have co-working spaces available in Cape Town and Kampala, for collaboration and connection and for the use of those who value and want to work out of an office. You have flexibility where to work from, as long as you have access to a reliable connection and are set up to work remotely. At JUMO, we believe that diversity strengthens our teams and we strive in our recruitment process to create an environment where people from every background can collaborate and prosper and be themselves.
About this job At JUMO, we’ve built a smart financial services platform to make finance accessible to everyone. Our Data engineering team ensures that our intelligent banking platform is able to acquire data from millions of users across globally distributed sources, such as GSM. Beyond ingress, the data is processed and stored in the cloud to support event-driven analytics and machine learning pipelines. As a JUMO Senior Data Engineer, you will design, build and maintain scalable data architectures. You will monitor their performance, perform root cause analysis and provide solutions to any issues that might arise. You will work with large, complex data sets, which means experience working with Big Data is essential. On a typical day, we use Spark, Kafka, Python, SQL, Airflow and AWS technologies, such as Redshift. We work with Docker and Kubernetes for automating deployment, scaling, and management of containerized applications. You will - Be responsible for creating robust, mission critical batch and streaming data processing capabilities - Design, implement, and maintain the data pipelines that constitute JUMO's data platform, enabling effective use of data across the organisation - Provide feedback on team members' output, encouraging skills development within the team - Work closely with Portfolio Managers and Decision Scientists to understand the real world problems we’re trying to solve - Mentor juniors with technical leadership - Be supported by senior leaders as you drive your own development You will need - 5 - 8 years of experience in a similar role with a BSc. in Computer Science, Electrical Engineering or equivalent tertiary degree - Real-world understanding & experience of data pipeline design and development, as well as data processing and storage (i.e. experience in designing systems to process and curate large data sets) - Experience in streaming technologies, specifically Spark and Kafka - Experience with cloud technologies, AWS preferred - Solid, proven experience working with big-data technologies such as Apache Spark, Flink, Hadoop, Kafka or Kinesis, Cassandra, DynamoDB, InfluxDB, MongoDB, Presto, Beam - Command of productionising and monitoring of data pipeline workflows, and working knowledge of the Data Product Lifecycle - Experience in application design and development with at least one of the following languages: Python (preferred), Scala, Java - RDBMS experience in any relevant technology such as MySQL, PostgreSQL, Redshift and SQL Server. This includes experience with relational database administration, technical architectures and infrastructure components - Understanding of CI/CD practices - Productive within a Linux command line environment - Proven ability to contribute software as part of a team. We are looking for someone that can effectively communicate technical concepts, and apply critical thinking under pressure Bonus if you have - Experience working with messaging systems (RabbitMQ, SNS) - Experience working with data pipeline orchestration (Airflow, Nifi, Streamsets) - Experience working with production BI environments and tools (Tableau, Superset, Looker) We ask a lot of each other at JUMO, but we give a lot too. You will love - Collaborating with smart, engaging people in an inspiring work environment - Working for impact - Growing and learning continuously, with loads of encouragement and support - Boldly taking risks as we navigate new challenges - Flexible work practices enabling your best delivery - Being autonomous and empowered to lead - A stack of leading-edge technologies Remote First This vacancy will be based in either South Africa or Portugal. We operate a remote first working approach, where working remotely is our default way of working. Our environment is designed to foster innovation and enable collaboration. You have flexibility where to work from in the country you are hired, as long as you are set up to work remotely and have access to data with a strong and reliable connection as we value online facetime for collaboration at JUMO. Diversity and Inclusion At JUMO, we firmly believe that diversity strengthens our teams. We are dedicated to fostering an inclusive recruitment process that cultivates an environment where all individuals can be authentic, collaborate, and thrive.
