Vattenfall

Role Description As our new Data Engineer, you will join a collaborative and skilled organization working with data at scale to enable automation of real business cases through data-driven and Machine Learning-based intelligence. We are hiring two Data Engineers for two closely collaborating teams with different focus areas: - The Data Science team, working directly with forecasting, analytics, and ML-based decision support. - The Data Streaming Platform (DSP) team, a newly formed team building and operating a central platform for real-time and event-driven data across the organization. We work in a Scrum setup and have in recent years been very successful in forecasting both electricity prices and electricity consumption, while at the same time scaling our real-time data capabilities. Your work will have direct and measurable impact. What will you do? - Design, build, and operate batch and real-time data pipelines using Azure, Databricks, and Python. - Ingest and process external and internal data such as weather data, market prices, grid telemetry, and event streams. - Work with streaming technologies including Kafka, Spark Structured Streaming, and Flink. - Ensure high data quality through validation, monitoring, and lineage. - Develop and improve data workflows, standards, and reliability in production systems. - Dependent on your area of expertise and interest, you will work in either the Data Science- or Data Streaming Platform team. Data Science team focus: - Enable forecasting, analytics, and ML use cases with high-quality, well-modelled data. - Collaborate closely with Data Scientists, ML Engineers, and other engineering teams. DSP team focus: - Build Kafka based central streaming data platform from scratch to enable traders, analysts and asset managers to publish and consume live and historical data at scale. - Operate and continuously enhance the components once the platform is in production. Location: Katowice or remotely in Poland Qualifications - Strong Python and SQL skills. - Practical experience with cloud data platforms (Azure, Databricks) and with different data storage solutions, including data lakes, lake houses and analytical databases. - Hands-on experience with streaming systems (Kafka, Spark or Flink). - Familiarity with Airflow, CI/CD, DevOps practices. - Experience with Docker, Kubernetes, and Infrastructure-as-Code. - Understanding of data contracts, time-series data, and event-driven systems. - Fluency in English and interest in the energy domain. Benefits - Good remuneration. - A challenging and international work environment. - The possibility to work with some of the best in the field. - Working in interdisciplinary teams with support from committed colleagues. - Attractive employment conditions and opportunities for personal and professional development.