A global IT engineering and consulting company specializing in custom software development.
Senior Data Engineer – Streaming, Big Data Platform
Location
Poland
Posted
11 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Streaming, Big Data Platform
Avenga
• Design, develop and maintain streaming data pipelines and transformations • Build and operate a big data platform used for real-time network observability and alerting • Ensure platform availability, reliability and performance • Design and implement monitoring solutions for data pipelines and platform components • Support platform evolution, upgrades and infrastructure improvements • Collaborate with internal stakeholders and translate business requirements into technical solutions • Troubleshoot complex platform and data processing issues • Create and maintain technical documentation and operational standards
Job Requirements
- 5+ years of experience in Data Engineering, Platform Engineering
- Strong experience with streaming data processing and distributed systems
- Experience with big data technologies and real-time data pipelines
- Hands-on experience with platform/infrastructure operations in Linux environments
- Experience with containers and modern infrastructure tooling
- Good understanding of monitoring, observability and platform reliability concepts
- Ability to work independently and take ownership of technical solutions
- Strong problem-solving and troubleshooting skills
- Very good English communication skills.
- Experience in telecom or network-related environments (nice-to-have)
- Experience with Kafka, Spark Streaming, Flink, Elasticsearch/OpenSearch, ClickHouse or similar (nice-to-have)
- Experience with Kubernetes and cloud/platform infrastructure (nice-to-have)
- Familiarity with GDPR, security and compliance requirements in enterprise environments (nice-to-have)
Benefits
- Equal opportunities in recruitment
- Career development
- Leadership support
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Lead, mentor, and develop a high-performing DataOps team • Set technical direction, priorities, and operating standards for DataOps • Own the operational health, performance, and availability of CloudPay’s data platforms • Design and implement monitoring, alerting, and incident management for data pipelines • Implement robust CI/CD processes for data pipelines and analytics assets • Ensure strong data access controls, auditability, and regulatory compliance (e.g., GDPR) • Act as a bridge between Data Engineering, Analytics, Product, and Operations
• Assist in identifying and resolving QA bugs related to data pipelines and processes • Validate data accuracy, completeness, and consistency • Participate in designing, building, and maintaining data pipelines • Implement data transformations, data cleansing, and ETL processes • Collaborate with senior engineers to enhance pipeline efficiency • Take ownership of small features within the data pipeline • Write code to handle data ingestion, processing, and storage • Lead a small independent project related to data engineering • Choose a topic of interest (e.g., optimizing pipeline performance, automating data validation) • Present your findings and recommendations to the team
Senior Data Engineer
Vantage Risk CompaniesVantage Group Holdings Ltd. (Vantage) was established in late 2020 as a re/insurance partner designed for the future. Driven by relentless curiosity, our team of trusted experts provides a fresh perspective on our clients’ risks. We add creativity to tech-enabled efficiency and robust analytics to address risks others avoid. Vantage provides specialty re/insurance through its operating subsidiaries in Bermuda and the U.S. Approximately 365 colleagues in both the United States and Bermuda. Offices in Chicago, IL, Norwalk, CT, Arlington, VA, Boston, MA, New York, NY, Atlanta, GA and Hamilton, Bermuda. Highly geographically diverse workforce with colleagues based in 35 states and counting. Fully support work flexibility including remote and hybrid work arrangements.
Role Description At Vantage, the Data Engineer is mainly responsible for ingesting and transforming replicated source data into data products that are easily consumed by the Reporting team and other downstream users. The role requires an expertise in Data Modeling and Data Pipeline development within an Analytical, column-oriented data warehouse. This role reports to our Data Engineering Lead and is a remote opportunity/hybrid opportunity where the candidate is in a commutable distance to one of Vantage’s office locations, with a preferred location in New York or Chicago. The base salary expectation for this role is between $120,000 - $160,000. Additionally, Vantage offers its colleagues performance-based bonus potential, strong health & welfare benefits, retirement plans with company match, competitive time off plans, a highly flexible work environment, and much more. Responsibilities & Accountabilities - Translate requirements and data mapping documents into a technical design. - SQL development using dbt. - Pipeline development using dagster. - Talent for identifying the specific records that are produced from SQL bugs. - Debug and troubleshoot issues found during testing or production. - Communicate project status, issues, and blockers with the team. - Contribute to continuous improvement by identifying and addressing opportunities. Qualifications - Bachelor’s degree in computer science or equivalent professional experience required. - Minimum of 5 years of experience in ELT development within a Data Warehouse. - Minimum of 5 years working with Dimensional Data Modeling, such as Kimball, Data Vault, or Medallion Architecture. - Strong familiarity with dbt and dagster/airflow. - Good working knowledge of distributed data and Kubernetes. - Expertise in data modeling using append-only replication methods to produce point-in-time data marts. - Strong understanding of test-driven development applied to Enterprise Data. - Strong familiarity with git and CI best practices. - Expertise in Python and SQL. - Expertise in column-oriented SQL modeling. - Experience in P&C Insurance or Financial Services Industry is a plus. - Ability to process and adapt to change in a rapid growth company environment. Our Vantage Values - Do What’s Right – be a force for good, for our customers, shareholders, colleagues, industry and communities. - Be Relentlessly Curious – ask questions, listen to understand, challenge the status quo, look for ways to say YES, drive innovation, be opportunistic and disciplined. - We’re Better Together – collaborate, be inclusive, supportive, respectful and humble, be a trusted partner internally and externally. - Be Transparent – in who you are, what you think, and how you work, seek and provide alternative points of view, be true to yourself. - Make A Difference – focus on impact, embrace technology, leverage data and analytics as critical assets, act intentionally to deliver on our mission.
Role Description This role is for you if you enjoy building real data solutions end-to-end — not just one piece of the puzzle. At NIVA Health, we’re growing our data capability and looking for a Full-Stack Data Engineer who’s comfortable working across the full data lifecycle: - Pulling data in, cleaning it up, building pipelines — all the way through to dashboards and usable insights. - Working on the back end and the front end of data, alongside a collaborative team, solving practical problems that impact healthcare operations every day. What you’ll be working on: - Designing, building, and maintaining data solutions that power reporting and decision-making across the business. - Building and maintaining data pipelines using Google Cloud Platform (BigQuery, Cloud Functions, Cloud Composer, Cloud Scheduler). - Cleaning, transforming, and organizing data from multiple sources (APIs, spreadsheets, internal systems). - Automating ETL / ELT workflows to improve reliability and efficiency. - Writing Python (and some Bash) scripts to support data processing and internal tools. - Building and maintaining dashboards and KPI reports using Looker Studio (and supporting data visualization needs). - Preparing datasets for simple predictive or forecasting use cases as the team evolves. This is a hands-on role — you’ll be writing code, fixing issues, improving pipelines, and seeing your work used by real teams. Qualifications - 2+ years’ experience in data engineering, analytics engineering, data science, or software engineering. - Comfortable working with GCP, especially BigQuery. - Confident in using Python for data processing and automation. - Solid SQL skills and understanding of data modeling basics. - Experience building or maintaining data pipelines (batch or streaming). - Experience with dashboards or BI tools (Looker / Looker Studio preferred). - Enjoy working across both technical backend tasks and user-facing reporting. Nice to have - Experience with Apache Airflow / Cloud Composer. - Exposure to Apache Beam. - Familiarity with Vertex AI, AutoML, or basic ML workflows. - Experience supporting operational or healthcare data. Salary $1500 - $1800. Final offer will depend on experience and technical depth.



