Job Closed

This listing is no longer active.

Cuculus GmbH logo
Cuculus GmbH

Affordable energy and water for everyone.

Senior Data Engineer – Integration Hub, Data Pipelines

Data EngineerData EngineerFull TimeRemoteSeniorTeam 51-200Since 2007H1B No SponsorCompany SiteLinkedIn

Location

India

Posted

125 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer – Integration Hub, Data Pipelines

Cuculus GmbH

• Design, build, and maintain robust ETL/ELT data pipelines for batch and streaming workloads. • Implement data ingestion and transformation workflows using Apache Airflow, Apache NiFi, Apache Spark, and Kafka. • Integrate data from multiple sources including REST APIs, files, relational databases, message queues, and external SaaS platforms. • Optimize pipelines for performance, scalability, reliability, and cost efficiency. • Develop and operate a centralized data integration hub that supports multiple upstream and downstream systems. • Build reusable, modular integration components and frameworks. • Ensure high availability, fault tolerance, and observability of data workflows. • Design and manage data warehouses, data lakes, and operational data stores using PostgreSQL and related technologies. • Implement appropriate data modeling strategies for analytical and operational use cases. • Manage schema evolution, metadata, and versioning. • Implement data validation, monitoring, and reconciliation mechanisms to ensure data accuracy and completeness. • Enforce data security best practices, access controls, and compliance with internal governance policies. • Establish logging, alerting, and auditability across pipelines. • Automate data workflows, deployments, and operational processes to support scale and reliability. • Monitor pipelines proactively and troubleshoot production issues. • Improve CI/CD practices for data engineering workflows. • Work closely with data scientists, analysts, backend engineers, and business stakeholders to understand data requirements. • Translate business needs into technical data solutions.

Job Requirements

  • 5+ years of hands-on experience as a Data Engineer or in a similar role.
  • Proven experience as an individual contributor on at least three end-to-end data engineering projects, from design to production.
  • Strong hands-on experience with: Apache Airflow / Dagster, Apache NiFi, Apache Spark, Apache Kafka, PostgreSQL.
  • Extensive experience integrating data from APIs, files, databases, and third-party systems.
  • Strong SQL skills and experience with data modeling.
  • Solid programming experience in Python and/or Java/Scala.
  • Experience with Linux environments and version control systems (Git).
  • Strong problem-solving, debugging, and performance-tuning skills.

Benefits

  • Professional development opportunities
  • Flexible working arrangements

Related Categories

Related Job Pages

More Data Engineer Jobs

OtherRemoteTeam 10,001+Since 1969H1B No Sponsor

• Design and develop conceptual, logical, and physical data models for various domains • Lead the development of data modeling standards, best practices, and guidelines • Develop end-to-end solution architectures for data-driven and AI-focused applications • Mentor and guide junior data modelers • Design data models to support enterprise and operational reporting • Collaborate with data scientists to develop features for machine learning models • Optimize data models for performance and scalability • Ensure data models comply with data governance policies and standards • Identify and define data quality checks and validation processes • Implement data quality monitoring and reporting mechanisms

California + 1 moreAll locations: California | Texas
Job Closed
Full TimeRemoteTeam 10,001+Since 1978H1B No Sponsor

• Análisis, diseño y desarrollo de soluciones de ingeniería de datos en entornos cloud. • Construcción y mantenimiento de pipelines de datos. • Tratamiento, integración y análisis de grandes volúmenes de información. • Optimización de procesos de datos en plataformas Microsoft Azure. • Data engineering y data analysis.

Spain
Smartbridge logo

Senior Azure Data Engineer

Smartbridge

Simplifying business transformation through thought leadership and innovation. Bring your digital agenda to reality.

Data Engineer126 days ago
OtherRemoteTeam 51-200H1B Sponsor

• Define the target-state Azure data architecture (ingestion, orchestration, storage zones, serving patterns), security/networking boundaries, cost/perf tradeoffs, and promotion strategy (Dev→Test→Prod). • Implement robust ELT/ETL with ADF/Synapse Pipelines (parameters, reusable templates, CI/CD). • Hands-on in T-SQL and Python/PySpark for transformations, utilities, and tests. • Physical/semantic modeling, partitioning, columnstore strategies, statistics management, query plan analysis, index design, concurrency & transaction isolation, workload management. • SLA/SLO definitions, Azure Monitor / Log Analytics / App Insights dashboards and alerts; error handling, retries/backoff, idempotency, CDC and schema drift strategies. • RBAC, Key Vault, managed identities, private endpoints/VNet, data masking patterns; document data contracts and access patterns. • Code reviews, PR discipline, mentoring, and crisp documentation/runbooks for client handoff.

United States
Job Closed
CARE logo

Senior Data Engineer

CARE

We work to fight poverty and achieve social justice by empowering women and girls. www.CARE.org

Data Engineer126 days ago
OtherRemoteTeam 5,001-10,000Since 1945H1B Sponsor

• Design and implement scalable, reliable, and efficient data pipelines to support clinical, operational, and business needs. • Develop and maintain architecture standards, reusable frameworks, and best practices across data engineering workflows. • Build automated systems for data ingestion, transformation, and orchestration leveraging cloud-native and open-source tools. • Optimize data storage and processing in data lakes and cloud data warehouses (Azure, Databricks). • Develop and monitor batch and streaming data processes to ensure data accuracy, consistency, and timeliness. • Maintain documentation and lineage tracking across datasets and pipelines to support transparency and governance. • Work cross-functionally with analysts, data scientists, software engineers, and business stakeholders to understand data requirements and deliver fit-for-purpose data solutions. • Review and refine work completed by other team members, ensuring quality and performance standards are met. • Provide technical mentorship to junior team members and collaborate with contractors and third-party vendors to extend engineering capacity. • Use Databricks, DBT, Azure Data Factory, and SQL to architect and deploy robust data engineering solutions. • Integrate APIs, structured/unstructured data sources, and third-party systems into centralized data platforms. • Evaluate and implement new technologies to enhance the scalability, observability, and automation of data operations. • Proactively suggest improvements to infrastructure, processes, and automation to improve system efficiency, reduce costs, and enhance performance.

United States
$120K - $160K / year