Sigma Software Group logo
Sigma Software Group

We support enterprises, product houses, and startups with custom software solutions development and IT consulting.

Senior Data Engineer – GovTech, Public Sector

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 2002H1B No SponsorCompany SiteLinkedIn

Location

Ukraine

Posted

66 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer – GovTech, Public Sector

Sigma Software Group

• Design and implement data ingestion pipelines for batch and streaming data • Configure and maintain data orchestration workflows (Airflow, NiFi) and CI/CD automation for data processes • Design and organize data layers within Data Lake architecture (HDFS, Iceberg, S3) • Build and maintain secure and governed data environments using Apache Ranger, Atlas, and SDX • Develop SQL queries and optimize performance for analytical workloads in Hive/Impala • Collaborate on data modeling for analytics and BI, ensuring clean schemas and dimensional models • Support machine learning workflows using Spark MLlib or Cloudera Machine Learning (CML)

Job Requirements

  • Proven experience in building and maintaining large-scale data pipelines (batch and streaming)
  • Strong knowledge of data engineering fundamentals: ETL/ELT, data governance, data warehousing, Medallion architecture
  • Strong SQL skills for Data Warehouse data serving
  • Minimum 3 years of experience in Python or Scala for data processing
  • Hands-on experience with Apache Spark, Kafka, Airflow, and distributed systems optimization
  • Experience with Apache Ranger and Atlas for security and metadata management
  • Upper-Intermediate English proficiency
  • BS in Computer Science or related field

Benefits

  • remote work options

Related Categories

Related Job Pages

More Data Engineer Jobs

AB InBev logo

Data Engineer

AB InBev

To a Future With More Cheers

Data Engineer66 days ago
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Implement ETL/ELT solutions and data integration between multiple systems and data sources. • Design, implement, and maintain data pipelines to ingest, store, and process large volumes of data. • Collaborate with other teams to ensure data security and compliance. • Design, develop, and implement solutions to optimize the performance and scalability of data processing systems.

Brazil
Airbnb logo

Staff Software Engineer, Data Engineering

Airbnb

Airbnb is a community based on connection and belonging.

Data Engineer66 days ago
Full TimeRemoteTeam 5,001-10,000Since 2007H1B Sponsor

• Architect and productionize batch and real-time data systems to support various products and business needs. • Ensure the quality, performance, and stability of data systems through robust quality systems and monitoring practices. • Design and optimize data models for efficient storage and retrieval to meet critical product and business requirements. • Collaborate with cross-functional teams, including product managers, engineers, data scientists, and business partners, to align on data requirements and develop scalable systems. • Tune, productionize, and optimize data systems and machine learning models to enhance their effectiveness and efficiency. • Build and maintain strong relationships with partner engineering teams, including backend, client, data science, and ML teams, to ensure seamless integration and support. • Contribute to the development of long-term data strategies and roadmaps, and influence the technical direction of data engineering practices within the organization. • Mentor and coach team members, providing guidance and support to enhance their skills and performance.

United States
$191K - $225K / year
Job Closed
Airbnb logo

Senior Software Engineer, Data Engineering

Airbnb

Airbnb is a community based on connection and belonging.

Data Engineer66 days ago
Full TimeRemoteTeam 5,001-10,000Since 2007H1B Sponsor

• Design, build, and maintain robust and efficient data pipelines that collect, process, and store data from various sources, including user interactions, listing details, and external data feeds. • Develop data models that enable the efficient analysis and manipulation of data for merchandising optimization. Ensure data quality, consistency, and accuracy. • Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executor framework • Collaborate with cross-functional teams, including Data Scientists, Product Managers, and Software Engineers, to define data requirements, and deliver data solutions that drive merchandising and sales improvements. • Contribute to the broader Data Engineering community at Airbnb to influence tooling and standards to improve culture and productivity • Improve code and data quality by leveraging and contributing to internal tools to automatically detect and mitigate issues

United States
$191K - $223K / year
Job Closed
XpertGroup logo

Ingeniero de datos, contabilidad y finanzas

XpertGroup

Acompañamos a nuestros clientes en su proceso de transformación digital a través de soluciones ágiles y de valor.

Data Engineer66 days ago
Full TimeRemoteTeam 51-200Since 2015H1B No Sponsor

• Desarrollar e Implementar Modelos Analíticos con Python, Flask o Django, Shell Linux, PostgreSQL o MySQL, SOLID. • Desarrollar modelos analíticos que involucren librerías de Python Avanzado como Numpy, Pandas, Scikit-Learn, Tensorflow, NLTK. • Extraer, procesar y consumir la información desde los repositorios de datos. • Crear el roadmap de los modelos analíticos que serán ejecutados. • Participar en el desarrollo de proyectos de analítica. • Trabajar con Lenguaje Fuente abierta. Python (pandas y matplot is), Open Refine remediación de datos normalización, SQL, algoritmos del blockchain Quicksha, códigos de sha 256 o sha 512. • Impulsar y mantener una arquitectura que permita asegurar datos de alta calidad, para que sean consumibles por los diferentes equipos. • Utilizar los datos para determinar qué tareas se pueden automatizar o emplear en modelos predictivos y prescriptivos. • Identificar cómo mejorar la calidad, eficiencia y confiabilidad de los datos. • Apoyar al equipo de trabajo del cliente y de XpertGroup en los temas asociados a la ingeniería de datos y que no se encuentren descritos en las funciones anteriores. • Generar estimaciones de proyectos y contribuir en la planificación y gestión de recursos necesarios para la ejecución exitosa del mismo. • Cumplir con las demás funciones que le sean asignadas afines a la descripción del cargo y de acuerdo con el nivel, naturaleza y área de desempeño.

Colombia
Job Closed