Job Closed
This listing is no longer active.
Data Engineer | SR AWS / Spark
Location
Worldwide
Posted
32 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer | SR AWS / Spark
Compass UOL
Role Description - Desenvolver e otimizar pipelines de dados em larga escala com foco em desempenho e confiabilidade; - Atuar na construção de soluções de streaming e batch para ingestão, transformação e consumo de dados; - Colaborar com arquitetos e desenvolvedores na definição de padrões técnicos e boas práticas; - Garantir observabilidade, versionamento e qualidade dos dados em produção; - Apoiar iniciativas de modernização e automação de processos dentro da tribo; Qualifications - Experiência sólida como Engenheiro(a) de Dados Sênior; - Proficiência em Scala, Spark (Streaming) e Python; - Conhecimento prático em arquiteturas distribuídas e microsserviços; - Vivência em processamento de dados em nuvem (AWS); - Capacidade analítica e pensamento voltado à eficiência e escalabilidade; Requirements - Experiência com ScyllaDB (ou Apache Cassandra); - Conhecimento em Kafka; - Vivência com serviços AWS como Lambda, SNS, SQS e Glue; - Experiência em ambientes de dados voltados a produtos digitais e times;
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Product Manager – Data Platform
ENSEKThe award-winning SaaS platform behind leading energy suppliers.
• Own the strategy, vision, and roadmap for the PDL as a governed data product — engaging external and internal consumers, the Analytics Chapter, and engineering leadership to define, validate, and continuously iterate the direction. • Own the commercial narrative for the PDL as an external product offer — defining what ENSEK’s data capability means to an energy retailer’s analytical team, how it is differentiated, and working with commercial partners to validate positioning and identify the investment required to productise it at scale. • Make evidence-based prioritisation decisions, estimating the effort and value of roadmap items, using AI-powered estimation tools to improve accuracy, and discussing trade-offs transparently with stakeholders. • Define and track key product outcomes — implementing dashboards for real-time performance visibility, and using data to drive continuous improvement in quality and user experience. • Represent users confidently in internal discussions, advocating for research across all user types and leveraging AI-powered tools to accelerate insight generation at scale. • Advocate for product-led thinking within your team, consult with stakeholders to protect the team from unnecessary directional changes, and champion AI tool adoption. • Build long-term stakeholder relationships, implementing communications strategies and influencing effectively to drive alignment and outcomes.
Staff Engineer – Data Platform
YunoYuno enables any company to manage all the payment methods and fraud providers through a single integration.
• Design and build data pipelines for large volumes of payment data that are performant, reliable, and correct — not just fast. • Own end-to-end data flows: from ingestion and transformation through to the outputs that Finance, Product, and clients depend on. • Drive data quality across your domain with tooling. • Work cross-functionally with Product, Finance and enable other Engineering teams via a 'consulting' style model. • Contribute to how the team works — code review culture, CI/CD standards, ADRs, how we handle incidents — we're building these practices now and senior engineers shape them. • Help onboard and level up engineers around you; there's real opportunity to make an impact here.
• Design and implement scalable data architecture for targeting and engagement • Serve as technical partner to program teams • Extend our foundational datasets to facilitate new outreach strategies • Load-test our systems to prepare for elections • Maintain key datasets for polling images and precinct cross-matching • Develop standardized experimental frameworks for testing campaign effectiveness • Build AI tooling that makes our own team faster — QC agents for data deliveries, the evals to trust them, and context systems that keep our repos self-documenting • Shape the pipelines and integrations feeding our RCT platform and our foundational voter datasets: the APIs, replication, federation layers, experiment randomization, and the failure modes that come with running data systems for hundreds of partner organizations on a non-negotiable political calendar • Extend our data platform and orchestration layer, and optimize it for cost, runtime, and reliability so it holds up during the elections, with millions of outreach attempts • Lead projects that span multiple systems and teams — scoping, designing, and driving them to real milestones • Hold the bar on quality — test and validate your own work, review teammates' code, and mentor through that review • Document and communicate — make your systems understandable to teammates, stakeholders, and non-engineers • Rotate through on-call and rapid response, including evenings and weekends during high-stakes moments like election cycles
Senior Data Engineer – Integrations, Data Platform
Sigma Software GroupWe support enterprises, product houses, and startups with custom software solutions development and IT consulting.
• Design, build, and maintain reliable data pipelines to ingest and process data from external systems • Implement integrations using Java or Scala, ensuring scalability and resilience • Handle both full data loads and incremental updates efficiently • Collaborate with engineering and product teams to align on integration and data needs • Develop and maintain a unified internal data model • Ensure data quality, detect and resolve incorrect or incomplete data • Utilize AI-assisted tools to improve data quality and operational efficiency • Monitor and optimize data pipelines for production readiness



