Leega logo
Leega

Inteligência, Inovação e Tecnologia.

Senior Data Engineer – GCP, DBT

Data EngineerData EngineerFull TimeRemoteSeniorTeam 201-500Since 2010H1B No SponsorCompany SiteLinkedIn

Location

Brazil

Posted

73 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer – GCP, DBT

Leega

• Analysis and Planning of Loads/Pipelines: • Assess the data warehouse architecture and requirements. • Map data, transformations and processes in GCP services (Cloud Storage, BigQuery, Dataproc). • Define data migration strategy (full load, incremental, CDC). • Develop a data architecture plan on GCP. • Design and Data Modeling on GCP: • Design table schemas in BigQuery, considering performance, cost and scalability. • Define partitioning and clustering strategies for BigQuery. • Model data zones in Cloud Storage (Bronze, Silver and Gold). • ELT/ETL Pipeline Development: • Create data transformation routines using Dataproc (Spark) or Dataflow to load data into BigQuery. • Translate business logic and existing transformations into GCP. • Implement data validation and data quality mechanisms. • Infrastructure Provisioning and Management: • Use IaC tools (Terraform) to provision and manage GCP resources (BigQuery datasets/tables, Cloud Storage buckets, Dataproc clusters). • Configure and optimize Dataproc clusters for different workloads. • Manage networking, security (IAM) and access in GCP. • Performance and Cost Optimization: • Optimize queries in BigQuery to reduce costs and improve performance. • Tune and optimize Spark jobs on Dataproc. • Monitor and optimize GCP resource usage to control costs. • Data Security and Governance: • Implement and ensure data security in transit and at rest. • Define and apply IAM policies to control access to data and resources. • Ensure compliance with data governance policies. • Monitoring and Support: • Troubleshoot performance and functional issues of data pipelines and GCP resources. • Documentation: • Document the architecture, data pipelines, data models and operational procedures. • Communication: • Communicate effectively with team members, stakeholders and other areas of the company. • Ensure clear communication between architecture definitions and software components, and the evolution and quality of the team's deliverables. • Jira / Agile Methodologies: • Be familiar with agile methodologies, their ceremonies, and be proficient with the Jira tool.

Job Requirements

  • Proven experience with DBT for at least 3 years.
  • Mastery of: models (staging, intermediate, marts)
  • ref() and source()
  • macros (Jinja)
  • seeds and snapshots
  • tests (not null, unique, custom)
  • Layered organization:
  • Staging → Transform → Mart (Data Warehouse)
  • Google Cloud Platform (GCP):
  • BigQuery: Deep knowledge in data modeling, query optimization, partitioning, clustering, data loading (streaming and batch), security and data governance.
  • Cloud Storage: Experience managing buckets, storage classes, lifecycle policies, access control (IAM) and data security.
  • Dataproc: Skill in provisioning, configuring and managing Spark/Hadoop clusters, job optimization, and integration with other GCP services.
  • Dataflow/Composer/DBT: Knowledge of orchestration and data processing tools for ELT/ETL pipelines.
  • Cloud IAM (Identity and Access Management): Implementation of security policies and fine-grained access control.
  • VPC, Networking and Security: Understanding of networks, subnets, firewall rules and cloud security best practices.
  • Programming Languages:
  • Python and PySpark: Essential for automation scripts, data pipeline development and integration with GCP APIs.
  • SQL (advanced): For BigQuery, DBT and data transformations.
  • Shell Scripting: For automation of routine tasks.
  • Version Control:
  • Git/GitHub/Bitbucket.

Benefits

  • 🏥 Porto Seguro Health Plan
  • 🦷 Porto Seguro Dental Plan
  • 💰 Profit Sharing (PLR)
  • 👶 Childcare Assistance
  • 🍽️ Alelo Meal and Food Vouchers
  • 💻 Home Office Allowance
  • 📚 Partnerships with Educational Institutions
  • 🚀 Support for Certifications, including Cloud
  • 🎁 Livelo Points
  • 🏋️‍♂️ TotalPass
  • 🧘‍♂️ Mindself

Related Categories

Related Job Pages

More Data Engineer Jobs

Full TimeRemoteTeam 10,001+H1B Sponsor

• Conduzir apresentações técnicas, workshops e treinamentos sobre soluções de dados na GCP; • Desenvolver e executar provas de conceito (PoCs) e demonstrações hands-on; • Apoiar clientes na construção de arquiteturas modernas de dados (data platforms); • Atuar na disseminação de soluções de Data Analytics, Machine Learning e AI; • Trabalhar em conjunto com Customer Engineers e Account Executives na qualificação de oportunidades; • Apoiar estratégias de adoção de produtos de dados da GCP; • Criar conteúdos técnicos e materiais de treinamento; • Interagir com clientes e parceiros em toda a América Latina; • Promover boas práticas em engenharia e arquitetura de dados na nuvem;

Brazil
Job Closed
Zipdev logo

Senior Data Engineer

Zipdev

Remote technical staffing from Latin America.

Data Engineer73 days ago
Full TimeRemoteTeam 51-200Since 2017H1B No Sponsor

• Architect and optimize the **Snowflake data platform**, including warehouse sizing, cost optimization, storage strategy, and access controls • Design and own **dbt project structure**, including models, macros, testing, documentation, and scalable data contracts • Build and maintain **ELT pipelines** using Fivetran and orchestration tools, ensuring reliable data ingestion across multiple sources • Implement and manage **data quality and observability frameworks** (tests, SLAs, lineage, monitoring, incident response) • Translate **business requirements into scalable data models and reusable datasets** • Partner with **Analytics, Product, and Marketing teams** to deliver high-quality, self-service data solutions • Establish and enforce **data modeling standards** (dimensional and ER models) • Optimize **query performance and warehouse costs** in Snowflake, providing insights to stakeholders • Define and enforce **data governance policies**, including RBAC, masking, and PII handling • Own end-to-end delivery of **complex data initiatives**, from design to production • Participate in **code reviews and technical design discussions**, raising engineering standards • Identify and reduce **technical debt** across pipelines, models, and infrastructure

Brazil
Job Closed
Waylin Partners logo

Manager – Data Engineering

Waylin Partners

Waylin Partners streamlines FP&A, Data Analytics, and Technical Accounting for private equity portfolio companies.

Data Engineer73 days ago
Full TimeRemoteTeam 11-50Since 2022H1B No Sponsor

• Design, build, and optimize data warehouses, pipelines, and ETL/ELT processes to support enterprise reporting and analytics • Translate ambiguous business questions into scalable, data-driven solutions that help leaders make informed decisions • Develop automated reporting and analytics workflows, ensuring data accuracy, consistency, and timeliness • Build and maintain dashboards and visualization tools that synthesize complex data for finance and operational leadership • Partner with finance, accounting, and operational teams across portfolio companies to understand data needs and implement effective solutions • Implement best practices for data governance, data quality, and system architecture • Support integration efforts during acquisitions, system migrations, and new tool onboarding

United States
Job Closed
Waylin Partners logo

Director – Data Engineering

Waylin Partners

Waylin Partners streamlines FP&A, Data Analytics, and Technical Accounting for private equity portfolio companies.

Data Engineer73 days ago
Full TimeRemoteTeam 11-50Since 2022H1B No Sponsor

• Design, build, and optimize data warehouses, pipelines, and ETL/ELT processes to support enterprise reporting and analytics • Translate ambiguous business questions into scalable, data-driven solutions that help leaders make informed decisions • Develop automated reporting and analytics workflows, ensuring data accuracy, consistency, and timeliness • Build and maintain dashboards and visualization tools that synthesize complex data for finance and operational leadership • Partner with finance, accounting, and operational teams across portfolio companies to understand data needs and implement effective solutions • Implement best practices for data governance, data quality, and system architecture • Support integration efforts during acquisitions, system migrations, and new tool onboarding • Designing and deploying a new cloud-based data warehouse (Snowflake, Redshift, BigQuery, etc.) • Optimizing an existing warehouse or data architecture to improve performance and reduce cost • Developing enterprise-grade dashboards (Power BI, Tableau, Looker, etc.) for monthly reporting, KPIs, and operational visibility • Automating reporting processes previously done in Excel or manual systems • Building financial and operational data models used for forecasting, profitability analysis, or executive dashboards • Supporting process improvement and system enhancement initiatives across finance, accounting, and operations • Managing ad-hoc technical projects such as API integrations, data migrations, or master data restructuring

United States
Job Closed