Soluções inteligentes em comunicação 360
Data Engineer
Location
Brazil
Posted
11 days ago
Salary
$2.5K / month
Seniority
Senior
Job Description
Data Engineer
Fertil Comunicacao e Marketing Ltda.
• Design, build and evolve the company’s data platform architecture. • Develop and maintain ETL/ELT pipelines, including API extraction, transformations, incremental loads and CDC (Change Data Capture). • Model data for Data Warehouse, Data Lake and Data Marts, applying dimensional modeling (Star Schema, Snowflake Schema). • Implement batch and streaming processing with event-driven architecture and asynchronous processing. • Ensure database performance and reliability: relational modeling, indexes, partitioning, query optimization and replication. • Establish Data Quality, Data Lineage, Data Catalog and Data Governance practices. • Orchestrate pipelines and handle historical data with versioning and traceability. • Implement pipeline observability: logs, metrics, monitoring and alerts. • Provide reliable and near real-time data for dashboards, operational analytics and business intelligence. • Prepare the data foundation for future Machine Learning initiatives. • Automate processes and deployments using GitHub Actions and Docker. • Collaborate with product, engineering and business teams to define requirements and KPIs.
Job Requirements
- Languages: Python and advanced SQL.
- Databases: PostgreSQL and MySQL; relational modeling, indexes, partitioning, query optimization, replication and performance tuning.
- Data Modeling: Data Warehouse, Data Lake, Star Schema, Snowflake Schema, Data Mart, dimensional modeling, normalization and denormalization.
- ETL / ELT: building pipelines, API extraction, data transformation, incremental load and CDC (Change Data Capture).
- Data Processing: batch processing, streaming, asynchronous processing and Event-Driven Architecture.
- AWS: S3, Lambda, SQS, RDS, CloudWatch and IAM.
- Messaging: Amazon SQS.
- Version control: Git and GitHub.
- Observability: logs, metrics, pipeline monitoring and alerts.
- Automation: GitHub Actions.
- Containers: Docker.
- Data Engineering: Data Quality, Data Lineage, Data Catalog, Data Governance, data versioning, pipeline orchestration and historical data handling.
Benefits
- Flexible working hours
- Career progression plan
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, develop and maintain scalable data pipelines to ingest, process and transform large volumes of structured and unstructured data from diverse sources. • Implement efficient processes to ensure accurate and timely delivery of data to downstream systems and applications. • Ensure data quality, security and governance across the entire pipeline.
Lead GCP Data Architect
Naveera Technology LLCEngineering Production-Ready Data, AI & Cloud Platforms - Scalable, Secure, and Built for Enterprise Growth.
• Collaborate with Product Managers, Business Analysts, DevOps teams, Engineering teams, PMO. • Deliver scalable enterprise cloud solutions. • Provide architectural options and technical recommendations. • Conduct cost-benefit analysis and risk assessments. • Lead the modernization of data platforms, ETL pipelines, and analytics ecosystems.
• Buscamos un Data Engineer con experiencia en el diseño, construcción y optimización de soluciones de datos escalables • Será responsable de habilitar la integración, procesamiento y disponibilidad de información para iniciativas analíticas, de inteligencia de negocios y ciencia de datos • Colaborar estrechamente con equipos multidisciplinarios y áreas de negocio.
Senior Data Engineer
FederatoWhen underwriters have real-time risk selection and portfolio insights at their fingertips, profitable growth follows!
• Collaborate with Data Science, Product Managers and Software Engineers to build robust ETL pipelines that enable the Product Support team to deliver compelling user-facing features • Contribute to architecture decisions, observability tooling, and data quality initiatives that keep our platform robust and maintainable. • Contribute to a scalable internal framework for managing prompt engineering pipelines and other AI workflows. • Enforce and elevate engineering best practices across the AI/ML org, including code quality, testing, and documentation.
