Job Closed
This listing is no longer active.
Inspiramos inovação, expiramos tecnologia
Engenheiro de Dados – Databricks
Location
Brazil
Posted
58 days ago
Salary
0
Seniority
Senior
Job Description
Engenheiro de Dados – Databricks
INDT - Instituto de Desenvolvimento Tecnológico
• Desenvolvimento e manutenção de pipelines de ingestão de dados • Implementação de cargas batch/full e incrementais • Integração de dados provenientes de sistemas legados e SAP Datasphere • Desenvolvimento de processos de carga na camada Raw/Bronze do Databricks • Apoio na sustentação, monitoração e evolução das demandas da área • Atuação em integração, tratamento técnico e disponibilização de dados • Construção e manutenção de pipelines utilizando PySpark e SQL • Garantia de rastreabilidade, governança e qualidade técnica das cargas
Job Requirements
- Experiência em engenharia de dados
- Experiência com Databricks
- Experiência com PySpark e Spark SQL
- Vivência com pipelines ETL/ELT e ingestão de dados
- Conhecimento em cargas incrementais e integrações de dados
- Conhecimento em Delta Lake
- Experiência com Git/versionamento
- Perfil com autonomia e capacidade de atuação em demandas contínuas do time
- Experiência com SAP Datasphere (diferencial)
- Experiência com SAP S/4HANA (diferencial)
- Conhecimento em DLT (Delta Live Tables) (diferencial)
Benefits
- Ambiente colaborativo
- Oportunidade de desenvolver seus talentos
- Liberdade para propor e desenvolver projetos inovadores
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Azure API, Data Engineer
HRinRecrutamento ágil e inteligente - Conectando talentos. Contato: alejandra.helen@hrin.com.br
• Evolve Azure API Management strategies • Support design of integration and data architecture • Develop and optimize data pipelines • Work with cloud environments (Azure/AWS) • Ensure governance, security, and scalability • Support automation initiatives, CI/CD, and technical documentation
• Design, build, and maintain scalable and secure cloud-native data platforms and data pipelines. • Lead the architecture, optimisation, and operational management of the Snowflake data warehouse platform. • Develop robust ELT/ETL pipelines to ingest, transform, and deliver high-quality data from multiple internal and external sources. • Build reusable and maintainable data frameworks, transformation models, and orchestration workflows. • Develop and maintain infrastructure-as-code and automation for data platform provisioning and management where appropriate. • Optimise performance, scalability, and cost efficiency across data storage, transformation, and query workloads. • Support near real-time and batch-based data processing requirements. • Own and continuously improve Snowflake architecture, performance tuning, security, governance, and operational best practices. • Design and optimise Snowflake schemas, warehouses, clustering strategies, and data sharing capabilities. • Monitor Snowflake usage, query performance, and cost consumption to drive optimisation initiatives. • Support data lifecycle management, retention, and governance policies within Snowflake. • Enable self-service analytics capabilities through well-structured semantic layers and governed datasets. • Collaborate closely with Product, Engineering, Operations, Finance, and Business stakeholders to deliver impactful data solutions.
Lead Data Engineer, Multiple Positions Available
Provident BankProvident Bank is a financial institution based in Jersey City, New Jersey, dedicated to simplifying customers' lives through innovative financial solutions, wi
• Responsible for the design, development, testing, and implementation of the new generation of data movement solutions for the Bank using state of the art technologies to optimize effectiveness of Provident Bank’s data warehouse. • Design, develop, test, and implement ETL/ELT data movement jobs based on business requirements to feed data into Provident’s data warehouse and the various downstream bank applications. • Provide technical guidance on data movement related issues and ensure optimal design for all aspects of the data warehouse. • Perform data analysis and sourcing to target mappings and verify accuracy of data within the Bank’s data warehouse. • Identify, design and implement process improvements (i.e. automate manual processes and optimize data delivery) to optimize the quality of data for the firm’s business intelligence team. • Implement, maintain, and extend best practices and assets in line with the market standards and guidelines to best ensure reliability of the Bank’s data.
Mid+ Data Engineer
Lean Solutions GroupLean Tech is a rapidly expanding organization situated in Medellín, Colombia. We pride ourselves on possessing one of the most influential networks within software development and IT services for the entertainment, financial, and logistics sectors. Our corporate projections offer many opportunities for professionals to elevate their careers and experience substantial growth. Joining our team means engaging with expansive engineering teams across Latin America and the United States, contributing to cutting-edge developments in multiple industries.
Role Description We are seeking a talented Analytics Engineer / Data Engineer to join our dynamic Data & Analytics team. This role focuses on building scalable data solutions that support business decision-making through advanced data integration, transformation, and visualization processes. The successful candidate will work on developing and optimizing ETL pipelines, leveraging Databricks, Python, SQL, and collaborating with cross-functional teams to drive innovative analytics initiatives. - Develop and optimize ETL pipelines using Databricks to support data integration and processing. - Collaborate with stakeholders across technical and business teams to gather requirements and design scalable data solutions. - Manage the data lifecycle, from ingestion into the data lake (bronze, silver, gold tiers) to preparation for visualization. - Implement and maintain data processing logic using SQL and Python. - Support advanced analytics and machine learning operations within the Databricks environment. - Build and enhance dashboards and reports, with Sigma Computing as a preferred platform (Qlik is not required). - Apply Agile methodologies to enhance collaboration and efficiency across the analytics team. Qualifications - BS degree in Computer Science or a related subject. - 3 years of experience in a Data Engineer, Analytics Engineer, or similar role. - Advanced proficiency in SQL and Python for data transformation and automation. - Experience working with PySpark. - Experience working with Databricks or other data lake platforms. - Familiarity with the medallion architecture (bronze, silver, gold tiers). - Strong analytical mindset with experience in advanced analytics techniques. - Intermediate experience with ETL pipeline development and machine learning operations. - Familiarity with Agile methodologies and Sigma Computing (preferred but not required). - Excellent communication and collaboration skills. Requirements - Knowledge of AI and machine learning frameworks. - Experience with R is a plus. - Experience with data visualization tools (Qlik or similar tools). Soft Skills - Excellent problem-solving and analytical skills. - Collaborate seamlessly with technical and non-technical stakeholders to understand and address their analytical needs. - Utilize agile methodologies to identify complex issues and implement effective solutions in a timely manner. Benefits - Join a powerful tech workforce and help us change the world through technology. - Professional development opportunities with international customers. - Collaborative work environment. - Career path and mentorship programs that will lead to new levels. Company Description Global Technology Services is a rapidly expanding organization situated in Medellín, Colombia. We pride ourselves on possessing one of the most influential networks within software development and IT services for the entertainment, financial, and logistics sectors. Our corporate projections offer a multitude of opportunities for professionals to elevate their careers and experience substantial growth. Joining our team means engaging with expansive engineering teams across Latin America and the United States, contributing to cutting-edge developments in multiple industries.


