Translating technology into your success
Lead Data Engineer
Location
Brazil
Posted
2 days ago
Salary
$30 / hour
Seniority
Senior
Job Description
Lead Data Engineer
Bertoni Solutions
• Work with one of our U.S.-based clients. • Develop and implement data pipelines using PySpark. • Collaborate with cross-functional teams including machine learning engineers, software developers, analytics engineers, and product managers. • Translate business requirements into highly available data solutions. • Maintain and improve existing data pipelines and systems.
Job Requirements
- 8+ Years of total experience collaborating with cross-functional teams, including machine learning engineers, software developers, analytics engineers, and product managers to translate business requirements into highly available data solutions
- 3+ years of experience developing and implementing data pipelines using PySpark.
- Proven experience with SQL
- Advanced written and Spoken English (B2 or C1/C2).
- Good to have Databricks Certification
- Good to have Experience providing practical direction within azure native services , implementing data migration and data processing using Azure services: ADLS, Azure Data Factory, Synapse/DW /Azure SQL DB, Fabric.
- Good to have Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, Cosmo Db knowledge.
- Good to have SAP Hana knowledge
- Intermediate knowledge on Power BI
- Good to have knowledge in DevOps and CI/CD deployments, Cloud migration methodologies and processes
Benefits
- Contract Type: Independent contractor (no paid holidays or vacation, tax deductions, or insurance)
- Compensation is based on gross monthly payment for hours worked
- Equipment: You must use your own laptop or PC.
- Onboarding process: Onboarding begins after candidate selection and includes an in-person identity verification due to the end client’s security requirements.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, develop, and maintain robust ELT/ETL pipelines • Build and optimize data models in cloud-based data warehouses • Implement and manage orchestration frameworks • Develop streaming and real-time data pipelines • Architect and provision cloud data infrastructure on GCP • Ensure all data systems comply with HIPAA, HITECH, and applicable state privacy regulations • Implement data quality frameworks • Mentor junior data engineers
• Collaboration with U.S-based client stakeholders on solution design/definition. Identify solutions that solve business problems, translating requirements into technical specifications and actionable work. • Writing code and implementing the proposed solutions • Creating data pipelines, versioning and change management • Manage the complexity inherent in versioned data pipelines • Develop ETL/ELT processes to help extract and manipulate data from multiple sources. • Design, build and maintain batch or real-time data pipelines in production. • Automate data workflows such as data ingestion, aggregation, and ETL processing. • Logging and instrumentation of pipelines and services. • Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
Senior Data Engineer – PowerBI, Data Integration
3Pillar GlobalBuilding digital businesses, together.
• Collaboration with U.S-based client stakeholders on solution design/definition. Identify solutions that solve business problems, translating requirements into technical specifications and actionable work. • Writing code and implementing the proposed solutions • Creating data pipelines, versioning and change management • Manage the complexity inherent in versioned data pipelines • Develop ETL/ELT processes to help extract and manipulate data from multiple sources. • Design, build and maintain batch or real-time data pipelines in production. • Automate data workflows such as data ingestion, aggregation, and ETL processing. • Logging and instrumentation of pipelines and services. • Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
• Collaboration with U.S-based client stakeholders on solution design/definition. Identify solutions that solve business problems, translating requirements into technical specifications and actionable work. • Writing code and implementing the proposed solutions • Creating data pipelines, versioning and change management • Manage the complexity inherent in versioned data pipelines • Develop ETL/ELT processes to help extract and manipulate data from multiple sources. • Design, build and maintain batch or real-time data pipelines in production. • Automate data workflows such as data ingestion, aggregation, and ETL processing. • Logging and instrumentation of pipelines and services. • Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.


