Job Closed
This listing is no longer active.
Bringing down borders in technology.
Data Engineer, SQL, ADF
Location
Brazil
Posted
170 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer, SQL, ADF
Upstart 13
• Code SQL for both on-premises and Azure SQL environments, including optimizing queries, data manipulation, and ensuring high performance. • Design and build data pipelines from various data sources (mainly databases) to a target Data Warehouse / Lakehouse, using batch data load strategies, and leveraging Microsoft Azure technologies (Data Factory and Databricks). • Implement data validation and quality assurance checks to maintain high data integrity standards, ensuring accurate and reliable data for analytical purposes. • Collaborate with a cross-functional team of business analysts, architects, engineers, data analysts, and key stakeholders to formulate and implement data pipelines. • Participate in architecture discussions, take ownership and responsibility over new projects or requirements, and perform audits to ensure data quality and integrity. • Utilize programming languages, databases & cloud Data Warehouses / Lakehouses. • Document database designs that include data models, metadata, ETL specifications, and process flows for business data project integrations. • Monitor, maintain, and optimize production systems. • Investigate and resolve incidents reported by users. • Identify opportunities to automate, consolidate, and simplify data solutions at scale. • Define best practices and create coding standards and patterns that can be reused by the rest of the data engineers.
Job Requirements
- 4+ years of work experience as a Data Engineer
- Advanced working knowledge of SQL
- Proficient in Python for data engineering tasks
- Working knowledge of Azure Data Factory (ADF)
- Familiarity with NoSQL databases
- Solid understanding of cloud technologies (Microsoft Azure)
- Experience with GitHub and CI/CD processes for data pipelines
- Working knowledge and understanding of SDLC
- Experience with Databricks or similar tools (Spark)
- English Proficiency (B2)
Benefits
- Project-based, full-time job
- Fully remote
- USD competitive salary
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Assist in building and maintaining EDP data pipelines using Python, Airflow, DBT, and Snowflake. • Support ingestion and transformation of financial datasets following established data standards. • Develop DBT models with basic testing, documentation, and data quality checks. • Create and monitor Airflow DAGs with dependencies, retries, alerts, and backfills under guidance. • Help troubleshoot pipeline issues, perform routine checks, and support schema changes. • Collaborate with analysts and platform teams; participate in reviews, sprints, POCs, and reusable frameworks.
• Architect, design, and deliver end-to-end EDP data solutions using Python, Airflow, DBT, and Snowflake • Lead complex, cross-system data pipeline initiatives • Define and govern enterprise-wide data modelling and transformation frameworks • Set strategic direction for orchestration, monitoring, reliability, and operational excellence • Establish robust data quality, governance, security, and compliance frameworks • Act as a technical leader and domain expert—mentoring engineers and influencing architecture
• Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity. • Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization. • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Definir el enfoque global de arquitectura de datos dentro del entorno de trabajo del proyecto. • Desarrollar modelos de datos curados alineados con el modelo de información corporativo. • Diseñar el enfoque de integración tras las ingestas iniciales. • Participar en la definición del marco de calidad del dato. • Diseñar la capa de consumo para dar soporte analítico completo a las distintas áreas del negocio. • Colaborar con equipos multidisciplinares en entornos ágiles. • Aplicar buenas prácticas de seguridad, gobierno y gestión del dato en arquitecturas cloud. • Dar soporte técnico y orientación a equipos relacionados con ingeniería y analítica.
