SysMap Solutions logo
SysMap Solutions

#sejaSysMap #SysMap #soulSysMap

Engenheiro de Dados SR – Databricks

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1,001-5,000Since 1999Company SiteLinkedIn

Location

Brazil

Posted

4 days ago

Salary

0

Seniority

Senior

Job Description

Engenheiro de Dados SR – Databricks

SysMap Solutions

• Desenvolver, evoluir e sustentar pipelines de dados escaláveis para ingestão, transformação e disponibilização de informações em ambiente Databricks, utilizando Python, SQL, Spark e DBT; • Projetar, implementar e optimizar processos de ETL/ELT, garantindo alta performance, confiabilidade, governança e qualidade dos dados ao longo do ciclo de vida das soluções; • Construir, manter e monitorar workflows e DAGs no Apache Airflow, assegurando a correta orquestração, automação e observabilidade das cargas de dados; • Atuar na modelagem e implementação das camadas Bronze, Silver e Gold seguindo a arquitetura Lakehouse e as melhores práticas de Data Engineering; • Integrar dados provenientes de múltiplas fontes, como APIs, bancos de dados relacionais e não relacionais, sistemas corporativos e serviços em nuvem; • Apoiar a definição de padrões técnicos, boas práticas de desenvolvimento, versionamento, testes e documentação de pipelines de dados; • Trabalhar em parceria com equipes de Analytics, BI, Data Science e áreas de negócio para entender requisitos e transformar necessidades em soluções de dados escaláveis; • Investigar e solucionar incidentes, problemas de performance e falhas em processos de dados, garantindo a estabilidade e disponibilidade da plataforma.

Job Requirements

  • Experiência sólida como Engenheiro(a) de Dados Sênior em projetos de dados corporativos e ambientes de grande volume de informações;
  • Domínio de Databricks, Apache Spark, Delta Lake, Python e SQL para desenvolvimento, sustentação e otimização de pipelines de dados;
  • Experiência prática com DBT para transformação, modelagem e gerenciamento de ativos de dados;
  • Conhecimento avançado em Apache Airflow para orquestração, monitoramento e automação de fluxos de dados;
  • Experiência com arquitetura Lakehouse e modelagem de dados nas camadas Bronze, Silver e Gold;
  • Vivência em integração de dados por meio de APIs REST, bancos de dados relacionais e outras fontes corporativas;
  • Conhecimento em versionamento de código utilizando Git e experiência com Azure DevOps ou ferramentas similares de CI/CD;
  • Experiência em práticas de qualidade, monitoramento, observabilidade e governança de dados;
  • Conhecimento em performance tuning e otimização de consultas e pipelines de processamento de dados;
  • Certificações Databricks, Azure Data Engineer Associate ou similares;
  • Experiência com Azure Data Lake Storage (ADLS), Azure Data Factory (ADF) e demais serviços de dados na nuvem Azure;
  • Conhecimento em DataOps, CI/CD para pipelines de dados e Infraestrutura como Código (IaC);
  • Experiência com ferramentas de Data Quality, Data Governance e Observabilidade de Dados.

Benefits

  • Vaga também para PcD

Related Categories

Related Job Pages

More Data Engineer Jobs

Smartcat logo

Senior Data Engineer – AI First

Smartcat

The essential language AI platform for global enterprise.

Data Engineer4 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor

• Evolve our architecture into Next Generation Data Platform • Turn Business Data into AI-Ready Assets • Drive AI-First Data Engineering • Improve Business Intelligence & Data Accessibility • Lead Through Technical Excellence

Europe
Sonny's Enterprises Inc. - Conveyorized Car Wash Equipment Leader logo

AI/ML Data Engineer

Sonny's Enterprises Inc. - Conveyorized Car Wash Equipment Leader

The world’s largest manufacturer of conveyorized car wash equipment, parts, and supplies. https://SonnysDirect.com

Data Engineer4 days ago
Full TimeRemoteTeam 1,001-5,000Since 1949H1B No Sponsor

• Build and maintain feature pipelines, training datasets, and forecast workflows for revenue, demand, delivery timing, customer behavior, inventory risk, process performance, and operational planning use cases. • Operationalize forecasting and machine learning models through repeatable training, evaluation, deployment, inference, and monitoring patterns in Databricks. • Deploy and support batch, near-real-time, and API-based inference outputs for dashboards, Databricks Apps, workflow automation, business alerts, and decision-support tools. • Implement model performance tracking, drift monitoring, validation checks, error handling, and traceability from source data through feature logic to prediction output. • Partner with Data Engineering and BI teams to ensure forecast outputs, KPIs, business logic, and AI-enabled metrics align to governed semantic structures and reporting standards. • Create reusable notebooks, libraries, feature engineering patterns, evaluation templates, and deployment frameworks that accelerate enterprise AI adoption while remaining supportable. • Support AI/BI and agent-based consumption by preparing structured, governed, business-readable outputs that can be used by reporting tools, applications, and AI assistants. • Translate forecasting and AI outputs into measurable operational or financial impact, including revenue opportunity, margin improvement, demand planning, service performance, inventory optimization, and process automation.

United States
ContractRemoteTeam 11-50Since 1998H1B No Sponsor

• Build, configure, and optimize email, SMS, and multi-channel marketing campaigns using Klaviyo. • Develop and maintain automated customer lifecycle flows, including Welcome Series, Abandoned Cart, Browse Abandonment, Post-Purchase, Re-Engagement, and Win-Back campaigns. • Create personalized customer experiences using advanced segmentation, behavioral data, dynamic content, and conditional logic. • Configure and manage Klaviyo Flows, Campaigns, Segments, Lists, Forms, and SMS Marketing capabilities. • Integrate Klaviyo with Shopify and other ecommerce platforms through native connectors, APIs, and webhooks. • Support data synchronization between Klaviyo, ecommerce platforms, customer data platforms, analytics tools, and corporate systems. • Monitor campaign performance and identify opportunities to improve engagement, conversion, retention, and revenue metrics. • Analyze customer behavior and campaign results using Klaviyo Analytics and Google Analytics 4. • Troubleshoot campaign, integration, tracking, data quality, and automation issues. • Implement and maintain email deliverability best practices, including SPF, DKIM, and DMARC configurations. • Collaborate with marketing, ecommerce, CRM, analytics, and technology teams to translate business requirements into scalable marketing automation solutions.

Brazil
TetraScience logo

Principal Architect, Platform & Data Lake

TetraScience

Open | Cloud-Native | Purpose-Built for Science

Data Engineer4 days ago
Full TimeRemoteTeam 51-200Since 2015H1B Sponsor

Role Description TetraScience is building the scientific data and AI cloud for biopharma. The platform is the innermost foundation for developing, delivering and operating our enterprise grade, secure, compliant Scientific Data and AI capabilities customers rely on. In this role, you will own the platform architecture, evolution and growth scaling across: - Enterprise Platform - Scientific Search - AI/ML Ops - Developer Platform - Developer Productivity - Lakehouse Platform - Partner Integrations - Cloud Infrastructure This is a senior IC leadership role. You set technical direction, own the decisions that cross team boundaries, and close architectural gaps before they become business risks. After achieving strong product-market fit and traction, we are entering a growth scaling phase where we are expanding our industry partnerships and developer experience to rapidly build the foundations of AI-native scientific data and workflows in production. The scope of this role is intentionally broad. We are looking for experienced candidates who cover a majority of these areas. Strong candidates bring deep fingerprints in one of two architectural profiles, with meaningful range across both: - Enterprise Data & AI Platforms - Data, Knowledge, and Developer Products Qualifications - 12+ years in software engineering, with at least 5 at staff or principal level in a SaaS platform or data infrastructure context. - Deep architecture ownership in at least one of the two fingerprint profiles above, with meaningful range across the other. - Demonstrated ownership of enterprise authentication and authorization systems at scale: SAML, OIDC, fine-grained RBAC across a multi-tenant SaaS product. - Hands-on experience with AI/ML serving infrastructure: you have built and operated model inference pipelines under production load. - Search architecture experience: you have designed and operated a search platform that handles diverse query types (keyword, semantic, or hybrid) across large structured or semi-structured datasets. - Hands-on experience with data lake architectures at scale: Delta Lake or Apache Iceberg, schema evolution patterns, partition pruning, and the trade-offs between query performance and storage cost. - Infrastructure fluency on AWS with Kubernetes or ECS. - Ability to write and defend architecture decisions: RFCs, trade-off documents, design reviews. - Strong cross-team communication. - Comfort operating across strategy, architecture, and operations in the same week. Requirements - Authn/Authz architecture is documented, consistent across services, and passing enterprise security reviews without heroics from a single engineer. - AI/ML infrastructure has a clear architecture and roadmap for MLE inference and training use cases, with strong operational telemetry and cost visibility. - The developer platform has clear SDKs and a set of standard templates for scientific use cases to start from, with adoption and delivery by multiple scientific use case teams. - Operational excellence based on a clear O11y architecture rolled out, with every production service having SLOs defined, monitored and managed. - Cost governance with customer chargeback attribution architecture and operationalized with the finance and field teams. - Lakehouse platform architecture and operational buildout as a Data Products Platform with strong DX and operational scaling. - Evolve IDS to open standards based schema and encoding with strongly typed data models and schema-on-write enforcement. - Published reference architecture for each partner class (lab instrument manufacturers and AI models), with one partner successfully onboarded against each without bespoke engineering support. Benefits - Competitive compensation with equity - Unlimited PTO - Company-paid Life Insurance, LTD/STD - 401(k)

United States