We operationalize intelligence.
Senior Data Engineer – Azure, Databricks
Location
Brazil
Posted
8 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Azure, Databricks
Keyrus
• As a Data Engineer, you will be responsible for developing, deploying and evolving scalable data pipelines, ensuring data quality, governance and availability to support analytics, digital products and AI initiatives.
Job Requirements
- Experience in Data Engineering using Microsoft Azure
- Experience with Azure Data Factory, Azure Data Lake, Azure Functions, Synapse Analytics or equivalent services
- Strong experience with Databricks
- Advanced knowledge of PySpark
- Experience with Airflow for pipeline orchestration
- Familiarity with DevOps practices (Git, CI/CD and deployment automation)
- Experience building and maintaining production data pipelines
- Knowledge of data modeling (dimensional, relational, Data Vault or similar)
- Proficiency in SQL
- Experience with Delta Lake and Lakehouse architecture (advantageous)
Benefits
- Collaborate with multidisciplinary teams, contributing to technical decisions and sharing knowledge
- Ensure data quality, governance and security through engineering best practices
Related Guides
Related Categories
Related Job Pages
More Cloud Engineer Jobs
Cloud Architect
Insight Enterprises, Inc.14,000+ engaged teammates globally #20 on Fortune’s World's Best Workplaces™ list $9.2 billion in revenue Received 35+ industry and partner awards in the past year $1.4M+ total charitable contributions in 2023 by Insight globally
Role Description We are seeking a motivated and passionate Cloud Architect to join Insight’s Enterprise Cloud Consulting Practice, focused on delivering innovative solutions for enterprise customers. This role requires a blend of technical expertise and consulting acumen, with the ability to provide thought leadership, design scalable and secure cloud architectures, and lead the implementation of complex solutions across AWS, Azure, and multi-cloud environments. The ideal candidate will have a strong background in Cloud and DevOps, a proven track record of success in customer-facing engagements, and the ability to translate business requirements into actionable cloud strategies that drive value for large-scale enterprise clients. - Design, plan, and implement Microsoft Azure Solutions including Edge/IO, Landing Zones, AI Platforms, Hybrid, and Multicloud architectures. - Lead and deliver enterprise Azure projects encompassing cloud architecture, migration assessments, proof-of-concept deployments, and full-scale implementations. - Serve as a trusted advisor to clients by providing Azure subject-matter expertise, recommending best-fit solutions, and aligning architecture decisions with business objectives and industry best practices. - Provide in-depth advisory on enterprise networking, hybrid connectivity, identity, and security architecture within Azure and hybrid cloud environments. - Demonstrate thought leadership through internal enablement, customer workshops, and executive briefings, focusing on Azure trends, cloud-native design patterns, and platform innovation. - Contribute to the growth of Insight’s knowledge base and IP by developing reusable templates, automation scripts, and documentation around Azure landing zones, DevOps, and infrastructure-as-code. - Collaborate with practice leadership to co-develop offerings, accelerators, and reference architectures, including Azure migration toolkits, cloud readiness frameworks, and pilot environments. - Engage with Microsoft and other strategic partners to co-develop Azure-centric solutions, ensuring alignment with partner roadmaps and Insight’s go-to-market strategy. - Stay passionate and current on Azure platform advancements, industry cloud adoption trends, and emerging technologies such as AI, edge, and platform engineering in the Azure ecosystem. Qualifications - Bachelor’s degree in information technology, Engineering, Business, or a related field preferred; MBA or advanced degree is a plus. - 7+ years of experience in cloud consulting or services delivery, with a strong focus on architecting and implementing cloud solutions in client environments. - 5+ years of hands-on experience with public cloud platforms (Azure, AWS, or GCP), including design, deployment, and operational management of cloud-native and hybrid workloads. - Proven track record of client-facing consulting engagements, including assessments, architecture design sessions, migration planning, and post-deployment advisory. - Demonstrated success in presales roles, including scoping, solution development, proposal support, and collaboration with sales teams to position cloud services and offerings. - 5+ years of experience working in or with cloud integrators, MSPs, or channel service providers, with a strong understanding of service packaging, go-to-market execution, and customer value realization. - 3+ years of experience leading or collaborating within cross-functional delivery teams on large-scale cloud transformation projects or modernization initiatives. Requirements - Hands-on experience delivering public cloud infrastructure solutions, with a focus on designing and deploying secure, scalable, and resilient architectures in Microsoft Azure environments. - Proven experience in large-scale application modernization efforts, including re-platforming and refactoring legacy applications using Azure-native services such as App Services, Azure Kubernetes Service (AKS), and Azure Functions. - Deep knowledge and hands-on experience with container architectures and deployment strategies, including Kubernetes orchestration, container registries, service mesh (e.g., Istio), and CI/CD integration using Azure DevOps or GitHub Actions. - Extensive expertise in Microsoft Azure, including core services, identity and access management, governance, hybrid connectivity, and cost management. - Experience with infrastructure as code using Terraform to define, provision, and manage Azure infrastructure at scale, including modular design, remote state management, and CI/CD integration. - Practical experience implementing DevOps practices and pipelines for infrastructure and application delivery, leveraging tools such as Azure DevOps, GitHub Actions, Terraform Cloud, and Bicep for streamlined automation and repeatability. - Solid networking foundation with hands-on experience in designing and implementing Azure networking components such as virtual networks, subnets, network security groups, firewalls, load balancers, and private endpoints. - Experience implementing enterprise-grade Azure Landing Zones using reference architectures aligned with Microsoft Cloud Adoption Framework (CAF), with embedded governance, security, and automation controls. Benefits - Freedom to work from another location—even an international destination—for up to 30 consecutive calendar days per year. - Full medical, dental, vision, and company 401k match. - Core values of Hunger, Heart, and Harmony, guiding relationships with teammates, partners, and clients.
Technical Support Engineer, Edge Cloud
GcorePowerful edge and cloud solutions for media business and the entertainment industry
• Handle customer and internal user requests via chat, email, and occasionally phone calls in English. • Provide consultations regarding our products and services in Edge Cloud. • Conduct technical diagnostics to resolve issues, gather information about customer and user incidents, and escalate to higher levels of technical support if needed. • Address complex technical issues and provide timely, effective solutions while considering the impact on customers' business operations, based on internal documentation. • Identify patterns and trends in technical issues and suggest improvements to prevent reoccurrences, sharing knowledge and expertise with other team members to enhance overall support capabilities.
• Assess on-premises hosts, applications, databases, middleware, and infrastructure components to perform technical discovery and migration readiness analysis. • Analyze CPU, memory, storage utilization, dependencies, integrations, runtime patterns, licensing considerations, and application architecture. • Identify migration risks, technical constraints, dependencies, and remediation requirements. • Analyze workloads based on functionality, business criticality, resiliency requirements, compliance needs, and interruption tolerance. • Define appropriate cloud placement strategies including rehost, replatform, refactor, retire, retain, or replace approaches. • Design and provision cloud infrastructure capacity required to support migration activities, testing environments, and production workloads. • Configure cloud resources including compute, storage, networking, identity, security controls, monitoring, and operational tooling. • Validate cloud environments against architecture standards, security requirements, and operational readiness criteria. • Implement infrastructure automation using Infrastructure as Code practices. • Execute migration waves following approved migration plans, runbooks, and change management procedures. • Perform User Acceptance Testing (UAT), production parallel testing, and technical validation activities. • Compare pre-migration and post-migration results, capture evidence, troubleshoot discrepancies, and drive issues through resolution. • Support production cutovers, release activities, rollback procedures, and post-release validation. • Perform post-migration health checks to ensure workload stability, performance, scalability, and operational readiness. • Perform workload tuning and optimization to ensure migrated applications are stable, scalable, reliable, and cost efficient. • Analyze performance metrics, resource utilization, logs, and operational data to identify optimization opportunities. • Support monitoring, alerting, backup, disaster recovery, patching, and operational support requirements. • Assist with cloud cost optimization through resource right-sizing and utilization analysis. • Partner with application engineering, infrastructure, security, database, and platform teams to define migration plans and execution strategies. • Collaborate with business stakeholders to establish validation criteria, acceptance requirements, and migration timelines. • Participate in architecture reviews, technical discussions, and solution design activities. • Communicate technical risks, dependencies, progress, and migration status effectively. • Design, enhance, and implement automation frameworks to enable repeatable, scalable, and efficient cloud migrations. • Develop AI-assisted solutions and automation capabilities to improve migration discovery, analysis, execution, and documentation processes. • Create reusable scripts, templates, tools, and frameworks supporting migration factory operations. • Document migration procedures, technical standards, operational learnings, risks, and best practices. • Continuously improve migration methodologies, processes, and delivery models.
Senior Cloud Operations Engineer
DragosDragos is a computer and network security company specializing in industrial cybersecurity, incident response, threat intelligence, and security software. Past
Role Description We are seeking an experienced Senior Cloud Operations Engineer to join our Cloud Skill Community. This role owns the operational health of the Dragos customer cloud fleet across Azure, AWS, and GCP -- both Dragos-managed and customer-managed environments. You will drive fleet reliability, deployment automation, and day-to-day operations at scale, bringing a strong infrastructure-as-code mindset and a bias toward automation and repeatability. Responsibilities - Operate, maintain, and improve the Dragos cloud fleet across Azure, AWS, and GCP - Own the full customer environment lifecycle -- onboarding, configuration, upgrades, and offboarding - Build and maintain Terraform-based infrastructure-as-code for customer environment provisioning and fleet standardization - Manage fleet health, drift detection, patching, and version lifecycle management at customer scale - Design and enforce multi-tenant isolation patterns -- blast radius containment, RBAC at scale, and cross-account access controls - Configure and maintain cloud networking components across all three providers (VPCs/VNets, peering, transit gateways, DNS, firewalls, load balancers) - Manage cloud-to-OT/on-prem connectivity for customer environments - Implement and maintain IAM, secrets management, and compliance posture across cloud providers - Build and maintain Datadog observability -- monitors, dashboards, log pipelines, and SLOs - Participate in an on-call rotation (PagerDuty) for the Dragos cloud fleet -- triage, respond to, and resolve production incidents across customer environments - Drive SRE practices: define SLOs, manage error budgets, maintain runbooks, and lead post-incident reviews - Support audit and compliance activities including evidence collection for FedRAMP, SOC2, and customer-specific requirements - Identify and eliminate toil through automation and process improvement Qualifications - 4+ years of hands-on cloud operations experience across one or more of Azure, AWS, or GCP - Cybersecurity Experience - Strong proficiency with Terraform for infrastructure-as-code (primary IaC tool at Dragos) - Experience operating cloud environments at scale -- fleet management, patching, upgrades, drift detection - Hands-on experience with multi-tenant cloud architectures and customer-facing environment management - Solid knowledge of cloud networking: VPCs/VNets, peering, transit gateways, DNS, firewalls, and hybrid connectivity - Experience with IAM across AWS, Azure Entra ID, and/or GCP -- roles, policies, federation, SSO - Proficiency with Datadog for monitoring, alerting, dashboards, and log pipelines - Comfort with on-call responsibilities -- this role participates in a PagerDuty rotation for the customer cloud fleet - Experience with SRE practices: SLO definition, error budget management, incident response, and blameless post-mortems - Strong scripting skills in Python and/or Bash - Familiarity with compliance frameworks (FedRAMP, SOC2, NIST CSF) and audit evidence collection - Strong ownership mindset and accountability for production stability - Excellent communication, documentation, and collaboration skills Preferred Qualifications - Experience with all three major cloud providers: Azure, AWS, and GCP - Experience managing cloud environments for external customers (customer-managed and vendor-managed models) - Familiarity with cloud-to-OT/ICS network connectivity and segmentation - Experience with secrets management tooling: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager - Experience with cloud-native fleet tools: AWS Systems Manager, Azure Arc, GCP Fleet Management - Proficiency with policy-as-code tools (OPA, Sentinel, AWS SCPs, Azure Policy) - Experience with FinOps practices and cloud cost optimization - Familiarity with Cloud Security Posture Management (CSPM) tooling - Experience with CI/CD pipeline authoring (GitHub Actions) - Passion for automation and continuously improving operational efficiency Compensation - Salary: $165,000.00 - Competitive Equity Package - Comprehensive Benefits Plan Equal Opportunity Employer Dragos is an Equal Opportunity Employer and considers applicants for employment without regard to race, color, religion, sex, orientation, national origin, age, disability, genetics, or any other basis forbidden under federal, state, or local laws. All new hires must pass a background check as a condition of employment.



