Data Architect, AWS, Databricks

Data EngineerData EngineerFull TimeRemoteSeniorTeam 10,001+H1B SponsorCompany SiteLinkedIn

Location

Brazil

Posted

3 days ago

Salary

0

Seniority

Senior

Job Description

Data Architect, AWS, Databricks

Compass

• Perform the complete migration of the current environment, today hosted on Databricks on Azure, to AWS, including creating a new data model and restructuring legacy pipelines and routines; • Define and evolve the Corporate Data Platform architecture (Lakehouse); • Ensure adherence to the target model based on AWS + Databricks; • Define architecture standards, frameworks and best practices; • Drive the definition of the migration strategy (waves, prioritization, dependencies); • Migration and Modernization: Lead the modernization of the legacy Data Warehouse (Azure/DataStage → AWS/Databricks); • Define migration approaches: Incremental vs Big Bang; • Ensure operational continuity during the transition; • Governance & Security: Define and implement standards for: Data governance/Access control/Data quality and lineage; • Ensure compliance with corporate policies and LGPD (Brazilian data protection law); • DataOps & Standardization: Structure standardized and reusable pipelines; • Implement best practices for CI/CD for data; • Reduce dependence on manual processes and low standardization; • Integration and Ecosystem: Design integrations with multiple sources and on-premises systems;

Job Requirements

  • Experience with Cloud & AWS Platform, S3, Glue, IAM, Lake Formation, CloudWatch, CloudTrail;
  • Experience with Databricks: Unity Catalog, Delta Lake, notebooks, clusters and policies;
  • Knowledge of modern Lakehouse-based architecture;
  • Experience with data modeling (DW, Lakehouse – Bronze/Silver/Gold);
  • Experience with data pipelines (ETL/ELT);
  • Experience with: advanced SQL, Python, tools such as: Airflow / Control-M / distributed orchestration;
  • Experience with ADF / DataStage (legacy);
  • Experience with CI/CD for data (Azure DevOps, Git, pipelines);
  • Experience with Data Quality, Data Contracts, Data Lineage;
  • Experience with data catalog and corporate governance;
  • Experience with security and compliance (LGPD, access control, sensitive data);
  • Knowledge of integration with multiple sources: APIs, relational databases, NoSQL, mainframe;
  • Experience in distributed and domain-driven architecture;
  • Migration strategies: Replatform, Refactor, Rewrite;
  • Knowledge of monitoring (Datadog, CloudWatch);
  • Definition of SLAs/SLOs;
  • Experience troubleshooting critical pipelines;

Related Categories

Related Job Pages

More Data Engineer Jobs

Naveera Technology LLC logo

Senior AWS Data Engineer / Lead Data Architect

Naveera Technology LLC

Engineering Production-Ready Data, AI & Cloud Platforms - Scalable, Secure, and Built for Enterprise Growth.

Data Engineer3 days ago
Full TimeRemoteTeam 201-500H1B No Sponsor

• Define and implement end-to-end Data Lakehouse solutions on AWS. • Lead the automation of cloud infrastructure using Terraform. • Orchestrate large-scale performance tuning initiatives. • Establish automated Data Quality gates using AWS Glue Data Quality. • Design complex, event-driven workflows using Step Functions and Airflow. • Serve as the primary technical liaison between Data Science, BI teams, and Business Stakeholders.

India
Full TimeRemoteTeam 51-200H1B No Sponsor

• Lead a distributed engineering team across platform/product engineering, connectors, QA, DevOps/infra, AI implementation, data governance, and support. • Own delivery, quality, release discipline, and execution of the technical roadmap. • Install engineering discipline where it's thin: automated testing, QA, CI/CD, release governance, and versioning. • Drive repeatable, perimeter-safe deployments, including containerization, infrastructure-as-code, secure deployment, and SOC 2 readiness. • Partner with the Chief Architect on the connector framework, canonical metadata model, architecture decisions, and product IP. • Build the team: assess current talent, retain the strong, hire the gaps, and align the team plan to the roadmap. • Push practical AI use across coding, review, testing, ops, and engineering productivity.

Alabama + 29 moreAll locations: Alabama | Arizona | California | Colorado | Connecticut | District Of Columbia | Florida | Idaho | Illinois | Iowa | Kansas | Nevada | New Hampshire | New Jersey | New York | North Carolina | Ohio | Oklahoma | Oregon | Maryland | Massachusetts | Michigan | Minnesota | Missouri | Pennsylvania | Tennessee | Texas | Virginia | Washington | Wisconsin
Full TimeRemoteTeam 51-200H1B No Sponsor

• Own the end-to-end product design vision for a data-dense enterprise platform. • Translate complex lineage, migration impact, governance, and blast-radius workflows into clear, navigable product experiences. • Partner closely with Product and Engineering to define what gets built, for whom, and why. • Design for two audiences at once: technical users who need depth and senior stakeholders who need confidence in decisions. • Build and maintain the design system, set the quality bar, and create repeatable standards for future product surfaces. • Run user discovery and research; turn what you learn into product direction, design decisions, and prioritization input. • Use AI across the design workflow, including research synthesis, ideation, prototyping, iteration, and productivity.

Texas
Full TimeRemoteTeam 201-500H1B No Sponsor

• Define the multi-year vision for the Data Engineering Practice, ensuring our technical capabilities are ahead of the curve for enterprise demand for Data & AI transformation. • Own the full P&L, including pipeline, pricing, delivery margin, and revenue growth, while reporting directly to executive leadership with clear commercial accountability. • Build and sustain senior relationships with Google Cloud partner teams and client executives to generate meaningful deal flow and expand strategic accounts. • Lead the practice's position on ethical AI and data democratisation while making principled, evidence-based bets on where advanced analytics and AI are creating real enterprise value. • Drive investment in reusable IP, data accelerators, and delivery assets that improve consistency, reduce time-to-value, and protect margins at scale. • Serve as a strategic advisor to client C-suite executives, helping them define data strategy, navigate AI adoption, and build the organisational capabilities to sustain it. • Set the standard for engineering and analytical excellence across the practice by hiring well, developing talent deliberately, and building teams that clients trust and return to.

Canada
CA$211K - CA$302K / year