Job Closed

This listing is no longer active.

Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 501-1,000H1B No SponsorCompany SiteLinkedIn

Location

Brazil

Posted

81 days ago

Salary

0

Seniority

Senior

Job Description

Senior Data Engineer

MTP Brasil

• Design and build scalable ETL/ELT data pipelines using dbt, PySpark and other modern data transformation tools. • Develop and maintain data ingestion pipelines for generative AI workloads, including document processing, chunking and embedding workflows. • Orchestrate workflows using Airflow, Dagster or cloud-native orchestration tools. • Plan and execute data migration projects, including source analysis, schema mapping, validation and rollback strategies. • Implement Change Data Capture (CDC) solutions using industry-standard tools. • Build and maintain data quality frameworks with automated tests and validations. • Ensure data governance, security and compliance, including proper handling of PII (personally identifiable information) and enforcement of RBAC (role-based access control) policies. • Collaborate with AI Engineers and Full Stack Developers to support RAG pipelines and applications based on generative AI. • Apply event-driven architecture concepts to design scalable and reliable data processing solutions.

Job Requirements

  • Strong proficiency in Python, PySpark and advanced SQL (window functions, CTEs, performance optimization) — 4+ years of experience.
  • Experience with at least one major cloud platform: AWS, Azure or GCP — 4+ years of experience.
  • Experience with Databricks, AWS data services or Microsoft Fabric for pipeline development — 4+ years of experience.
  • Experience with relational databases (PostgreSQL, MySQL) and NoSQL databases (MongoDB, DynamoDB) — 4+ years of experience.
  • Hands-on experience with Apache Spark / PySpark and workflow scheduling (AWS Glue or similar tools) — 4+ years of experience.
  • Proficiency with modern data warehouses such as Snowflake, BigQuery, Redshift or Databricks.
  • Experience with data migration tools for on-premises or cloud platforms (e.g., SSIS).
  • Practical experience in data migration projects.
  • Experience in data migration strategy and planning.
  • Experience implementing Change Data Capture (CDC) solutions.
  • Knowledge of streaming technologies such as Kafka or Kinesis.
  • Understanding of event-driven architectures and integration with microservices.
  • Familiarity with Infrastructure as Code and containerization tools (Terraform, Docker).
  • Experience with CI/CD pipelines (preferably GitHub Actions).
  • Strong knowledge of data modeling (Star Schema, Data Vault, Dimensional Modeling).

Benefits

  • Health insurance
  • Dental insurance
  • Meal voucher
  • Mobility voucher
  • Culture voucher
  • Education voucher
  • Life insurance
  • Mental health and wellness program
  • Childcare assistance
  • Partner discounts

Related Categories

Related Job Pages

More Data Engineer Jobs

ECS Tech Inc logo

Databricks Pipeline Architect

ECS Tech Inc

All candidates must meet the following criteria: Must be a US Citizen, no dual Citizenships. Must be able to secure a Public trust clearance. Must be able to work across multiple programs across the Federal and DOD space. The core values that ECS looks for in an engagement manager include: Teamwork, Respect, Accountability, Integrity, and Leadership.

Data Engineer81 days ago
OtherRemoteH1B No Sponsor

Role Description ECS is seeking a Databricks Pipeline Architect to work remotely. This position is contingent upon additional funding. The role will support the design, implementation, and long-term operation of Databricks within a complex Government enterprise environment. Responsibilities include: - Lead data engineering efforts within Databricks, including pipeline development on AWS in an enterprise environment. - Provide unification for organization and enterprise data capabilities to break down operational silos, including mechanisms to standardize and migrate existing data to a unified data plane/warehouse. - Support the integration of Databricks with existing analytics platforms and metadata solutions. - Serve as a technical advisor to stakeholders and help upskill internal teams through guidance/documentation. - Assist and develop customer use case applications of Databricks. Qualifications - Must be a US Citizen. - Ability to successfully maintain a DHS Entry on Duty (EOD)/Public Trust. - Minimum 7 years of hands-on experience architecting, designing, and data engineering with Databricks and AWS in a government setting. - Strong understanding and experience in areas such as Apache Spark, Lakebase, Lakehouse architectures, Delta Lake, Delta Sharing, Medallion Architecture, Data & AI Governance, and use case acceleration. - Demonstrated hands-on experience with AWS Cloud Native data and ETL capabilities (S3, Lambdas, Glue, etc.). - Demonstrated experience leveraging data to provide search, enrichment, correlation, AI/ML Model Training, in alignment with government mission and business use cases. - Experience integrating Databricks with other analytics platforms. - Prior consulting or customer-facing experience in data architecture or modernization initiatives. - Strong Python skills. - Experience in Agile environments and managing cross-functional technical delivery. Requirements - Strong Databricks and AWS experience. - Familiarity with architecting solutions for scale and governance. - Comfortable integrating Databricks with other data platforms. Benefits - Salary Range: $150,000 - $180,000. - General Description of Benefits.

United States
$150K - $180K / year
InnovativeDev logo

Senior Data Engineer

InnovativeDev

👋 Welcome to InnovativeDev! We help startups and industry leaders to design and develop amazing software solutions.

Data Engineer81 days ago
OtherRemoteTeam 11-50H1B No Sponsor

• Responsible for integrating, analyzing raw data, developing, and maintaining datasets. • Improving data quality and efficiency. • Integrating different data sources, transforming data into useful business drivers. • Developing and maintaining scalable data pipelines, integration tools, and building new API integrations. • Building analytical tools to provide actionable insights into key performance metrics. • Designing data integrations, data quality frameworks, and integrating them with monitoring services. • Integrating data from different SOR’s and creating resulting models ready for consumption. • Performing data analysis to troubleshoot issues and assist in resolution. • Identifying, designing, and implementing internal process improvements. • Building solution prototypes as needed and writing algorithms against data. • Implementing processes and systems to monitor data quality. • Collaborating with analytics and business teams to improve data models that feed business intelligence tools.

Pennsylvania
Job Closed

AWS Data Architect/Engineer Company Overview Random Bit is a system modernization partner that specializes in enterprise architecture, solution architecture, data architecture, cloud transformation, data governance, compliance framework solutions, and organizational change management for both public and private sector clients. Job Summary Random Bit is seeking an AWS Data Architect with experience in Medicaid, Medicare, or Health and Human Services, particularly within claims and financial management data domains. This role provides technical leadership and supports implementation teams by designing high-quality, cloud-native data solutions on AWS. The AWS Data Architect is responsible for defining the data strategy and technical solutions for managing data across transactional, relational, and operational data stores within AWS. This role collaborates with Enterprise Architects and Business Analysts who support enterprise strategy and analysis across Medicaid and healthcare data domains. This position requires proficiency with AWS Cloud technologies and includes hands-on responsibilities such as establishing canonical models, defining interface specifications, and mapping data structures. The AWS Data Architect will be relied upon to: - Design information architecture to support an operational data store for data exchange across the Medicaid enterprise - Design and engineer 100% AWS cloud-native application components - Develop solution prototypes to demonstrate suitability and interoperability of chosen components, and to establish patterns for adoption by implementation team - Review data architectures to determine effectiveness, efficiency, and alignment with enterprise objectives - Develop comprehensive implementation and remediation strategies for data architecture non-compliance - Develop an architectural strategy for metadata management, including tooling and metadata governance - Manage metadata models that support data quality and mastering processes - Develop and maintain Conceptual, Logical, and Physical data models at the enterprise level and for specific data domains within Medicaid (Provider, Member, Claim, Reference) - Review data models with business owners and obtain approval from the data governance committee - Define, and maintain, business rules for data elements in conjunction with business owners and the data governance committee - Research emerging technologies, data modelling methods, and information management systems - Perform data mapping between the canonical models and data formats (both input and output) such as X12 EDI, FHIR, and other structured JSON, XML, or proprietary formats - Support database administrators, network designers and IT personnel on designing and managing the organization's data infrastructure and overall data strategy - Participate in Data Governance Committee meetings, representing the architecture team - Collaborate with internal and external personnel, including solution architects, software developers, database administrators, design analysts and information modelling experts to determine project capabilities, requirements, and timelines Required Experience and Skills: - Bachelor’s degree in information management, data science, statistics, public health, health informatics, or a related field or commensurate work experience - Proficiency with object-oriented design principles and software design patterns - Experience with the following AWS services: - Glue - Lambda - S3 - CloudFormation - CloudWatch and CloudTrail - CodeCommit - EC2. SNS, SQS, KMS, VPC, and IAM - Experience working with data models and relational, transactional, and NoSQL databases - At least five years of experience with healthcare data, particularly in Medicare, Medicaid, Health and Human Services, medical claims, and financial management - Familiarity with federal, state, and organizational policies that safeguard Protected Health Information (PHI) and Personally Identifiable Information (PII) - At least five years of experience with UML 2.0 and Archimate notations - Creative problem-solver with strong communication skills and a collaborative mindset - Experience with data visualization tools - Experience designing data warehouses and domain-specific data models - Experience modeling transactional data stores using relational and document data patterns - Experience modeling data for REST APIs - Experience with data transformation and Domain-Driven Design Desired Certifications: - AWS Solution Architect - Associate This position can be remote and work hours must be based on Eastern Standard Time business hours. This is a W-2 position and we offer a competitive salary and benefits package.

United States
$130K - $165K / year
CVS Health logo

Senior Data Engineer

CVS Health

Bringing our heart to every moment of your health.

Data Engineer81 days ago
OtherRemoteTeam 10,001+Since 1963H1B No Sponsor

We’re building a world of health around every individual — shaping a more connected, convenient and compassionate health experience. At CVS Health®, you’ll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger – helping to simplify health care one person, one family and one community at a time. Position Summary Designs, builds, and maintains large-scale data infrastructure and data processing systems. Implements robust and scalable solutions to support data-driven applications, analytics, and business intelligence. - Ensures seamless integration of data from different sources, such as databases, application programming interfaces (APIs), or streaming platforms. - Optimizes data processing and query performance by fine-tuning data pipelines, database configurations, and data partitioning strategies. - Establishes data quality checks and validations to identify and resolve data issues, ensuring high-quality and reliable data for downstream applications and analytics. - Implements security measures to protect sensitive data throughout the data lifecycle by working closely with security teams to ensure data encryption, access controls, and compliance with data protection regulations. - Collaborates with cross-functional teams, including data scientists, analysts, software engineers, and business stakeholders. - Designs and develops data infrastructure, including data warehouses, data lakes, and data pipelines. - Establishes auditing and monitoring mechanisms to track data access and maintain data governance standards. - Establishes monitoring and alerting mechanisms to identify bottlenecks, latency, or failures, and troubleshoots data-related problems, investigates root causes, and implements remediation measures. - Explores new frameworks, platforms, or cloud services that can enhance data processing capabilities. - Leverages DevSecOps practices Required Qualifications - 5-7 years work experience - Strong IBM DataStage experience - Experience with SQL-based relational database - Participation in a 24x7 on-call rotation (approximately every 6 weeks) Preferred Qualifications - DB2 experience - SQL Server experience - Python development experience - Experience with SQL Server Integration Services (SSIS) or other ETL tools - Experience with Windows and Linux environments - Experience using AI-assisted development tools (e.g., code generation, automated testing, or AI-driven data analysis) - Experience working with cloud platforms such as Azure or Google Cloud (GCP) - Familiarity with Agile methodologies - Adept at problem solving and decision making skills - Adept at collaboration and teamwork - Adept at growth mindset (agility and developing yourself and others) skills - Adept at execution and delivery (planning, delivering, and supporting) skills - Adept at business intelligence Education Bachelor's degree preferred/specialized training/relevant professional qualification. Computer Science or Equivalent Years of Relevant Experience Anticipated Weekly Hours 40 Time Type Full time Pay Range The typical pay range for this role is: $92,700.00 - $185,400.00 This pay range represents the base hourly rate or base annual full-time salary for all positions in the job grade within which this position falls. The actual base salary offer will depend on a variety of factors including experience, education, geography and other relevant factors. This position is eligible for a CVS Health bonus, commission or short-term incentive program in addition to the base pay range listed above. Our people fuel our future. Our teams reflect the customers, patients, members and communities we serve and we are committed to fostering a workplace where every colleague feels valued and that they belong. Great benefits for great people We take pride in our comprehensive and competitive mix of pay and benefits – investing in the physical, emotional and financial wellness of our colleagues and their families to help them be the healthiest they can be. In addition to our competitive wages, our great benefits include: - Affordable medical plan options, a 401(k) plan (including matching company contributions), and an employee stock purchase plan. - No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching. - Benefit solutions that address the different needs and preferences of our colleagues including paid time off, flexible work schedules, family leave, dependent care resources, colleague assistance programs, tuition assistance, retiree medical access and many other benefits depending on eligibility. For more information, visit https://jobs.cvshealth.com/us/en/benefits We anticipate the application window for this opening will close on: 04/26/2026 Qualified applicants with arrest or conviction records will be considered for employment in accordance with all federal, state and local laws.

United States
$92.7K - $185K / year
Job Closed