SWORD Health logo
SWORD Health

SWORD Health is a virtual musculoskeletal care provider that is on a mission to free 2 million people from post-surgical and chronic pain. The company’s platf

Senior Data Engineer – Data Platform

Location

Portugal

Posted

157 days ago

Salary

€39.5K - €62.1K / year

Seniority

Senior

Job Description

Senior Data Engineer – Data Platform

SWORD Health

• Spearhead the migration of existing workloads to the Iceberg format, establishing and maturing the foundational lakehouse architecture. • Architect and construct robust batch and streaming data pipelines utilizing Spark and Flink technologies. • Collaborate closely with the Backend Engineering team on API integrations and the establishment of formal data contracts. • Contribute to the development of a unified lineage and governance framework utilizing DataHub. • Provide comprehensive support to the Core Team in the successful adoption of new data platform capabilities.

Job Requirements

  • Demonstrated proficiency with Python and PySpark.
  • Hands-on experience with data lake formats (e.g., Iceberg, Delta Lake, or Hudi).
  • Solid understanding of Kafka and event-driven architectures.
  • Experience in building and orchestrating data pipelines at scale.
  • Strong SQL proficiency and comprehensive data modeling knowledge.
  • Familiarity with workflow orchestration tools (e.g., Airflow, Dagster, or similar).
  • Platform-oriented mindset: developing solutions for broad organizational use, not solely individual purposes.
  • Ownership mentality: committed to seeing problems through to resolution.
  • Clear communication skills: ability to articulate complex technical concepts to non-technical stakeholders.
  • Highly collaborative: excels in working alongside backend engineers, data engineers, and analysts.
  • Pragmatic approach: adept at balancing ideal solutions with practical delivery timelines.
  • Bonus: Demonstrated expertise with Flink or comparable streaming frameworks.
  • Bonus: Proficiency in DBT and familiarity with the modern data stack.
  • Bonus: Experience with modern data platforms such as BigQuery, Trino, Snowflake, or Databricks.
  • Bonus: Proven background in developing self-service data platforms.

Benefits

  • Health, dental and vision insurance
  • Meal allowance
  • Equity shares
  • Remote work allowance
  • Flexible working hours
  • Work from home
  • Discretionary vacation
  • Snacks and beverages
  • English class

Related Categories

Related Job Pages

More Data Engineer Jobs

Rocky Mountain Elk Foundation logo

Data Engineer

Rocky Mountain Elk Foundation

Ensuring the future of elk, other wildlife, their habitat and our hunting heritage.

Data Engineer157 days ago
OtherRemoteTeam 51-200H1B No Sponsor

• Design and implement enterprise data architecture, including data models and integration patterns, to establish a single source of truth for member and donor data. • Implement and manage NetSuite Analytics Warehouse (NSAW) as the primary analytics platform for the organization. • Develop and maintain high-value Power BI dashboards for leadership and operational teams to enable data-driven decision making. • Design and build automated data integration pipelines (ETL/ELT) between NetSuite, M360suite, Klaviyo, and payment gateways. • Define and document data governance policies, including metric definitions, data quality rules, and access controls. • Monitor data pipeline performance, troubleshoot failures, and implement automated error handling to ensure system reliability. • Investigate and resolve data quality incidents (e.g., duplicates, sync failures) and implement monitoring to detect anomalies. • Enable self-service analytics by creating user-friendly data models and training staff on tools to reduce ad-hoc report requests. • Ensure data handling practices comply with relevant regulations (e.g., PCI-DSS) and implement appropriate access controls. • Serve as technical advisor to IT leadership data strategy, recommending technologies and best practices to advance organizational capabilities.

Montana
Job Closed
CourtAvenue logo

Senior Data Engineer

CourtAvenue

CourtAvenue is a marketing services company that is “accelerating digital transformation” for the most ambitious brands in the world. The company, as an emp

Data Engineer158 days ago

• Design, build, and maintain scalable data pipelines and workflows in Snowflake to support CDP and customer data initiatives. • Integrate and ingest data from multiple systems (CRM, digital analytics, marketing platforms, APIs, flat files, etc.) into Snowflake. • Develop and optimize SQL queries, views, and materialized datasets to support analytics, segmentation, and activation. • Collaborate with data strategy and MarTech teams to translate business requirements into technical data models and pipelines. • Support data standardization and transformation logic for customer profiles, identities, and behavioral data. • Implement and maintain data validation and quality checks within pipelines to ensure data accuracy, consistency, and reliability across ingestion and transformation processes. • Document workflows, schemas, and logic to maintain transparency and support scalability. • Manage and optimize Snowflake environments for performance, cost efficiency, and reliability.

United States
Job Closed
McKesson logo

Master Data Management Engineer

McKesson

Sarah Cannon Research Institute (SCRI) is one of the world’s leading oncology research organizations conducting community-based clinical trials. Focused on advancing therapies for patients over the last three decades, SCRI is a leader in drug development. In 2022, SCRI formed a joint venture with former US Oncology Research to expand clinical trial access across the country. It has conducted more than 850 first-in-human clinical trials since its inception and contributed to pivotal research that has led to the majority of new cancer therapies approved by the FDA in the past decade. SCRI’s research network brings together more than 1,300 physicians who are enrolling patients into clinical trials at more than 200 locations in 20+ states across the U.S.

Data Engineer158 days ago
OtherRemoteTeam 10,001+Since 1833H1B Sponsor

• Collaborate with business and technology stakeholders to understand and capture data requirements, ensuring master data meets user and system needs. • Lead and support the design, development, and enrichment of master data domains to ensure consistency and accuracy. • Configure, implement, and maintain MDM solutions, including data matching, survivorship, and hierarchy management. • Perform hands-on data profiling, cleansing, and enrichment activities to ensure master data quality. • Develop and execute workflows within MDM tools to support business processes. • Oversee governance policies and standards for master data creation, maintenance, and quality. • Develop training materials and provide training to business users about MDM processes, tools, and standards. • Regularly assess the MDM processes, tools, and policies and recommend improvements. • Ensure that master data management processes are compliant with relevant regulations and that data is securely handled.

Ohio
$105K - $175K / year
Job Closed
LifeStance Health logo

Senior Data Engineer

LifeStance Health

Reimagining Mental Health

Data Engineer158 days ago
OtherRemoteTeam 5,001-10,000H1B Sponsor

• Provide primary data engineer for real-time production support, ensuring minimal downtime of mission-critical data pipelines. • Rotate the role and on call requirements. • Monitor, troubleshoot, and optimize data pipeline failures, query performance bottlenecks, and data discrepancies using AWS CloudWatch, Redshift, and PostgreSQL. • Automate Root Cause Analysis (RCA) reporting, reducing resolution time by 30%+ through proactive alerting and system monitoring. • Implement best practices in IAM roles, encryption, and regulatory compliance (HIPAA, GDPR) to ensure data security and governance. • Design, develop, and maintain scalable ETL pipelines using AWS Glue, Lambda, Redshift, S3, and PostgreSQL. • Optimize query performance through partitioning, indexing, query tuning, and materialized views, achieving significant performance improvements. • Implement CI/CD pipelines for automated deployments using AWS CodePipeline, Terraform, and CloudFormation to improve system stability and deployment efficiency. • Support streaming data pipelines using Kafka, Kinesis, or Spark Streaming for real-time data ingestion and processing. • Assist in scaling, performance tuning, and automation to optimize data platform efficiency. • Collaborate with business intelligence teams and analysts to develop high-quality data models that support analytics and decision-making. • Build interactive dashboards and reports using Power BI, Tableau, or Looker to enable real-time insights for stakeholders. • Automate data validation, cleansing, and quality checks, ensuring 99.9%+ data accuracy across critical business functions.

United States
$110K - $135K / year
Job Closed