Offshoring as a service. Hire the top 1% of flexible, global talent. $0 fees to get started.
Data Engineer
Location
South Africa
Posted
1 day ago
Salary
$700 - $3K / month
Seniority
Senior
Job Description
Data Engineer
Hire Hangar Global
• Design, build, and maintain robust, scalable data pipelines and ETL/ELT workflows • Develop and manage data warehousing solutions to support business intelligence and analytics needs • Ensure data quality, integrity, and availability across all data systems and sources • Collaborate with data scientists, analysts, and product teams to understand and fulfill data requirements • Optimize query performance and database architecture for speed and efficiency • Monitor, troubleshoot, and resolve data pipeline failures and incidents • Implement and enforce data governance, security, and compliance best practices • Document data models, processes, and infrastructure to support team knowledge sharing
Job Requirements
- 4+ years of experience as a Data Engineer or in a similar data infrastructure role within a tech company (non-negotiable)
- Strong proficiency in SQL and at least one programming language such as Python or Scala
- Hands-on experience building and managing ETL/ELT pipelines using tools such as Apache Airflow, dbt, or similar
- Experience with cloud data platforms such as AWS, GCP, or Azure, and data warehouses such as Snowflake, BigQuery, or Redshift
- Solid understanding of data modeling, schema design, and database optimization
- Strong problem-solving skills with the ability to manage complex, high-volume data environments
- Must have prior remote work experience, be fluent with remote collaboration tools and platforms (such as Slack, Zoom, Google Workspace, Asana, or similar), and have ideally worked with US or UK-based companies. Applications without this experience will not be considered.
Benefits
- Health insurance
- 401(k) matching
- Flexible work hours
- Paid time off
- Remote work options
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer, Underwriting Technical Lead
TravelersFounded in 1853, Travelers is a financial services company that offers comprehensive personal, business, and specialty insurance coverage to individuals and org
Senior Data Engineer, Underwriting Technical Lead remote type Hybrid locations CT - Hartford MN - St. Paul time type Full time job requisition id R-50825 Who Are We? Taking care of our customers, our communities and each other. That’s the Travelers Promise. By honoring this commitment, we have maintained our reputation as one of the best property casualty insurers in the industry for over 170 years. Join us to discover a culture that is rooted in innovation and thrives on collaboration. Imagine loving what you do and where you do it. Job Category Data Analytics, Data Science, Technology Compensation Overview The annual base salary range provided for this position is a nationwide market range and represents a broad range of salaries for this role across the country. The actual salary for this position will be determined by a number of factors, including the scope, complexity and location of the role; the skills, education, training, credentials and experience of the candidate; and other conditions of employment. As part of our comprehensive compensation and benefits program, employees are also eligible for performance-based cash incentive awards. Salary Range $139,400.00 - $230,000.00 Target Openings 1 What Is the Opportunity? Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Senior Data Engineer you will accelerate growth and transformation of our analytics landscape. You will bring a strong desire to guide team members' growth and develop data solutions that translate complex data into user-friendly terminology. You will leverage your ability to design, build and deploy data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning and business intelligence/insights. What Will You Do? - Build and operationalize complex data solutions, correct problems, apply transformations, and recommending data cleansing/quality solutions. - Design complex data solutions, including incorporating new data sources and ensuring designs are consistent across projects and aligned to data strategies. - Perform analysis of complex sources to determine value and use and recommend data to include in analytical processes. - Incorporate core data management competencies including data governance, data security and data quality. - Act as a data and technology subject matter expert within lines of business to support delivery and educate end users on data products/analytic environment. - Perform data and system analysis, assessment and resolution for defects and incidents of high complexity and correct as appropriate. - Collaborate across team to support delivery and educate end users on complex data products/analytic environment. - Perform other duties as assigned. What Will Our Ideal Candidate Have? - Bachelor’s Degree in STEM related field or equivalent. - Ten years of related experience. - Demonstrated expertise designing, building, and maintaining scalable data pipelines on AWS, with hands-on experience across core services such as S3, Glue, Redshift, Lambda, and/or EMR. - Proficiency in Databricks, including development and optimization of data workflows and leveraging the Lakehouse architecture for large-scale data processing. - Solid understanding of AI/ML concepts, with the ability to collaborate with data science teams and support the deployment or operationalization of machine learning models within the data ecosystem. - Hands-on experience with dbt for data modeling, transformation, and documentation, applying strong software engineering best practices to analytics workflows. - Advanced proficiency in PySpark for distributed data processing, with the ability to write performant, production-grade code for large-scale batch and/or streaming workloads. - Candidates coming from an Azure environment (e.g., Azure Data Factory, Synapse Analytics, Azure Databricks) will be considered, provided they demonstrate a clear ability to operate effectively within an AWS-based stack. - Willingness to serve in a technical leadership capacity, including mentoring engineers, driving architectural decisions, and partnering with stakeholders across engineering and the business. What is a Must Have? - Bachelor’s degree in computer science, related STEM field, or its equivalent in education and/or work experience. - 6 additional years of data engineering experience. What Is in It for You? - Health Insurance: Employees and their eligible family members – including spouses, domestic partners, and children – are eligible for coverage from the first day of employment. - Retirement: Travelers matches your 401(k) contributions dollar-for-dollar up to your first 5% of eligible pay, subject to an annual maximum. If you have student loan debt, you can enroll in the Paying it Forward Savings Program. When you make a payment toward your student loan, Travelers will make an annual contribution into your 401(k) account. You are also eligible for a Pension Plan that is 100% funded by Travelers. - Paid Time Off: Start your career at Travelers with a minimum of 20 days Paid Time Off annually, plus nine paid company Holidays. - Wellness Program: The Travelers wellness program is comprised of tools, discounts and resources that empower you to achieve your wellness goals and caregiving needs. In addition, our mental health program provides access to free professional counseling services, health coaching and other resources to support your daily life needs. - Volunteer Encouragement: We have a deep commitment to the communities we serve and encourage our employees to get involved. Travelers has a Matching Gift and Volunteer Rewards program that enables you to give back to the charity of your choice. Employment Practices Travelers is an equal opportunity employer. We value the unique abilities and talents each individual brings to our organization and recognize that we benefit in numerous ways from our differences. In accordance with local law, candidates seeking employment in Colorado are not required to disclose dates of attendance at or graduation from educational institutions. If you are a candidate and have specific questions regarding the physical requirements of this role, please send us an email so we may assist you. Travelers reserves the right to fill this position at a level above or below the level included in this posting. To learn more about our comprehensive benefit programs please visit http://careers.travelers.com/life-at-travelers/benefits/.
• Ensure monitoring of Data Quality; • Implement processes to monitor data accuracy, consistency, and integrity; • Data Security: Protect data against unauthorized access through security measures such as encryption and access control; • Collaboration with Teams: Work closely with data professionals and business areas to understand their needs; • Monitoring and Optimization: Monitor the performance of the data platform and implement improvements to optimize efficiency and scalability; • Documentation: Document data engineering processes and procedures to ensure practices are reproducible and understandable for other team members;
Role Description As one of the founding members of Prolific's newly formed AI Data Services team, you'll help build the quality systems behind some of the world's most advanced AI models. Data quality is a strategic priority for Prolific, so this is a high-visibility role with direct exposure to senior stakeholders. This isn't a traditional QA role; we are looking for an innovative thinker that can leverage their expertise to define what good means where no definition exists yet. Your primary focus is working directly with clients and alongside frontier AI labs, translating what their models need into robust human data and evaluation strategies. Rather than checking quality at the end of a project, you'll engineer quality into every stage of the lifecycle, from study design and participant strategy through to evaluation, launch readiness, and client delivery. You will also work alongside our product engineering and supply teams to define and build the quality infrastructure that will enable us to deliver high-quality human data at scale. Much of the work you'll tackle won't have an existing playbook; you'll help create it. What You’ll Be Doing - Design the quality frameworks that underpin complex human data programmes, from evaluation rubrics through to launch readiness. - Work with clients as a strategic thought partner, challenging annotation schemas and data requirements when they won't produce the signal the model needs. - Advise on project design and how the choice of schema can impact data quality. - Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines and running calibration sessions. - Build scalable quality systems, measurement frameworks, and automated checks using Python and SQL. - Partner with product and engineering to build the quality infrastructure that delivers high-quality human data at scale. - Investigate data quality and integrity issues, identifying root causes and turning insights into scalable improvements. - Architect and build dashboards, monitoring, and reporting that provide clear visibility into quality and operational performance. - Raise the quality capability across the company, upskilling operations and acting as a thought mentor to junior analysts. - Help define how Prolific approaches quality across new AI domains, shaping best practice as the team grows. Qualifications - 5+ years of experience in building quality, evaluation, or annotation systems within AI, machine learning, LLMs, or human data environments. - Strong Python and SQL skills, with a passion for using data to solve complex quality problems. - A solid understanding of machine learning pipelines and how human data impacts model performance. - Strong analytical and statistical thinking, with experience designing scalable quality frameworks. - The confidence and credibility to interact with stakeholders at frontier labs and act as a partner. - The ability to leverage your experience and expertise to influence and guide stakeholders at every level, both client-side and internally. - The ability to turn your own data analysis and quality methodology into requirements that product and engineering can build into systems. - The ability to explain your data analysis and findings clearly to non-technical stakeholders, so they can act on them. - A proactive, builder's mindset - you enjoy creating new systems, navigating ambiguity, and improving how things work. Requirements - Experience with LLM evaluation, RLHF, AI safety, or red teaming. - Experience translating vague model or evaluation goals into clear annotation specifications. - Experience working with human annotation programmes or human data operations. - Familiarity with calibration, inter-rater agreement, drift detection, or other evaluation methodologies. Benefits - Competitive salary, benefits, and remote working within an impactful, mission-driven culture. - Compensation packages for eligible roles include base salary, equity, and benefits. - Opportunity to earn a cash variable element, such as a bonus or commission. - Salary range reflects the minimum and maximum target for new hires, based on the role’s location as well as skills, experience, and relevant education or training.
Principal Data Engineer
Abacus InsightsImproving people’s lives by harnessing the healthcare data explosion through an intelligent data integration platform.
• Architect Enterprise‑Scale Data Solutions: Design, build, and evolve high‑volume batch and real‑time data pipelines using PySpark, SparkSQL, Databricks Workflows, and distributed processing frameworks. • Own Platform‑Level Integrations: Develop end‑to‑end ingestion and transformation frameworks integrating Databricks, Snowflake, AWS services (such as S3, SQS, Lambda), and external data provider APIs, with a strong focus on data quality, lineage, and schema evolution. • Lead Technical Design for Clients: Serve as the technical lead for complex client implementations, defining highly available, fault‑tolerant architectures across multi‑account cloud environments. • Translate Business Needs into Architecture: Convert complex business and regulatory requirements into scalable technical designs, detailed specifications, and reusable engineering patterns. • Set Engineering Standards: Establish and champion best practices across CI/CD, code quality, testing, orchestration, monitoring, logging, and observability for data platforms. • Ensure Security & Compliance: Design and implement security‑first data solutions, including RBAC, encryption, PHI handling, auditability, and alignment with HIPAA and SOC 2 requirements. • Optimize Performance & Cost: Profile and tune compute workloads, cluster configurations, partitioning strategies, indexing, and caching across Databricks and Snowflake environments. • Provide Technical Mentorship: Mentor senior and junior engineers, conduct design and code reviews, and raise the overall technical bar across teams. • Produce Technical Artifacts: Create clear documentation, including architecture diagrams, runbooks, and operational standards that support scalable delivery.




