Lead Data Engineer
Location
South Africa
Posted
19 hours ago
Salary
0
Seniority
Lead
Job Description
Lead Data Engineer
DevologyX
Role Description We're looking for an experienced Lead Data Engineer to design, build, and scale modern data platforms while leading a high-performing engineering team. - Design and develop scalable data pipelines and integration solutions. - Build and optimize ETL/ELT processes, data warehouses, lakehouses, and analytics platforms. - Implement data modelling, governance, quality, and performance best practices. - Lead technical delivery, mentor engineers, and establish engineering standards. - Collaborate with business and technical stakeholders to translate requirements into scalable data solutions. - Drive end-to-end delivery of enterprise data projects using Agile methodologies. Qualifications - 8+ years of experience in Data Engineering, Data Warehousing, or Cloud Data Platforms. - 3+ years of technical leadership or team management experience. - Strong experience with Microsoft Fabric, Databricks, Azure SQL/SQL Server, and modern data architectures. - Proficiency in Python, PySpark, SQL, and data orchestration tools such as Azure Data Factory, Fabric Data Factory, or similar. - Experience with DevOps practices, CI/CD pipelines, and version control. - Exposure to Power BI and semantic modelling is advantageous. Requirements - Bachelor's degree in Computer Science, IT, Data Science, Engineering, or a related field. - Relevant cloud or data engineering certifications are advantageous. Key Skills - Data Engineering & Architecture - ETL/ELT Development - Data Modelling & Lakehouse Design - Cloud Analytics Platforms - Technical Leadership & Mentorship - Performance Optimization - Stakeholder Management - Agile Delivery
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
Globaldev GroupBuilding remote teams and providing software development solutions for tech businesses 🇺🇸🇮🇱🇩🇪🇺🇦🇵🇹🇵🇱
• Design and implement streaming and batch data pipelines for data integration, ingestion, data quality management (quality gating), transformations, and auditing. • Perform feature engineering and data exploration. • Collaborate with business and operations teams to align on requirements, and to request and standardize waveform/sensor data. • Build and maintain the Gold data layer, key KPIs, and PoC evaluation frameworks. • Set up and maintain solid engineering practices, including CI/CD pipelines, Infrastructure as Code (IaC), version control (Git), unit and integration testing, and proper documentation.
Data Architect
ZensarAt Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: One Zensar, Nurturing, Empowering, and Client Focus. Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to race, creed, color, ancestry, religion, sex, national origin, citizenship, age, sexual orientation, gender identity, disability, marital status, family medical leave status, or protected veteran status.
Role Description - Design and govern conceptual, logical, and physical data models for banking domains (Customer, Account, Transaction, Product, Risk, Compliance); experience in data modeling tools like ER Studio. - Define enterprise data architecture standards and modeling best practices aligned with Azure/Snowflake services (ADLS, Azure SQL, Snowflake, etc.). - Experience in mapping data elements from source systems to target systems; experience in FIS Horizon or IBIS is preferable. - Partner with business, data engineering, and platform teams to enable financial reporting solutions. - Proficient in overall Data architecture and design of data components; support Data engineers in the overall development lifecycle. - Support Azure-based data platforms and data products, optimizing models for performance, cost, and future extensibility. - Hands-on experience in SQL and Excel, with strong skills in reverse engineering and developing clear and well-structured documents and presentations. Knowledge of Snowflake is preferable. Qualifications - Experience in data modeling tools like ER Studio. - Knowledge of Azure/Snowflake services (ADLS, Azure SQL, Snowflake, etc.). - Experience in FIS Horizon or IBIS is preferable. - Proficient in SQL and Excel. - Strong skills in reverse engineering and documentation. - Knowledge of Snowflake is preferable. Requirements - Experience in data architecture and design. - Ability to partner with various teams for financial reporting solutions. - Support for Azure-based data platforms and products. Benefits - Inclusive workplace with a focus on individuality and growth. - Commitment to equal employment opportunity (EEO) and affirmative action. Company Description At Zensar, we’re “experience-led everything.” We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. - Part of the $4.8 billion RPG Group. - Community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. - Explore Life at Zensar and join us to Grow. Own. Achieve. Learn.
Senior Data Engineer
TrustonicWe embed world-leading trusted cybersecurity technology into smart devices and connected vehicles.
Role Description We’re looking for a Senior Data Engineer to architect and maintain cutting-edge data systems that power analytics, AI, and operational decision-making. In this role, you’ll take ownership of end-to-end data lifecycles, designing pipelines, models, and architectures that support real-time insights and machine learning at scale. - Design Scalable Data Architecture: Build modern, cloud-native data platforms (AWS, Snowflake, Databricks) supporting batch and streaming use cases. - Develop Efficient Data Pipelines & Models: Automate ETL/ELT workflows, optimise data models, and enable self-serve analytics and AI. - End-to-End Data Ownership: Manage ingestion, storage, processing, and delivery of structured and unstructured data. - Performance & Cost Optimisation: Continuously tune infrastructure for high concurrency, low latency, and cost efficiency. - Real-Time Integration & Analytics: Ingest telemetry, API, and application data in real time to power dashboards and AI-driven tools. - ML & AI Enablement: Provision datasets for ML/AI workloads, integrating with SageMaker, Snowflake ML, and MLOps best practices. - Data Governance & Security: Ensure robust data governance, compliance (GDPR, SOC 2), and enterprise-grade security. - Collaboration & Strategy: Work closely with Product, Engineering, DevOps, and Analytics teams to align data solutions with business goals. Qualifications - Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or related field (Master’s preferred) - Relevant certifications (AWS Certified Data Analytics, Solutions Architect, SnowPro Core) Requirements - Significant experience in technology roles, with 5+ years in data engineering on real-time, scalable cloud platforms (AWS & Snowflake preferred). - Experience in SaaS/product companies managing large-scale IoT, telemetry, or digital datasets is highly desirable. - Technical Expertise: - AWS (S3, Glue, Lambda, Athena, Kinesis) - Snowflake (data pipelines, schema design, query optimization) - Data modeling, ETL/ELT, real-time streaming (Kafka, Kinesis) - Big data processing (Spark, Airflow), SQL, Python, Java/Scala - BI & analytics platforms (Tableau, Looker) - ML/AI integration (SageMaker, TensorFlow, Snowflake ML, feature stores) - Data governance, security, and compliance frameworks - Attributes: Strong communicator, collaborative, analytical, and strategic. Ability to balance multiple projects while driving innovation and operational excellence. Benefits - Competitive compensation - A base salary that reflects your expertise and impact. - Bonus scheme to share in our success. - Flexibility & work-life balance - Remote working arrangements to support your lifestyle. - An open holiday policy with no upper limit—take the time you need, when you need it. - Support for volunteering and causes that matter to you. - Growth & development - Opportunities for professional development and career progression. - Exposure and interaction with global teams. - Inclusive, supportive culture - A commitment to diversity, equity, and inclusion. - A collaborative environment where your voice is valued and your wellbeing is prioritised. - Impact & purpose - The chance to work on technology that drives global economic inclusion and makes a real difference in people’s lives.
Data Engineer
ClearCaptions LLCClearCaptions was founded in 2011 with a driving force to serve the hard-of-hearing to enable them to communicate again after they lose the ability to use the telephone. Since our inception, we have existed for one purpose, to improve the lives of our customers. Enabling communication is our passion. We believe in providing the highest levels of service and products to our customers. Our goal is to change lives by re-enabling social connections and independence through using the telephone today and through other services in the future. We are a technology service company that utilizes groundbreaking automatic speech recognition, human captioning, product development, and customer center marketing, sales, and service to deliver easy-to-use products to our mostly senior customer base. As a Federal Communications Commission (FCC) – certified telephone captioning provider, ClearCaptions follows the highest industry standards for privacy, security, and professionalism on all calls.
Role Description The Data Engineer - Fabric collaborates with cross-functional teams to gather data requirements, develops efficient data solutions, and ensures the accurate formatting and storage of enterprise data leveraging OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and Notebooks. The role works closely with the reporting team to provide actionable insights based on reliable and repeatable data. In addition, the role is responsible for documentation, data governance, and providing level II/III support for ETL and/or warehousing solutions. This role reports to the Sr. Data Engineering & Report Visualization Manager. What This Role Does: - Collaborates with cross-functional teams to understand data and analytical requirements and translates them into effective Microsoft Fabric–based data solutions. - Designs, develops, and maintains end-to-end data ingestion and transformation processes using Microsoft Fabric OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and Notebooks. - Formats, cleanses, and stores data in a structured manner to ensure data quality and accessibility for reporting and analysis, following Medallion Architecture. - Works closely with reporting and analytics teams to ensure datasets are accurate, well-modeled, and optimized for Power BI and other analytical workloads. - Creates comprehensive documentation for data pipelines, processes, and solutions. - Participates in data governance activities to ensure data integrity, security, and compliance. - Utilizes source control to manage and track changes to data pipelines and codebase. - Provides level II/III support to troubleshoot and resolve data-related issues and inquiries. - Provides ancillary support for Machine Learning (ML) and Artificial Intelligence (AI) processes. - Stays updated with industry trends and best practices related to data warehousing, ETL, and data engineering. - Manages and optimizes Microsoft Fabric platform performance by monitoring capacity utilization, query performance, Lakehouse and Warehouse workloads, Delta table maintenance (OPTIMIZE/VACUUM), data partitioning, storage consumption, and pipeline execution to ensure scalable, reliable, and cost-effective data operations. - Develops and maintains monitoring, alerting, and observability solutions for Microsoft Fabric pipelines, notebooks, Lakehouses, Warehouses, semantic models, and related data platform components to ensure operational reliability and rapid issue resolution. - Performs other duties as assigned. Qualifications - Bachelor's degree in computer science, information technology, or a related field (or equivalent experience). - A minimum of three (3) years of proven experience in data engineering, ETL development, and data warehousing. - Highly experienced with Microsoft Fabric, including OneLake, Lakehouses, Data Pipelines, Dataflows Gen2, and/or Notebooks. - Solid understanding of Medallion Architecture, Delta Lake concepts, data modeling, and modern analytics patterns. - Experience with data governance, data quality, and data security frameworks and standards. - Experience monitoring and optimizing Microsoft Fabric Capacity utilization, workload performance, and resource consumption is preferred. - Solid grasp of Continuous Integration Continuous Delivery (“CI/CD”), including source control, deployment pipelines, and release management practices. - Experience with Git-based source control and collaborative development workflows. - Expertise in integrating Fabric solutions with Power BI, Azure services, and external data sources. - Experience with Apache Spark and PySpark within Microsoft Fabric. - Excellent verbal and written communication skills, presentation, problem-solving skills and attention to detail. - Self-starter with strong organizational and time management skills, self-directed and able to handle multiple priorities with demanding timeframes. - Ability to guide and support others through collaboration, including assisting teammates and contributing to shared processes, to foster a high-quality, team-oriented, fast-paced environment that drives results. - Demonstrated ability to use discretion, make sound decisions, and maintain confidentiality. - Willingness and ability to work flexible hours. - Proficiency in MS Office (including Excel, Outlook, PowerPoint, Visio, & Word) and modern communication tools for virtual teams (e.g., MS Teams, Slack). Preferred - Expertise in developing data pipelines to support Machine Learning (ML) and / or Artificial Intelligence (AI) tools. Physical Demands - Sitting, standing, and walking (95-100%) - Keyboarding (70-90%) - Viewing computer monitor, tablet, and cell phone requiring close vision (70-90%) Work Environment - 100% Remote: Work environment is at home. Compensation $120,000 to $135,000 plus 5% determined by competitive market analysis and internal equity considerations. Final compensation will be based on the candidate’s qualifications, experience, and business needs. For details on our comprehensive benefits program, visit www.clearcaptions.com/careers to explore our total rewards package. Intrigued to learn more? When you apply for this role, your information will be personally reviewed by our talent acquisition team (not by a robot). You can expect to hear back from us if we think there could be a fit and what next steps look like. Company Description ClearCaptions is an equal opportunity employer committed to inclusion and diversity. All employment decisions are based on business needs, job requirements, and individual qualifications, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics.

