Building digital businesses, together.
Lead Data Engineer
Location
India
Posted
10 days ago
Salary
0
Seniority
Senior
Job Description
Lead Data Engineer
3Pillar Global
• Understanding business requirements and writing code to implement solutions. • Creating data pipelines and managing versioning and change management. • Developing batch or real-time data pipelines. • Logging and instrumentation of pipelines and services. • Ensuring data accuracy, integrity, and compliance through quality control procedures.
Job Requirements
- Proficiency in MongoDB, MySQL, Snowflake SQL.
- Experience with ETL processes.
- Knowledge of Python is good to have.
- NoSQL-to-RDBMS Mapping: Flattening and normalizing JSON data.
- Experience in building and managing ETL pipelines.
Benefits
- Flexible work environment – whether it's the office, your home, or a blend of both.
- Part of a global team, learning from top talent around the world.
- Well-being focused with fitness offerings and mental health plans.
- Professional services model enabling career growth and development opportunities.
- Equal opportunity employer promoting diversity.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Lead Data Engineer
Encora DigitalEncora, a leader in digital engineering, drives innovation by crafting cutting-edge, cloud-first, data-first, and AI-first solutions that redefine industries. Since its inception i
Lead Data Engineer Bolivia Job Description: Job Title: Lead Data Engineer Key Skills: Snowflake, Azure Databricks, DBT, Terraform, SQL, ETL/ELT, Azure Cloud, CI/CD, Data Migration, Data Architecture Experience: +8 YOE. Location Bolivia Mode: Remote We at Coforge are hiring Lead Data Engineer (#20978) with the following skill set. Key Responsibilities - Lead and mentor a team of data engineers, ensuring best practices and high-quality deliverables. - Own and drive the end-to-end migration strategy from Snowflake on AWS to Azure Databricks and DBT. - Architect and oversee scalable, secure, and reliable data pipelines and infrastructure in Azure. - Design and implement Infrastructure-as-Code solutions using Terraform and manage CI/CD workflows. - Collaborate with cross-functional stakeholders to align data initiatives with business and technical goals. Required Skills & Qualifications - 8+ years of experience in Data Engineering, including leadership or technical lead experience. - Strong hands-on expertise with Snowflake, Azure Databricks, and DBT. - Deep experience with Terraform and cloud infrastructure automation. - Proven experience leading large-scale cloud data migration projects, preferably AWS to Azure. - Advanced proficiency in SQL, data modeling, and ETL/ELT frameworks. - Strong understanding of scalable data architecture and performance optimization. - Experience with CI/CD pipelines and Git-based version control. - Mandatory experience within the insurance industry and knowledge of insurance data domains. - Excellent communication, stakeholder management, and leadership skills. - Ability to work effectively in fast-paced and collaborative environments. Preferred Skills: - Experience with data governance and observability frameworks. - Knowledge of security and compliance standards related to insurance or financial data. - Familiarity with Kafka, Event Hubs, or real-time data processing solutions. - Experience designing lakehouse architectures and cost-optimization strategies. - Exposure to multi-cloud environments and advanced automation practices. Posted On: 8-05-26 At Coforge, we hire professionals based solely on their skills and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
• Join a team of innovative engineers building and scaling the core data infrastructure, pipelines, and services that power our products • Design and implement a robust data warehouse to support a wide range of analytics and operational use cases • Develop and maintain efficient data pipelines and curated data sets by working closely with stakeholders to gather requirements and translate them into technical solutions • And of course—write code every day!
Senior Data Engineer
Ford Motor CompanyAt Ford Motor Company, we believe freedom of movement drives human progress. We also believe in providing you with the freedom to define and realize your dreams. With our incredible plans for the future of mobility, we have a wide variety of opportunities for you to accelerate your career potential as you help us define tomorrow’s transportation.
Role Description We are looking for a skilled GCP Data Engineer to join our EPEO - Data and AI Ops team. In this role, you will play a critical part in designing, developing, and maintaining our Security Data Lake and associated data products. The core requirement for this role is deep technical expertise in Google Cloud Platform (GCP) and hands-on experience building scalable cloud data pipelines (ETL/ELT). Additionally, experience with Cribl (for log stream routing, shaping, and reduction) is considered a strong asset and a major plus. - Design, develop, and maintain robust data pipelines using GCP native services. - Build and manage data quality frameworks to ensure the integrity and reliability of security data assets. - Integrate diverse data sources and security tools via APIs to centralize security oversight. - Optimize database performance, query efficiency, and storage costs within Google BigQuery. - (Preferred) Utilize Cribl to route, shape, and enrich incoming security telemetry and log data. - Develop, deploy, and monitor automated data pipelines (ETL/ELT) using Python, SQL, and GCP services (such as Cloud Functions, Cloud Scheduler, and Dataflow). - Manage and optimize schema designs, partitioning, and clustering in Google BigQuery to ensure cost-effective and high-performance querying. - Implement and scale data quality and auditing frameworks using GCP Dataplex with centralized rules metadata configuration. - Design and maintain robust API integrations (e.g., EAMS, TrendMicro, and other threat detection platforms) to ingest critical security logs. - [Preferred/Good to Have] Configure and manage Cribl Stream pipelines (sources, destinations, routes, and functions) to parse, mask, enrich, and route security logs. - [Preferred/Good to Have] Implement log reduction strategies in Cribl to optimize data ingestion and lower downstream storage costs. - Partner with security teams to deliver actionable data products, reporting views, and tactical dashboards to prevent service outages. Qualifications - Bachelor’s degree in Computer Science, Computer Engineering, Data Science, Information Technology, or a related technical field (or equivalent combination of education and experience). - 8+ years of professional experience in Data Engineering, Cloud Data Warehousing, or software development. - 5+ years of hands-on experience designing and implementing production-grade solutions on Google Cloud Platform (GCP), specifically utilizing native services such as Google BigQuery, Cloud Run/Functions, and Cloud Storage, Dataflow, PubSub. - High proficiency in Python and advanced SQL for building, optimizing, and troubleshooting complex ETL/ELT pipelines. - Excellent written and verbal communication skills, with a proven ability to collaborate effectively with cross-functional teams in an agile environment. Benefits - Immediate medical, dental, vision and prescription drug coverage. - Flexible family care days, paid parental leave, new parent ramp-up programs, subsidized back-up child care and more. - Family building benefits including adoption and surrogacy expense reimbursement, fertility treatments, and more. - Vehicle discount program for employees and family members and management leases. - Tuition assistance. - Established and active employee resource groups. - Paid time off for individual and team community service. - A generous schedule of paid holidays, including the week between Christmas and New Year’s Day. - Paid time off and the option to purchase additional vacation time.
• Define and drive the product vision, strategy, and roadmap for the Data Curation Pipeline team • Lead the modernization of healthcare data pipelines that ingest and transform raw claims, prescription, and EHR data into standardized enterprise datasets • Identify opportunities to improve data quality, scalability, usability, speed, and downstream product readiness • Partner with internal stakeholders to understand business needs and translate them into product priorities and roadmap decisions • Own and manage the team backlog, ensuring clear prioritization of high-impact initiatives, technical enablers, defects, and enhancements • Serve as the primary product point of contact for engineering, architecture, and cross-functional stakeholders • Define acceptance criteria and participate in user acceptance testing and release validation • Work closely with software developers, data engineers, architects, QA, analytics teams, and business stakeholders throughout the product lifecycle • Lead Agile product processes including backlog refinement, sprint planning input, prioritization, and stakeholder communication • Build strong relationships with internal business stakeholders, product teams, and data consumers across the organization



