Job Closed
This listing is no longer active.
Digital Transformation Consulting Leader with expertise in business/ technology advisory and digital platform solutions
Lead Data Engineer
Location
India
Posted
96 days ago
Salary
0
Seniority
Senior
Job Description
Lead Data Engineer
Exavalu
• Data Pipeline Engineering: Architect, build, and maintain complex, real-time, and batch data pipelines using Azure Data Factory, Python/PySpark, and Databricks. • Architecture & Modelling: Design and implement modern data warehouse solutions, data models, and data lakes, optimizing for performance and scalability. • Data Ingestion & Integration: Ingest, cleanse, and transform data from diverse sources into usable data structures for analytics. • Security & Governance: Implement security features, including role-based access control (RBAC), data encryption, and governance via Azure Purview.
Job Requirements
- Performance Optimization & Monitoring: Troubleshoot and tune data systems and SQL queries for efficiency; monitor data workflows.
- Technical Leadership & Mentorship: Lead code reviews, mentor junior engineers, and define technical standards and best practices.
- Collaboration: Work with data scientists, analysts, and stakeholders to deliver actionable business insights.
- Azure Services: Azure Data Factory, Databricks, Synapse Analytics, Data Lake Storage.
- Languages & Tools: Python/PySpark, SQL, Scala, CI/CD (DevOps) tools.
- Processes: ETL/ELT, Data Modelling, Data processing.
Benefits
- Diversity Inclusion: At Exavalu, we are committed to building a diverse and inclusive workforce.
- Flexibility depending on the needs of employees, customers, and the business.
- Welcome back program to help people get back to mainstream after a long break due to health or family reasons.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Own and architect the end-to-end reporting lifecycle in the diamond matching project • Inherit a replicated production environment • Transform raw Python application data into a high-performance analytics layer • Bridge the gap between backend data structures and business-ready visualizations
• Deliver end-to-end data migration activities across SAP ECC and HR platforms • Extract, transform, and load data using ETL tools (SAP BODS preferred) • Perform data mapping, cleansing, validation, and reconciliation • Develop and execute SQL queries for data analysis and validation • Support testing cycles including SIT and UAT • Work closely with HR, IT, and functional stakeholders • Document data migration processes and technical specifications • Ensure data integrity, compliance, and audit readiness
Senior Data Engineer, Platform & Pipelines
NateraWe are a global leader in cell-free DNA (cfDNA) testing, dedicated to oncology, women’s health, and organ health.
• Architect, implement, and maintain data ingestion and transformation pipelines using modern workflow orchestration tools (e.g. Dagster). • Identify, catalog, and integrate internal and external data sources used across research efforts. • Operationalize bioinformatics pipelines that support large-scale batch processing, incremental updates, and backfills within AWS. • Normalize and structure heterogeneous data into consistent, reusable representations that support downstream analysis, modeling, and querying. • Populate and maintain patient-centric data models in shared storage systems (e.g., graph and relational databases). • Collaborate with backend and AI engineers to design data-access patterns that support analytics applications and AI-driven interactions. • Contribute to backend services and APIs that expose integrated data to internal tools and applications. • Participate in the evolution of AI-enabled analysis workflows, including tooling that supports LLM- or agent-based interactions with data. • Contribute to system-level design decisions around data flow, service boundaries, reliability, and scalability. • Write clean, tested, and well-documented Python code that meets production software engineering standards. • Debug and resolve complex data quality, pipeline, backend, and infrastructure issues in a distributed environment.
• Enterprise Data Strategy & MDM Design • MDM Architecture: Design the strategy and logical architecture for linking "Core Entities" including but not limited to Providers, Practices, Patients across the enterprise ecosystem to enable a holistic view of the business and to eliminate redundant manual workflows. • Business Data Warehouse Design: Architect the centralized data hub within Snowflake, ensuring it is structured to support complex cross-functional reporting on revenue and performance. • Automation Roadmap: Partner with the Integrations team to define how MuleSoft should be leveraged to automate data flows between systems (e.g., ensuring a signed contract in Ironclad triggers the appropriate data synchronization across Salesforce and financial systems). • Lifecycle Mapping: Create end-to-end data flow diagrams that capture the full customer journey, identifying "golden record" sources for every data point. • Cross-Functional Governance: Lead the effort to document and standardize business definitions of data. You will serve as the bridge between technical teams and stakeholders in Product, Security, Compliance, Growth, Provider Networks, and Finance. • Data Cataloging: Leverage DataHub to build and maintain a comprehensive data catalog, ensuring technical and non-technical users understand data lineage, definitions, and ownership. • Governance Framework: Establish the standards for data quality, security, and privacy in collaboration with the Security and Compliance (CRG) teams. • Insights Readiness: Ensure the data architecture provides the BI team with clean, reliable, and well-documented datasets to drive analysis on business performance and success metrics. • Technical Consulting: Act as the internal consultant for business units looking to integrate new data types or systems into the enterprise landscape. • Organizational Scaling: Define the long-term vision for the Data Architecture function, including the future hiring plan and organizational structure as the complexity of our data ecosystem grows. • Mentorship: Provide high-level technical guidance to engineers and systems analysts across the Enterprise Applications team.




