Job Closed
This listing is no longer active.
Digital Transformation Consulting Leader with expertise in business/ technology advisory and digital platform solutions
Data Engineer
Location
India
Posted
97 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
Exavalu
• Data Pipeline Development: Design, build, and optimize robust ETL/ELT pipelines using Azure Databricks, Spark, and SQL. • Data Processing & Transformation: Utilize Python, PySpark, and SQL to clean, transform, and aggregate complex data for analytics. • Azure Data Lake Management: Manage and optimize data storage and retrieval in Azure Data Lake Storage (ADLS) Gen2. • Delta Lake Implementation: Implement Delta Lake for ACID compliance, data versioning, and high-performance data lake operations. • Integration with Azure Services: Integrate Databricks with Azure Data Factory for orchestration, Azure Synapse Analytics for warehousing, and Azure Key Vault for security. • Performance Optimization: Monitor, troubleshoot, and optimize Databricks clusters and spark jobs to manage costs and performance. • Data Security & Governance: Implement Role-Based Access Control (RBAC), data encryption, and data lineage tracking. • Requirements Collaboration: Work with data scientists and analysts to support machine learning models and business intelligence (BI) reporting.
Job Requirements
- Databricks & Spark: Strong proficiency in PySpark, Spark SQL, and Databricks Jobs.
- Azure Infrastructure: Familiarity with Azure Data Factory, ADLS, and Synapse Analytics.
- Data Modelling: Experience in building data models (e.g., Star Schema, Snowflake).
- Programming: Proficient in Python and SQL.
- DevOps: Experience with Git and CI/CD tools.
Benefits
- Diversity Inclusion: At Exavalu, we are committed to building a diverse and inclusive workforce.
- We nurture a culture that embraces all individuals and promotes diverse perspectives, where you can make an impact and grow your career.
- Exavalu also promotes flexibility depending on the needs of employees, customers and the business. It might be part-time work, working outside normal 9-5 business hours or working remotely.
- We also have a welcome back program to help people get back to mainstream after a long break due to health or family reasons.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Own and architect the end-to-end reporting lifecycle in the diamond matching project • Inherit a replicated production environment • Transform raw Python application data into a high-performance analytics layer • Bridge the gap between backend data structures and business-ready visualizations
• Deliver end-to-end data migration activities across SAP ECC and HR platforms • Extract, transform, and load data using ETL tools (SAP BODS preferred) • Perform data mapping, cleansing, validation, and reconciliation • Develop and execute SQL queries for data analysis and validation • Support testing cycles including SIT and UAT • Work closely with HR, IT, and functional stakeholders • Document data migration processes and technical specifications • Ensure data integrity, compliance, and audit readiness
Senior Data Engineer, Platform & Pipelines
NateraWe are a global leader in cell-free DNA (cfDNA) testing, dedicated to oncology, women’s health, and organ health.
• Architect, implement, and maintain data ingestion and transformation pipelines using modern workflow orchestration tools (e.g. Dagster). • Identify, catalog, and integrate internal and external data sources used across research efforts. • Operationalize bioinformatics pipelines that support large-scale batch processing, incremental updates, and backfills within AWS. • Normalize and structure heterogeneous data into consistent, reusable representations that support downstream analysis, modeling, and querying. • Populate and maintain patient-centric data models in shared storage systems (e.g., graph and relational databases). • Collaborate with backend and AI engineers to design data-access patterns that support analytics applications and AI-driven interactions. • Contribute to backend services and APIs that expose integrated data to internal tools and applications. • Participate in the evolution of AI-enabled analysis workflows, including tooling that supports LLM- or agent-based interactions with data. • Contribute to system-level design decisions around data flow, service boundaries, reliability, and scalability. • Write clean, tested, and well-documented Python code that meets production software engineering standards. • Debug and resolve complex data quality, pipeline, backend, and infrastructure issues in a distributed environment.
• Enterprise Data Strategy & MDM Design • MDM Architecture: Design the strategy and logical architecture for linking "Core Entities" including but not limited to Providers, Practices, Patients across the enterprise ecosystem to enable a holistic view of the business and to eliminate redundant manual workflows. • Business Data Warehouse Design: Architect the centralized data hub within Snowflake, ensuring it is structured to support complex cross-functional reporting on revenue and performance. • Automation Roadmap: Partner with the Integrations team to define how MuleSoft should be leveraged to automate data flows between systems (e.g., ensuring a signed contract in Ironclad triggers the appropriate data synchronization across Salesforce and financial systems). • Lifecycle Mapping: Create end-to-end data flow diagrams that capture the full customer journey, identifying "golden record" sources for every data point. • Cross-Functional Governance: Lead the effort to document and standardize business definitions of data. You will serve as the bridge between technical teams and stakeholders in Product, Security, Compliance, Growth, Provider Networks, and Finance. • Data Cataloging: Leverage DataHub to build and maintain a comprehensive data catalog, ensuring technical and non-technical users understand data lineage, definitions, and ownership. • Governance Framework: Establish the standards for data quality, security, and privacy in collaboration with the Security and Compliance (CRG) teams. • Insights Readiness: Ensure the data architecture provides the BI team with clean, reliable, and well-documented datasets to drive analysis on business performance and success metrics. • Technical Consulting: Act as the internal consultant for business units looking to integrate new data types or systems into the enterprise landscape. • Organizational Scaling: Define the long-term vision for the Data Architecture function, including the future hiring plan and organizational structure as the complexity of our data ecosystem grows. • Mentorship: Provide high-level technical guidance to engineers and systems analysts across the Enterprise Applications team.




