Senior Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 1-10H1B No SponsorCompany SiteLinkedIn

Location

India

Posted

29 days ago

Salary

0

Seniority

Senior

Bachelor Degree8 yrs expEnglishAWSAzureCloudERPETLKafkaPythonScalaSparkSQL

Job Description

Senior Data Engineer

Summit Global Health

• The Senior Data Engineer designs, builds and maintains the data ingestion and transformation pipelines that populate Inchcape’s central data platforms. • This role ingests raw data from diverse systems, curates the bronze and silver layers of the medallion architecture, ensures data quality and performance, and delivers reliable, reusable datasets for modelling, analysis, reporting and BI. • Develop and maintain ETL/ELT pipelines to ingest data from ERP, CRM, operational, financial, and third‑party sources into the Microsoft Fabric Platform - Lakehouse (OneLake) using Delta Lake/Parquet. • Implement data quality standards and controls, incremental loading, CDC and error handling to ensure reliability and correctness. • Create bronze tables, curated Silver and gold tables by transforming, cleansing and conforming raw data, document lineage and transformations. • Optimise performance through partitioning, clustering and tuning of MS Fabric, SQL and Spark workloads, monitor pipeline health and costs. • Collaborate with Data Architects, Modellers and Data BAs to design schemas, naming conventions and integration patterns to ensure business self-service enablement through Gold data. • Build reusable components, templates and automation scripts using Fabric Dataflows Gen2, Spark notebooks or equivalent tools. • Maintain metadata and sensitivity labels via Purview, ensure proper access controls and compliance. • Research and evaluate new features and patterns in Microsoft Fabric and make recommendations for adoption, enabling an AI-driven data strategy. • Ensure NFR requirements like resilience, availability and security standards are applied as per agreed NFR metrics.

Job Requirements

  • 8+ years in data engineering or data integration roles, working with SQL, Python and or Scala.
  • Experience of ETL/ELT patterns, CDC, incremental loading and data warehousing principles using Microsoft Fabric.
  • Practical experience with Delta Lake, Lakehouse/Medallion architecture and cloud data platforms (Azure and AWS).
  • Proficient in building pipelines with tools like MS Fabric Dataflows Gen2, Spark/SQL notebooks or equivalent.
  • Experienced in data quality frameworks and monitoring/observability tools.
  • Ability to collaborate across teams and communicate technical concepts to non‑engineers.
  • Experience with streaming data (Kafka, Event Hubs) or real-time ingestion.
  • Knowledge of master data management and data governance practices.
  • Familiarity with version control and CI/CD for data pipelines.
  • Experience in, and assisted in, the implementation of governance, controls, and standards.
  • Any Shipping or Maritime experience Big Data Ecosystem prior experience is a plus.
  • Bachelor’s degree in computer science, engineering or related discipline.
  • Certification in Azure Data Engineering is a MUST.
  • Certifications in AWS cloud data platform or other data platform.

Benefits

  • Working in a diverse society, we recognize that our customers, colleagues, and contractors are central to our success.

Related Categories

Related Job Pages

More Data Engineer Jobs

Tech Holding logo

ML / AI Data Engineer – Contract

Tech Holding

Think big, Start small, Scale fast!!!

Data Engineer29 days ago
ContractRemoteTeam 51-200H1B No Sponsor

• Design, deploy, and scale large-scale ML and data processing pipelines across cloud infrastructure • Build systems to ingest, process, and serve 250,000+ hours of multimodal data (video, audio, metadata) • Architect and optimize GPU-based compute environments (e.g., NVIDIA Tesla clusters) for distributed training and inference • Develop high-throughput backend systems for video ingestion from desktop and mobile platforms • Implement distributed processing workflows, including job scheduling, fault tolerance, and resource allocation • Design and build human-in-the-loop and automated annotation systems to ensure data quality and scalability • Translate ML and multimodal research into scalable, production-grade cloud architectures • Optimize pipelines for performance, reliability, and cost efficiency across compute, storage, and networking layers • Collaborate with ML, data, and engineering teams to deliver end-to-end data workflows.

India
Miratech logo

Senior Data Engineer

Miratech

Helping Visionaries Change the World

Data Engineer29 days ago
Full TimeRemoteTeam 501-1,000Since 1989H1B No Sponsor

• Design, develop, and maintain databases supporting IVR and contact center systems • Support databases used by both runtime IVR services and analytics platforms • Design and maintain relational data models for IVR event, routing, and call data • Write, maintain, and optimize SQL for application logic, configuration changes, and investigations • Ensure database availability, integrity, performance, and scalability in production environments • Support storage and availability of IVR event logs, call lifecycle data, and routing outcomes for analytics and reporting • Monitor and troubleshoot production databases and support incident investigations • Collaborate with architects, application developers, DevOps, QA, and Analytics teams on platform changes and improvements

Canada

Senior Associate, Data Exchange Specialist

New York Life Insurance Co

At New York Life, our 180-year legacy of purpose and integrity fuels our future. As we evolve into a more technology-, data-, and AI-enabled organization, we remain grounded in the values that drive lasting impact. Our diverse business portfolio creates opportunities to make a difference across industries and communities—inviting bold thinking, collaborative problem-solving, and purpose-driven innovation. As a Fortune 100 mutual company, we offer a place to grow your skills, contribute to meaningful work, and deliver solutions that matter. Your ideas drive what’s next, and your growth powers it.

Data Engineer29 days ago

Role Description The Complex Data Exchange Specialist is responsible for managing the overall process, practice, and project implementation of electronic data interfaces with New York Life clients in support of Life, Accident, Disability, Family Medical Leave products, and various service offerings, with a strong focus on National Accounts and highly complex data integrations. This role serves as both a technical and consultative resource, ensuring high-quality data exchange while educating clients on how their data powers downstream services including administration, billing, eligibility, enrollment, and reporting. The Specialist understands how data flows through New York Life systems—from file receipt and validation to internal processing and operational impact—and proactively advises clients and TPAs on how data structure, file dependencies, and transmission timing affect service delivery and administration outcomes. They balance New York Life standards regarding layout, data integrity, and transmission protocols with client requirements to deliver compliant, scalable, and efficient solutions. The Complex Data Exchange Specialist manages multi-file data environments where multiple client files must be validated, sequenced, reconciled, and merged in order to support accurate system ingestion. These integrations often involve complex file structures, dependent file relationships, and conditional data mapping logic, requiring advanced technical expertise in file architecture, merge capabilities, and data validation. This role also drives inventory control of active interfaces and milestone management in partnership with internal and external stakeholders. The Specialist demonstrates strong analytical capabilities to audit incoming files across multiple data sources, identify discrepancies or systemic errors, and implement corrective actions that improve data integrity and client experience. The Complex Data Exchange Specialist partners closely with Implementation IT, Family Medical Leave Operations, Account Management, and external Ben Tech vendors to ensure deadlines are met, key milestones are achieved, and implementation efforts stay aligned with business objectives. Because this role supports large National Accounts and marquee clients, it requires the ability to manage high-volume, complex integrations with multiple vendors, benefit programs, and file dependencies. The Specialist must proactively assess data risk, ensure data governance standards are met, and maintain operational stability across interconnected systems. Additionally, they act as a leader among peers by proactively contributing expertise, mentoring team members, taking on complex work, driving process improvements, and independently leading projects and training initiatives. Qualifications - Bachelor’s Degree strongly preferred or minimum of 3 years relevant work experience required. - Prior experience working with clients, producers, TPAs, or Ben Tech partners preferred. - Strong understanding of complex data structures, file layouts, and electronic transmission methods preferred. - Demonstrated ability to analyze, audit, and interpret complex and high-volume data files, including multi-file data environments. - Experience managing file dependencies, merge logic, and reconciliation across multiple data sources preferred. - Strong planning, project management, and inventory control skills. - Proficient/advanced Microsoft Excel skills required (VLOOKUP, Pivot Tables, Macros). - Proficiency in Access preferred. - Experience with mainframe TSO and JCL scripting preferred. - Excellent organizational skills with ability to manage multiple complex implementations simultaneously and meet tight deadlines. - Excellent written and verbal communication skills with the ability to explain technical concepts to non-technical audiences. - Strong interpersonal and consultative skills. - Demonstrated ability to drive results and meet key milestones. - Ability to work independently while collaborating effectively across internal business partners. - Demonstrated leadership capability in a fast-paced, evolving environment. - Proven adaptability and willingness to embrace change and continuously improve processes. Requirements - Integrate with TPAs and Ben Tech partners in support of eligibility, enrollment, remittance, and billing data for new and inforce business. - Serve as a consultative partner to clients and TPAs by educating them on data structure, file specifications, and the operational impact of their data across downstream administration systems. - Implement complex electronic interfaces based on established timelines while maintaining strict inventory control of active integrations, dependent file streams, and in-flight implementations. - Manage multi-file integration environments, ensuring files are received, validated, and sequenced appropriately for system ingestion and downstream processing. - Review, audit, and validate test data across multiple related files and data sources to ensure quality, completeness, and alignment with business rules prior to production implementation. - Identify root causes of file discrepancies, mapping errors, transmission issues, or data inconsistencies across merged files, and recommend corrective actions. - Understand and articulate end-to-end data flow, including how files are received, validated, transformed, merged across multiple sources, and processed within New York Life systems. - Balance New York Life/LDEX standards regarding layout, data, and transmission protocols with client requirements, delivering compliant and scalable solutions for complex client environments. - Prepare and maintain documentation confirming the method of data exchange between New York Life and client/TPA, including file dependencies, mapping logic, merge requirements, and audit controls. - Partner with Production Support and IT to ensure successful receipt, mapping, merging, and processing of data. - Articulate complex file mapping and merge logic requirements clearly to IT and technical partners. - Drive accountability with internal teams and Ben Tech partners to meet key deliverables and milestone deadlines. - Troubleshoot and diagnose issues related to electronic data exchange, multi-file integrations, and system processing impacts. - Track file status and ensure metrics are reported accurately within Salesforce. - Develop and maintain support documentation, SOPs, and training materials for complex integration processes and data governance practices. - Proactively review and suggest improvements to existing procedures to enhance efficiency, data integrity, and scalability of complex integrations. - Serve as a subject matter expert for complex data exchange implementations and National Account integrations, providing guidance and mentorship to peers. Benefits - We provide a full package of benefits for employees – and have unique offerings for a modern workforce, including leave programs, adoption assistance, and student loan repayment programs. - Based on feedback from our employees, we continue to refine and add benefits to our offering, so that you can flourish both inside and outside of work.

United States
$81K - $115.5K / year
Zensar logo

Lead Engineer – Full Stack & Data Platforms

Zensar

At Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: One Zensar, Nurturing, Empowering, and Client Focus. Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including Milpitas, Seattle, Princeton, Cape Town, London, Zurich, Singapore, and Mexico City. We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to race, creed, color, ancestry, religion, sex, national origin, citizenship, age, sexual orientation, gender identity, disability, marital status, family medical leave status, or protected veteran status.

Data Engineer29 days ago
Full TimeRemoteTeam 10,001

Role Description The ideal candidate will have hands-on experience with: - Spring Boot - BigQuery/BigTable - Spark/Scala - Workflow orchestration tools like Airflow/Composer Exposure to the following is essential: - Authblue - OneIdentity - EAG - Event-driven architectures You will also drive DevOps practices using: - GitHub Actions Manage deployments on: - GCP Hydra Company Description At Zensar, we’re “experience-led everything”. We are committed to conceptualizing, designing, engineering, marketing, and managing digital solutions and experiences for over 130 leading enterprises. We are a company driven by a bold purpose: Together, we shape experiences for better futures. Whether for our clients, our people, or the world around us, this belief powers everything we do. At the heart of our culture is ONE with Client - a set of four core values that reflect who we are and how we work: - One Zensar - Nurturing - Empowering - Client Focus Part of the $4.8 billion RPG Group, we’re a community of 10,000+ innovators across 30+ global locations, including: - Milpitas - Seattle - Princeton - Cape Town - London - Zurich - Singapore - Mexico City Explore Life at Zensar and join us to Grow. Own. Achieve. Learn. to be the best version of yourself. We believe the best work happens when individuality is celebrated, growth is encouraged, and well-being is prioritized. We are an equal employment opportunity (EEO) and affirmative action employer, committed to creating an inclusive workplace. All qualified applicants will be considered without regard to: - Race - Creed - Color - Ancestry - Religion - Sex - National origin - Citizenship - Age - Sexual orientation - Gender identity - Disability - Marital status - Family medical leave status - Protected veteran status

India
Job Closed