Job Closed
This listing is no longer active.
We are a global BI Consultancy & Contingent Labor provider, focusing on contract IT recruitment and staff augmentation
Senior Data Engineer – Azure, Apache Kafka
Location
India
Posted
98 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Azure, Apache Kafka
Billigence
• Design, develop, and maintain scalable data pipelines on Microsoft Azure • Build and manage real-time and batch data processing solutions using Kafka, Spark, and Python • Develop and optimize ETL/ELT workflows using Azure Data Factory (ADF) • Work extensively with Databricks for large-scale data processing and analytics • Design, develop, and optimize Snowflake data models and queries • Write efficient and optimized SQL for data transformation and analysis • Process and manage structured and semi-structured data, including XML and JSON formats • Collaborate with cross-functional teams to support data-driven solutions • Ensure data quality, reliability, performance, and scalability of data systems • Follow best practices for cloud infrastructure, security, and performance optimization
Job Requirements
- Apache Kafka (Queue-based messaging / streaming) – Strong hands-on experience
- Python – Advanced development skills
- Apache Spark – Strong experience in distributed data processing
- Databricks – Hands-on experience in development and data engineering workflows
- Azure Data Factory (ADF) – ETL/ELT pipeline development
- Snowflake – Data warehousing and SQL optimization
- SQL – Advanced querying and performance tuning
- XML & JSON – Strong understanding of data structures and transformations
- Scala – Basic working knowledge
Benefits
- Hybrid/remote working environment, allowing you a flexible work-life balance to thrive both in the office and from the comfort of your home.
- Competitive compensations package + performance bonus.
- Referral bonus scheme.
- Coaching, mentoring, and buddy scheme (for faster integration during the probationary period).
- Certification opportunities throughout your time with us.
- Career growth support, internal moves, and career advancement opportunities.
- Team building and networking events.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
Digible, IncDigible provides advanced digital marketing and technology solutions for the multifamily housing industry.
• Own our Bronze data layer: build and maintain ingestion pipelines from third-party APIs, transform via dbt, orchestrate via Prefect • Partner with Product and Engineering on Silver-layer modeling — ensuring data is clean, documented, and governed as it moves toward consumption • Enable upstream teams to own their Gold-layer data well — establishing best practices, governance standards, pipeline automation, and tooling so teams can build confidently without it becoming the wild west • Troubleshoot pipeline failures and data quality issues, driving toward root cause and long-term fixes • Contribute to platform evolution — identifying opportunities to optimize, refactor, or scale our data infrastructure • Stay informed on developments in the modern data stack and introduce tools and processes that improve our development workflows
Senior Data Engineer
YunoYuno enables any company to manage all the payment methods and fraud providers through a single integration.
• Design and build data pipelines for large volumes of payment data that are performant, reliable, and correct — not just fast. • Own end-to-end data flows: from ingestion and transformation through to the outputs that Finance, Product, and clients depend on. • Drive data quality across your domain with tooling. • Work cross-functionally with Product, Finance and enable other Engineering teams via a ‘consulting’ style model. • Contribute to how the team works — code review culture, CI/CD standards, ADRs, how we handle incidents — we're building these practices now and senior engineers shape them. • Help onboard and level up engineers around you; there's real opportunity to make an impact here.
• Lead the development and optimization of clinical data systems • Drive data standardization and build validation frameworks • Design high-performance databases and pipelines across global trials • Architect scalable validation frameworks to proactively address systemic data issues • Build and maintain modular codebases, set coding standards, and mentor junior engineers • Oversee regulatory alignment with GCP and FDA 21 CFR Part 11 • Develop dynamic dashboards and analytics tools to deliver actionable insights into trial performance and data quality • Act as a strategic liaison across Biostatistics, Clinical Operations, and Regulatory Affairs
• Lead the development and optimization of clinical data systems • Drive data standardization initiatives • Build validation frameworks and design high-performance databases and pipelines across global trials • Serve as a strategic partner to Clinical Data Managers and cross-functional teams • Lead compliance initiatives to ensure all data systems and workflows meet GCP, FDA 21 CFR Part 11, and evolving global regulatory requirements • Develop and operationalize dynamic dashboards and analytics tools that provide real-time insights into data quality, trial progress, and operational KPIs • Act as a strategic liaison between Clinical Data Engineering and other departments




