We are a global BI Consultancy & Contingent Labor provider, focusing on contract IT recruitment and staff augmentation
Senior Data Engineer – Azure Databricks
Location
India
Posted
82 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Azure Databricks
Billigence
• Build and optimise data pipelines within Azure Databricks • Support migration of existing Azure Synapse notebook pipelines • Design scalable data models and storage patterns within Delta Lake • Integrate historical data from on-premise data warehouses • Build new ingestion pipelines from SQL Server stored procedures • Implement scalable ingestion frameworks and reusable pipeline patterns • Collaborate with global data engineering teams across the UK and India • Contribute to best practices and platform standards for future pipeline development
Job Requirements
- Strong experience working as a Data Engineer or Senior Data Engineer
- Hands-on experience with Azure Databricks
- Strong understanding of Delta Lake architecture
- Experience with Azure Synapse, Spark, and modern cloud data platforms
- Strong SQL and experience working with enterprise data systems
- Experience building data pipelines and ETL/ELT frameworks
- Ability to work effectively in distributed global teams
- Experience within Financial Services or regulated industries is beneficial
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Azure Data Architect
AccentureAccenture Federal Services, a division of Accenture, provides technology and consulting services to U.S. federal agencies, delivering solutions that enhance performance and efficie
Accenture Flex offers you the flexibility of local fixed-duration project-based work powered by Accenture, a leading global professional services company. Accenture is consistently recognized on FORTUNE’s 100 Best Companies to Work For and Diversity Inc’s Top 50 Companies For Diversity lists. As an Accenture Flex employee, you will apply your skills and experience to help drive business transformation for leading organizations and communities. In addition to delivering innovative solutions for Accenture’s clients, you will work with a highly skilled, diverse network of people across Accenture businesses who are using the latest emerging technologies to address today’s biggest business challenges. You will receive competitive rewards and access to benefits programs and world-class learning resources. The Role: - The Senior Data Architect is responsible for designing, governing, and evolving enterprise-scale data platforms that support analytics, reporting, and advanced data use cases. - This role partners closely with business stakeholders, data engineers, analytics teams, and cloud platform teams to translate business requirements into scalable, secure, and high-performing data architectures. - The ideal candidate brings strong experience in cloud-based data warehousing, modern data platforms (Snowflake, Databricks), and Azure-native services, along with excellent communication and documentation skills. - This role plays a critical part in defining data models, data flows, architectural standards, and best practices across the organization. Responsibilities: - Design and maintain enterprise data architectures across Azure cloud data platforms. - Translate business and analytical requirements into logical and physical data models. - Define and document end-to-end data flows, integrations, and dependencies. - Architect and guide implementation of data warehouses and lakehouse solutions in Azure. - Provide architectural leadership for Snowflake and Databricks-based solutions. - Collaborate with BI, data engineering, and analytics teams to ensure architectural alignment. - Establish data standards, patterns, and best practices for scalability, performance, and security. - Produce high-quality technical documentation for architecture, data models, and data flows. - Communicate complex technical concepts clearly to both technical and business audiences. - Document data flows and system integrations. Basic Qualifications: - Minimum of 4 years of Data Platform experience building enterprise data solutions in Azure, understanding of Azure data services and cloud architecture principles. - Minimum 4 years of experience working as an Azure Data Architect level, dependent on role seniority - Minimum of 2 years of hands on experience designing or architecting Snowflake solutions for enterprise scale data warehouses - Minimum of 2 years of experience working with Databricks in production environments with modern data processing and lakehouse patterns. - Minimum of 2 years of advanced Erwin data modeling expertise (conceptual, logical, and physical models). Preferred Qualifications: - Bachelors or Associates degree - Excellent communication skills for both technical and business stakeholders. - Strong documentation and data architecture design skills. - Experience architecting or supporting MicroStrategy Strategy One environments. - Azure Solution Architect Certification - Azure Database Certification - Azure AI Certification - Snowflake Certification - Databricks certification Compensation at Accenture varies depending on a wide array of factors, which may include but are not limited to the specific office location, role, skill set, and level of experience. As required by local law, Accenture provides a reasonable range of compensation for roles that may be hired as set forth below. We anticipate this job posting will be posted until 04/28/2026. Accenture Flex offers a market competitive suite of benefits including medical, dental, vision, and long-term disability coverage, a 401(k) plan, and paid time off. See more information on our benefits here: Accenture Flex Jobs Role Location Hourly Salary Range California $66.05 to $76.05 Cleveland $66.05 to $76.05 Colorado $66.05 to $76.05 District of Columbia $66.05 to $76.05 Illinois $66.05 to $76.05 Maryland $66.05 to $76.05 Massachusetts $66.05 to $76.05 Minnesota $66.05 to $76.05 New York $66.05 to $76.05 New Jersey $66.05 to $76.05 Washington $66.05 to $76.05 About Accenture Accenture is a leading global professional services company that helps the world’s leading businesses, governments and other organizations build their digital core, optimize their operations, accelerate revenue growth and enhance citizen services—creating tangible value at speed and scale. We are a talent- and innovation-led company with approximately 791,000 people serving clients in more than 120 countries. Technology is at the core of change today, and we are one of the world’s leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, data and AI with unmatched industry experience, functional expertise and global delivery capability. Our broad range of services, solutions and assets across Strategy & Consulting, Technology, Operations, Industry X and Song, together with our culture of shared success and commitment to creating 360° value, enable us to help our clients reinvent and build trusted, lasting relationships. We measure our success by the 360° value we create for our clients, each other, our shareholders, partners and communities.Visit us at www.accenture.com What We Believe We have an unwavering commitment to diversity with the aim that every one of our people has a full sense of belonging within our organization. As a business imperative, every person at Accenture has the responsibility to create and sustain an inclusive environment. Inclusion and diversity are fundamental to our culture and core values. Our rich diversity makes us more innovative and more creative, which helps us better serve our clients and our communities. Read more here Requesting An Accommodation Accenture is committed to providing equal employment opportunities for persons with disabilities or religious observances, including reasonable accommodation when needed. If you are hired by Accenture and require accommodation to perform the essential functions of your role, you will be asked to participate in our reasonable accommodation process. Accommodations made to facilitate the recruiting process are not a guarantee of future or continued accommodations once hired. If you would like to be considered for employment opportunities with Accenture and have accommodation needs such as for a disability or religious observance, please call us toll free at 1 (877) 889-9009 or send us an email or speak with your recruiter. Equal Employment Opportunity Statement We believe that no one should be discriminated against because of their differences. All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, military veteran status, sexual orientation, gender identity or expression, genetic information, marital status, citizenship status or any other basis as protected by applicable law. Our rich diversity makes us more innovative, more competitive, and more creative, which helps us better serve our clients and our communities. For details, view a copy of the Accenture Equal Opportunity Statement Accenture is an EEO and Affirmative Action Employer of Veterans/Individuals with Disabilities. Accenture is committed to providing veteran employment opportunities to our service men and women. Other Employment Statements Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States. Candidates who are currently employed by a client of Accenture or an affiliated Accenture business may not be eligible for consideration. Job candidates will not be obligated to disclose sealed or expunged records of conviction or arrest as part of the hiring process. Further, at Accenture a criminal conviction history is not an absolute bar to employment. The Company will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. Additionally, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the Company's legal duty to furnish information. California requires additional notifications for applicants and employees. If you are a California resident, live in or plan to work from Los Angeles County upon being hired for this position, please click here for additional important information. Please read Accenture’s Recruiting and Hiring Statement for more information on how we process your data during the Recruiting and Hiring process.
Cloud Data Engineer
2innovateEnabling Tomorrow’s Payments Ecosystem. Deliver the Experiences your Customers and Partners Desire
• Diseñar y desarrollar pipelines ETL con PySpark para procesamiento de grandes volúmenes de datos • Administrar y optimizar infraestructura de datos en AWS (S3, Glue, EMR, Redshift u otros servicios de analítica) • Trabajar con equipos funcionales para traducir requerimientos de negocio en soluciones de datos y métricas clave • Asegurar el rendimiento, estabilidad y escalabilidad de la plataforma de datos • Participar en despliegues en entornos cloud y soporte a incidentes junto con equipos de operaciones y delivery
Senior Data Engineer, Core Data Platform
1upHealth, Inc.Healthcare's most complete FHIR® data platform.
• Design, develop, and implement robust ETL/ELT data pipelines for large-scale data ingestion, transformation, and storage. • Ensure data quality, integrity, and governance by implementing validation techniques, data monitoring, and automated testing. • Collaborate with cross-functional teams, including data scientists, analysts, platform engineers, and business stakeholders, to develop scalable and reusable data solutions. • Automate deployments and testing using CI/CD pipelines with Git, Terraform, GitHub Actions, or Jenkins. • Design and build custom data tools and abstractions to support analytics, machine learning, and real-time data processing. • Work with DevOps and platform teams to establish efficient deployment and monitoring processes for internal and external data products. • Develop and implement alerting, monitoring, and observability frameworks for data pipelines to ensure reliability and proactive issue resolution. • Contribute to the data architecture and strategy, driving improvements in scalability, performance, and cost optimization. • Stay up to date with emerging technologies and industry best practices to continuously enhance data engineering capabilities.
Senior Data Engineer (Core Data Platform)
1upHealthAt 1upHealth, our mission is to unlock health data and improve industry outcomes. As leaders in FHIR® interoperability, our platform makes it easier for partners to access, integrate, aggregate, and share data across a variety of systems. 1upHealth is building a data ecosystem to promote the digital transformation of the industry and encourage insight-driven healthcare.
1upHealth is powering the digital transformation of the healthcare industry with our industry leading platform. By making healthcare data easily accessible and computable, we are on a mission to improve the patient experience, improve the quality of care, and reduce the overall cost of care. We are seeking an experienced Senior Data Engineer to join our team and play a crucial role in designing, implementing, and optimizing scalable data solutions. In this role, you will work with modern data technologies, distributed systems, and cloud platforms to build efficient, high-performance data pipelines that support analytics, AI/ML, and business intelligence initiatives. You will: - Design, develop, and implement robust ETL/ELT data pipelines for large-scale data ingestion, transformation, and storage. - Ensure data quality, integrity, and governance by implementing validation techniques, data monitoring, and automated testing. - Collaborate with cross-functional teams, including data scientists, analysts, platform engineers, and business stakeholders, to develop scalable and reusable data solutions. - Automate deployments and testing using CI/CD pipelines with Git, Terraform, GitHub Actions, or Jenkins. - Design and build custom data tools and abstractions to support analytics, machine learning, and real-time data processing. - Work with DevOps and platform teams to establish efficient deployment and monitoring processes for internal and external data products. - Develop and implement alerting, monitoring, and observability frameworks for data pipelines to ensure reliability and proactive issue resolution. - Contribute to the data architecture and strategy, driving improvements in scalability, performance, and cost optimization. - Stay up to date with emerging technologies and industry best practices to continuously enhance data engineering capabilities. We are looking for people who have: - 5+ years of hands-on experience in data engineering, with expertise in distributed data processing and big data frameworks (e.g., Apache Spark, Apache Iceberg, Trino, Apache Airflow, dbt, Dagster). - Advanced programming skills in Scala or Python for data transformation and automation. - Experience with real-time data streaming technologies such as Apache Flink, Spark Streaming, or Kafka. - Strong experience in performance tuning for Spark and optimizing large-scale data workflows. - Proficiency in SQL and database management, with hands-on experience in Massively Parallel Processing (MPP) databases such as Amazon Redshift, Snowflake, Teradata. - Familiarity with cloud-based data services (AWS, RDS, DynamoDB) and containerized infrastructure (EKS, Docker, Kubernetes). - Hands-on experience integrating DevOps and CI/CD practices in data engineering using GitHub Actions, Jenkins, or Terraform. - Proven ability to build monitoring, alerting, and observability tools for data pipelines to ensure high availability and reliability. - Experience in data mapping, validation, and testing frameworks to ensure accuracy and consistency. - Exposure in machine learning/deep learning using PyTorch, Tensorflow, or Keras (PyTorch preferred) - Exposure to machine learning workflows and familiarity with MLOps tools for model deployment and lifecycle management. - Self-starter with strong problem-solving skills who thrives in a fast-paced, agile environment. Nice to Have: - Experience working with healthcare data standards (e.g., HL7, FHIR) and regulatory compliance. - Familiarity with reporting and visualization tools such as Superset, Grafana, Tableau. - Understanding of data security, compliance, and governance frameworks (e.g., HIPAA, GDPR, SOC 2). Why Join Us? ✅ Impact-Driven Work – Be a key player in building data solutions that advance analytics and enable AI-driven innovations in the healthcare sector. ✅ Cutting-Edge Technology Stack – Work with modern big data, cloud, and AI-driven solutions. ✅ Collaborate with Experts – Join a team of top engineers to solve complex data challenges. ✅ Career Growth – Opportunity to lead projects and drive technical innovations. If you're passionate about building scalable data solutions and working with the latest in data engineering and AI-driven analytics, we’d love to hear from you! 🚀 Don't meet every requirement? Studies have shown that women, communities of color and historically underrepresented talent are less likely to apply to jobs unless they meet every single qualification. At 1upHealth we are dedicated to building a diverse, inclusive and authentic workplace. We encourage folks to apply if they meet at least 50% of the qualifications above. About 1upHealth At 1upHealth, our mission is to unlock health data and improve industry outcomes. As leaders in FHIR® interoperability, our platform makes it easier for partners to access, integrate, aggregate, and share data across a variety of systems. 1upHealth is building a data ecosystem to promote the digital transformation of the industry and encourage insight-driven healthcare. Benefits 100% Paid BCBS Medical and Dental Insurance for Employees Vision Insurance Flexible PTO Equity for All Employees 401(k) Home Office Stipend Lifestyle Savings Account Parental Leave (6 weeks bonding leave for all new parents + 10 weeks for birthing parents) Security Alert: 1upHealth only uses email domains of First Name. Last Name@1up.health or no-reply@1up.health to communicate with prospects. You will never receive an email from a third-party email service such as gmail. In addition, we will never ask a candidate for employment to share personal information (such as banking information, social security numbers, passport, etc), purchase their own equipment, or pay to apply to an open position.



