Job Closed
This listing is no longer active.
We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
Director – Data Engineering
Location
India
Posted
52 days ago
Salary
₹6,000K - ₹8,500K / year
Seniority
Lead
Job Description
Director – Data Engineering
Weekday (YC W21)
• This role is for one of the Weekday's clients • We are seeking an experienced and visionary Director – Data Engineering to lead and scale our data engineering function. • You will play a critical role in building robust, scalable, and efficient data platforms that power analytics, machine learning, and business intelligence across the organization.
Job Requirements
- Min Experience: 12 years
- Lead the design, development, and optimization of scalable data pipelines, data lakes, and data warehouses.
- Define and implement the overall data engineering strategy aligned with business goals and future growth.
- Architect modern data platforms leveraging cloud technologies such as AWS, Azure, or GCP.
- Drive best practices in data modeling, ETL/ELT processes, data governance, and data quality management.
- Collaborate closely with data scientists, analysts, and business stakeholders to enable data-driven decision-making.
- Build and mentor a high-performing team of data engineers, fostering a culture of innovation, ownership, and continuous improvement.
- Oversee large-scale data integration efforts involving structured and unstructured data sources.
- Ensure data security, compliance, and privacy standards are adhered to across all systems.
- Evaluate and implement new tools, frameworks, and technologies to enhance data capabilities.
- Establish performance benchmarks and continuously improve system reliability, scalability, and efficiency.
- Strong expertise in Data Engineering fundamentals including distributed data processing, data architecture, and pipeline optimization.
- Hands-on experience with big data technologies such as Spark, Hadoop, Kafka, and Airflow.
- Proficiency in SQL, Python, Scala, or Java for building and managing data workflows.
- Deep understanding of cloud-based data platforms and services (e.g., AWS Redshift, BigQuery, Azure Data Factory).
- Experience with real-time and batch data processing systems.
- Strong knowledge of data warehousing concepts, dimensional modeling, and data lake architectures.
- Familiarity with DevOps practices, CI/CD pipelines, and infrastructure as code.
- Proven ability to design systems that handle high data volume, velocity, and variety.
- Demonstrated experience in leading and scaling data engineering teams in fast-paced environments.
- Strong stakeholder management and communication skills, with the ability to translate technical concepts into business value.
- Strategic thinker with a problem-solving mindset and attention to detail.
- Ability to drive cross-functional collaboration and influence decision-making at senior levels.
- Passion for innovation and staying updated with emerging trends in data and analytics.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Experience in building data platforms supporting AI/ML initiatives is a plus.
- Prior experience in a leadership role within high-growth or enterprise environments.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineer
The Hershey CompanyThe Hershey Company is an Equal Opportunity Employer. The policy of The Hershey Company is to extend opportunities to qualified applicants and employees on an equal basis regardless of an individual's race, color, gender, age, national origin, religion, citizenship status, marital status, sexual orientation, gender identity, transgender status, physical or mental disability, protected veteran status, genetic information, pregnancy, or any other categories protected by applicable federal, state or local laws. The Hershey Company is an Equal Opportunity Employer - Minority/Female/Disabled/Protected Veterans. You may request a reasonable accommodation if you are unable or limited in your ability to use or access our online application process as a result of a disability. You can request an accommodation via phone or email. To request an accommodation via phone, please call +1 877-804-1794 and leave a voicemail with your contact information. You may also email a request for accommodation to ApplicationHelp@hersheys.com. Please be sure to include “Accommodation Needed” in the subject line. This will ensure that your email is routed to the appropriate contact who will handle your request.
Job Location: Remote This position can be Remote based in US, Dallas,TX in our Frisco office or at 19E in Hershey, PA Summary: The Data Engineer, Data Products supports the design, build, and operation of data pipelines and models that power Hershey’s enterprise data products. Working within established architectures and with guidance from Senior Data Engineers and Architects, this role develops reliable, scalable data transformations that enable analytics, reporting, and AI across Hershey’s business domains. Data Engineers collaborate closely with Senior Data Engineers, Architects, Platform Engineering, and domain teams to translate defined business requirements into high quality technical solutions using modern cloud-native tools such as Azure and Databricks. They contribute to key stages of the data product lifecycle, including ingestion, transformation, modeling, documentation, and operational support. By helping build reusable and well engineered data assets, the Data Engineer strengthens Hershey’s broader data strategy—improving data quality, consistency, and accessibility while supporting the development of trusted, long-term data products. What We Are Building for Hershey This role contributes to Hershey’s enterprise data strategy by engineering reusable, governed data assets that scale across domains and reduce duplication. By applying engineering discipline and governance-by-design, Data Engineers help evolve one-off solutions into durable, production-grade data products that improve trust and speed of decision-making. Major Duties & Responsibilities Data Product Engineering & Delivery: • Build and maintain scalable data ingestion and transformation pipelines on Azure and Databricks; develop curated/semantic models for analytics and AI. • Translate requirements into technical designs and acceptance criteria with Product Managers. Technical & Architectural Implementation • Apply best practices for performance, cost, security, and reliability; follow enterprise standards and shared infrastructure. • Implement efficient patterns such as Delta Lake, medallion architecture, and orchestration frameworks. Governance, Quality & Operations • Embed governance-by-design including metadata, lineage, documentation, certification, and automated data quality checks. • Operate and troubleshoot production pipelines; contribute improvements to standards and automation. Collaboration Across Domains • Partner with domain teams to validate definitions and ensure products meet business needs. • Work with Platform Engineering on pipeline frameworks and optimization; coordinate with Data Ops & Enablement on documentation and certification. Minimum Knowledge, Skills, and Abilities • Data Engineering & Pipelines: ETL/ELT development, distributed processing, and workflow optimization. • Cloud & Platforms: Hands-on with Databricks and Azure (preferred) or AWS data platforms. • Programming & Development: Proficient in Python and SQL; experience with modular coding, APIs, automation, and source control. • Data Modeling & Quality: Dimensional/semantic modeling; familiarity with data quality, metadata, lineage, and catalog tools. • Collaboration & Communication: Ability to translate business needs into technical solutions and communicate effectively across teams. Experience & Education • Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or related field • 2–5 years in data engineering or analytics engineering roles. • Experience building pipelines and models in Azure/Databricks or equivalent AWS tooling. • Working knowledge of SQL and NoSQL data stores (PostgreSQL, MySQL, MongoDB). #LI-AM1 #LI-Remote The Hershey Company is an Equal Opportunity Employer. The policy of The Hershey Company is to extend opportunities to qualified applicants and employees on an equal basis regardless of an individual's race, color, gender, age, national origin, religion, citizenship status, marital status, sexual orientation, gender identity, transgender status, physical or mental disability, protected veteran status, genetic information, pregnancy, or any other categories protected by applicable federal, state or local laws. The Hershey Company is an Equal Opportunity Employer - Minority/Female/Disabled/Protected Veterans. You may request a reasonable accommodation if you are unable or limited in your ability to use or access our online application process as a result of a disability. You can request an accommodation via phone or email. To request an accommodation via phone, please call +1 877-804-1794 and leave a voicemail with your contact information. You may also email a request for accommodation to ApplicationHelp@hersheys.com. Please be sure to include “Accommodation Needed” in the subject line. This will ensure that your email is routed to the appropriate contact who will handle your request.
Senior Data Engineer
Intus CareCatalyzing data-driven change in the care for low-income, older adults.
• Define and drive the technical vision, architecture, and best practices for new and existing products. • Lead design discussions, code reviews, and ensure high standards for code quality, performance, and security. • Collaborate with product managers, designers, and stakeholders to translate requirements into scalable technical solutions. • Plan, prioritize, and delegate tasks to ensure timely and high-quality project delivery. • Mentor and support engineers, fostering a culture of knowledge sharing, growth, and continuous improvement. • Ensure solutions are robust, scalable, and compliant with healthcare industry standards (e.g., HIPAA). • Identify and address technical risks, bottlenecks, and opportunities for process improvement. • Stay current on emerging technologies and advocate for their adoption where appropriate. • Communicate complex technical concepts clearly to both technical and non-technical audiences. • Foster a collaborative, innovative, and inclusive team environment.
• Shaping the architecture of data products designed for data analytics and data science specifically focusing on use cases like forecasting, feature engineering, customer behaviour, and integration of new data sources. • Leading the way in data transformation by setting up best practices in areas like Data modelling, performance optimisation, Data Governance etc, ensuring that the data used within Prima is consistent, available and reliable. • Build reusable technology that enables teams to ingest, store, transform, and serve their own data products. • Engaging with data scientists and machine learning engineers to explore the product landscape and refine data requirements for enhanced data infrastructure. • Embrace continuous learning and experimentation to stay updated on emerging technologies, from testing open source tools to engaging in community-building activities like Meetups. Your passion for staying at the forefront of the field will drive your journey.
Data & AI Warsaw Tech Summit 2026: Data Platform Engineer – Build the Backbone of AI
CapcoCapco, a Wipro company, is a management & technology consultancy dedicated to the financial services & energy industries
Data Platform Engineer – Build the Backbone of AICapco at Data & AI Warsaw Tech Summit 2026About Capco Capco drives digital transformation across the financial industry. A global consulting firm focused on financial services, Capco partners with leading banks, fintechs, and financial institutions to design and deliver next-generation data platforms, AI solutions, and digital ecosystems. From data strategy and modern platforms to AI-powered decision systems and GenAI innovation, teams unlock measurable value from data. What defines Capco? A fast, flexible, and entrepreneurial environment. Quick decision-making, creative thinking, and real ownership enable people to push the boundaries of what technology can achieve. Capco stands for: • Trusted partnerships with banks, payments providers, and financial institutions • Delivery of modern data platforms and AI-powered systems • Innovation across cloud, data engineering, machine learning, and GenAI • A community of engineers, architects, and consultants solving complex challenges Meet Capco at the Data & AI Warsaw Tech Summit 🚀At this year’s Data & AI Warsaw Tech Summit, Capco will share how financial institutions can move from experimentation to production-grade AI and scalable data ecosystems. Our experts will explore how organizations can: • Build AI-native architectures on modern cloud platforms • Scale machine learning and generative AI solutions across enterprise environments • Transform fragmented data into high-value data products • Embed AI into real business workflows and decision-making systems Capco Speakers at Data & AI Warsaw Tech Summit 🚀Andrzej Worona & Laura Żusin-KaczmarekTopic: From Data to Meaning: Educating AI in Banking with Ontologies: Lessons from FIBO and Conversational Banking Time: 11:50-12:10 CET Intro: Many AI solutions still fall short when it comes to understanding and reasoning about complex financial concepts. The real challenge is about how financial knowledge is represented and shared with machines. Why does AI still misunderstand basic banking terms despite having access to vast amounts of data? How can AI truly understand financial concepts? Using the Financial Industry Business Ontology (FIBO) as an example of structured domain knowledge, we will discuss how formal, machine-readable definitions can provide the contextual foundation AI needs. By analysing selected conversational banking scenarios and example solutions, we will invite participants to reflect together on what the right semantic layer for AI in banking should look like. Join us to discover why the next leap in AI for banking isn’t just about more data or better models, but about building a structured understanding of financial meaning. Looking for Data Engineers Role OverviewWe are looking for a Data Engineer to join our Data & Analytics team. In this role, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures. You will work closely with data analysts, data scientists, and business stakeholders to ensure reliable and high-quality data is available for decision-making. Key Responsibilities - Design, develop, and maintain ETL/ELT data pipelines - Integrate data from multiple sources (APIs, databases, external systems) - Build and optimize data warehouses and data lakes - Ensure data quality, consistency, and availability - Monitor and improve performance of data processing systems - Collaborate with data scientists and analysts to deliver datasets - Create and maintain technical documentation - Implement and manage cloud-based data solutions (AWS, Azure, GCP) Requirements - Proven experience as a Data Engineer or similar role (3+ years) - Strong SQL skills - Proficiency in Python or Scala - Experience with ETL tools (e.g., Airflow, dbt, Informatica, Talend) - Experience with relational and NoSQL databases - Familiarity with cloud platforms (AWS, Azure, or GCP) - Understanding of data warehousing concepts and data modeling - Experience working with large-scale data (e.g., Spark, Hadoop) - Strong problem-solving and communication skills Nice to Have - Experience with BI tools (e.g., Power BI, Tableau) - Knowledge of DataOps and CI/CD practices - Experience working in Agile environments - Familiarity with Docker and Kubernetes Online Recruitment Process - Screening call with the Recruiter - Hiring Manager Technical Interview - Feedback - Offer We offer a flexible collaboration model based on a B2B contract with the opportunity to work on diverse projects.




