We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
Senior Data Engineer, Microsoft Fabric Engineer
Location
India
Posted
2 days ago
Salary
Rs1,100K - Rs5,000K / year
Seniority
Senior
Job Description
Senior Data Engineer, Microsoft Fabric Engineer
Weekday (YC W21)
• Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. • Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. • Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. • Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. • Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. • Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. • Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. • Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. • Implement best practices for data engineering, governance, documentation, testing, and operational support. • Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes.
Job Requirements
- 5+ years of hands-on experience as a Data Engineer with expertise in enterprise-scale data engineering solutions.
- Strong experience with Microsoft Azure Fabric, Databricks, and Azure data services in production environments.
- Solid understanding of SQL, data modeling, data warehousing concepts, and cloud-based data platforms.
- Proven experience designing and implementing scalable ETL/ELT pipelines and enterprise data integration solutions.
- Proficiency in Python, Apache Spark, or similar technologies for large-scale data processing.
- Experience optimizing cloud-based data workflows for performance, scalability, reliability, and cost efficiency.
- Strong analytical and problem-solving skills with the ability to troubleshoot complex production issues effectively.
- Comfortable using AI-assisted development tools and automation techniques to improve engineering productivity and solution quality.
- Excellent communication and stakeholder management skills, with the ability to collaborate effectively across global, cross-functional teams.
- A proactive, ownership-driven mindset with the flexibility to adapt to evolving priorities while consistently delivering high-quality, business-critical data solutions.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer / Microsoft Fabric Engineer
Weekday (YC W21)We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
Role Description We are looking for an experienced Data Engineer to design, develop, and optimize scalable data platforms and modern cloud-based data solutions. This role is ideal for professionals with strong expertise in Microsoft Azure Fabric, Databricks, Azure data services, and enterprise data engineering, along with a passion for building reliable data pipelines and leveraging AI-assisted development to improve engineering productivity. As a Data Engineer, you will take ownership of the complete data engineering lifecycle—from solution design and implementation to production support and continuous optimization. You will collaborate closely with business stakeholders, architects, analysts, and development teams to transform complex data requirements into scalable, high-performance data solutions that enable analytics and informed business decision-making. Key Responsibilities - Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. - Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. - Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. - Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. - Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. - Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. - Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. - Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. - Implement best practices for data engineering, governance, documentation, testing, and operational support. - Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes. Qualifications - 5+ years of hands-on experience as a Data Engineer with expertise in enterprise-scale data engineering solutions. - Strong experience with Microsoft Azure Fabric, Databricks, and Azure data services in production environments. - Solid understanding of SQL, data modeling, data warehousing concepts, and cloud-based data platforms. - Proven experience designing and implementing scalable ETL/ELT pipelines and enterprise data integration solutions. - Proficiency in Python, Apache Spark, or similar technologies for large-scale data processing. - Experience optimizing cloud-based data workflows for performance, scalability, reliability, and cost efficiency. - Strong analytical and problem-solving skills with the ability to troubleshoot complex production issues effectively. - Comfortable using AI-assisted development tools and automation techniques to improve engineering productivity and solution quality. - Excellent communication and stakeholder management skills, with the ability to collaborate effectively across global, cross-functional teams. - A proactive, ownership-driven mindset with the flexibility to adapt to evolving priorities while consistently delivering high-quality, business-critical data solutions.
Role Description We’re looking for a Data Engineer to join our Data & Analytics team and take ownership of the data infrastructure that powers internal analytics, reporting, and operational insights. This is a hands‑on role where you’ll manage and evolve our ELT pipelines, Snowflake environment, and Python‑based integrations while partnering closely with analysts and systems administrators who rely on high‑quality, reliable data. - Own the day‑to‑day operation, maintenance, and enhancement of ELT pipelines and the Snowflake data warehouse - Troubleshoot and resolve pipeline failures, data quality issues, and performance bottlenecks - Design, build, and document new data models and transformations as analytical needs evolve - Ensure data reliability, consistency, and availability for downstream consumers - Support and extend API integration workflows built with Boomi and Python - Maintain and contribute to custom Python applications used across the team - Collaborate with internal stakeholders to scope and deliver new data integration projects - Participate in evaluating modern data tooling as we consider improvements to our architecture - Document systems, processes, and decisions to build long‑term institutional knowledge Qualifications - Bachelor’s degree in Computer Science, Information Systems, Statistics, or a related field (preferred) - 3–5 years of hands‑on experience in a data engineering role - Strong SQL skills and a solid understanding of data modeling for analytics - Practical experience with Snowflake or a similar cloud data warehouse - Familiarity with ELT/ETL concepts and tools (WhereScape RED, Matillion, Airbyte, dbt, etc.) - Working knowledge of Python for data processing and scripting - Experience with REST APIs and integrating external data sources - Comfort digging into unfamiliar systems to diagnose root‑cause issues - Experience using GitHub for version control Nice to Have - Exposure to enterprise integration platforms (Boomi or similar iPaaS tools) - Experience with Oracle EBS or Salesforce - Familiarity with data pipeline orchestration concepts - Experience partnering closely with BI or analytics teams
• Design, build, and maintain data pipelines that serve models, data, and AI workflows to internal and client-facing applications. • Work across database types — relational, vector, and graph — to model and store data appropriately for each access pattern, in partnership with data scientists. • Build and maintain dbt models — writing transformation logic, tests, and documentation that ensure data quality and traceability. • Operate what you build : instrument pipelines with logging, metrics, and tracing, and help diagnose and resolve production data issues. • Write clean, tested, production-quality code and contribute to CI/CD pipelines and infrastructure-as-code. • Participate in code reviews, design discussions, and retrospectives.
• architect and build production data pipelines and data platforms that serve models, data, and AI workflows to internal and client-facing applications, accountable for them under live traffic • own non-functional quality – latency and throughput budgets, scalability, reliability, observability, and cost – for the systems in your domain • lead the design and operation of multi-model data stores – relational databases (PostgreSQL, MySQL), vector databases (Pinecone, Weaviate, pgvector), and graph databases (Neo4j, Neptune) – ensuring the right tool for each access pattern • set technical direction: write design docs, make build-vs-buy decisions, and defend your approach with evidence • work across the stack when needed – services, data access, infrastructure-as-code, CI/CD – and debug it when things drift in production • mentor and set the quality standards for mid-level and junior data engineers

