We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent
Senior Data Engineer
Location
India
Posted
9 days ago
Salary
₹1,100K - ₹5,000K / year
Seniority
Senior
Job Description
Senior Data Engineer
Weekday (YC W21)
• Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. • Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. • Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. • Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. • Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. • Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. • Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. • Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. • Implement best practices for data engineering, governance, documentation, testing, and operational support. • Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes.
Job Requirements
- 5+ years of hands-on experience as a Data Engineer with expertise in enterprise-scale data engineering solutions.
- Strong experience with Microsoft Azure Fabric, Databricks, and Azure data services in production environments.
- Solid understanding of SQL, data modeling, data warehousing concepts, and cloud-based data platforms.
- Proven experience designing and implementing scalable ETL/ELT pipelines and enterprise data integration solutions.
- Proficiency in Python, Apache Spark, or similar technologies for large-scale data processing.
- Experience optimizing cloud-based data workflows for performance, scalability, reliability, and cost efficiency.
- Strong analytical and problem-solving skills with the ability to troubleshoot complex production issues effectively.
- Comfortable using AI-assisted development tools and automation techniques to improve engineering productivity and solution quality.
- Excellent communication and stakeholder management skills, with the ability to collaborate effectively across global, cross-functional teams.
- A proactive, ownership-driven mindset with the flexibility to adapt to evolving priorities while consistently delivering high-quality, business-critical data solutions.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design and develop distributed systems in a principled way and take part in high impact architectural decisions • Engage in stimulating technical challenges in topics such as distributed systems, very large databases, SQL processing, data modelling, or systems performance • Assume ownership of components within our blockchain data platform such as data ingestion and decoding • Write code mostly in Kotlin and Go with some Java and Rust with a strong emphasis on good design and performance • Contribute to the internal developer experience by improving local test setups, integration tests, and incremental deployment processes. • Continually improve how we operate the platform, including seeking opportunities to automate incident diagnosis and resolution using AI • Work with one of the fastest engineering teams in the industry. • Building AI agents for automation into the team.
Data and AI Senior Engineer
AmtrakAmtrak, the national rail operator, also known as The National Railroad Passenger Corporation, is a publicly funded railroad service headquartered in Washington
Title: Data & AI Senior Engineer - 90405345 - Remote Location: United States Job Description: Washington, DC, US, 20001 Wilmington, DE, US, 19801 Chicago, IL, US, 60607 Philadelphia, PA, US, 19104 Company: Amtrak Your success is a train ride away! As we move America’s workforce toward the future, Amtrak connects businesses and communities across the country. We employ more than 20,000 diverse, energetic professionals in a variety of career fields throughout the United States. The safety of our passengers, our employees, the public and our operating environment is our priority, and the success of our railroad is due to our employees. Are you ready to join our team? Our values of ‘Do the Right Thing, Excel Together and Put Customers First’ are at the heart of what matters most to us, and our Core Capabilities, ‘Building Trust, Accountability, Effective Communication, Customer Focus, and Proactive Safety & Security’ are what every employee needs to know and do to be most impactful at Amtrak. By living the Amtrak values, focusing on our capabilities, and actively embracing and fostering diverse ideas, backgrounds, and perspectives, together we will honor our past and make Amtrak a company of the future. JOB SUMMARY: Work Environment: Remote The Data & AI Senior Engineer advances Amtrak’s mission to make trusted, high-quality, and intelligent data broadly accessible for decision-making, automation, and innovation. This role is responsible for leading the design and delivery of complex data and AI solutions that power Amtrak’s analytics, operational systems, and digital experiences. Senior Engineers apply deep technical expertise to architect and optimize pipelines, integrations, and data services. They ensure solutions meet performance, security, and governance standards while mentoring junior engineers and influencing design decisions across teams. The Senior Data & AI Engineer acts as both a hands-on developer and a technical leader, helping to scale Amtrak’s data and AI capabilities through standardization, reusability, and continuous improvement. ESSENTIAL FUNCTIONS: - Lead the design and evolution of scalable data and AI platform capabilities across Databricks (lakehouse), SAP data environments (e.g., Datasphere/S/4), and virtualization layers (e.g., Denodo) to deliver a unified, governed, self-service ecosystem. - Provide hands-on technical guidance in coding, API-first integrations, model lifecycle management, pipeline development, and integration patterns. Architect and enforce engineering standards across ingestion, transformation, feature engineering, model deployment, and governance-as-code controls embedded directly into pipelines. - Mentor junior and mid-level engineers, fostering skill development, collaboration, and engineering excellence. - Partner with architects, product owners, and governance leads to align solutions with Amtrak’s enterprise data strategy and roadmap. - Design and implement reusable platform accelerators including APIs, templates, and feature engineering patterns that enable self-service analytics and AI across domains. MINIMUM QUALIFICATIONS: • Education: Bachelor’s degree in Computer Science, Data Engineering, or a related technical field; equivalent experience may be considered. • Experience: 4–6 years of experience in data engineering, software development, or data architecture. PREFERRED QUALIFICATIONS: - Experience in a leadership capacity within a scaled agile environment or data platform modernization initiative. - Familiarity with enterprise data governance, metadata management, and security standards. - Exposure to MLOps, cloud data platforms, and automation tools that accelerate delivery. - Experience implementing production-grade MLOps pipelines, including model versioning, CI/CD, monitoring, and governance controls embedded into data and AI workflows. - Prior mentorship or technical leadership of multi-functional project teams. REQUIRED KNOWLEDGE, SKILLS and ABILITIES: - Advanced proficiency in Python and SQL with deep experience in distributed data processing (e.g., Spark) and modern lakehouse architectures (Databricks strongly preferred), including integration of SAP data platforms and virtualization technologies into enterprise-scale solutions. - Strong understanding of data quality, observability, and performance optimization. - Demonstrated ability to lead teams through technical challenges while remaining hands-on in design and coding. - Experience working within agile product teams, coordinating across multiple stakeholders. - Strong understanding of API-first and event-driven architecture patterns, including secure service-to-service communication and role-based access control (RBAC). - Experience embedding data quality validation, schema enforcement, lineage tracking, and policy-as-code controls directly into pipelines and AI workflows. - Excellent communication, collaboration, and problem-solving abilities. - Experience building reusable cost effective, feature stores, semantic layers, or internal platform services that enable self-service analytics and AI. The salary/hourly range is $86,500.00 – $111,996.00. Pay is based on several factors including but not limited to education, work experience, certifications, etc. Depending on an employee’s assigned worksite or location, Amtrak may consider a geo-pay differential to be applied to the employee’s base salary. Amtrak may offer additional incentive and pay programs to recognize and reward our employees, including a short-term incentive bonus based upon factors such as individual and company performance that is commensurate with the level of the position. In addition to your salary, Amtrak offers a comprehensive benefit package that includes health, dental, and vision plans; health savings accounts; wellness programs; flexible spending accounts; 401K retirement plan with employer match; life insurance; short and long term disability insurance; paid time off; back-up care; adoption assistance; surrogacy assistance; reimbursement of education expenses; Public Service Loan Forgiveness eligibility; Railroad Retirement sickness and retirement benefits; and rail pass privileges. Learn more about our benefits offerings here. Requisition ID:166067 Work Arrangement:06-Onsite 4/5 Days Click here for more information about work arrangements at Amtrak. Relocation Offered:No Travel Requirements:Up to 25% You power our progress through your performance. We want your work at Amtrak to be more than a job. We want your career at Amtrak to be a fulfilling experience where you find challenging work, rewarding opportunities, respect among colleagues, and attractive compensation. Amtrak maintains a culture that values high performance and recognizes individual employee contributions. Amtrak is committed to a safe workplace free of drugs and alcohol. All Amtrak positions requires a pre-employment background check that includes prior employment verification, a criminal history check and a pre-employment drug screen. Candidates who test positive for marijuana will be disqualified, regardless of any state or local statute, ordinance, regulation, or other law that legalizes or decriminalizes the use or possession of marijuana, whether for medical, recreational, or other use. Amtrak's pre-employment drug testing program is administered in accordance with DOT regulations and applicable law. In accordance with DOT regulations (49 CFR § 40.25), Amtrak is required to obtain prior drug and alcohol testing records for applicants/employees intending to perform safety-sensitive duties for covered Department of Transportation positions. If an applicant/employee refuses to provide written consent for Amtrak to obtain these records, the individual will not be permitted to perform safety-sensitive functions. In accordance with federal law governing security checks of covered individuals for providers of public transportation (Title 6 U.S.C. §1143), Amtrak is required to screen applicants for any permanent or interim disqualifying criminal offenses. Note that any education requirement listed above may be deemed satisfied if you have an equivalent combination of education, training and experience. Amtrak is an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race/color, to include traits historically associated with race, including but not limited to, hair texture and hairstyles such as braids, locks and twists, religion, sex (including pregnancy, childbirth and related conditions, such as lactation), national origin/ethnicity, disability (intellectual, mental and physical), veteran status, marital status, ancestry, sexual orientation, gender identity and gender expression, genetic information, citizenship or any other personal characteristics protected by law.
Principal Data Engineer
CitizensCitizens, its parent, subsidiaries, and related companies (Citizens) provide equal employment and advancement opportunities to all colleagues and applicants for employment without regard to age, ancestry, color, citizenship, physical or mental disability, perceived disability or history or record of a disability, ethnicity, gender, gender identity or expression, genetic information, genetic characteristic, marital or domestic partner status, victim of domestic violence, family status/parenthood, medical condition, military or veteran status, national origin, pregnancy/childbirth/lactation, colleague’s or a dependent’s reproductive health decision making, race, religion, sex, sexual orientation, or any other category protected by federal, state and/or local laws. At Citizens, we are committed to fostering an inclusive culture that enables all colleagues to bring their best selves to work every day and everyone is expected to be treated with respect and professionalism. Employment decisions are based solely on merit, qualifications, performance and capability.
Role Description - Set a clear and comprehensive vision for the credit and debit card servicing experiences, based on market analysis, business needs, and technical feasibility. - Develop, prioritize, and communicate the product backlog to the development team and relevant stakeholders. - Collaborate with UX/UI experts to ensure the creation of intuitive, user-friendly product experiences. - Leverage UML modeling to create structural and behavioral diagrams for applications, facilitating a clear understanding of the product's architecture and functionality. - Implement and manage robust CI/CD processes to streamline the product development and deployment pipeline. - Work closely with domain architects and subject matter experts to ensure efficient project development strategies. - Utilize data analytics to measure development progress and effectiveness against objective key results (OKRs). - Collaborate with Software Test Engineers to identify areas for automation testing and ensure high-quality products through black-box, functional, regression, and user acceptance testing (UAT). - Gather technical requirements for application programming interfaces (APIs), web services, batch jobs, and mobile applications. - Coordinate with external vendors to ensure smooth API integration and management of API request and response parameters. - Ensure that products meet all security and compliance standards. - Implement security best practices and conduct regular security assessments to identify and mitigate potential risks. Qualifications - Master’s degree in Information Technology, Computer Science, or related field and 3 years of experience as Digital Experience Owner, Product Owner, or a closely related position. - In the alternative, employer will accept a Bachelor’s degree in the specified fields and 5 years of the specified progressive, post-baccalaureate experience. - Must have at least 3 years of experience with: - Utilizing Jira, HP Quality Center, TFS - Project Management, and Microsoft Project to efficiently manage tasks, track progress, and ensure successful project execution. - Utilizing Confluence, Google Docs, and Microsoft SharePoint to effectively document project requirements, designs, and architectural diagrams. - Utilizing Figma, Sketch, and Adobe XD to create product experiences and visually represent product architecture and functionality through UML modeling. - Utilizing Splunk, Google Analytics, and Adobe Analytics to gain insights into user behavior and make data-driven decisions to improve product features. - Utilizing Git, GitHub, and Bitbucket to manage code and promote collaboration among developers. - Implementing CI/CD processes using Jenkins and GitLab CI/CD to streamline the product development and deployment pipeline. - Utilizing Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) to deploy scalable and reliable products and services. - Utilizing Postman and Swagger (OpenAPI) to effectively design, test, and document APIs. - Utilizing Selenium and TestRail to conduct automated testing for web applications, ensuring efficient and repeatable testing processes. - Utilizing Datadog to identify and address performance issues and bottlenecks in web applications and services. - Utilizing Lucidchart, Draw.io, MS Visio, and Visual Paradigm to create flowcharts, diagrams, and visual representations of processes and systems. - Utilizing Salesforce administration tasks, including user management, security settings, and data access controls, to secure and organize Salesforce environments. - Utilizing Tableau Prep for data preparation, transformation, cleaning, and structuring. - Utilizing change management tools, including ServiceNow and Jira Service Management, to facilitate transitions during product implementations and updates. - Utilizing Tableau and Power BI to conduct financial and business analysis. Benefits - 100% remote work permitted from any US location. - Worksite: One Citizens Bank Way, Johnston, RI 02919 United States, 40 hrs/wk. Company Description For summary of benefits and to apply visit our website at https://jobs.citizensbank.com and apply online (Job ID: 48179) or email resume and cover letter to Recruiting@Citizensbank.com with subject line: 48179. Direct applicants only. Equal Employment Opportunity Citizens, its parent, subsidiaries, and related companies (Citizens) provide equal employment and advancement opportunities to all colleagues and applicants for employment without regard to age, ancestry, color, citizenship, physical or mental disability, perceived disability or history or record of a disability, ethnicity, gender, gender identity or expression, genetic information, genetic characteristic, marital or domestic partner status, victim of domestic violence, family status/parenthood, medical condition, military or veteran status, national origin, pregnancy/childbirth/lactation, colleague’s or a dependent’s reproductive health decision making, race, religion, sex, sexual orientation, or any other category protected by federal, state and/or local laws. At Citizens, we are committed to fostering an inclusive culture that enables all colleagues to bring their best selves to work every day and everyone is expected to be treated with respect and professionalism. Employment decisions are based solely on merit, qualifications, performance and capability. Equal Employment and Opportunity Employer Job Applicant Data Privacy Policy Background Check: Any offer of employment is conditioned upon the candidate successfully passing a background check, which may include initial credit, motor vehicle record, public record, prior employment verification, and criminal background checks. Results of the background check are individually reviewed based upon legal requirements imposed by our regulators and with consideration of the nature and gravity of the background history and the job offered. Any offer of employment will include further information.
• Du entwickelst, implementierst und optimierst skalierbare Data Pipelines für zuverlässige und sichere Datenflüsse. • Du modellierst und transformierst Daten nach modernen Best Practices, inklusive Versionierung, Historisierung und sauberer Dokumentation. • Du entwickelst und betreibst ETL und ELT Prozesse sowie orchestrierte Datenpipelines. • Du arbeitest mit modernen Cloud Technologien, bevorzugt AWS und unterstützt den Ausbau unserer Data Platform. • Du nutzt Infrastructure as Code, idealerweise mit Pulumi oder Terraform, um unsere Datenplattform stabil und skalierbar weiterzuentwickeln. • Du analysierst und optimierst Datenprozesse im Hinblick auf Performance, Skalierbarkeit, Stabilität und Datenqualität. • Du arbeitest eng mit Data Scientists, Analysten, Produktteams und Entwickler:innen zusammen, um datengetriebene Entscheidungen im Unternehmen zu ermöglichen. • Du bringst dich aktiv in die Weiterentwicklung unserer Data Engineering Standards ein und hilfst dabei, unsere bestehende BI und Data Landschaft zu modernisieren.

