Full‑Stack data engineer
Location
India
Posted
59 days ago
Salary
0
Seniority
Mid Level
Job Description
Full‑Stack data engineer
Millennium
Full‑Stack data engineer We are seeking a highly self‑sufficient, motivated engineer based with strong full‑stack data engineering skills to join our innovative, dynamic team. This is a remote/offshore role that requires autonomy, excellent communication, and the ability to deliver high‑quality work with limited supervision while collaborating with a predominantly US‑based team. You will build reliable, scalable data products and user experiences that power AI/ML modeling, agentic workflows, and reporting—working end‑to‑end from data ingestion and transformation through to UI. Our Python‑based data platform is undergoing a major evolution toward a modern, cloud‑native ELT architecture. We are standardizing on Snowflake as our central data platform and dbt as our core transformation framework, implementing scalable, maintainable ELT practices that simplify ingestion, modeling, and deployment. This role will be pivotal in independently designing and building robust data pipelines and semantic layers that directly power our AI and machine learning initiatives—delivering clean, reliable, and well‑modeled data assets to our data science team for feature engineering, model training, and production inference. You will collaborate closely (primarily via remote channels) with data scientists and ML engineers to ensure our data ecosystem is optimized for experimentation speed, model performance, and seamless integration into downstream products and services. Key Responsibilities - Remote collaboration & communication: Operate effectively as an offshore member of a distributed team, proactively communicating status, risks, and blockers across time zones and coordinating overlap with US working hours as needed. - Full‑stack data engineering: Build across the entire stack, including data ingestion/acquisition and transformation, APIs, front‑end components, and automated test suites, delivering production‑grade solutions with minimal hand‑holding. - Autonomous delivery & ownership: Take end‑to‑end ownership of features and projects—clarifying requirements, breaking work into milestones, estimating timelines, and delivering high‑quality, well‑documented solutions. - Specification and design: Translate short‑ and long‑term business requirements, architectural considerations, and competing timelines into clear, actionable technical specifications and design documents. - Code quality: Write clean, maintainable, efficient code that adheres to evolving standards and quality processes, including unit tests and isolated integration tests in containerized environments. - Continuous improvement: Contribute to agile practices and provide input on technical strategy, architectural decisions, and process improvements, continuously suggesting better tools, patterns, and automation. Required Skills & Experience - Professional experience: 5+ years in software engineering, with a full‑stack background building complex, scalable data‑engineering pipelines using data warehouse technology, SQL with dbt, Python, AWS with Terraform, and modern UI technologies. - Modern data engineering: Strong experience with medallion data architecture patterns using data warehouse technologies (e.g., Snowflake), data transformation tooling (e.g., dbt), BI tooling, and NoSQL data marts (e.g., Elasticsearch/OpenSearch). - Testing and QA: Solid understanding of unit testing, CI/CD automation, and quality assurance processes for both data pipeline testing and operational data quality tests. - Remote work & autonomy: Proven track record working in a remote or distributed environment, demonstrating self‑motivation, reliable execution, and the ability to make sound technical decisions independently. - Agile methodology: Working knowledge of Agile development practices and workflows (e.g., sprint planning, stand‑ups, retrospectives) in a distributed team setting. - Education: Bachelor’s or Master’s degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field. Preferred Skills & Experience - Machine learning and AI: Hands‑on experience with large language models (LLMs) and agentic frameworks/workflows. - Search and analytics: Familiarity with the ELK stack (Elasticsearch, Logstash, Kibana) for search and analytics solutions. - Cloud expertise: Experience with AWS cloud services; familiarity with SageMaker; and CI/CD tooling such as GitHub Actions or Jenkins. - Front‑end expertise: Experience building user interfaces with Angular or a modern UI stack. - Financial domain knowledge: Broad understanding of equities, fixed income, derivatives, futures, FX, and other financial instruments.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Software Engineer II - Azure Data Lake Storage Gen 2
MicrosoftMicrosoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to any characteristic protected by applicable local laws, regulations, and ordinances.
Overview Are you excited about the possibility of working on the largest Cloud Storage service in the world? Azure Storage team is chartered with building, managing and running the Persistent Cloud Storage for Microsoft Azure Cloud. Azure Storage consists of a Storage Platform which currently powers five different storage services: Blob Storage, Table Storage (NoSQL Key Value store), Queue Storage, Data Lake Storage, File Storage and Disk Storage. We are one of the foundational services in the Azure Cloud. The service hosts data from some of the largest companies in the world plus supports all of Microsoft’s largest online businesses including Xbox, Teams and Bing and has scaled to store exabytes of customer data with the roadmap looking even better in terms of growth. We are looking for a Software Engineer II - Azure Storage with strong coding and problem solving skills and a solid foundation in designing and building reliable software. This position will contribute to building and improving key components of Azure Storage, delivering customer and partner facing features and reliability improvements with high quality. The candidate will work on modern server technologies and highly distributed systems, with a focus on reliability, scalability, performance, consistency and debuggability. If you are ready to take on the challenge of solving impactful problems, designing and implementing well scoped components, and debugging live systems in partnership with experienced engineers, then this is the position for you. Strong collaboration skills, the ability to communicate clearly with teammates and partners, and a strong commitment to our customers will be required. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. Responsibilities - Collaborates with appropriate stakeholders to determine user requirements for a scenario, asks clarifying questions, and helps translate requirements into engineering tasks. - Contributes to the development of design documents for a product, application, service, or platform by proposing options, identifying tradeoffs, and reviewing designs with senior engineers. - Creates, implements, optimizes, debugs, refactors, and reuses code to improve correctness, performance, and maintainability. - Writes and maintains unit, integration, and scenario tests, and uses telemetry and diagnostics to validate health and quality in production. - Delivers well scoped work items end to end, including implementation, code reviews, testing, safe rollout, and follow up based on signals and feedback. - Partners with appropriate stakeholders (for example, project managers and partner teams) to execute on project plans, release plans, and work items for owned areas. - Participates in on call rotation and live site operations by following established playbooks, escalating appropriately, and contributing to mitigation and post incident follow up. Qualifications Required Qualifications: - Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR equivalent experience. Other Requirements: - Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: - Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter. Preferred Qualifications: - Master's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR Bachelor's Degree in Computer Science or related technical field AND 5+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python - OR equivalent experience. #azurecorejobs This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled. Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
WHAT YOU'LL BE WORKING ON - Set up a data migration process from one database to another - Assist with automation of BI processes by building new ETL and ELT scripts - Support and improve current Analytics infrastructure (incl. Airflow) - Review scheduled jobs and queries, perform costs optimization - Work with different partners API sources and implement data upload - Improve analytics processes, set up cross checks and data lineage Our Tech Stack: DB - BigQuery; Language - Python; BI - Tableau WHAT YOU NEED TO SUCCEED IN THIS ROLE - Databases: experience with transactional and analytics databases (Google BigQuery, Snowflake), advanced SQL knowledge - API: working with different APIs and setting up data loads - ETL/ELT Data Pipelines: build and support piplines with Python - Python: data Analysis and preprocess with Python - Airflow: DAGs creation, sensors, Airflow infrustructure support - Additional (would be a benefit): Tableau experience, working with clouds (AWS), Data Science skills - Strong analytical and problem-solving skills - Bachelor's degree in a related field such as Statistics, Mathematics, Computer Science, or a similar quantitative discipline would be a plus - Good level of English and russian language. WHAT WE OFFER - An opportunity to make something great even greater, you can be the reason why we grow, develop, and become the best fintech company on the market! - Career prospects - we are young, we have huge ambitions, and it is important that our employees grow with us - Work with coworkers who are passionate about their business - Compensation that will fully correspond to the competence and knowledge, with yearly performance reviews - The opportunity to attend conferences, courses, and professional development at the company's expense - we favor the continuous development of our employees - Remote type of work - 20 days of vacation time; Latvian bank holidays; Sick leaves; Additional birthday day off You can find out more about our team, our values, and the unique ways we celebrate our successes here: FYST Culture
WHAT YOU'LL BE WORKING ON - Set up a data migration process from one database to another - Assist with automation of BI processes by building new ETL and ELT scripts - Support and improve current Analytics infrastructure (incl. Airflow) - Review scheduled jobs and queries, perform costs optimization - Work with different partners API sources and implement data upload - Improve analytics processes, set up cross checks and data lineage Our Tech Stack: DB - BigQuery; Language - Python; BI - Tableau WHAT YOU NEED TO SUCCEED IN THIS ROLE - Databases: experience with transactional and analytics databases (Google BigQuery, Snowflake), advanced SQL knowledge - API: working with different APIs and setting up data loads - ETL/ELT Data Pipelines: build and support piplines with Python - Python: data Analysis and preprocess with Python - Airflow: DAGs creation, sensors, Airflow infrustructure support - Additional (would be a benefit): Tableau experience, working with clouds (AWS), Data Science skills - Strong analytical and problem-solving skills - Bachelor's degree in a related field such as Statistics, Mathematics, Computer Science, or a similar quantitative discipline would be a plus - Good level of English and russian language. WHAT WE OFFER - An opportunity to make something great even greater, you can be the reason why we grow, develop, and become the best fintech company on the market! - Career prospects - we are young, we have huge ambitions, and it is important that our employees grow with us - Work with coworkers who are passionate about their business - Compensation that will fully correspond to the competence and knowledge, with yearly performance reviews - The opportunity to attend conferences, courses, and professional development at the company's expense - we favor the continuous development of our employees - Remote type of work - 20 days of vacation time; Latvian bank holidays; Sick leaves; Additional birthday day off You can find out more about our team, our values, and the unique ways we celebrate our successes here: FYST Culture
Senior Associate, Complex Data Exchange Specialist
New York LifeAt New York Life, our 180-year legacy of purpose and integrity fuels our future. As we evolve into a more technology-, data-, and AI-enabled organization, we remain grounded in the values that drive lasting impact. Our diverse business portfolio creates opportunities to make a difference across industries and communities—inviting bold thinking, collaborative problem-solving, and purpose-driven innovation. As a Fortune 100 mutual company, we offer a place to grow your skills, contribute to meaningful work, and deliver solutions that matter. Our Benefits We provide a full package of benefits for employees – and have unique offerings for a modern workforce, including leave programs, adoption assistance, and student loan repayment programs. Our Commitment to Inclusion At New York Life, fostering an inclusive workplace is fundamental to who we are and how we serve our communities. We have a longstanding commitment to creating an environment where individuals can contribute their best and succeed together.
Location Designation: Fully Remote Role Overview: The Complex Data Exchange Specialist is responsible for managing the overall process, practice, and project implementation of electronic data interfaces with New York Life clients in support of Life, Accident, Disability, Family Medical Leave products, and various service offerings, with a strong focus on National Accounts and highly complex data integrations. This role serves as both a technical and consultative resource, ensuring high-quality data exchange while educating clients on how their data powers downstream services including administration, billing, eligibility, enrollment, and reporting. The Specialist understands how data flows through New York Life systems—from file receipt and validation to internal processing and operational impact—and proactively advises clients and TPAs on how data structure, file dependencies, and transmission timing affect service delivery and administration outcomes. They balance New York Life standards regarding layout, data integrity, and transmission protocols with client requirements to deliver compliant, scalable, and efficient solutions. The Complex Data Exchange Specialist manages multi-file data environments where multiple client files must be validated, sequenced, reconciled, and merged in order to support accurate system ingestion. These integrations often involve complex file structures, dependent file relationships, and conditional data mapping logic, requiring advanced technical expertise in file architecture, merge capabilities, and data validation. This role also drives inventory control of active interfaces and milestone management in partnership with internal and external stakeholders. The Specialist demonstrates strong analytical capabilities to audit incoming files across multiple data sources, identify discrepancies or systemic errors, and implement corrective actions that improve data integrity and client experience. The Complex Data Exchange Specialist partners closely with Implementation IT, Family Medical Leave Operations, Account Management, and external Ben Tech vendors to ensure deadlines are met, key milestones are achieved, and implementation efforts stay aligned with business objectives. Because this role supports large National Accounts and marquee clients, it requires the ability to manage high-volume, complex integrations with multiple vendors, benefit programs, and file dependencies. The Specialist must proactively assess data risk, ensure data governance standards are met, and maintain operational stability across interconnected systems. Additionally, they act as a leader among peers by proactively contributing expertise, mentoring team members, taking on complex work, driving process improvements, and independently leading projects and training initiatives. What You'll Do: - Integrate with TPAs and Ben Tech partners in support of eligibility, enrollment, remittance, and billing data for new and inforce business. - Serve as a consultative partner to clients and TPAs by educating them on data structure, file specifications, and the operational impact of their data across downstream administration systems. - Implement complex electronic interfaces based on established timelines while maintaining strict inventory control of active integrations, dependent file streams, and in-flight implementations. - Manage multi-file integration environments, ensuring files are received, validated, and sequenced appropriately for system ingestion and downstream processing. - Review, audit, and validate test data across multiple related files and data sources to ensure quality, completeness, and alignment with business rules prior to production implementation. - Identify root causes of file discrepancies, mapping errors, transmission issues, or data inconsistencies across merged files, and recommend corrective actions. - Understand and articulate end-to-end data flow, including how files are received, validated, transformed, merged across multiple sources, and processed within New York Life systems. - Balance New York Life/LDEX standards regarding layout, data, and transmission protocols with client requirements, delivering compliant and scalable solutions for complex client environments. - Prepare and maintain documentation confirming the method of data exchange between New York Life and client/TPA, including file dependencies, mapping logic, merge requirements, and audit controls. - Partner with Production Support and IT to ensure successful receipt, mapping, merging, and processing of data. - Articulate complex file mapping and merge logic requirements clearly to IT and technical partners. - Drive accountability with internal teams and Ben Tech partners to meet key deliverables and milestone deadlines. - Troubleshoot and diagnose issues related to electronic data exchange, multi-file integrations, and system processing impacts. - Track file status and ensure metrics are reported accurately within Salesforce. - Develop and maintain support documentation, SOPs, and training materials for complex integration processes and data governance practices. - Proactively review and suggest improvements to existing procedures to enhance efficiency, data integrity, and scalability of complex integrations. - Serve as a subject matter expert for complex data exchange implementations and National Account integrations, providing guidance and mentorship to peers. What You'll Bring: - Bachelor’s Degree strongly preferred or minimum of 3 years relevant work experience required. - Prior experience working with clients, producers, TPAs, or Ben Tech partners preferred. - Strong understanding of complex data structures, file layouts, and electronic transmission methods preferred. - Demonstrated ability to analyze, audit, and interpret complex and high-volume data files, including multi-file data environments. - Experience managing file dependencies, merge logic, and reconciliation across multiple data sources preferred. - Strong planning, project management, and inventory control skills. - Proficient/advanced Microsoft Excel skills required (VLOOKUP, Pivot Tables, Macros). - Proficiency in Access preferred. - Experience with mainframe TSO and JCL scripting preferred. - Excellent organizational skills with ability to manage multiple complex implementations simultaneously and meet tight deadlines. - Excellent written and verbal communication skills with the ability to explain technical concepts to non-technical audiences. - Strong interpersonal and consultative skills. - Demonstrated ability to drive results and meet key milestones. - Ability to work independently while collaborating effectively across internal business partners. - Demonstrated leadership capability in a fast-paced, evolving environment. - Proven adaptability and willingness to embrace change and continuously improve processes. Pay Transparency Salary Range: $90,000-$130,000 Overtime eligible: Exempt Discretionary bonus eligible: Yes Sales bonus eligible: No Actual base salary will be determined based on several factors but not limited to individual’s experience, skills, qualifications, and job location. Additionally, employees are eligible for an annual discretionary bonus. In addition to base salary, employees may also be eligible to participate in an incentive program. Company Overview At New York Life, our 180-year legacy of purpose and integrity fuels our future. As we evolve into a more technology-, data-, and AI-enabled organization, we remain grounded in the values that drive lasting impact. Our diverse business portfolio creates opportunities to make a difference across industries and communities—inviting bold thinking, collaborative problem-solving, and purpose-driven innovation. Here, you’ll find the rare balance of long-standing stability and forward momentum, supported by an inclusive team that honors tradition while embracing progress. As a Fortune 100 mutual company, we offer a place to grow your skills, contribute to meaningful work, and deliver solutions that matter. Your ideas drive what’s next, and your growth powers it. Our Benefits We provide a full package of benefits for employees – and have unique offerings for a modern workforce, including leave programs, adoption assistance, and student loan repayment programs. Based on feedback from our employees, we continue to refine and add benefits to our offering, so that you can flourish both inside and outside of work. Click here to discover more about our comprehensive benefit options or visit our NYL Benefits Site. Our Commitment to Inclusion At New York Life, fostering an inclusive workplace is fundamental to who we are and how we serve our communities. We have a longstanding commitment to creating an environment where individuals can contribute their best and succeed together. This foundation is rooted in our core values of humanity and integrity, ensuring that every employee feels valued and supported. By embracing a broad range of perspectives and experiences, we achieve greater success and fulfill our promise of providing financial security and peace of mind to families across all communities. Click here to learn more about New York Life’s leadership in this space. Recognized as one of Fortune’s World’s Most Admired Companies, New York Life is committed to improving local communities through a culture of employee giving and volunteerism, supported by the Foundation. We're proud that due to our mutuality, we operate in the best interests of our policy owners. To learn more about career opportunities at New York Life, please visit the Careers page of www.NewYorkLife.com. Visit our LinkedIn to see how our employees and agents are leading the industry and impacting communities. Visit our Newsroom to learn more about how our company is constantly evolving to meet our clients' and employees’ needs. Job Requisition ID: 93686

