AgileEngine logo
AgileEngine

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

Data Engineer (GCP) ID56375

Data EngineerData EngineerFull TimeRemoteMid LevelTeam 1,001-5,000

Location

India

Posted

57 days ago

Salary

0

Seniority

Mid Level

Job Description

Data Engineer (GCP) ID56375

AgileEngine

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE WHAT YOU WILL DO - Build and maintain scalable, distributed, fault-tolerant data pipelines on GCP; - Develop and manage lakehouse layers and Delta Lake workflows using BigQuery and Dataproc; - Collaborate with stakeholders across data engineering, compliance, and business teams; - Design and implement pipelines to acquire, normalise, transform, and release large volumes of financial data; - Design and implement bitemporal data models on BigQuery for regulatory-grade time-series datasets; - Build and maintain testing frameworks for data pipelines and transformation logic; - Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails; - Contribute to shared platform services in a collaborative environment; - Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search using Vertex AI. MUST HAVES - 6–8 years of experience in data engineering; - Proficiency in Python for data pipelines, transformation logic, and automation; - Proficiency in SQL with hands-on experience in BigQuery including partitioning, clustering, and time-series queries; - Experience with Cloud Composer (Apache Airflow) for pipeline orchestration; - Working knowledge of Dataproc (Apache Spark) for batch ingestion and incremental processing; - Experience with AI-assisted development tools such as GitHub Copilot or similar; - Experience with Git version control and collaboration workflows; - Familiarity with REST APIs for integrations; - Familiarity with GCP technologies (Cloud Storage, Pub/Sub, Datastream, Cloud Monitoring, IAM, VPC Service Controls); - Understanding of financial data concepts related to equities and& other asset classes; - Upper-intermediate English level. NICE TO HAVES - Knowledge of data libraries such as pandas or PySpark; - Experience with columnar storage and time-series analytics tools such as ClickHouse; - Familiarity with Dataplex for data governance and lineage; - Understanding of Change Data Capture (CDC) using Datastream; - Understanding of bitemporal data modeling concepts; - Knowledge of financial reference data such as equities, fixed income, or corporate actions; - Experience with BigQuery cost management techniques; - Experience with CI/CD pipelines and Terraform for infrastructure as code; - Exposure to LLMs and& Agentic AI using Vertex AI for data-related use cases. PERKS AND BENEFITS - Remote work & Local connection: Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts. - Legal presence in India: We ensure full local compliance with a structured, secure work environment tailored to Indian regulations. - Competitive Compensation in INR: Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness. - Innovative Projects: Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.

Related Categories

Related Job Pages

More Data Engineer Jobs

Centene Corporation logo

Data Engineer II

Centene Corporation

Transforming the health of the communities we serve, one person at a time.

Data Engineer57 days ago
Full TimeRemoteTeam 10,001+Since 1984H1B No Sponsor

You could be the one who changes everything for our 28 million members by using technology to improve health outcomes around the world. As a diversified, national organization, Centene's technology professionals have access to competitive benefits including a fresh perspective on workplace flexibility. Position Purpose: Develops and operationalizes data pipelines to make data available for consumption (reports and advanced analytics), including data ingestion, data transformation, data validation / quality, data pipeline optimization, and orchestration. Engages with the DevSecOps Engineer during continuous integration and continuous deployment. - Designs and implements standardized data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction (scripts, programs, automation, assisted by automation, etc.) - Designs, develops, implements, tests, documents, and operates large-scale, high-volume, high-performance data structures for business intelligence analytics - Designs, develops, and maintains real-time processing applications and real-time data pipelines - Ensure quality of technical solutions as data moves across Centene’s environments - Provides insight into the changing data environment, data processing, data storage, and utilization requirements for the company and offers suggestions for solutions - Develops, constructs, tests, and maintains architectures using programming language and tools - Identifies ways to improve data reliability, efficiency, and quality; use data to discover tasks that can be automated - Performs other duties as assigned - Complies with all policies and standards Education/Experience: A Bachelor's degree in a quantitative or business field (e.g., statistics, mathematics, engineering, computer science) and requires 2 – 4 years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: - One or more of the following skills are desired. - - SQL Server - SSIS - Healthcare experience Pay Range: $63,600.00 - $114,600.00 per year Centene offers a comprehensive benefits package including: competitive pay, health insurance, 401K and stock purchase plans, tuition reimbursement, paid time off plus holidays, and a flexible approach to work with remote, hybrid, field or office work schedules. Actual pay will be adjusted based on an individual's skills, experience, education, and other job-related factors permitted by law, including full-time or part-time status. Total compensation may also include additional forms of incentives. Benefits may be subject to program eligibility. Centene is an equal opportunity employer that is committed to diversity, and values the ways in which we are different. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other characteristic protected by applicable law. Qualified applicants with arrest or conviction records will be considered in accordance with the LA County Ordinance and the California Fair Chance Act

United States + 1 moreAll locations: United States | Tunisia
$63.6K - $114K / year
Centene Corporation logo

Data Engineer II

Centene Corporation

Transforming the health of the communities we serve, one person at a time.

Data Engineer57 days ago
Full TimeRemoteTeam 10,001+Since 1984H1B No Sponsor

You could be the one who changes everything for our 28 million members by using technology to improve health outcomes around the world. As a diversified, national organization, Centene's technology professionals have access to competitive benefits including a fresh perspective on workplace flexibility. Position Purpose: Develops and operationalizes data pipelines to make data available for consumption (reports and advanced analytics), including data ingestion, data transformation, data validation / quality, data pipeline optimization, and orchestration. Engages with the DevSecOps Engineer during continuous integration and continuous deployment. - Designs and implements standardized data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction (scripts, programs, automation, assisted by automation, etc.) - Designs, develops, implements, tests, documents, and operates large-scale, high-volume, high-performance data structures for business intelligence analytics - Designs, develops, and maintains real-time processing applications and real-time data pipelines - Ensure quality of technical solutions as data moves across Centene’s environments - Provides insight into the changing data environment, data processing, data storage, and utilization requirements for the company and offers suggestions for solutions - Develops, constructs, tests, and maintains architectures using programming language and tools - Identifies ways to improve data reliability, efficiency, and quality; use data to discover tasks that can be automated - Performs other duties as assigned - Complies with all policies and standards Education/Experience: A Bachelor's degree in a quantitative or business field (e.g., statistics, mathematics, engineering, computer science) and requires 2 – 4 years of related experience. Or equivalent experience acquired through accomplishments of applicable knowledge, duties, scope and skill reflective of the level of this position. Technical Skills: One or more of the following skills are desired. - ETL - SQL - Snowflake Pay Range: $63,600.00 - $114,600.00 per year Centene offers a comprehensive benefits package including: competitive pay, health insurance, 401K and stock purchase plans, tuition reimbursement, paid time off plus holidays, and a flexible approach to work with remote, hybrid, field or office work schedules. Actual pay will be adjusted based on an individual's skills, experience, education, and other job-related factors permitted by law, including full-time or part-time status. Total compensation may also include additional forms of incentives. Benefits may be subject to program eligibility. Centene is an equal opportunity employer that is committed to diversity, and values the ways in which we are different. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or other characteristic protected by applicable law. Qualified applicants with arrest or conviction records will be considered in accordance with the LA County Ordinance and the California Fair Chance Act

United States
$63.6K - $114K / year
Centene Corporation logo

Data Engineer II

Centene Corporation

Transforming the health of the communities we serve, one person at a time.

Data Engineer57 days ago
Full TimeRemoteTeam 10,001+Since 1984H1B No Sponsor

• Develops and operationalizes data pipelines to make data available for consumption (reports and advanced analytics) • Engages with the DevSecOps Engineer during continuous integration and continuous deployment • Designs and implements standardized data management procedures around data staging, data ingestion, data preparation, data provisioning, and data destruction • Designs, develops, implements, tests, documents, and operates large-scale, high-volume, high-performance data structures for business intelligence analytics • Designs, develops, and maintains real-time processing applications and real-time data pipelines • Ensure quality of technical solutions as data moves across Centene’s environments • Provides insight into the changing data environment, data processing, data storage, and utilization requirements for the company and offers suggestions for solutions • Identifies ways to improve data reliability, efficiency, and quality; use data to discover tasks that can be automated

Illinois + 3 moreAll locations: Illinois | Nebraska | Tennessee | Wisconsin
$63.6K - $114.6K / year
Job Closed

Role Description Ready to own the data pipeline powering the voice of the next generation of AI characters? You'll be joining a well-funded startup building AI character technology, where speech is a core part of the product experience. Think super natural conversations, handling interruptions, personality shifts and more! You'll own the datasets that power their speech systems — from raw, messy audio through to clean, versioned training corpora that directly drive TTS and ASR model performance. - Own the full data lifecycle — defining specs, auditing and curating large-scale audio and text corpora - Build automated quality metrics and dashboards across SNR, VAD, WER, speaker verification and safety, validated against listening tests - Train and deploy lightweight classifiers for noise detection, diarisation, language ID, and content moderation Qualifications - Deep experience working with speech and audio data at scale — 1M+ hours - Strong ML engineering skills in Python and PyTorch, including training and fine-tuning models like Whisper or Wav2Vec - Practical knowledge of audio processing — torchaudio, librosa, spectrograms, DSP basics - A solid understanding of audio quality metrics — MOS, WER, PESQ/STOI, SNR, speaker verification Requirements - Experience with Spark/Beam, Airflow, SQL or similar data engineering tools - Open-source contributions or publications in speech or audio ML - Background in denoising and enhancement, and how it affects downstream model quality Benefits - Remote, with a preference for European or overlapping timezones - Competitive compensation and equity

Europe + 1 moreAll locations: Europe | EMEA