AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
Data Engineer (GCP) ID56375
Location
India
Posted
43 days ago
Salary
0
Seniority
Mid Level
Job Description
Data Engineer (GCP) ID56375
AgileEngine
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards. WHY JOIN US If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! ABOUT THE ROLE WHAT YOU WILL DO - Build and maintain scalable, distributed, fault-tolerant data pipelines on GCP; - Develop and manage lakehouse layers and Delta Lake workflows using BigQuery and Dataproc; - Collaborate with stakeholders across data engineering, compliance, and business teams; - Design and implement pipelines to acquire, normalise, transform, and release large volumes of financial data; - Design and implement bitemporal data models on BigQuery for regulatory-grade time-series datasets; - Build and maintain testing frameworks for data pipelines and transformation logic; - Own end-to-end solutions including ingestion pipelines, QA workflows, correction management, and audit trails; - Contribute to shared platform services in a collaborative environment; - Support implementation of AI solutions including data ingestion, anomaly detection, and semantic search using Vertex AI. MUST HAVES - 6–8 years of experience in data engineering; - Proficiency in Python for data pipelines, transformation logic, and automation; - Proficiency in SQL with hands-on experience in BigQuery including partitioning, clustering, and time-series queries; - Experience with Cloud Composer (Apache Airflow) for pipeline orchestration; - Working knowledge of Dataproc (Apache Spark) for batch ingestion and incremental processing; - Experience with AI-assisted development tools such as GitHub Copilot or similar; - Experience with Git version control and collaboration workflows; - Familiarity with REST APIs for integrations; - Familiarity with GCP technologies (Cloud Storage, Pub/Sub, Datastream, Cloud Monitoring, IAM, VPC Service Controls); - Understanding of financial data concepts related to equities and& other asset classes; - Upper-intermediate English level. NICE TO HAVES - Knowledge of data libraries such as pandas or PySpark; - Experience with columnar storage and time-series analytics tools such as ClickHouse; - Familiarity with Dataplex for data governance and lineage; - Understanding of Change Data Capture (CDC) using Datastream; - Understanding of bitemporal data modeling concepts; - Knowledge of financial reference data such as equities, fixed income, or corporate actions; - Experience with BigQuery cost management techniques; - Experience with CI/CD pipelines and Terraform for infrastructure as code; - Exposure to LLMs and& Agentic AI using Vertex AI for data-related use cases. PERKS AND BENEFITS - Remote work & Local connection: Work where you feel most productive and connect with your team in periodic meet-ups to strengthen your network and connect with other top experts. - Legal presence in India: We ensure full local compliance with a structured, secure work environment tailored to Indian regulations. - Competitive Compensation in INR: Fair compensation in INR with dedicated budgets for your personal growth, education, and wellness. - Innovative Projects: Leverage the latest tech and create cutting-edge solutions for world-recognized clients and the hottest startups.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Serve as a principal software engineer for AdSmart products • Architect and develop mission-critical backend services using microservices, serverless, and event-driven patterns under the leadership of the VP of Engineering • Participate in scrum ceremonies and perform peer code reviews • Utilize cutting-edge cloud computing technologies to solve problems • Drive integration of LLMs, AI agents, vector search, and ML-based personalization • Supporting products with the overall roadmap and providing updates to senior leadership
Senior Data Engineer
AcadiaAcadia Healthcare is a leading provider of behavioral healthcare services nationwide. Our organization values input from employees and fosters collaboration to create a team-oriented service delivery system.
Job DetailsJob Location: Brazil - State/ProvinceAbout the role: Please submit a resume in English and respond to application questions to be considered. The Senior Data Engineer delivers the data strategy that powers analytics at Acadia, developing and implementing solutions that support data analysis, machine learning, and other advanced analytics techniques. They are responsible for building and maintaining ETL/ELT processes and data warehouses to ensure data is accurate, performant, and reliable for client projects and analyses. The role also includes automating procedures for client reporting and data exports, as well as extracting, transforming, and loading datasets from various sources. The Senior Data Engineer works closely with Senior, Mid, and Associate Engineers/Developers, as well as Business Intelligence and Data Science teams. Responsibilities: Building and maintaining client data warehouses - within cloud-based environments Develop, optimize, and maintain ETL/ELT pipelines (e.g., dbt, Azure DevOps) Maintains company procedures for change control and database project tracking and reporting Develop, implement and maintain application layer data protection solutions to ensure the security and privacy of Acadia’s confidential data Functions as a data owner and assigns database level access privileges Supporting analytics and modeling with Business Intelligence and Data Science teams Developing automated procedures to support overall analytics projects Researching potential solutions for future implementations Staying up to date on the latest cyber security threats through company provided training Review queries and/or database updates for quality assurance Database documentation (e.g., data dictionaries, ETL/ELT documentation) QualificationsRequirements: 5–8 years of experience in database development or data engineering Hands-on experience with Snowflake and dbt Advanced proficiency in SQL language and solid experience with Python Demonstrated experience in database and ETL/ELT design and implementation Experience working in AWS environments; Airflow experience preferred Strong attention to detail and commitment to data quality Excellent problem-solving and critical thinking skills Knowledge of media/marketing data (e.g., ad platforms, impressions, clicks, conversions) and how it flows into reporting and analytics is preferred Fluency in English is a must Please submit a resume in English and respond to application questions to be considered.
• Analyze user behavior, product performance, and key business metrics to identify trends and opportunities. • Create and maintain interactive dashboards and reports using tools like Tableau, Looker, Power BI, or Google Data Studio. • Collect, clean, and structure large datasets to ensure accuracy and usability. • Define and track KPIs for marketing, product, and business performance. • Work with SQL, Python, or R to extract, analyze, and manipulate data. • Collaborate with product managers, engineers, and marketing teams to provide data-driven insights. • Conduct A/B testing and experiment analysis to improve product features and user engagement. • Identify bottlenecks and opportunities in user journeys and customer funnels. • Present findings in a clear and compelling way to stakeholders and leadership teams. • Stay up to date with industry trends, analytics best practices, and emerging data tools.
• Analyze user behavior, product performance, and key business metrics to identify trends and opportunities. • Create and maintain interactive dashboards and reports using tools like Tableau, Looker, Power BI, or Google Data Studio. • Collect, clean, and structure large datasets to ensure accuracy and usability. • Define and track KPIs for marketing, product, and business performance. • Work with SQL, Python, or R to extract, analyze, and manipulate data. • Collaborate with product managers, engineers, and marketing teams to provide data-driven insights. • Conduct A/B testing and experiment analysis to improve product features and user engagement. • Identify bottlenecks and opportunities in user journeys and customer funnels. • Present findings in a clear and compelling way to stakeholders and leadership teams. • Stay up to date with industry trends, analytics best practices, and emerging data tools.


