Redefining Excellence. Your Partner in Sustainable Software Development.
Senior GCP Data Engineer
Location
United States
Posted
8 days ago
Salary
0
Seniority
Senior
Job Description
Senior GCP Data Engineer
KANINI
• Design, develop, and maintain scalable batch and real-time data pipelines on GCP • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing • Build high-performance data transformations using Python and PySpark • Develop and optimize complex SQL queries for analytical workloads • Work extensively with BigQuery for large-scale data processing and performance tuning • Develop and deploy pipelines using Cloud Dataflow • Orchestrate workflows using Cloud Composer (Apache Airflow) • Manage data storage and lifecycle using Google Cloud Storage (GCS) • Implement version control and CI/CD pipelines using Git-based tools • Ensure data security, governance, and access control using GCP IAM • Optimize data solutions for performance, scalability, reliability, and cost-efficiency
Job Requirements
- Strong hands-on experience with Google Cloud Platform (GCP)
- Expertise in BigQuery (partitioning, clustering, query optimization)
- Proven experience implementing Medallion Data Architecture
- Strong programming skills in Python and PySpark
- Hands-on exposure on Java
- Advanced proficiency in SQL (complex joins, window functions, performance tuning)
- Hands-on experience with Cloud Dataflow
- Experience with Cloud Composer (Airflow) for orchestration
- Experience working with Google Cloud Storage (GCS)
- Knowledge of version control systems (Git) and CI/CD practices
- Strong understanding of GCP IAM and cloud security best practices
Benefits
- Health insurance
- Flexible work arrangements
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Improve code base and data model of the data estate • Assist with maintenance tasks and daily data operations • Monitor data loading process for Datawarehouse ETL/ELT • Migrate Central Backend to Central repository • Collaborate on migration of architecture to Microsoft Fabric • Involve in web scripting data analytics project • Enhance data pipeline and auditing processes • Support production issues across pipelines • Conduct ad hoc data analysis for business stakeholders
Software Engineer – Data Platforms
smartclipWe are shaping the future of online video and TV advertising
• Join our team at smartclip as a Software Engineer, Data Platforms. • You will design and build the engines that process, transform, and refine terabytes of TV and advertising data every single day. • Ensure our data platform is scalable, resilient, and cutting-edge. • Design and implement technical solutions for Big Data applications in an agile, high-performance environment. • Continuously develop and refine our data aggregation pipelines (TV data, advertising, and forecasting) primarily based on Spark and Scala. • Host, manage, and scale your solutions using Docker and Kubernetes within the AWS cloud. • Lead the way in test automation – from unit tests to complex integration tests. • Evaluate new frameworks and tools, share your findings with the team, and help us stay at the forefront of Big Data technology.
Master's Fellow – Data Engineering, Automated Ingestion, AWS S3, Data Governance
Sistema FibraPelo Futuro da Indústria | Pelo Futuro do Trabalho
• The fellow will support the structuring, development, and evolution of a data solution based on Retrieval-Augmented Generation (RAG), with a focus on organizing, governing, and intelligently providing access to the area's information.
• Design, develop, and maintain secure, scalable ETL/ELT pipelines processing structured and unstructured data from multiple sources • Build and optimize data models and processing workflows to support BI, analytics, and reporting needs • Develop monitoring and alerting solutions to ensure pipeline performance and data quality • Build insightful dashboards, reports, and datasets in Domo • Integrate data from external APIs and third-party systems (e.g., Salesforce) • Collaborate with Software Engineering, Architecture, Product, Analytics, and IT teams • Translate business requirements into robust technical solutions • Conduct code reviews and provide feedback to engineering team members • Mentor other data engineers and contribute to onboarding efforts • Participate in Agile ceremonies including standups, retrospectives, and planning




