TechBiz Global is a leading IT recruitment and software development company
Data Engineer
Location
India
Posted
177 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
TechBiz Global
• Design, develop, and maintain data ingestion pipelines using Kafka Connect and Debezium for real-time and batch data integration. • Ingest data from MySQL and PostgreSQL databases into AWS S3, Google Cloud Storage (GCS), and BigQuery. • Implement best practices for data modeling, schema evolution, and efficient partitioning in the Bronze Layer. • Ensure reliability, scalability, and monitoring of Kafka Connect clusters and connectors. • Collaborate with cross-functional teams to understand source systems and downstream data requirements. • Optimize data ingestion processes for performance and cost efficiency. • Contribute to automation and deployment scripts using Python and cloud-native tools. • Stay updated with emerging data lake technologies such as Apache Hudi or Apache Iceberg.
Job Requirements
- 5+ years of hands-on experience as a Data Engineer or similar role.
- Strong experience with Apache Kafka and Kafka Connect (sink and source connectors).
- Experience with Debezium for change data capture (CDC) from RDBMS.
- Proficiency in working with MySQL and PostgreSQL.
- Hands-on experience with AWS S3, GCP BigQuery, and GCS.
- Proficiency in Python for automation, data handling, and scripting.
- Understanding of data lake architectures and ingestion patterns.
- Solid understanding of ETL/ELT pipelines, data quality, and observability practices.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Architect, build, and maintain next-generation data pipelines • Design and build robust, scalable ELT pipelines to ingest data into Snowflake • Own the dbt project structure, developing complex SQL-based data models • Manage the Snowflake environment for cost-efficiency and performance • Champion data integrity and implement observability tools • Mentor junior engineers and establish best practices for SQL and version control
• Design, build, and maintain scalable, high-quality data pipelines for structured and unstructured data. • Implement robust data ingestion, transformation, and storage using cloud-based technologies. • Collaborate with stakeholders to understand business goals and translate them into data engineering solutions. • Monitor, troubleshoot, and optimize data pipelines for reliability and performance. • Support data validation, testing, and documentation processes. • Contribute to the design and deployment of modern data architectures (e.g., data lakes, lakehouses, data warehouses). • Apply Infrastructure-as-Code (IaC) practices for provisioning and managing cloud resources. • Integrate emerging tools and frameworks to modernize existing data environments. • Ensure security, governance, and compliance in all stages of data handling. • Work in agile teams, contributing to continuous improvement and mentoring junior team members.
• Define and lead end-to-end data architecture for complex ecosystems, balancing technical depth with business outcomes. • Translate business strategy into scalable technology solutions through discovery workshops, assessments, and roadmaps. • Act as technical sponsor for CI&T’s most strategic accounts, working closely with C-level clients and non-technical stakeholders. • Lead architectural design, technology selection, and proofs of concept for critical platforms and innovation programs. • Govern technical quality across delivery squads, driving adherence to security, performance, scalability, and privacy standards. • Support pre-sales, proposal development, RFPs, and technical visioning with account teams and clients. • Coach and mentor senior architects and technical leads, shaping technical career development paths.
Data Architect
Xenon SevenHuman Experts Implementing Artificial Intelligence #AI #ArtificialIntelligence #HumanIntelligence
• Design, implement, and optimize data pipelines using both batch and streaming processing frameworks. • Architect and maintain data lakehouse solutions using Apache Iceberg and object storage such as S3. • Implement scalable Data Vault and Star Schema models. • Build and manage real-time ingestion pipelines with Kafka, Spark, or Flink. • Integrate and orchestrate workflows using tools like Airflow, dbt, NiFi, or Airbyte. • Enforce data governance, data quality, and access control policies. • Troubleshoot pipeline performance and reliability issues.



