Centific Global Solutions

Centific, founded in 2020, is a technology company specializing in artificial intelligence and digital transformation services, focusing on developing scalable

Data Engineer

Location

India

Posted

165 days ago

Salary

0

Seniority

Senior

Bachelor Degree3 yrs expEnglishAzureETLPySparkPythonApache Spark

Job Description

Data Engineer

Centific Global Solutions

• Develop and optimize PySpark-based ETL pipelines for processing large-scale data • Develop, test, and maintain robust Python applications • Collaborate with data engineers, analysts, and other developers to design and implement data solutions • Work with Azure Data Lake, Azure Databricks, and Azure Data Factory to build scalable data solutions • Implement data classification, policy enforcement, and metadata extraction within Purview • Collaborate with data engineers and business teams to ensure smooth data flow across systems • Troubleshoot performance bottlenecks in Spark jobs and improve data pipeline efficiency • Ensure data security, compliance, and quality following best practices.

Job Requirements

  • 3+ years of experience in Python and PySpark for big data processing
  • Proven experience as a Python Developer or similar role
  • Strong experience with Azure Data Services (Data Lake, Data Factory)
  • Excellent problem-solving skills and ability to work in an agile environment
  • Excellent communication and teamwork abilities.

Benefits

  • Work-life balance for all employees
  • Opportunities for skills enhancement

Related Categories

Related Job Pages

More Data Engineer Jobs

CareOregon logo

IS Data Warehouse Architect II

CareOregon

Making health care work for everyone.

Data Engineer165 days ago
Full TimeRemoteTeam 501-1,000Since 1994H1B Sponsor

• Perform Data Warehouse Architecture at an intermediate level • Focus on design and development of data models and ETL specifications • Conduct data analysis, system design and architecture, standards and policy administration • Coordinate vendor relations and conduct product/vendor research • Create moderate to advanced data models and ETL specifications • Maintain documentation on design and operation • Analyze data sources and recommend solutions • Participate in long-term data warehouse and dashboard architecture planning • Manage existing systems compliance with established standards

Oregon
$113.9K - $139.3K / year
Pearson VUE logo

Data Engineer III

Pearson VUE

The potential of every professional. The promise of every industry.

Data Engineer165 days ago
OtherRemoteTeam 1,001-5,000Since 1994H1B No Sponsor

• Develop repeatable ETL pipelines to feed team dashboards, reports, and analytics • Investigate underlying data to assist in production support efforts • Assist platform engineering team with in-app reporting and dashboard development • Work with adjacent teams to develop reliable data extracts that support their work • Implement validation checks and monitoring to maintain data integrity and consistency • Build tools to fabricate data sets for standing up product demos and to assist QA efforts • Perform other duties as assigned

Texas
$85K - $115K / year
Job Closed
Comfrt logo

Senior Data Engineer

Comfrt

"The Only Hoodie Worth Wearing"

Data Engineer166 days ago
OtherRemoteTeam 51-200H1B No Sponsor

Role Description We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team. The ideal candidate will be responsible for designing, developing, and optimizing our data pipeline architecture to support our analytics, machine learning, and business intelligence initiatives. This role requires a strong background in big data technologies, cloud infrastructure, and a passion for building robust, scalable, and efficient data solutions. Responsibilities - Data Architecture and Development: - Design, construct, install, test, and maintain highly scalable data management systems and processing pipelines using cloud-native services (e.g., AWS, GCP, Azure). - Develop and optimize ETL/ELT processes to ingest, transform, and load data from various internal and external sources into our data warehouse/data lake. - Implement data governance, security, and quality controls across all data pipelines. - Ensure data architecture supports the needs of data scientists, analysts, and other business stakeholders. - Optimization and Performance: - Monitor, tune, and optimize data warehouse performance (e.g., Snowflake, BigQuery, Redshift). - Troubleshoot and resolve complex data-related issues and performance bottlenecks in the data platform. - Drive continuous improvement in data platform reliability, efficiency, and cost management. - Collaboration and Mentorship: - Collaborate closely with data scientists, software engineers, and product managers to understand data requirements and deliver solutions. - Define and enforce best practices for data engineering, including coding standards, documentation, and operational procedures. - Mentor and guide junior data engineers, fostering a culture of technical excellence and continuous learning. Qualifications - Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field. - 5+ years of professional experience in data engineering, software engineering, or a related role focused on data infrastructure. - Proven experience designing and building production-grade, highly reliable, and scalable data pipelines. - Expert proficiency in SQL and at least one high-level programming language (e.g., Python, Scala, Java). - Deep expertise with major cloud platforms (AWS, GCP, or Azure), specifically their data and storage services (e.g., S3, Google Cloud Storage, Azure Data Lake Storage). - Solid experience with modern data warehousing solutions (e.g., Snowflake, Google BigQuery, Amazon Redshift). - Experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). - Familiarity with distributed data processing frameworks (e.g., Apache Spark, Dask). - Experience with version control systems (e.g., Git). - Experience with AI tools (Claude, Vertex AI, Gemini). Preferred Qualifications - Experience with stream processing technologies (e.g., Apache Kafka, Kinesis, Pub/Sub). - Exposure to GenAI Integration with development process. - Knowledge of data modeling techniques (e.g., 3NF, Dimensional Modeling). - Familiarity with machine learning pipelines (MLOps). - Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation). - Experience/Familiarity with D2C/E-Commerce Domain. Key Competencies - Problem-Solving: Exceptional analytical and problem-solving skills with a high degree of attention to detail. - Communication: Excellent verbal and written communication skills, with the ability to explain complex technical concepts to non-technical audiences. - Ownership: A proactive mindset and a strong sense of ownership over the data platform and its integrity. Benefits - Generous paid time off. - Company-covered health insurance. - 5% 401k match. - Discounts on all Comfrt products.

United States
$160K - $180K / year
Job Closed
Full TimeRemoteTeam 10,001+H1B No Sponsor

• Diseñar, implementar y mejorar soluciones de ingenieria de Datos • Contribuir en la arquitectura de datos • Liderar al equipo en buenas prácticas • Colaborar con otros equipos, y aprender continuamente.

Colombia
₱10,500K - ₱13,500K / month