Data Engineer
Location
India
Posted
9 days ago
Salary
0
Seniority
Mid Level
Job Description
Data Engineer
V4C.ai
Role Description We're seeking an experienced Data Engineer/ Sr Data Engineer / Lead Data Engineer with expertise in data engineering across major data platforms. The ideal candidate will have a strong background in Python, SQL, ETL, and data modeling, with experience in tools like Teradata, Informatica, Hadoop, Spark, PySpark, ADF, Snowflake, and Big Data. Cloud knowledge (AWS, Azure, or GCP) is a plus. The role requires a willingness to transition and upskill into Databricks & AI/ML projects. - Design, develop, and maintain large-scale data systems - Develop and implement ETL processes using various tools and technologies - Collaborate with cross-functional teams to design and implement data models - Work with big data tools like Hadoop, Spark, PySpark, and Kafka - Develop scalable and efficient data pipelines - Troubleshoot data-related issues and optimize data systems - Transition and upskill into Databricks & AI/ML projects Qualifications - Relevant years of experience in data engineering - Strong proficiency in Python, SQL, ETL, and data modeling - Experience with one or more of the following: - Teradata - Informatica - Hadoop - Spark - PySpark - ADF - Snowflake - Big Data - Scala - Kafka - Cloud knowledge (AWS, Azure, or GCP) is a plus - Willingness to learn and adapt to new technologies, specifically Databricks & AI/ML Requirements - Experience with Databricks - Knowledge of AI/ML concepts and tools - Certification in relevant technologies Benefits - Competitive salary and benefits - Opportunity to work on cutting-edge projects - Collaborative and dynamic work environment - Professional growth and development opportunities - Remote work opportunities & flexible hours
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineer
Encora DigitalEncora, a leader in digital engineering, drives innovation by crafting cutting-edge, cloud-first, data-first, and AI-first solutions that redefine industries. S
Role Description Location: Brazil Job Mode: Full-time Work Mode: Work from home Essential Skills: - Desire to work at high level with stakeholders to devise, understand and communicate clearly requirements and architecture design for data platforms in an efficient manner. - Large Experience with architecture, governance, security, design, business mapping and understanding, performance and tuning for data lake, data warehouse and other data storage systems and transformation. - Experience with talking with customers and stakeholders and extracting business and technical requirements in high and low level. - Strong knowledge of Data frameworks: Hadoop (YARN, HDFS), Hive, Spark, Kafka, Pentaho, Airflow, AWS data tools. - Experience manipulating data using SQL, NoSQL and unstructured data sources, including metadata. - Strong knowledge of ETL frameworks and workflow processes (data ingestion, clean up and preparation). Highly Desirable Skills: - Experience with Python and Java for system administration. - Large experience with data modeling and data design. - System administration experience on Linux platforms. - Financial and/or banking marketing knowledge is a plus. - Experience with AWS platform. Additional Skills: - Machine learning algorithms and workflow. - Pipeline and workflow orchestration (Oozie, Luigi). - Experience with Kubernetes. Company Description Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include: - Product Engineering & Development - Cloud Services - Quality Engineering - DevSecOps - Data & Analytics - Digital Experience - Cybersecurity - AI & LLM Engineering At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
Principal Data Operations Engineer
State of ColoradoThe State of Colorado is located in the Rocky Mountain region of the western United States. It entered the 100-year-old Union in 1876, earning the nickname "Cen
Role Description This position is term limited with an anticipated end date of approximately two (2) years from the date of hire. This position is eligible for State employee benefits and may be extended as the situation warrants. We are looking for a team player who is passionate about data and finding ways to improve solutions for a diverse customer base. As our new Principal Data Operations Engineer you will be responsible for: - Implementing designs and standards provided by the Data and Integrations Architect. - Providing platform and application administration, configuration management, end user management, security and application level patching and upgrades, as well as source code control, deployment and release management. - Providing operational support ranging from minor bug fixes to major enhancements. - Participating in planning for application replacement and modernization. - Collaborating across departments, establishing configurations and tools for efficient data and integration service consumption. - Driving continuous improvement and managing vendor interactions to achieve organizational goals. Some of the day-to-day opportunities include: - Consulting with Data and Integrations Architects, Principal Developers, and other Data Operations and OIT team members to maintain and enhance existing platforms. - Performing platform administration and support for applications within Data Operations. - Working with Data Architects, Data Engineers, and Integration Developers on data ingestion, transformation, and presentation tasks. - Establishing automation of manual processes, including code deployment and environment provisioning. - Acting as Tier-2 escalation point for on-call/break-fix efforts. - Working with SecOps resources to ensure network security policy is established consistently. - Collaborating with Business Analysts, Customers, Project Managers, and others to assist in the creation of estimates and timelines. - Performing coding or configuration management in accordance with standards and best practices. - Coordinating update releases and other system changes. - Organizing, building, and validating all segments of the code and configurations related to a specific build through CI/CD pipelines. - Ensuring application maintenance and configuration activities are consistent with established service portfolio policies. - Identifying and recommending changes to application and platform policies to improve service quality. Qualifications - A minimum of seven (7) years of experience as a data engineer, DevOps Engineer, or similar software engineering role. - A minimum of one year (1) of experience designing, building, implementing, and maintaining data and system integrations. - Experience with MS Azure DevOps CI/CD, Terraform, and Python. Requirements - Additional appropriate education will substitute for the required experience on a year-for-year basis. - Training or Certification related to the work assigned to the position will be assigned credit towards substitution for experience and/or education. - If the minimum qualifications include a degree requirement, additional appropriate paid or unpaid experience will substitute for the required education on a year-for-year basis. Benefits - Eligible for State employee benefits. - Support for a healthy balance of work and personal time.
Clinical Data Engineer
Oregon Health & Science UniversityWe are Oregon's only public academic health center. In addition to caring for patients, we lead groundbreaking research. We also train the next generation of health care professionals. As Portland's largest employer, we give you opportunities to learn and advance in a system of hospitals and clinics across Oregon and Southwest Washington. All are welcome. OHSU welcomes people of all ages, ethnicities, genders, national origins, religions and sexual orientations. We are striving to build an anti-racist, multicultural institution and encourage people with diverse backgrounds to apply. To request reasonable accommodation, contact askhr@ohsu.edu.
Role Description The Clinical Data Engineer sits on the clinical data team within the broader Clinical Business Intelligence unit, alongside analysts, engineers, administrators, and architects. This position builds new clinical data warehousing solutions, data transformations, and data integration assets, and supports the changes, enhancements, and maintenance of existing assets in support of OHSU clinical data initiatives. You will work closely with cross-functional teams — clinical and operational stakeholders, data architects, and IT specialists — to develop robust data pipelines, implement data quality controls, and deliver trustworthy data that supports clinical decision-making. Development happens primarily in the Epic Caboodle Console, Microsoft SQL Server tools, and Microsoft Fabric Data Engineering tools, with Azure DevOps for version control, code management, and deployment. Duties may extend to other cloud data engineering tools, such as Apache Airflow, as needed. - Design and develop ETL pipelines that extract, transform, and load clinical data from a variety of sources into structures suitable for analysis. - Implement data quality controls that validate, monitor, and maintain the accuracy, completeness, and consistency of clinical data across the pipeline lifecycle. - Contribute to the growth of the OHSU Caboodle Data Warehouse by designing, developing, testing, and implementing custom clinical data models. - Partner with BI architects, developers, analysts, and customers (practice managers, data scientists, quality analysts) to build and publish data models, ETL processes, Lakehouses, Warehouses, Notebooks, and metadata using the Epic Caboodle Console and third-party ETL tools. - Develop data feeds using SSIS or similar tools, ensuring appropriate security review and transport consistent with information privacy and security requirements, business associate agreements, and data use agreements. - Document warehouse content in the Caboodle Console and the Analytics Marketplace so users can determine what data exists, how it is defined, and how it traces back to Epic Clarity. - Troubleshoot ETL failures, data anomalies, and warehouse issues surfaced by automated monitoring, other developers, and end users; resolve or escalate through established processes and communicate status to affected groups. - Recommend improvements to ETL processes, tool sets, data models, and monitoring techniques that increase reliability and efficiency. - Deploy warehouse content using approved Azure DevOps systems and processes, and follow approved SDLC practices throughout. - Manage assigned projects by building timelines, identifying risks and milestones, and reporting status. - Respond to and track issues in Jira Service Desk, gathering information from customers and triaging to resolution. Qualifications - Bachelor’s degree in computer science, a related field, or a clinical field and six years of work-related experience in the information technology field or a combination of clinical or operational healthcare environments; - OR Associate’s degree in computer science, a related field, or a clinical field and seven years of work-related experience in the information technology field or a combination of clinical or operational healthcare environments; - OR Eight years work related experience in the information technology field or a combination of clinical or operational healthcare environments; - OR Equivalent combination of education and experience where one year of experience will be substituted for an Associate’s degree and two years of experience will be substituted for a Bachelor’s degree. Requirements - Minimum of two (2) years of experience as an Application Engineer or Developer (or equivalent classification) developing data warehouse objects and data integration ETL solutions. - Minimum of three (3) years SQL Server Experience, including SSIS and T-SQL, coding, performance tuning, and system optimization. - Minimum of five (5) years with Microsoft SQL Server T-SQL. - Minimum of two (2) years of experience in a medallion architecture data warehouse environment. - One year of experience with Microsoft Fabric using OneLake and Data Engineering tools. - One year of experience with Python or PySpark. Skills and Abilities - Knowledge of data warehousing architecture and dimensional modeling concepts. - Knowledge of data validation and testing methodologies for ETL processes. - Familiarity with data governance and cataloging practices. - Proven communication, analytical, and problem-solving skills. - Ability to manage competing priorities and communicate progress on an ongoing basis with excellent attention to detail. - Ability to accurately document system technical artifacts at a level of detail sufficient for ongoing production support. Certifications - Epic Clarity Data Model Certifications and Epic Caboodle Developer Certification within 6 months of hire. - Microsoft DP-700 Fabric Data Engineer certification within 9 months of hire. Preferred Qualifications - Experience with the Epic Clarity and Caboodle data models. - Experience planning and managing small projects. - Experience with HIPAA and PHI compliance. - Microsoft DP-700 Fabric Data Engineer Certification. Benefits - Healthcare for full-time employees covered 100% and 88% for dependents. - $50K of term life insurance provided at no cost to the employee. - Two separate above market pension plans to choose from. - Vacation - up to 200 hours per year dependent on length of service. - Sick Leave - up to 96 hours per year. - 9 paid holidays per year. - Substantial Tri-Met and C-Tran discounts. - Employee Assistance Program. - Childcare service discounts. - Tuition reimbursement. - Employee discounts to local and national businesses. Why apply to OHSU? We are Oregon's only public academic health center. In addition to caring for patients, we lead groundbreaking research. We also train the next generation of health care professionals. As Portland's largest employer, we give you opportunities to learn and advance in a system of hospitals and clinics across Oregon and Southwest Washington. All are welcome. OHSU welcomes people of all ages, ethnicities, genders, national origins, religions and sexual orientations. We are striving to build an anti-racist, multicultural institution and encourage people with diverse backgrounds to apply. To request reasonable accommodation, contact askhr@ohsu.edu.
Role Description - Fabric Ecosystem Management: - Design and maintain Workspaces, Capacities, and Lakehouses within the MS Fabric environment. - Architecture & Modeling: - Build scalable Medallion Architectures (Bronze/Silver/Gold) using OneLake. - Data Integration: - Develop complex ETL/ELT pipelines using Fabric Data Factory and Spark Notebooks. - Warehousing: - Optimize SQL Analytics Endpoints and Synapse Data Warehouses for high-performance querying. - Governance: - Implement fine-grained security, lineage tracking, and Purview integration. Qualifications - Hands-on experience with Lakehouses, Warehouses, and Power BI integration. - Strong background in Delta Lake formats, Parquet, and Star Schema modeling. - Proficiency in T-SQL, PySpark, and DAX. - Familiarity with Azure Data Lake Storage (ADLS Gen2) and Azure Synapse Analytics. Company Description
