CSpring logo
CSpring

Unlocking the power and potential of data.

Enterprise Data Warehouse ETL/Data Engineer

Location

Illinois

Posted

10 days ago

Salary

0

Seniority

Senior

Bachelor Degree5 yrs expEnglishAzureCloudPySparkPythonSparkSQLVault

Job Description

Enterprise Data Warehouse ETL/Data Engineer

CSpring

• Design and develop reusable, parameter-driven ingestion and transformation pipelines • Build and maintain medallion architecture solutions • Develop performant ELT workflows • Create and optimize PySpark notebooks and distributed processing jobs • Design dimensional data models • Implement data vault patterns • Optimize distributed SQL workloads • Implement CI/CD processes • Build monitoring, logging, and auditing solutions • Lead or contribute to cloud modernization initiatives

Job Requirements

  • Deep hands-on expertise with Azure Data Factory
  • Strong experience with Azure Databricks and PySpark
  • Production experience with Azure Synapse Analytics
  • Strong understanding of Azure Data Lake Storage Gen2
  • Experience with Azure Key Vault and Azure AD / Entra ID
  • Experience monitoring and troubleshooting data solutions
  • Advanced SQL skills
  • Strong Python skills
  • Proficiency with T-SQL and familiarity with Spark SQL

Benefits

  • Health insurance
  • Professional development opportunities

Related Categories

Related Job Pages

More Data Engineer Jobs

Data Engineer

UnitedHealth Group

UnitedHealth Group is a healthcare and well-being company that’s dedicated to improving the health outcomes of millions around the world. We are comprised of

Data Engineer10 days ago

Role Description We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data and AI platforms on Azure. This role will focus on enabling enterprise data pipelines, real-time processing, and AI/ML model integration using Databricks and modern cloud technologies. You will enjoy the flexibility to telecommute* from anywhere within the U.S. as you take on some tough challenges. - Design and develop scalable data pipelines using Databricks, Apache Spark, and Python on Azure - Build cloud-native solutions leveraging Azure Data Lake, Azure Data Factory, and Delta Lake - Collaborate with Data Science and AI teams to operationalize ML models and embed them into production workflows - Develop and maintain feature stores, model input pipelines, and real-time/streaming frameworks - Ensure data quality, governance, and security across the full data lifecycle - Build reusable frameworks, accelerators, and automation scripts to improve engineering efficiency - Optimize performance, scalability, and reliability of data workflows and batch/streaming pipelines - Participate in Agile development processes, including sprint planning, code reviews, and CI/CD pipelines - Provide production support and on-call coverage, ensuring system stability and rapid issue resolution - Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI Qualifications - Bachelor’s degree in Computer Science, Engineering, or IT related field - 6+ years of experience in Data Engineering with Python/PySpark - 6+ years of experience in building ETL/ELT pipelines using Databricks - 6+ years of experience working in Agile environments - 5+ years of strong experience in SQL / PL-SQL - 4+ years of experience with Azure Databricks and Delta Lake architecture - 4+ years of hands-on experience with CI/CD (GitHub Actions, Azure DevOps) - 3+ years of hands-on experience with Azure cloud services (ADF, ADLS, Databricks) - 2+ years of experience with Databricks Delta Live Tables (DLT) - 2+ years of experience with unit testing, validation, and pipeline testing frameworks Requirements - Familiarity with medallion architecture and SCD2 implementations - AI builder: Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI - Experience building enterprise-scale data platforms - Strong skills in performance tuning and debugging large-scale pipelines - Experience with real-time/streaming frameworks (Structured Streaming) - Ability to work in distributed, cross-functional global teams - Exposure to GenAI tools (e.g., GitHub Copilot) for engineering productivity - Strong understanding of secure coding practices and vulnerability remediation - Proven ability to analyze logs, troubleshoot production issues, and optimize performance - Demonstrated capability to design and deploy AI-powered solutions responsibly Benefits - Comprehensive benefits package - Incentive and recognition programs - Equity stock purchase - 401k contribution (all benefits are subject to eligibility requirements)

United States
$72.8K - $130K / year
Job Closed
Full TimeRemoteTeam 11-50H1B No Sponsor

• 8–10 years in Data Engineering and Data Analysis. • Strong hands-on experience in Informatica PowerCenter/IDQ for ETL design, development, and optimization. • Advanced skills in PySpark for large-scale data processing, transformation, and analytics. • Solid working knowledge of Hadoop technologies (HDFS, Hive, Sqoop, MapReduce). • Proficiency in Python and Kafka for streaming and batch data pipelines. • Strong understanding of database concepts, data design, data modeling, and ETL workflows. • Experience in analyzing, designing, and coding ETL programs including data extraction, ingestion, quality checks, normalization, and loading. • Hands-on experience with Agile methodology and Jira for project delivery. • Proven ability in client-facing roles with strong communication and leadership skills to coordinate across SDLC. • Exposure to AWS data components and analytics. • Familiarity with machine learning models and AI concepts. • Experience with data modeling tools such as Erwin.

Ohio + 1 moreAll locations: Ohio | Pennsylvania
Capgemini logo

FBS Data Engineer, SQL, Power BI

Capgemini

Founded in 1967, Capgemini is revered as one of the world's leading consulting, technology, and outsourcing agencies. In 2016 alone, the company reported global

Data Engineer10 days ago

• Responsible for acquiring, curating, and publishing data both on prem and in the cloud for analytical or operational uses. • Ensures the data is in a ready-to-use form. • Utilizes skills to translate business analytic requests/requirements. • Works with various technologies from big data, relational and non-relational databases. • Consults on data projects of intermediate complexity. • Practices code management and integration with current architectural and data governance guidelines. • Produces data building blocks and data flows for varying client requests. • Creates business user access methods to data. • Utilizes techniques such as mapping data and transforming data to satisfy business rules. • Translates business data stories into a technical story breakdown structure. • Develops and maintains moderately complex scalable data pipelines. • Executes on mid size projects and identifies opportunities to optimize data workflows.

Mexico
Sales Consulting logo

Data Engineer

Sales Consulting

#recruitment #consultancy #agency #HR #personnelleasing

Data Engineer10 days ago
Full TimeRemoteTeam 51-200Since 1998H1B No Sponsor

• Design, develop, and maintain scalable data pipelines for data ingestion, processing, and transformation • Integrate data from multiple source systems (e.g., ERP, MES, engineering systems) • Ensure data quality, consistency, and reliability across the data platform • Implement and optimize data storage and processing solutions • Collaborate with Data Architects to align with target architecture and standards • Support Data Scientists and business teams by providing well-structured and accessible data • Monitor and troubleshoot data pipelines and workflows • Contribute to the development of a modern, cloud-based data platform • Apply software engineering best practices in data development

Romania