• Design and develop reusable, parameter-driven ingestion and transformation pipelines • Build and maintain medallion architecture solutions • Develop performant ELT workflows • Create and optimize PySpark notebooks and distributed processing jobs • Design dimensional data models • Implement data vault patterns • Optimize distributed SQL workloads • Implement CI/CD processes • Build monitoring, logging, and auditing solutions • Lead or contribute to cloud modernization initiatives

Role Description We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data and AI platforms on Azure. This role will focus on enabling enterprise data pipelines, real-time processing, and AI/ML model integration using Databricks and modern cloud technologies. You will enjoy the flexibility to telecommute* from anywhere within the U.S. as you take on some tough challenges. - Design and develop scalable data pipelines using Databricks, Apache Spark, and Python on Azure - Build cloud-native solutions leveraging Azure Data Lake, Azure Data Factory, and Delta Lake - Collaborate with Data Science and AI teams to operationalize ML models and embed them into production workflows - Develop and maintain feature stores, model input pipelines, and real-time/streaming frameworks - Ensure data quality, governance, and security across the full data lifecycle - Build reusable frameworks, accelerators, and automation scripts to improve engineering efficiency - Optimize performance, scalability, and reliability of data workflows and batch/streaming pipelines - Participate in Agile development processes, including sprint planning, code reviews, and CI/CD pipelines - Provide production support and on-call coverage, ensuring system stability and rapid issue resolution - Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI Qualifications - Bachelor’s degree in Computer Science, Engineering, or IT related field - 6+ years of experience in Data Engineering with Python/PySpark - 6+ years of experience in building ETL/ELT pipelines using Databricks - 6+ years of experience working in Agile environments - 5+ years of strong experience in SQL / PL-SQL - 4+ years of experience with Azure Databricks and Delta Lake architecture - 4+ years of hands-on experience with CI/CD (GitHub Actions, Azure DevOps) - 3+ years of hands-on experience with Azure cloud services (ADF, ADLS, Databricks) - 2+ years of experience with Databricks Delta Live Tables (DLT) - 2+ years of experience with unit testing, validation, and pipeline testing frameworks Requirements - Familiarity with medallion architecture and SCD2 implementations - AI builder: Design, develop, and deploy AI-powered solutions to address complex business challenges with emphasis on responsible use of AI - Experience building enterprise-scale data platforms - Strong skills in performance tuning and debugging large-scale pipelines - Experience with real-time/streaming frameworks (Structured Streaming) - Ability to work in distributed, cross-functional global teams - Exposure to GenAI tools (e.g., GitHub Copilot) for engineering productivity - Strong understanding of secure coding practices and vulnerability remediation - Proven ability to analyze logs, troubleshoot production issues, and optimize performance - Demonstrated capability to design and deploy AI-powered solutions responsibly Benefits - Comprehensive benefits package - Incentive and recognition programs - Equity stock purchase - 401k contribution (all benefits are subject to eligibility requirements)

• 8–10 years in Data Engineering and Data Analysis. • Strong hands-on experience in Informatica PowerCenter/IDQ for ETL design, development, and optimization. • Advanced skills in PySpark for large-scale data processing, transformation, and analytics. • Solid working knowledge of Hadoop technologies (HDFS, Hive, Sqoop, MapReduce). • Proficiency in Python and Kafka for streaming and batch data pipelines. • Strong understanding of database concepts, data design, data modeling, and ETL workflows. • Experience in analyzing, designing, and coding ETL programs including data extraction, ingestion, quality checks, normalization, and loading. • Hands-on experience with Agile methodology and Jira for project delivery. • Proven ability in client-facing roles with strong communication and leadership skills to coordinate across SDLC. • Exposure to AWS data components and analytics. • Familiarity with machine learning models and AI concepts. • Experience with data modeling tools such as Erwin.

• Responsible for acquiring, curating, and publishing data both on prem and in the cloud for analytical or operational uses. • Ensures the data is in a ready-to-use form. • Utilizes skills to translate business analytic requests/requirements. • Works with various technologies from big data, relational and non-relational databases. • Consults on data projects of intermediate complexity. • Practices code management and integration with current architectural and data governance guidelines. • Produces data building blocks and data flows for varying client requests. • Creates business user access methods to data. • Utilizes techniques such as mapping data and transforming data to satisfy business rules. • Translates business data stories into a technical story breakdown structure. • Develops and maintains moderately complex scalable data pipelines. • Executes on mid size projects and identifies opportunities to optimize data workflows.