Datamaxis is a WMBE corporation and committed to provide IT services to commercial and government organizations.
Senior Azure Data Engineer
Location
Illinois
Posted
59 days ago
Salary
0
Seniority
Senior
Job Description
Senior Azure Data Engineer
DATAMAXIS, Inc
• Design and build robust, reusable, parameter-driven ingestion and transformation pipeline using Azure Data Factory • Implement medallion architecture on Azure Data Lake Storage Gen2 • Build performant ELT workflows that leverage pushdown to source systems • Develop and optimize PySpark notebooks and jobs on Azure Databricks or Synapse Spark • Design dimensional models and data vault patterns for analytics consumption • Implement Slowly Changing Dimensions and Change Data Capture • Tune distributed SQL workloads in Synapse Dedicated SQL Pool • Implement CI/CD for data pipelines using Azure DevOps • Instrument pipelines with robust logging and monitoring • Lead or contribute to legacy-to-cloud migrations
Job Requirements
- 5+ years of data warehouse development experience
- 5+ years of data modeling experience using ERWIN or similar tools
- 2+ years of experience with Azure Data Factory and Snowflake
- Deep hands-on expertise with Azure Data Factory
- Strong Experience in Data Bricks and PySpark
- Strong working knowledge of Azure Data Lake Storage Gen2
- Experience with Azure Key Vault, Azure AD / Entra ID
- Monitoring and troubleshooting with Azure Monitor
- Advanced SQL — window functions, CTEs, query optimization
- Strong Python for data engineering — pandas, PySpark
Benefits
- Flexible work arrangements
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Architect event-driven pipelines (Kafka) and develop new data models that ensure transactional integrity (ACID) for commercial events like invoices, payments, and adjustments • Automate scalable ETL processes and refactor next-generation data architectures to improve quality, security, and coverage for rapidly growing business demands • Collaborate across teams to codify business processes into self-measuring systems, debugging complex challenges to ensure the reliability of financial operations
• Design and build of ETL processes in collaboration with software and model development teams. • Create and maintain scalable data infrastructure. • Own full pipeline and infrastructure lifecycle including performance monitoring and optimization. • Maintain and improve existing pipelines, ensuring stability over existing requirements and adapting to new needs.
• Design, develop, and maintain scalable data pipelines focused on ingestion via CDC (Change Data Capture) using Oracle GoldenGate; • Configure and manage real-time and near-real-time data replication between source systems and cloud environments; • Ensure data consistency, integrity, and synchronization between source and target systems; • Monitor ingestion pipelines, perform troubleshooting, and optimize CDC process performance; • Support full and incremental (delta) load strategies; • Develop and maintain data processing pipelines using Azure and Databricks (Spark); • Implement transformations following modern data architecture patterns using Bronze, Silver, and Gold layers; • Optimize pipelines for performance, scalability, and cost efficiency; • Work with structured and semi-structured data for analytical consumption, reporting, and AI/ML initiatives; • Collaborate with data architects to define modern Lakehouse architectures; • Support data governance, data catalog, lineage, and compliance initiatives; • Ensure data availability, reliability, security, and quality for downstream consumption.
• As part of the Data Engineering team, you will be responsible for design, development and operations of large-scale data systems operating at petabytes scale. • You will be focusing on real-time data pipelines, streaming analytics, distributed big data and machine learning infrastructure. • You will interact with the engineers, product managers, BI developers and architects to provide scalable robust technical solutions.




