• The Lead Azure Data Engineer is responsible for migration, governance, and data integration for the implementation of new Microsoft Cloud based document routing and task management system. • They will ensure alignment with the technical, security, and document management requirements such as leading data discovery and mapping, validating metadata standards, managing data migration into Dataverse and SharePoint Online, and ensuring proper implementation of retention, auditability, and role-based access controls within the Microsoft SaaS architecture.

• Design, build, and optimize high-volume, high-performance ELT pipelines for centralized data warehousing • Collaborate with Product Managers, ML Engineers, Data Scientists, and DevOps to define and enforce a reliable, scalable, and secure data platform architecture • Ensure adherence to data warehousing standards, data quality best practices, and metadata management processes • Take ownership of key data warehouse components and drive improvements in performance and reliability • Conduct architecture reviews and participate in system design discussions to provide technical leadership • Mentor and guide junior engineers, fostering a culture of quality and continuous learning

• Design and develop scalable, reliable data platforms and pipelines for analytics and operations • Implement data processing workflows using distributed frameworks (Apache Spark/pySpark, Databricks) • Evolve the organization’s cloud data platform leveraging Azure technologies (Data Factory, Synapse, Event Hub, Azure Data Lake Storage) • Model data to support analytics, reporting, and downstream consumption • Integrate and process data from multiple sources ensuring quality and consistency • Monitor and optimize pipeline performance, ensuring reliability and scalability • Collaborate with analysts, scientists, and engineers to translate requirements into solutions • Improve data engineering standards, testing, and operational reliability • Evaluate and introduce new tools and technologies in the data engineering ecosystem

• Design, build, and optimize scalable data pipelines in Apache NiFi to automate ingestion, cleansing, and enrichment. • Architect and manage datasets on HDFS for high-volume storage. • Develop distributed processing workflows in PySpark and Hive. • Maintain and tune Postgres databases for high availability and performance. • Partner with Data Scientists to deliver clean datasets for model training. • Implement monitoring and alerting frameworks for pipeline health.