• Develop and deploy data pipelines for streaming and batch data sources • Create data models and implement business logic using required tools and languages • Collaborate with teams for performance tuning and data processing • Utilize Azure and Spark technologies in daily tasks

• Design and build a green field data mesh platform on AWS. • Create a scalable data foundation for a large distribution company. • Help in building stream processing backend using AWS services, Spark and Kafka. • Ensure that the data pipelines are robust and handle low-quality raw data effectively. • Work collaboratively within a small team to achieve project goals.

• Create and maintain optimal data pipeline architecture. • Assemble large, complex data sets that meet functional / non-functional business requirements. • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. • Build analytics tools that utilize the data pipeline to provide actionable insights for Information Asset as well as Information Asset Customers’ operational efficiency and other key business performance metrics. • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. • Create data tools for analytics to build and optimize offering into an industry leading solution. • Work with data and analytics experts to strive for greater functionality data systems.

• Design and implement data ingestion pipelines for batch and streaming data • Configure and maintain data orchestration workflows (Airflow, NiFi) and CI/CD automation for data processes • Design and organize data layers within Data Lake architecture (HDFS, Iceberg, S3) • Build and maintain secure and governed data environments using Apache Ranger, Atlas, and SDX • Develop SQL queries and optimize performance for analytical workloads in Hive/Impala • Collaborate on data modeling for analytics and BI, ensuring clean schemas and dimensional models • Support machine learning workflows using Spark MLlib or Cloudera Machine Learning (CML)