Scale Faster, Reduce Costs, Meet Diversity Targets
Senior Data Engineer – ETL Data Modeling
Location
India
Posted
66 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – ETL Data Modeling
RemoteStar
• Implementing ETL/ELT pipelines within and outside of a data warehouse using Python, Pyspark and Snowflakes Snow SQL. • Support Redshift DWH to Snowflake Migration. • Design, implement, and support data warehouse/data lake infrastructure using AWS big data stack, Python, Redshift, Snowflake, Glue/lake formation, EMR/Spark/Scala etc. • Work with data analysts to scale value-creating capabilities, including data integrations and transformations, model features, and statistical and machine learning models. • Work with Product Managers, Finance, Service Engineering Teams and Sales Teams on a day-to-day basis to support their new analytics requirements. • Implement data quality and data governance measures and execute data profiling and data validation procedures • Implement and uphold data governance practices to maintain data quality, integrity, and security throughout the data lifecycle. • Leverage open-source technologies to build robust and cost-effective data solutions. • Develop and maintain streaming pipelines using technologies like Apache Kafka etc.
Job Requirements
- Must have total 5+ yrs. of IT experience and 3+ years' experience in data Integration, ETL/ETL development, and database design or Data Warehouse design
- Broad expertise and experience with distributed systems, streaming systems, and data engineering tools, such as Kubernetes, Kafka, Airflow, Dagster, etc.
- Experience in data transformation, ETL/ELT tool and technologies such as AWS Glue, DBTetc for transforming structured/semi structured and unstructured datasets.
- Experience in ingesting and integrating data from APIs/JDBC/CDC sources.
- Deep knowledge of Python, SQL, relational/ non-relational database design, and master data strategies.
- Experience defining, architecting, and rolling out data products, including ownership of data products through their entire lifecycle.
- Deep understanding of Star and Snowflake dimensional modeling.
- Experience with relational databases, including SQL queries, database definition, and schema design.
- Experience with data warehouses, distributed data platforms, and data lakes.
- Strong proficiency in SQL and at least one programming language (e.g., Python,Scala, JS).
- Familiarity with data orchestration tools, such as Apache Airflow, and the ability to design and manage complex data workflows.
- Familiarity with agile methodologies, sprint planning, and retrospectives.
- Proficiency with version control systems, Bitbucket/Git.
- Ability to work in a fast-paced startup environment and adapt to changing requirements with several ongoing concurrent projects.
- Excellent verbal and written communication skills.
- Preferred/bonus skills: Redshift to Snowflake migration experience.
- Experience with DevOps technologies such as Terraform, CloudFormation, and Kubernetes.
- While not mandatory, experience or knowledge in machine learning techniques is highly preferable, enriching our data engineering capabilities.
- Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases
Benefits
- Dynamic working environment in an extremely fast-growing company
- Work in an international environment
- Work in a pleasant environment with very little hierarchy
- Intellectually challenging, play a massive role in client’s success and scalability
- Flexible working hours
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineer, Intern
MactoresMactores is a trusted leader among businesses in providing modern data platform solutions.
• Write efficient code in the chosen technology for the project - For example - Spark, Apache Beam • Explore new technologies and learn new techniques to solve business problems creatively • Collaborate with many teams - engineering and business, to build better data products and services • Deliver the projects along with the team collaboratively and manage updates to customers on time
Senior AWS Data Engineer
MactoresMactores is a trusted leader among businesses in providing modern data platform solutions.
• Develop and maintain data pipelines using Amazon EMR or Amazon Glue. • Create data models and end-user querying using Amazon Redshift or Snowflake, Amazon Athena, and Presto. • Build and maintain the orchestration of data pipelines using Airflow. • Collaborate with other teams to understand their data needs and help design solutions. • Troubleshoot and optimize data pipelines and data models. • Write and maintain PySpark and SQL scripts to extract, transform, and load data. • Document and communicate technical solutions to both technical and non-technical audiences. • Stay up-to-date with new AWS data technologies and evaluate their impact on our existing systems.
Senior AWS Data Engineer – Freelancer
MactoresMactores is a trusted leader among businesses in providing modern data platform solutions.
• Develop and maintain data pipelines using Amazon EMR or Amazon Glue. • Create data models and end-user querying using Amazon Redshift or Snowflake, Amazon Athena, and Presto. • Build and maintain the orchestration of data pipelines using Airflow. • Collaborate with other teams to understand their data needs and help design solutions. • Troubleshoot and optimize data pipelines and data models. • Write and maintain PySpark and SQL scripts to extract, transform, and load data. • Document and communicate technical solutions to both technical and non-technical audiences. • Stay up-to-date with new AWS data technologies and evaluate their impact on our existing systems.
Senior AWS Data Engineer
MactoresMactores is a trusted leader among businesses in providing modern data platform solutions.
• Develop and maintain data pipelines using Amazon EMR or Amazon Glue. • Create data models and end-user querying using Amazon Redshift or Snowflake, Amazon Athena, and Presto. • Build and maintain the orchestration of data pipelines using Airflow. • Collaborate with other teams to understand their data needs and help design solutions. • Troubleshoot and optimize data pipelines and data models. • Write and maintain PySpark and SQL scripts to extract, transform, and load data. • Document and communicate technical solutions to both technical and non-technical audiences. • Stay up-to-date with new AWS data technologies and evaluate their impact on our existing systems.
