Get the Future You Want
Senior Data Engineer, Airflow
Location
Mexico
Posted
110 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer, Airflow
Capgemini
• Build and maintain automated data workflows and orchestrations using Apache Airflow • Build automation processes using Copilot to generate Airflow DAGs • Implement at least two major end-to-end data pipeline projects using Airflow • Design and optimize complex DAGs for scalability, maintainability, and reliability • Create reusable, parameterized, and modular Airflow components (operators, sensors, hooks) to streamline workflow development • Ensure effective monitoring, alerting, and logging of Airflow DAGs for quick issue resolution • Document workflows, solutions, and processes for team knowledge sharing and training • Mentor and support other team members in Airflow usage and adoption • Explain best practices, identify pros and cons, and communicate technical decisions to team members • Develop reusable frameworks, leveraging reusable concepts for efficiency and scalability • Implement and utilize reusable ecosystem components, including Python & Apache Airflow, DynamoDB, Amazon RDS • Develop reusable frameworks to enforce data governance and data quality standards • CI/CD pipeline development using re-usable frameworks and Jenkins
Job Requirements
- Between 4-6 years of experience in a similar role
- Bachelor's degree in IT, Information systems, Computer Science or a related field
- Insurance Experience (Desirable)
- Fluency in English
- Airflow (MUST) / Astronomer (PLUS) - Advanced (5 Years)
- Python - Advanced (4-6 Years) (MUST)
- Snowflake – Intermediate (MUST)
- DBT - Entry Level (PLUS)
- AWS Glue - Entry Level (PLUS)
- DynamoDB - Intermediate
- Amazon RDS - Intermediate
- Jenkins - Intermediate
- Work Independently
- Strategic Thinking
- Guide Others
- Documentation
- Explain best practices
- Communicate Technical Decisions
Benefits
- A competitive salary and performance-based bonuses.
- Comprehensive benefits package.
- Flexible work arrangements (remote and/or office-based).
- You will also enjoy a dynamic and inclusive work culture within a globally renowned group.
- Private Health Insurance.
- Paid Time Off.
- Training & Development opportunities in partnership with renowned companies.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• DecisionPoint seeks a Data Architect to lead the design, governance, and validation of enterprise data architectures for a Department of Defense (DoD) modernization program supporting Casualty and Mortuary Affairs (CMA) operations across all Service components. • The Data Architect will define and maintain logical and physical data models across Impact Level (IL) 2, 4, and 5 environments, ensuring high availability, data integrity, and secure data exchange within a cloud-hosted ecosystem. • The Data Architect will work closely with system engineers, database administrators, and developers to standardize schemas, improve data quality, and align data governance with DoD enterprise architecture standards.
Senior Data Engineer
General DynamicsA business unit of General Dynamics, General Dynamics Information Technology (GDIT) supports some of the United States' most complex government, defense, and in
• Provide advanced analytical, machine learning, and data engineering support. • Design, develop, and deploy machine learning models for classification, regression, time series forecasting, and natural language processing applications. • Build and optimize automated, scalable ETL/ELT pipelines using Python, SQL, and cloud-based tools. • Develop and maintain production ML systems including model deployment, monitoring, versioning, and performance tracking. • Design, develop, and deploy interactive dashboards and data visualizations. • Perform end-to-end model development including exploratory data analysis, feature engineering, hyperparameter tuning, model validation, and documentation. • Develop and maintain data pipelines and workflows using tools such as AWS services, Databricks, and GitLab CI/CD. • Conduct data mining, cleaning, and manipulation using SQL, Python (Pandas, NumPy), or R.
• As a Lead Data Engineer, you will drive the successful development of solution architecture and the completion of data pipeline connectors that automate the flow of data between client data platforms and our analytic health solution platform. • Improving the process of how we design and document data pipelines • Identifying high leverage improvement projects and driving them to completion • Support of our processes in partaking in peer code reviews, sprint planning, product grooming, maintaining Jira tasks and peer test reviews • You will be expected to contribute to multiple implementations simultaneously, which will include both new customer setup as well as support and enhancements for existing customers
Junior Data Engineer
Refinery89Helping publishers and advertisers to drastically increase their results from ads #ForPublishersByPublishers
• Design, build, and maintain data pipelines supporting Refinery89’s core data platform. • Contribute to the design and evolution of the company’s data architecture and tooling. • Implement reliable ingestion, transformation, and storage processes for structured and semi-structured data. • Ensure data reliability, consistency, and basic data quality controls across systems. • Collaborate with engineers and internal stakeholders to support analytics and operational use cases. • Maintain clear, consistent technical documentation for data systems and workflows.




