We’re a tech consultancy, expert in software engineering and cloud transformation.
Senior Data Engineer, Python, AWS, ML Pipelines
Location
Portugal
Posted
65 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer, Python, AWS, ML Pipelines
Vigil
• Building and maintaining data and machine learning pipelines in production • Developing ETL and data processing workflows using Python • Orchestrating workflows using tools such as Airflow and AWS Step Functions • Working with AWS Glue for data processing and pipeline execution • Supporting the deployment of machine learning models into production using SageMaker • Working closely with Data Scientists and ML Engineers to productionise ML models • Monitoring, troubleshooting, and improving pipeline reliability and performance • Contributing to architecture and technical decisions around data platform and ML infrastructure • Working in an agile environment with distributed teams
Job Requirements
- Strong experience with Python
- Experience building data pipelines and/or ML pipelines in production
- Hands-on experience with Airflow
- Experience with AWS Step Functions and AWS Glue
- Experience deploying ML models using AWS SageMaker
- Strong experience with AWS cloud
- Experience working with production systems at scale
- Experience working in cross-functional teams with Data Scientists
- Strong communication skills and ability to work in a distributed team environment.
- Strong English communication skills, both written and verbal
Benefits
- A culture in which we don’t criticise failure but ensure we learn from our mistakes
- An Agile environment where your ideas are welcome
- Ongoing Training & Mentoring
- The possibility of travel
Related Guides
Related Job Pages
More Backend Engineer Jobs
• Design and write with the cutting edge Elixir language • Improve the availability, scalability, latency, and efficiency of company’s products • Work with the engineering team to explore and create new designs/architectures • Participate in code and design reviews • Engage in service capacity and demand planning, software performance analysis, tuning and optimization • Collaborate with product and experience teams to define and prototype feature specifications • Work closely with the infrastructure team in building and scaling back-end services • Perform root cause analysis investigations • Design, build, analyze and fix large-scale distributed systems
• Building products and infrastructure that powers many businesses • Working as a Senior Software Engineer in Python
• Ownership over large domains and features related to card processing/issuing, and core ledgering of financial transactions • Play a critical role in the development, design, and maintenance of Flex's platform and services. • Architect well thought-out services and data models that ensure Flex's continued growth • Own problems end-to-end, thinking through everything from api contracts, data, scalability, security, observability, and metrics. • Lead by example and mentor engineers in all aspects of planning, development and testing. • Be a catalyst for driving best-in-class technology frameworks and tooling while staying engaged with the latest technology trends. • Be on call, with the expectation to ability to solve problems and find solutions for the servicing team
Senior Ruby on Rails Backend Developer
ColibriCodeSince then our team of software designers has worked with startups to veteran international business organizations.
• Writing production-level code in Ruby on Rails • Building high-scale cloud services • Automating repetitive tasks • Packaging reusable code and sharing with others • Collaborating with the team on project tasks



