Job Closed
This listing is no longer active.
We are all Humans!
Data Engineer – Snowflake, AWS
Location
Portugal
Posted
103 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer – Snowflake, AWS
HumanIT Digital Consulting
• Responsible for building, maintaining, and scaling big data platforms • Work closely with Data Science teams to set up and automate machine learning models and algorithms for production use • Build and optimize ETL/ELT data pipelines that power business intelligence and analytics initiatives • Transform raw, structured, and semi-structured data into actionable insights • Leverage cloud services (AWS preferred) including Lambda, S3, SageMaker, and more to build scalable data solutions • Pair program with engineers, support analytics teams, and work within CI/CD pipelines
Job Requirements
- Strong proficiency in object-oriented programming languages (Python, Java, or similar)
- Experience building and optimizing data pipelines for ingestion, transformation, and loading
- Strong working knowledge of SQL and experience with relational databases
- Familiarity with cloud-based data warehouses such as Snowflake, Amazon Redshift, or Google BigQuery
- Experience working with raw data, structured and semi-structured data formats (JSON, Parquet, Avro)
- Experience with source control (Git), Continuous Integration, Delivery, and Deployment through CI pipelines
- Ability to support and work effectively with Business Intelligence and Analytics teams in dynamic environments
- Strong collaborative skills with ability to effectively pair program with other engineers
- Ability to solve complex problems and present solutions in a business-oriented manner
- Fluent English (mandatory)
Benefits
- National and international travel varies by project (0-15% estimated)
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
BrahmaThe only account you'll ever need to secure, transact, and explore onchain like never before.
• Feature engineering at scale • Architect unstructured data pipelines • Orchestrate ML workflows using Dagster or Airflow • Optimize compute & cost • Build "dataset-as-code" • Infrastructure ownership with Kubernetes for scalability
• Modernizing and developing e-commerce applications • Develop, test, and maintain high-quality Magento applications and extensions. • Customize Magento themes and templates to create tailored user experiences. • Integrate third-party modules and APIs to enhance platform functionality. • Optimize Magento sites for performance, scalability, and security. • Troubleshoot and debug issues to ensure seamless operation of e-commerce platforms. • Collaborate with designers and frontend developers to implement new features and enhancements. • Stay updated with the latest Magento developments, trends, and best practices to recommend improvements. • Participate in code reviews and share knowledge with team members to foster a collaborative environment.
Unstructured Data Engineer
LeidosLeidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.
• Design, build, and manage end-to-end RAG pipelines for enterprise AI applications. • Lead preprocessing of unstructured data, including discovery, classification, cleansing, redaction, and metadata enrichment. • Develop and optimize document chunking, embedding, and vectorization strategies for structured and unstructured datasets. • Coordinate ingestion of curated datasets into vector databases and AI platforms. • Package curated unstructured datasets as governed, reusable data products for enterprise consumption. • Define and implement metadata tagging strategies to align with Collibra governance standards. • Partner with Data Governance and Data Quality teams to ensure AI-ready data meets enterprise standards for lineage, classification, and compliance. • Evaluate and optimize embedding models, retrieval strategies, and indexing performance. • Monitor and tune RAG pipeline performance, including latency, retrieval accuracy, and cost efficiency. • Implement automation for document ingestion, transformation, and publishing workflows. • Support integration with enterprise AI platforms (e.g., ChatGPT Enterprise, AskSage, Moveworks). • Conduct cost analysis and capacity planning for vector storage and processing workloads. • Provide technical guidance on AI data readiness and unstructured data lifecycle management. • Design, implement, and optimize enterprise-grade RAG and prompt engineering frameworks, including context engineering strategies (chunking, metadata enrichment, semantic filtering, dynamic context management) to improve retrieval accuracy, grounding, and response quality. • Develop and maintain scalable multi-modal data pipelines that ingest, preprocess, embed, and integrate text, documents, images, audio, and structured data into governed vectorized data products consumable by enterprise AI platforms.
• Wspieraniu klientów w tematyce inżynierii i analityki dużych zbiorów danych w środowisku chmurowym – tworzenie rozwiązań od koncepcji po wdrożenie oraz inne fazy procesu SDLC/DDLC • Definiowaniu rozwiązań Data Intelligence i zarządzaniu ich wdrażaniem pod kątem technicznym i metodycznym • Wspieraniu celów biznesowych naszych klientów poprzez opracowywanie i wdrażanie rozwiązań analitycznych • Budowaniu Proof of Concept (PoC) i nadzorowaniu architektury Microsoft dla naszych klientów • Dzieleniu się wiedzą i pełnieniu funkcji trenerskiej w zakresie Azure Data Intelligence, oferowaniu wsparcia pracownikom, związanego z tematyką BI i analityką danych • Współpracy z innymi działami biznesowymi w celu dostarczania przydatnych rozwiązań analitycznych • Proponowaniu nowych możliwości wykorzystania danych, wdrażaniu nowoczesnych rozwiązań analitycznych oraz dbaniu o najwyższą jakość analiz i raportów, które pomagają kształtować strategiczne decyzje biznesowe




