Job Closed
This listing is no longer active.
We are all Humans!
Staff Data Engineer, Databricks
Location
Portugal
Posted
105 days ago
Salary
0
Seniority
Lead
Job Description
Staff Data Engineer, Databricks
HumanIT Digital Consulting
• Join a dynamic technology consultancy where data engineering excellence drives business transformation • Architect and build robust data pipelines using cutting-edge Azure technologies • Lead the design of scalable data solutions that transform raw information into valuable business assets • Work on sophisticated data platforms built on Azure with Databricks at the core • Design end-to-end data pipelines handling complex ETL and ELT workflows • Transform raw data into analytics-ready datasets • Architect solutions using modern data modeling methodologies including Kimball, Inmon, and Data Vault, ensuring scalability and maintainability across multiple business domains
Job Requirements
- 9+ years of professional experience as a Data Engineer in fast-paced production environments
- Expert-level SQL proficiency with SQL and SQL-like query languages for complex data manipulation
- Advanced Python expertise with proven experience organizing Python-based data projects
- Azure Databricks mastery (mandatory): Vast hands-on experience building data solutions
- Data pipeline development with proven track record building scalable, reliable pipelines
- ETL/ELT expertise designing processes to transform raw data into golden datasets
- Azure cloud knowledge with strong experience across Microsoft Azure data services
- Data modeling expertise in Kimball, Inmon, and Data Vault 2.0 methodologies
- Language requirement : Fluent English (mandatory)
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
BrahmaThe only account you'll ever need to secure, transact, and explore onchain like never before.
• Feature engineering at scale • Architect unstructured data pipelines • Orchestrate ML workflows using Dagster or Airflow • Optimize compute & cost • Build "dataset-as-code" • Infrastructure ownership with Kubernetes for scalability
• Modernizing and developing e-commerce applications • Develop, test, and maintain high-quality Magento applications and extensions. • Customize Magento themes and templates to create tailored user experiences. • Integrate third-party modules and APIs to enhance platform functionality. • Optimize Magento sites for performance, scalability, and security. • Troubleshoot and debug issues to ensure seamless operation of e-commerce platforms. • Collaborate with designers and frontend developers to implement new features and enhancements. • Stay updated with the latest Magento developments, trends, and best practices to recommend improvements. • Participate in code reviews and share knowledge with team members to foster a collaborative environment.
Unstructured Data Engineer
LeidosLeidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.
• Design, build, and manage end-to-end RAG pipelines for enterprise AI applications. • Lead preprocessing of unstructured data, including discovery, classification, cleansing, redaction, and metadata enrichment. • Develop and optimize document chunking, embedding, and vectorization strategies for structured and unstructured datasets. • Coordinate ingestion of curated datasets into vector databases and AI platforms. • Package curated unstructured datasets as governed, reusable data products for enterprise consumption. • Define and implement metadata tagging strategies to align with Collibra governance standards. • Partner with Data Governance and Data Quality teams to ensure AI-ready data meets enterprise standards for lineage, classification, and compliance. • Evaluate and optimize embedding models, retrieval strategies, and indexing performance. • Monitor and tune RAG pipeline performance, including latency, retrieval accuracy, and cost efficiency. • Implement automation for document ingestion, transformation, and publishing workflows. • Support integration with enterprise AI platforms (e.g., ChatGPT Enterprise, AskSage, Moveworks). • Conduct cost analysis and capacity planning for vector storage and processing workloads. • Provide technical guidance on AI data readiness and unstructured data lifecycle management. • Design, implement, and optimize enterprise-grade RAG and prompt engineering frameworks, including context engineering strategies (chunking, metadata enrichment, semantic filtering, dynamic context management) to improve retrieval accuracy, grounding, and response quality. • Develop and maintain scalable multi-modal data pipelines that ingest, preprocess, embed, and integrate text, documents, images, audio, and structured data into governed vectorized data products consumable by enterprise AI platforms.
• Wspieraniu klientów w tematyce inżynierii i analityki dużych zbiorów danych w środowisku chmurowym – tworzenie rozwiązań od koncepcji po wdrożenie oraz inne fazy procesu SDLC/DDLC • Definiowaniu rozwiązań Data Intelligence i zarządzaniu ich wdrażaniem pod kątem technicznym i metodycznym • Wspieraniu celów biznesowych naszych klientów poprzez opracowywanie i wdrażanie rozwiązań analitycznych • Budowaniu Proof of Concept (PoC) i nadzorowaniu architektury Microsoft dla naszych klientów • Dzieleniu się wiedzą i pełnieniu funkcji trenerskiej w zakresie Azure Data Intelligence, oferowaniu wsparcia pracownikom, związanego z tematyką BI i analityką danych • Współpracy z innymi działami biznesowymi w celu dostarczania przydatnych rozwiązań analitycznych • Proponowaniu nowych możliwości wykorzystania danych, wdrażaniu nowoczesnych rozwiązań analitycznych oraz dbaniu o najwyższą jakość analiz i raportów, które pomagają kształtować strategiczne decyzje biznesowe




