Job Closed

This listing is no longer active.

HumanIT Digital Consulting logo
HumanIT Digital Consulting

We are all Humans!

Staff Data Engineer, Databricks

Data EngineerData EngineerFull TimeRemoteLeadTeam 51-200Since 2020H1B No SponsorCompany SiteLinkedIn

Location

Portugal

Posted

105 days ago

Salary

0

Seniority

Lead

Bachelor Degree9 yrs expEnglishAzureETLPythonSQLHashiCorp Vault

Job Description

Staff Data Engineer, Databricks

HumanIT Digital Consulting

• Join a dynamic technology consultancy where data engineering excellence drives business transformation • Architect and build robust data pipelines using cutting-edge Azure technologies • Lead the design of scalable data solutions that transform raw information into valuable business assets • Work on sophisticated data platforms built on Azure with Databricks at the core • Design end-to-end data pipelines handling complex ETL and ELT workflows • Transform raw data into analytics-ready datasets • Architect solutions using modern data modeling methodologies including Kimball, Inmon, and Data Vault, ensuring scalability and maintainability across multiple business domains

Job Requirements

  • 9+ years of professional experience as a Data Engineer in fast-paced production environments
  • Expert-level SQL proficiency with SQL and SQL-like query languages for complex data manipulation
  • Advanced Python expertise with proven experience organizing Python-based data projects
  • Azure Databricks mastery (mandatory): Vast hands-on experience building data solutions
  • Data pipeline development with proven track record building scalable, reliable pipelines
  • ETL/ELT expertise designing processes to transform raw data into golden datasets
  • Azure cloud knowledge with strong experience across Microsoft Azure data services
  • Data modeling expertise in Kimball, Inmon, and Data Vault 2.0 methodologies
  • Language requirement : Fluent English (mandatory)

Related Categories

Related Job Pages

More Data Engineer Jobs

Brahma logo

Senior Data Engineer

Brahma

The only account you'll ever need to secure, transact, and explore onchain like never before.

Data Engineer105 days ago
Full TimeRemoteTeam 11-50Since 2022H1B No Sponsor

• Feature engineering at scale • Architect unstructured data pipelines • Orchestrate ML workflows using Dagster or Airflow • Optimize compute & cost • Build "dataset-as-code" • Infrastructure ownership with Kubernetes for scalability

United Kingdom
MaxanaPay logo

Data Engineer

MaxanaPay

Empowering businesses one transaction at a time

Data Engineer105 days ago
OtherRemoteTeam 11-50Since 2019

• Modernizing and developing e-commerce applications • Develop, test, and maintain high-quality Magento applications and extensions. • Customize Magento themes and templates to create tailored user experiences. • Integrate third-party modules and APIs to enhance platform functionality. • Optimize Magento sites for performance, scalability, and security. • Troubleshoot and debug issues to ensure seamless operation of e-commerce platforms. • Collaborate with designers and frontend developers to implement new features and enhancements. • Stay updated with the latest Magento developments, trends, and best practices to recommend improvements. • Participate in code reviews and share knowledge with team members to foster a collaborative environment.

United States
$130K - $170K / year
Leidos logo

Unstructured Data Engineer

Leidos

Leidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.

Data Engineer105 days ago
OtherRemoteTeam 10,001+Since 1969H1B Sponsor

• Design, build, and manage end-to-end RAG pipelines for enterprise AI applications. • Lead preprocessing of unstructured data, including discovery, classification, cleansing, redaction, and metadata enrichment. • Develop and optimize document chunking, embedding, and vectorization strategies for structured and unstructured datasets. • Coordinate ingestion of curated datasets into vector databases and AI platforms. • Package curated unstructured datasets as governed, reusable data products for enterprise consumption. • Define and implement metadata tagging strategies to align with Collibra governance standards. • Partner with Data Governance and Data Quality teams to ensure AI-ready data meets enterprise standards for lineage, classification, and compliance. • Evaluate and optimize embedding models, retrieval strategies, and indexing performance. • Monitor and tune RAG pipeline performance, including latency, retrieval accuracy, and cost efficiency. • Implement automation for document ingestion, transformation, and publishing workflows. • Support integration with enterprise AI platforms (e.g., ChatGPT Enterprise, AskSage, Moveworks). • Conduct cost analysis and capacity planning for vector storage and processing workloads. • Provide technical guidance on AI data readiness and unstructured data lifecycle management. • Design, implement, and optimize enterprise-grade RAG and prompt engineering frameworks, including context engineering strategies (chunking, metadata enrichment, semantic filtering, dynamic context management) to improve retrieval accuracy, grounding, and response quality. • Develop and maintain scalable multi-modal data pipelines that ingest, preprocess, embed, and integrate text, documents, images, audio, and structured data into governed vectorized data products consumable by enterprise AI platforms.

United States
$107.9K - $195.1K / year
Job Closed
Full TimeRemoteTeam 1,001-5,000Since 2015H1B Sponsor

• Wspieraniu klientów w tematyce inżynierii i analityki dużych zbiorów danych w środowisku chmurowym – tworzenie rozwiązań od koncepcji po wdrożenie oraz inne fazy procesu SDLC/DDLC • Definiowaniu rozwiązań Data Intelligence i zarządzaniu ich wdrażaniem pod kątem technicznym i metodycznym • Wspieraniu celów biznesowych naszych klientów poprzez opracowywanie i wdrażanie rozwiązań analitycznych • Budowaniu Proof of Concept (PoC) i nadzorowaniu architektury Microsoft dla naszych klientów • Dzieleniu się wiedzą i pełnieniu funkcji trenerskiej w zakresie Azure Data Intelligence, oferowaniu wsparcia pracownikom, związanego z tematyką BI i analityką danych • Współpracy z innymi działami biznesowymi w celu dostarczania przydatnych rozwiązań analitycznych • Proponowaniu nowych możliwości wykorzystania danych, wdrażaniu nowoczesnych rozwiązań analitycznych oraz dbaniu o najwyższą jakość analiz i raportów, które pomagają kształtować strategiczne decyzje biznesowe

Poland