Vigil logo
Vigil

We’re a tech consultancy, expert in software engineering and cloud transformation.

Senior Data Engineer, Microsoft Fabric

Data EngineerData EngineerFull TimeRemoteSeniorTeam 51-200H1B No SponsorCompany SiteLinkedIn

Location

Portugal

Posted

3 days ago

Salary

0

Seniority

Senior

Bachelor DegreeEnglishAzureETLPySparkPythonSQLVault

Job Description

Senior Data Engineer, Microsoft Fabric

Vigil

• Design, build and maintain enterprise data solutions using Microsoft Fabric • Develop scalable data ingestion pipelines from APIs, databases and file-based sources • Build and maintain Lakehouse architectures using the Medallion (Bronze, Silver and Gold) approach • Develop robust data models and semantic layers to support business reporting • Create and maintain Power BI semantic models and dashboards • Implement data quality, validation and reconciliation frameworks • Design secure, governed data solutions including Row-Level Security and access controls • Build and maintain PySpark notebooks for data transformation • Develop SQL and Python-based ETL/ELT processes • Configure and maintain Fabric workspaces, OneLake environments and deployment pipelines • Work closely with solution architects to implement scalable platform designs • Produce technical documentation and engineering standards • Collaborate with UK-based stakeholders and delivery teams • Support future client projects across the TXP Data & AI Practice

Job Requirements

  • Strong commercial experience with Microsoft Fabric
  • Experience with Fabric Data Factory, Dataflows Gen2, Lakehouse and OneLake
  • Strong SQL skills
  • Strong Python and/or PySpark experience
  • Experience designing enterprise data models
  • Strong understanding of Medallion Architecture
  • Experience building scalable ETL/ELT pipelines
  • REST API integration experience
  • Experience implementing data quality and reconciliation processes
  • Solid Power BI experience, including semantic modelling and DAX
  • Understanding of Azure fundamentals, including Entra ID and Key Vault
  • Experience working independently on complex technical projects
  • Excellent communication skills in English

Benefits

  • Be part of our collegial environment where responsibility and authority are shared equally amongst colleagues and help create our company culture
  • A culture in which we don’t criticise failure but ensure we learn from our mistakes
  • An Agile environment where your ideas are welcome
  • The possibility to grow and experience different projects
  • Ongoing Training & Mentoring
  • The possibility to travel

Related Categories

Related Job Pages

More Data Engineer Jobs

Enroute logo

Data Engineer – Data Conversions, Junior-Mid

Enroute

We deliver IT services and solutions provided by a team of passionate problem solving individuals highly skilled.

Data Engineer3 days ago
Full TimeRemoteTeam 51-200H1B Sponsor

• Play a critical role in migrating insurance policy data from administration systems to a modern policy administration platform. • Perform analysis-first role with a data engineering component. • Deep-dive into data, trace and reconcile information across multiple data layers (databases and files). • Build mappings and transformations that move legacy data into the target format. • Resolve data quality issues to deliver validated loads. • Utilize strong, hands-on command of SQL for analysis, tracing, and transforming data. • Use AI tooling to accelerate work processes. • Receive mentorship from an experienced conversion manager. • Work through entire data migration lifecycle, from initial discovery through production cutover. • Support iterative delivery cycles with effective communication.

Mexico
Sofka Technologies logo

Arquitecto de Datos

Sofka Technologies

To transform people’s lives being the most trusted technology partner

Data Engineer3 days ago
Full TimeRemoteTeam 1,001-5,000Since 2013H1B No Sponsor

• Definir la estrategia analítica y el modelo funcional de la solución. • Diseñar y construir el pipeline de ingesta y transformación sobre AWS. • Configurar la seguridad de la infraestructura (IAM). • Diseñar el modelo dimensional (star schema). • Actuar como autoridad técnica frente al cliente.

Colombia
3Cloud logo

Director, Data Architecture – Engineering

3Cloud

Delivering the ultimate Microsoft Azure experience.

Data Engineer3 days ago
Full TimeRemoteTeam 501-1,000H1B No Sponsor

• Maintain a clear view of each team member's delivery commitments, growth plans, and internal contributions. • Develop and grow team members through regular coaching, clear expectations, and constructive feedback. Build a culture of trust, inclusion, and collaboration. • Monitor team health, morale, and workload balance, acting early to address engagement or performance concerns in partnership with practice leadership. • Maintain regular one-to-one connections with each team member, provide clear feedback, and help them remove blockers related to client work, pursuits, or internal initiatives. • Oversee staffing and resource alignment across teams, balancing utilization, development goals, client needs, and sustainable workloads. • Communicate practice priorities and organizational objectives clearly, ensuring teams understand how their work contributes to broader organizational goals. • Define interview standards and mentorship strategies, training others to apply them consistently and improving hiring outcomes and talent growth at scale. • Create and maintain development plans for team members, including stretch assignments, shadowing, certifications, and opportunities for external visibility. • Lead or support performance reviews, promotion recommendations, and compensation input for your teams, using consistent standards that reflect both impact and behavior. • Represent your teams in leadership forums, communicate expectations and decisions clearly back to the group, and bring forward patterns, risks, and successes that should shape practice strategy. • Lead architecture and technical strategy for large Azure programs and portfolios so solutions meet client goals for value, security, scalability, reliability, performance, and cost. • Define modernization blueprints and reference architectures across applications, platforms, and data, including system and subsystem designs, patterns, and standards that simplify delivery and operations over time. • Translate business and product strategy into cloud architecture roadmaps and execution plans that drive measurable outcomes and adapt based on production feedback and telemetry. • Guide major architecture decisions across programs, promote shared design standards and reusable models, and resolve high-impact technical trade-offs. • Work across teams to connect solutions into cohesive end-to-end architectures with clear ownership boundaries and integration contracts. • Provide architectural oversight during delivery through design reviews, forums, and governance, keeping cross-cutting concerns (security, performance, resilience, availability, compliance, observability, and cost) aligned across teams. • Promote modular design, clear boundaries, testability, CI/CD, and observability so non-functional requirements are treated as first-class and systems remain reliable and maintainable. • Use AI tools to accelerate discovery, design, code, tests, documentation, and analysis, and scale successful practices so teams adopt them consistently. • Define and own enterprise data platform strategy centered on Databricks or equivalent lakehouse architectures. • Establish reference architectures spanning ingestion, lakehouse, analytics, and AI/ML. • Set standards and best practices for: Databricks lakehouse design and workload isolation. Delta Lake optimization and data lifecycle management. Streaming and batch processing patterns. Unity Catalog–aligned governance and security models. Guide multi-workspace and multi-region Databricks deployments. • Ensure tight integration between Databricks, Azure-native services, and downstream analytics tools. • Oversee architectural alignment with complementary platforms (e.g., Snowflake, Fabric, Power BI).

United States
$162.8K - $244.2K / year
Full TimeRemoteTeam 201-500H1B No Sponsor

• Design, develop, and maintain scalable data pipelines to support ingestion, transformation, and delivery into centralized feature stores, model-training workflows, and real-time inference services. • Build and optimize workflows for extracting, storing, and retrieving semantic representations of unstructured data to enable advanced search and retrieval patterns. • Architect and implement lightweight analytics and dashboarding solutions that deliver natural language query experience and AI-backed insights. • Define and execute processes for managing prompt engineering techniques, orchestration flows, and model fine-tuning routines to power conversational interfaces. • Oversee vector data stores and develop efficient indexing methodologies to support retrieval-augmented generation (RAG) workflows. • Partner with data stakeholders to gather requirements for language-model initiatives and translate into scalable solutions. • Create and maintain comprehensive documentation for all data processes, workflows and model deployment routines. • Should be willing to stay informed and learn emerging methodologies in data engineering, MLOps and LLM operations.

Colombia