We help companies grow by creating digital products people love.
Data Engineer, Databricks
Location
Peru
Posted
1 day ago
Salary
0
Seniority
Senior
Job Description
Data Engineer, Databricks
Livefront
• Design and build production data pipelines using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming, with end-to-end ownership of ingestion, transformation, data quality expectations, and CI/CD deployment via Declarative Automation Bundles. • Architect and implement Lakehouse solutions on Databricks — medallion architecture, Delta Lake, Unity Catalog — tailored to the client's analytics, AI, and application needs. • Build and maintain Databricks transformation layers — DLT pipelines, PySpark notebooks, and dbt — with data quality constraints and SLAs baked in. • Design and maintain the data and AI foundations — Unity Catalog, Feature Store, MLflow, and Model Serving — that power production ML, agent workflows, and AI-enabled digital products. • Collaborate with product and backend engineers to design data models, APIs, and application data contracts — ensuring the platform serves the product, not just the warehouse. • Consult with clients to understand their data challenges, develop data strategies, and implement sustainable solutions. • Adapt your approach based on project needs — sometimes leading data architecture discussions with clients, other times supporting internal teams with specialized data expertise. • Work within multi-cloud environments — primarily AWS and Azure — anchoring data platform recommendations around Databricks where it fits the client's architecture and goals. • Champion data governance through Unity Catalog — access control, lineage, data quality policies, and compliance — as a first-class part of every engagement, not an afterthought. • Design data-to-application architectures — including Lakebase-backed services and Databricks Apps — that connect governed data to AI workflows, digital products, and user-facing experiences. • Help build Livefront's Databricks practice — contributing to accelerators, internal enablement, certification goals, and Databricks partner go-to-market materials alongside delivery work.
Job Requirements
- 3-5 years of data engineering experience with at least 2 years in production Databricks environments, preferably in a consulting or client delivery context.
- Solid working knowledge of AWS and Azure cloud services relevant to Databricks deployments — storage, networking, IAM, and compute — with GCP familiarity a plus.
- Deep, production-grade Databricks expertise: Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, Lakeflow Jobs, Unity Catalog (including fine-grained access control and lineage) — demonstrated through shipped production workloads, not prototypes.
- Proven experience designing Lakehouse architectures — medallion patterns, Delta Lake table design, partitioning, Z-ordering, and query optimization — at production scale.
- Hands-on experience with data pipeline testing, observability, and CI/CD for data — including unit testing, data quality frameworks, and version-controlled deployments via Git and Declarative Automation Bundles.
- Strong proficiency in SQL and Python, with the ability to write clean, performant, and maintainable code.
- Understanding of data modeling, schema design, and query optimization.
- Excellent communication skills with the ability to explain complex data concepts to both technical and non-technical stakeholders.
- Strong problem-solving skills with the ability to navigate ambiguous requirements and deliver pragmatic solutions.
- Above-average discipline and personal organization skills.
- Obvious comfort with critique and peer review in the context of an iterative development process.
- A demonstrated hunger for personal and professional growth.
- A self-evident love and care for the craft of data engineering.
Benefits
- You want to work with passionate and talented people who are always looking for ways to make things better.
- You desire a work environment where respect, mutual trust, and egoless collaboration are paramount.
- You want colleagues who take their work seriously but not themselves, and who know how to let loose and have a good time.
- You like being part of a team with a reputation for excellence that gives back to the community by educating, mentoring, and sponsoring.
- You want to work on products and accounts that have outsized impact and reach.
- You believe in sweating the details, giving a damn about quality, and taking pride in going the extra mile.
- You want to help build a data practice specialization from the ground up — shaping how we go to market with Databricks, what we build as accelerators, and what it means to do this kind of work at a digital product company.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• As a Senior Data Engineer, you will drive the end-to-end development of our core data infrastructure through AI-native engineering. • Championing our AI DevEx approach, you will orchestrate AI agents to rapidly build, scale, and refactor high-performance systems. • Developing and driving the architecture of complex data systems that prioritize scalability, reliability, and long-term maintainability. • Designing and optimizing production-grade data pipelines , with a primary focus on high-throughput, real-time streaming. • Driving Spec-Driven Development (SDD) using OpenSpec or GitHub Spec Kit to create strict engineering contracts that ensure predictable, high-quality AI code generation. • Orchestrating agentic AI workflows with Claude Code and the Model Context Protocol (MCP) to rapidly build, refactor, and scale our data infrastructure. • Taking full accountability and ownership of system components, working in a self-sufficient manner to solve deep technical challenges. • Implementing rigorous testing and monitoring frameworks to ensure the integrity of mission-critical data. • Mentoring junior engineers and fostering a culture of technical excellence through open feedback and architectural reviews.
• The Senior Data Engineer owns the design, delivery, and continuous improvement of Empower’s enterprise data foundation, enabling faster decisions, stronger quality, and scalable growth across a highly regulated 503A/503B pharmacy environment. • This role converts complex business, operational, and compliance needs into secure, reliable, AI-enabled data products, pipelines, models, and platforms that accelerate speed, scale, quality, and decision-making. • The role has end-to-end ownership for critical data engineering outcomes, from architecture and integration through automation, governance, and performance optimization. • Success requires P80-P90 talent: strategic thinking, execution rigor, learning agility, technical depth, and the ability to raise standards while partnering across Technology, Operations, Quality, Finance, and Commercial teams in a hyper-growth environment where data reliability directly impacts patients, providers, and enterprise performance under urgent demand, evolving priorities, and high executive visibility every day.
Data Migration / Integrations Specialist
OpendateOne connected system to ticket, promote, and manage live events.
• Own customer data migration projects from kickoff through QA and launch. • Review customer exports from legacy ticketing, CRM, POS, and marketing systems. • Clean, normalize, and map data into Opendate’s structure. • Import and validate records such as events and recurring events, ticket types and pricing, customers, fans, and contacts, orders, donations, and transaction history, marketing opt-ins and audience segments, holds, comps, guest lists, and venue-specific notes. • Build repeatable migration checklists, QA processes, and rollback plans. • Identify data inconsistencies before they become customer-facing issues. • Partner with customer success and implementation teams to ensure customers understand what was imported, where it lives, and how to use it. • Help configure, monitor, and troubleshoot integrations across Opendate’s ecosystem. • Support integrations involving systems such as POS, marketing tools, CRM platforms, calendars, reporting tools, SMS/email tools, advertising pixels, and distribution partners. • Investigate sync failures, missing records, duplicate records, broken mappings, stale data, and API/webhook issues. • Run and validate approved scripts, imports, backfills, and data correction workflows. • Maintain a high bar for data accuracy, customer communication, and operational safety.
Data Engineer
NearaIntelligence for infrastructure ⎮ Register for our upcoming webinar: bit.ly/447oixq
• Contribute to the end-to-end deployment of the Neara platform for utility customers worldwide. • Ingest and manipulate massive datasets (LiDAR, GIS, and asset databases). • Perform rigorous QA/QC to ensure the digital twin accurately reflects real-world physics and customer specifications. • Build custom reports and dashboards within the platform to drive actionable insights for utility engineers. • Work closely with Product and Engineering to provide feedback on process and tool improvements.
