Data Engineer (Azure Databricks)

Location

Spain

Posted

60 days ago

Salary

0

Seniority

Mid Level

Job Description

Data Engineer (Azure Databricks)

Miratech

Company Description Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989. Miratech has coverage across 5 continents and operates in over 25 countries around the world. Miratech retains nearly 1000 full-time professionals, and our annual growth rate exceeds 25%. Job Description We are looking for a Mid-Senior Data Engineer specialized in Azure Databricks to join our data platform team. The candidate will design and develop modern data pipelines and Lakehouse architectures, leveraging Azure Databricks, Spark, and Azure Data Factory, while integrating with existing SQL Server-based data warehouse environments, also evolving our data platform towards scalable, cloud-based data architectures, enabling advanced analytics and business intelligence. Key responsibilities: - Design, develop, and maintain data pipelines using Azure Databricks - Build and optimize data transformations using PySpark and SQL in Databricks - Implement and maintain Lakehouse architectures using Delta Lake - Develop ETL/ELT pipelines orchestrated through Azure Data Factory - Integrate data from multiple sources into the data platform and analytical layers - Design and maintain data models and data warehouse structures for analytics - Ensure data quality, scalability, and performance of large-scale data processing pipelines - Collaborate with BI teams to support Power BI and reporting platforms - Support and evolve existing SQL Server data platforms and ETL solutions (SSIS) when required - Contribute to the design of modern cloud-based data architectures Qualifications - 3+ years of experience in Data Engineering or Data Warehouse development - Strong experience with Azure Databricks - Experience developing data pipelines using PySpark and Spark SQL - Solid understanding of distributed data processing and big data concepts - Experience working with Delta Lake and Lakehouse architectures - Strong SQL skills and experience with SQL Server relational databases - Experience building data pipelines using Azure Data Factory - Experience handling large datasets and performance optimization Nice to have - Experience with Spark optimization techniques (partitioning, caching, cluster tuning) - Experience with structured streaming in Databricks - Knowledge of CI/CD pipelines for data platforms (Azure Devops) - Familiarity with Power BI - Experience in migrating from traditional ETL process to cloud architectures Soft Skills - Strong analytical and problem-solving skills. - Ability to work in collaborative environments and to adapt - Committed to continuous learning and professional development, with a keen focus on advancing cloud computing expertise - Team player - Good communication skills with technical and non-technical roles - Is proactive rather than reactive, actively identifying improvements and proposing solutions - Is comfortable working in dynamic environments where priorities and technologies evolve Additional Information We offer: - Culture of Relentless Performance: join an unstoppable technology development team with a 99% project success rate and more than 30% year-over-year revenue growth. - Competitive Pay and Benefits: enjoy a comprehensive compensation and benefits package, including health insurance, and a relocation program. - Work From Anywhere Culture: make the most of the flexibility that comes with remote work. - Growth Mindset: reap the benefits of a range of professional development opportunities, including certification programs, mentorship and talent investment programs, internal mobility and internship opportunities. - Global Impact: collaborate on impactful projects for top global clients and shape the future of industries. - Welcoming Multicultural Environment: be a part of a dynamic, global team and thrive in an inclusive and supportive work environment with open communication and regular team-building company social events. - Social Sustainability Values: join our sustainable business practices focused on five pillars, including IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality. * Miratech is an equal opportunity employer and does not discriminate against any employee or applicant for employment on the basis of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity, or any other protected status under applicable law.

Related Categories

Related Job Pages

More Data Engineer Jobs

Visa logo

Staff Software Engineer, Data Platform

Visa

Based in Foster City, California, Visa is a global payments technology organization. Visa was founded in 1958, coinciding with Bank of America’s launch of the

Data Engineer60 days ago

• Design and build event processing services and pipelines on AWS serverless (Lambda, Kinesis, EventBridge, Step Functions) • Write production-quality Golang and/or Python code with comprehensive testing and observability • Own services end-to-end: development, deployment, monitoring, incident response, and SLA adherence • Make sound architectural decisions for somewhat complex problems • Drive code quality through thorough code reviews, testing standards, and documentation • Investigate and resolve production incidents independently; contribute to postmortem analysis • Design and implement CI/CD improvements for better developer experience and deployment safety • Mentor Analyst-level engineers through pairing, design guidance, and constructive review feedback

Brazil
Job Closed

• Design, develop, and maintain data pipelines using Azure Databricks • Build and optimize data transformations using PySpark and SQL in Databricks • Implement and maintain Lakehouse architectures using Delta Lake • Develop ETL/ELT pipelines orchestrated through Azure Data Factory • Integrate data from multiple sources into the data platform and analytical layers • Design and maintain data models and data warehouse structures for analytics • Ensure data quality, scalability, and performance of large-scale data processing pipelines • Collaborate with BI teams to support Power BI and reporting platforms • Support and evolve existing SQL Server data platforms and ETL solutions (SSIS) when required • Contribute to the design of modern cloud-based data architectures

Spain
AppQuantum logo

Data Engineer

AppQuantum

Mobile F2P Game Publisher

Data Engineer60 days ago
Full TimeRemoteTeam 51-200Since 2017H1B No Sponsor

• Maintain and enhance our high-load DWH built on ClickHouse • Design, develop, and optimize low-level data marts for business-critical reporting • Support and improve our ETL/ELT pipelines for operational analytics • Ensure fault tolerance of our data infrastructure and cover key components with alerting • Support and optimize data storage, ensure cost-efficiency and scalability • Monitor and improve real-time data streaming via Push API

Cyprus
Job Closed
CAQH logo

Lead Data Strategist

CAQH

CAQH delivers technology-enabled solutions, operating rules and research to the healthcare industry.

Data Engineer60 days ago
Full TimeRemoteTeam 51-200Since 1998H1B No Sponsor

Position Summary The Lead Data Strategist is a senior individual contributor responsible for defining and advancing CAQH’s strategy for provider and member data, including reporting and analytics capabilities. This role operates with a high degree of autonomy, shaping how provider and member data are structured, governed, and leveraged to deliver actionable insights, improve data quality, and support enterprise and market-facing initiatives. The Lead Data Strategist leads work in complex and ambiguous problem spaces, translating business needs and market opportunities into scalable data strategies and reporting solutions. The role partners closely with Data Engineering, Data Governance, Data Science, Product, Technology, and Growth teams to ensure alignment between strategic data objectives and execution. This includes engaging external stakeholders to validate strategic direction and ensure continued market relevance. The Lead Data Strategist is a full-time, remote, exempt position and reports to the SVP, Data & Analytics. Specific Responsibilities - Define and lead CAQH’s enterprise strategy for provider and member data, with a focus on data integration, standardization, and quality improvement across internal and external sources. - Drive initiatives to unify fragmented data assets across the organization, establishing a cohesive and scalable approach to provider and member data management. - Identify, evaluate, and recommend third-party data vendors and partnerships to enhance CAQH’s data assets, coverage, and accuracy. - Develop frameworks for assessing external data sources, including data quality, interoperability, cost-benefit, and strategic fit. - Partner with Data Engineering, Data Governance, and Data Science teams to ensure alignment between data strategy, architecture, and analytical capabilities. - Collaborate with Product, Technology, and Growth teams to ensure data assets effectively support business use cases, client needs, and market expansion. - Establish and promote enterprise standards for data quality, data stewardship, and data lifecycle management across provider and member domains. - Define and implement approaches for data harmonization, normalization, and linkage across disparate systems and datasets. - Lead efforts to improve completeness, accuracy, and timeliness of provider and member data through both internal process improvements and external data enrichment. - Develop and maintain a strategic roadmap for enterprise data assets, with a focus on long-term scalability and reuse. - Conduct impact and value assessments for data initiatives, including operational efficiency, data quality improvements, and downstream business outcomes. - Serve as a subject matter expert on provider and member data strategy, advising senior leadership on data investments, risks, and opportunities. - Engage with industry stakeholders, including providers, health plans, and data vendors, to stay aligned with evolving standards and data ecosystem trends. Skills - Deep expertise in provider and member data, including data structures, data exchange, and healthcare administrative workflows (e.g., credentialing, enrollment, and roster management). - Strong understanding of data integration, data modeling, and data architecture concepts, particularly in complex, multi-source environments. - Experience evaluating and integrating third-party data sources, including vendor assessment and data due diligence. - Expertise in data quality management, including profiling, monitoring, and remediation strategies. - Strong knowledge of data governance principles, including stewardship models, standards, and controls. - Ability to connect data strategy to business outcomes, including growth, operational efficiency, and client value. - Demonstrated ability to influence cross-functional stakeholders without direct authority. - Strong analytical and structured problem-solving skills, particularly in ambiguous and complex environments. - Excellent communication skills, with the ability to translate data concepts into clear recommendations for executive and non-technical audiences. - Experience working across data, technology, and business teams to drive alignment and execution. Experience - 8 to 12 years of experience in data strategy, data management, data architecture, or related roles within healthcare or healthcare technology. - Direct experience working with provider and/or member data, including data integration, data quality, or data governance initiatives strongly preferred. - Experience evaluating or managing third-party data vendors and external data partnerships preferred. - Proven experience working across siloed data environments and leading efforts to unify and standardize data assets. - Advanced degree in data analytics, information systems, healthcare administration, public health, or related fields preferred. Who We Are CAQH is the trusted data connector at the core of healthcare. For more than 25 years, we have powered the industry with the largest and most complete healthcare data foundation in the U.S., including more than 4.8 million provider data records sourced directly from providers and member data representing 75% of covered lives supplied by health plans. By improving how essential information flows across the system, CAQH helps healthcare operate more efficiently, accurately, and with greater confidence. What You Get At CAQH, you will do meaningful work at the intersection of healthcare, data, and technology, helping solve complex problems that make the healthcare system work better. You will collaborate with experienced professionals who care deeply about accuracy, trust, and meaningful impact in a fully remote environment. CAQH offers competitive compensation and a comprehensive benefits package for full-time employees, including medical, dental, and vision coverage, a 401(k) with company contributions and matching, paid parental leave, tuition assistance, and generous paid time off. We are committed to investing in our people and supporting professional growth over time. Equal Opportunity Employer CAQH is proud to be an equal opportunity employer and is committed to fostering a workplace where all individuals are valued, respected, and empowered. Employment decisions at CAQH are made without regard to race, color, religion, sex, national origin or ancestry, age, marital status, disability, protected veteran status, personal appearance, sexual orientation, gender identity or expression, familial status, family responsibilities, matriculation, political affiliation, genetic information, source of income, place of residence, or any other characteristic protected by law. CAQH does not tolerate unlawful discrimination or harassment of any kind. Applicants have rights under the Family and Medical Leave Act (FMLA), Equal Employment Opportunity (EEO), and the Employee Polygraph Protection Act (EPPA). If you need a reasonable accommodation to apply for a posted position, please contact the CAQH People & Culture team at Careers@caqh.org or 202-517-0436.

United States
$165K - $180K / year
Job Closed