Software House focused on results since 1999
Senior Data Engineer, Azure
Location
Poland
Posted
107 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer, Azure
Software Mind
• Develop reusable, metadata-driven data pipelines • Automate and optimize any data platform related processes • Build integrations with data sources and data consumers • Add data transformation methods to shared ETL libraries • Write unit tests • Develop solutions for the Databricks data platform monitoring • Proactively resolve any performance or quality issues in ETL processes • Cooperate with infrastructure engineering team to set up cloud resources • Contribute to data platform wiki / documentation • Perform code reviews and ensures code quality • Initiate and implements improvements to the data platform architecture
Job Requirements
- Programming: Python/PySpark, SQL
- Proficient in building robust data pipelines using Databricks Spark
- Experienced in dealing with large and complex datasets
- Knowledgeable about building data transformations modules organized as libraries (Python packages)
- Familiar with Databricks Delta optimization techniques (partitioning, z-ordering, compaction, etc.)
- Experienced in developing CI/CD pipelines
- Experienced in leveraging event brokers (Kafka /Event Hubs / Kinesis) to integrate with data sources and data consumers
- Understanding of basic networking concepts
- Familiar with Agile Software Development methodologies (Scrum)
Benefits
- Flexible employment and remote work
- International projects with leading global clients
- International business trips
- Non-corporate atmosphere
- Language classes
- Internal & external training
- Private healthcare and insurance
- Multisport card
- Well-being initiatives
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Architecture, Governance Intern
LeidosLeidos is an innovation company rapidly addressing the world’s most vexing challenges in national security and health.
• Supporting data modeling and data product teams with documentation, metadata enrichment, and asset organization • Assisting with data governance activities, including business glossary management, data domain alignment, and stewardship workflows • Building Collibra content and business community views • Helping synchronize the Collibra DEV environment to make it closer to production (views, glossary terms, and data assets) to support testing and development • Creating and updating standard operating procedures (SOPs) and training materials, including short how-to guides and videos • Writing and validating Snowflake SQL queries to support data quality checks, analysis, and governance use cases • Assisting with data quality initiatives, including profiling, rule definition, and issue tracking • Collaborating with engineers and data scientists to build one or two simple APIs or data integrations • Participating in team meetings, design discussions, and governance forums to gain exposure to enterprise data decision-making
Data Architect
Independence Pet HoldingsTogether, we are building a better future for pets and pet parents.
Established in 2021, Independence Pet Holdings is a corporate holding company that manages a diverse and broad portfolio of modern pet health brands and services, including insurance, pet education, lost recovery services, and more throughout North America. We believe pet insurance is more than a financial product and build solutions to simplify the pet parenting journey and help improve the well-being of pets. As a leading authority in the pet category, we operate with a full stack of resources, capital, and services to support pet parents. Our multi-brand and omni-channel approach include our own insurance carrier, insurance brands and partner brands. Clarus is a pioneering pet health technology company on a mission to transform the future of pet healthcare. We are dynamic entrepreneurs improving pet health by providing trusted and transparent information across the entire pet health landscape. Join us in shaping a smarter, healthier future for pets everywhere. As we expand, we are looking for a seasoned Data Architect , in the data team, who can hit the ground running by building data models across the operational and analytics data store. This role will be reporting to the Senior Director of Data and Analytics. Role Overview This is a hands-on role for an experienced Data Architect with a proven track record of implementing a scalable data lake/medallion architecture and building robust data models in partnership with our product and engineering teams. The role is focused on designing and implementing scalable data solutions using Azure Databricks, acting as a subject matter expert in the platform to create efficient data pipelines, manage data lakes, and ensure data quality while leveraging the full capabilities of the Azure cloud ecosystem. Key Responsibilities: Design and implement data architecture solutions using Databricks, Spark, and Azure data services. Develop and optimize data models for both operational and analytical systems. Ensure data integrity, security, and accessibility across various platforms. Integrate data pipelines with event-based and microservice architectures leveraging Service Bus, Event Hub, and Functions. Create and maintain data standards, guidelines, and best practices. Conduct data quality assessments and implement data governance policies. Provide guidance on data storage solutions and ELT/ETL processes. Collaborate with developers, analysts, and data scientists to deliver trusted datasets for reporting, analytics, and machine learning use cases. Participate in architecture discussions, backlog refinement, estimation, and sprint planning. Contribute to defining and maintaining best practices, coding standards, and quality guidelines for data engineering. Perform code reviews, provide technical mentorship, and foster knowledge sharing within the team. Continuously evaluate and enhance data engineering tools, frameworks, and processes in the Azure environment. Qualifications: Company paid short-term and long-term disability 12 weeks of 100% paid Parental Leave Health Savings Account (HSA) Flexible Spending Accounts (FSA) Retirement savings plan Personal Paid Time Off Paid holidays and company-wide Wellness Day off Paid time off to volunteer at nonprofit organizations Pet friendly office environment Commuter Benefits Group Pet Insurance On the job training and skills development Employee Assistance Program (EAP)
• Define the future of AI + human healthcare with innovative data engineering solutions • Architect data platforms in a fast-paced, high-growth environment • Collaborate with teams on multi-stakeholder projects working extensively with healthcare data and standards (e.g. HL7 FHIR, CDS, ICD-10)
Senior Software Engineer, Data
Element 84Accelerating and scaling impactful projects with great software and design. Geospatial, cloud, and petabyte-scale data.
• Participate in all aspects of the software and data product development lifecycle from user story generation, through design, development, automated testing and operational support • Improve quality by actively participating in code-reviews and adhering to team quality standards • Work alongside other engineers on the team to elevate technology and consistently apply best practices • Own the execution of medium-to-large sized features with higher-level technical support • Describe and document the details of your work fluidly and accurately for technical peers and non-technical stakeholders • Think holistically about the application and build with an eye towards long term maintainability and efficiency • Collaboratively provide estimates and other input to the client, project managers designers, or others about features to help determine their feasibility, complexity, cost, and priority level • Contribute to a culture of positivity, curiosity, and respect for all individuals




