Passionate music fans. Innovative tech pros. Perfect harmony. Join our band.
Staff Data Engineer – Subscriptions User Understanding
Location
United Kingdom
Posted
4 days ago
Salary
0
Seniority
Lead
Job Description
Staff Data Engineer – Subscriptions User Understanding
Spotify
• Define and drive the long-term data engineering strategy for the Subscriptions User Understanding domain, identifying opportunities that improve scalability, reliability, and business impact. • Design scalable batch and streaming data platforms that power analytics, experimentation, machine learning, and subscriber experiences. • Partner with Product, Engineering, Data Science, Analytics, and Platform teams to turn complex business challenges into durable, well-designed data solutions. • Lead architectural decisions and establish engineering best practices for data quality, governance, observability, reliability, and operational excellence across multiple squads. • Design data models and platform architecture that support sustainable growth toward one billion users while balancing performance, cost, and maintainability. • Simplify complex systems, reduce technical debt, and improve the developer experience across the broader data ecosystem. • Mentor senior engineers, influence technical direction through collaboration, and help foster a culture of thoughtful engineering, knowledge sharing, and continuous improvement.
Job Requirements
- Extensive experience designing, building, and operating large-scale cloud-native data platforms.
- Deep experience building reliable distributed data pipelines using technologies such as Scio, BigQuery, Dataflow, GCS, SQL, Python, or similar cloud-native tools.
- Strong understanding of modern data modeling, metadata management, governance, and data quality practices.
- Ability to balance immediate business needs with long-term architectural sustainability.
- Effective communication with engineers, product managers, data scientists, analysts, and senior leadership.
- Enjoy mentoring engineers and helping technical teams grow through coaching, technical guidance, and thoughtful feedback.
Benefits
- Flexible work arrangements
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, build, and maintain scalable data pipelines for feature extraction, training data generation, and model monitoring. • Develop and enhance data systems that support analytics and machine learning workloads, including data lakehouse and feature store technologies. • Monitor production data pipelines, identify data quality issues or pipeline failures, and implement improvements to ensure reliability and freshness. • Participate in engineering design discussions and contribute to technical decisions around data architecture and pipeline implementation. • Build reusable data engineering components, including automated data quality checks, schema validation, and testing frameworks. • Translate business requirements into scalable data solutions that enable analytics and machine learning use cases. • Optimize SQL queries, Spark workloads, and data processing pipelines to improve performance and scalability. • Collaborate with ML Engineers and cross-functional partners to support MLOps best practices, including data versioning, lineage, and reproducibility. • Break down technical work into manageable tasks and deliver high-quality solutions within an agile team.
• You create data maps across heterogeneous inventories and assess the maturity of the digital foundation. • You unlock unstructured data sources (PDFs, reports, publications) for use in AI systems. • You develop metadata and identifier concepts, data models, and embedding pipelines, and build the retrieval layer for RAG applications. • You handle sensitive data responsibly and coordinate closely on AI governance, data protection, EU AI Act, and sovereignty requirements. • You will build and operate ingest and retrieval pipelines for proprietary software products in the future.
Mid-level Data Engineer
SunnyDataWith a strong Data Engineering backbone, we deliver Databricks projects from concept to production.
• Be part of a team responsible for supporting new and existing customers in their data engineering needs. • Guide customers to make the best technical decisions to achieve their goals. • Actively work across multiple customer accounts which you would need to track and report on their progress. • Design, build and operationalize complex data solutions, fix issues, apply transformations, and recommend data cleansing / quality solutions. • Analyze data sources to determine their value and recommend them to include in analytical processes. • Incorporate core data management competencies including data governance, data security and data quality. • Collaborate within and across teams to support delivery and educate end users on data products / analytic environments. • Perform data and system analysis, assessment and resolution for defects and incidents of moderate complexity and correct as appropriate. • Test data movement, transformation code, and data components.
• Build and maintain high-throughput ETL/ELT pipelines • Drive the adoption of dbt for transformation and PySpark • Implement "Data SLAs" and build monitoring systems • Work closely with the BI team and Data Scientists • Deep-dive into SQL and Spark query plans for optimization • Implement a local development workflow with DuckDB • Build specialized pipelines for client reporting • Explore use of DuckDB for internal tools or edge processing




