We enable our clients to thrive, thanks to a combination of technical proficiency and domain-specific knowledge.
Senior/Lead Data Engineer, Industrial Knowledge
Location
Poland
Posted
7 days ago
Salary
zł140 - zł200 / hour
Seniority
Senior
Job Description
Senior/Lead Data Engineer, Industrial Knowledge
Spyrosoft
• Design and implement ingestion and transformation pipelines from industrial source systems into clean, auditable datasets • Work directly with data from MES, SCADA, historians, PLC/OT systems, LIMS/LAB/OPS platforms, and other operational sources • Perform data quality audits and identify: Anomalies, Dead or inactive signals, Data gaps, Inconsistencies • Develop datasets and validation logic supporting KPI definitions and PoC delivery • Build PoC components such as: Batch analytics pipelines, Event detection logic, Time-series transformations • Create lightweight PoC tooling, dashboards, applications, or visualizations when required • Support presales and consulting teams by shaping technical solutions and identifying business value hidden in industrial data • Produce delivery-grade documentation, handover materials, and implementation support assets
Job Requirements
- 5+ years of experience in Data Engineering, industrial analytics, or data solution delivery
- Strong Python and SQL skills for building ingestion pipelines, transformations, and validation logic
- Proven experience in building reproducible, auditable, and scalable data products
- Hands-on experience with industrial and operational data, including: MES, SCADA, Historians, PLC / OT systems, Operational time-series data
- Solid background in data profiling and data quality assessment, including: Anomaly detection, Gap analysis, Dead signal analysis, Inconsistency checks
- Ability to design datasets aligned with business KPIs and PoC objectives
- Strong engineering discipline: Git-based workflows, Code reviews, Testing practices, Documentation and runbooks
- Experience working in PoC-driven, KPI-oriented project environments
- English level: B2 or higher
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Director – Enterprise Data Platforms
Trillium Health ResourcesTransforming Lives. Building Community Well-Being.
• Perform strategic leadership and oversight of data management staff • Direct performance, security, and scalability of enterprise data environments • Establish strategic roadmap for database and data warehouse services • Establish governance structures, policies, and standards for data management and architecture • Collaborate with senior leadership to align data initiatives with business objectives • Oversee administration of enterprise databases
Role Description - Design, develop, and maintain ETL pipelines using DBT and other data integration tools. - Collaborate with data engineers, analysts, and business teams to define requirements and deliver data solutions. - Build and optimize data models (star/snowflake schemas, fact/dimension tables) to support reporting and analytics. - Write, test, and document DBT models and macros for data transformation. - Implement data quality checks, testing, and validation frameworks within DBT. - Optimize query performance and troubleshoot pipeline bottlenecks. - Manage version control (Git), deployment, and CI/CD pipelines for DBT projects. - Ensure compliance with data governance, security, and privacy standards. - Stay updated with the latest modern data stack technologies (Snowflake, BigQuery, Redshift, Airflow, Fivetran, etc.). Qualifications - Proven experience as an ETL Developer / Data Engineer with hands-on DBT expertise. - Strong knowledge of SQL and relational databases (Snowflake, Redshift, BigQuery, PostgreSQL, etc.). - Experience in cloud data warehouses and modern ELT/ETL frameworks. - Proficiency in data modeling (Kimball/Inmon methodologies). - Familiarity with version control (Git), CI/CD pipelines, and collaborative workflows. - Understanding of data governance, security, and quality best practices. - Strong problem-solving skills with the ability to optimize and debug pipelines.
• Design end-to-end data architectures (batch/streaming ingestion, processing, storage in Lakehouse/Warehouse and consumption via BI/APIs/ML) • Build and maintain the Data Map (inventory of sources, pipelines and consumers) and perform domain-oriented conceptual/logical modeling • Define taxonomies, classify sensitive data, and support privacy and compliance requirements • Produce technical artifacts (HLD, LLD, ADRs, C4/DFD diagrams) and present/defend solutions in architecture committees • Maintain continuous collaboration with Data Engineering, Security, Platform teams and Business areas.
• Diseñar e implementar soluciones analíticas y de datos en Microsoft Azure • Desarrollar pipelines y notebooks (PySpark, Spark SQL, SparkR, Python, T-SQL) para la integración, transformación y explotación de datos • Valoración de experiencia en Fabric y con gobernanza y catalogación de datos con Microsoft Purview
