Evolved Ideas logo
Evolved Ideas

We don't just build technology, we build highly scalable technology-enabled businesses.

Data Engineer

Data EngineerData EngineerFull TimeRemoteSeniorTeam 51-200Since 2007H1B No SponsorCompany SiteLinkedIn

Location

Ukraine

Posted

1 day ago

Salary

0

Seniority

Senior

Bachelor Degree5 yrs expEnglishAirflowBigQueryCloudPythonSQL

Job Description

Data Engineer

Evolved Ideas

• Own the end-to-end pipeline that creates the unified customer_uuid across Books & Media and Fashion • Maintain and evolve our customer identity master data with a strong focus on accuracy, reliability, and production quality • Improve our probabilistic identity resolution model and make matching decisions measurable, transparent, and explainable • Build scalable and cost-efficient data pipelines across BigQuery, GCS, and Cloud Run Jobs • Introduce diagnostics, monitoring, and structured validation for every relevant model change • Identify and resolve edge cases in customer matching logic before they become production issues • Work closely with business and technical stakeholders to turn complex matching challenges into robust data solutions

Job Requirements

  • 5+ years of experience in production data engineering
  • Strong experience with BigQuery and advanced SQL in large-scale analytical environments
  • Strong Python skills for production-grade data engineering
  • Solid Airflow experience and a strong understanding of reliable orchestration patterns
  • Hands-on experience with incremental pipelines and idempotent data processing
  • Experience with probabilistic record linkage or entity resolution in production
  • Strong understanding of data quality, matching logic, and precision/recall trade-offs
  • A careful, structured, and ownership-driven way of working
  • Strong communication skills and the ability to explain technical decisions clearly

Benefits

  • Healthcare insurance
  • Educational budget
  • Challenging tasks and professional development, knowledge & best practice sharing

Related Categories

Related Job Pages

More Data Engineer Jobs

Data Engineer

Navtech, Inc.

NAVTECH INC 1600 Golf Road. Suite 1200, Rolling Meadows, IL 60008 Ph: (224) 348-1340 Email: alex@navtechusa.com Website: www.navtechusa.com E-Verified Company

Data Engineer1 day ago

Role Description We are seeking Data Engineers with extensive, hands-on experience in Databricks and PySpark to join our dynamic team. - Provide L2/L3 support for data pipelines running in production environments. - Work on enhancements and optimizations to existing pipelines and workflows. - Design, develop, and maintain robust, scalable data pipelines to support business needs. - Ensure data quality, reliability, and performance across all stages of data processing. - Implement and enforce data governance policies and best practices. - Participate in an on-call rotation, including providing support over weekends when required. Qualifications - Proven expertise in Databricks and PySpark. - Strong background in data engineering, ETL processes, and production support. - Solid understanding of data governance, data quality frameworks, and monitoring tools. - Ability to work collaboratively in a fast-paced environment and troubleshoot complex data issues. Requirements - Location: Toronto, Canada or REMOTE - Duration: 6 Months Company Description NAVTECH INC - Contact: (224) 348-1340 - Email: Alex@navtechusa.com - Address: 1600 Golf Road, Suite 1200, Rolling Meadows, IL 60008 - Website: www.Navtechusa.com - E-Verified Company.

Canada

Role Description The OT Data Engineer will play a central role in building, scaling, and maintaining the data infrastructure that supports digital manufacturing, real-time operations, and advanced analytics across our pharmaceutical production network. This role focuses on integrating shop-floor systems, implementing a robust Unified Namespace (UNS) architecture, and leveraging HighByte Intelligence Hub to enable contextualized, reliable, and compliant data flows for manufacturing, quality, and business stakeholders. - Design, implement, and maintain a Unified Namespace (UNS) to standardize real-time data access across OT, IT, and enterprise systems. - Deploy and configure High Byte Intelligence Hub for data modeling, contextualization, transformation, and secure data movement between OT and cloud platforms. - Integrate data from PLC/SCADA systems, historians (e.g., PI, IP.21), MES, LIMS, and IoT devices into the UNS and analytics platforms. - Build and maintain data pipelines that support real-time monitoring, predictive analytics, digital twins, and process optimization. - Collaborate with automation, process engineering, quality, and IT teams to ensure 21 CFR Part 11, GxP, and data integrity compliance. - Develop and maintain data models, ontologies, and asset hierarchies aligned with ISA-95/ISA-88 standards. - Implement data governance, including metadata management, lineage tracking, and access control. - Support advanced analytics and machine learning initiatives by ensuring high-quality, contextualized data availability. - Troubleshoot data flow issues across OT/IT boundaries and optimize system performance. - Contribute to the roadmap for Industry 4.0, smart manufacturing, and digital transformation initiatives. Qualifications - Bachelor's degree in engineering, Computer Science, Information Systems, or related field. - 3-7 years of experience in OT data engineering, industrial data systems, or manufacturing IT. - Hands-on experience with: - Unified Namespace (UNS) architectures - HighByte Intelligence Hub (data modeling, flows, connectors) - MQTT brokers (Ignition, HiveMQ, EMQX) - Industrial protocols (OPC UA, Modbus, Ethernet/IP) - Strong understanding of pharma manufacturing environments, including GxP, validation, and data integrity requirements. - Proficiency in Python, SQL, and modern data engineering tools. - Experience with cloud platforms (Azure, AWS, or GCP) and edge-to-cloud architectures. - Experience with ISA-95/ISA-88 modeling and enterprise-to-shop-floor integration. - Knowledge of data analytics, statistical process control, or machine learning workflows. - Experience with Ignition, Kepware, or similar industrial middleware. - Familiarity with historians, MES, Tulip and automation systems. Skills and Ability - Strong problem-solving and root-cause analysis skills. - Ability to translate manufacturing needs into scalable data solutions. - Excellent communication skills for cross-functional collaboration. - Detail-oriented mindset with a focus on reliability and compliance. - Passion for digital transformation and modern OT/IT convergence.

India

Role Description Client is seeking an experienced Data Spoke Technical Lead to support the People Data domain within a federated enterprise data organization. This role will serve as the primary liaison between business stakeholders, IT, data engineering, and data governance teams, ensuring successful delivery of People Data initiatives while driving governance adoption and technical alignment. This is primarily a technical leadership and coordination role (70%), with a focus on governance execution (20%) and limited hands-on technical support (10%). Responsibilities - Data Spoke Leadership & Delivery - Support execution of the People Data roadmap and priorities. - Manage intake, triage, prioritization, and tracking of data-related work requests. - Maintain and govern backlogs, ensuring alignment with business priorities and enterprise strategy. - Coordinate across business, IT, data engineering, and governance stakeholders. - Facilitate Agile ceremonies including backlog reviews, sprint planning, and status updates. - Provide visibility into delivery progress, risks, and dependencies. - Data Governance - Drive adoption of governance practices within the People domain. - Support metadata management, data ownership, stewardship, classification, and retention activities. - Partner with Data Owners and Data Stewards to execute governance responsibilities. - Monitor and report on governance and data quality KPIs. - Technical Leadership - Provide technical leadership for People Data solutions and assets. - Support SQL-based data modeling, including dimensional modeling and Slowly Changing Dimensions (SCD). - Guide data pipeline and integration initiatives using Azure Data Factory or similar technologies. - Support data warehouse and semantic model design. - Assist with troubleshooting and solution design in partnership with engineering teams. - Contribute to DevOps processes including backlog management, epics, and user stories. Qualifications - 10+ years of experience in Data Management, Data Engineering, Analytics, or Business Intelligence. - Experience working within enterprise data governance frameworks. - Proven experience leading or supporting a data domain or business-facing data function. - Strong experience operating within a hub-and-spoke or federated data model. - Demonstrated success coordinating across business, IT, data engineering, and governance teams. - Advanced SQL and data modeling expertise, including dimensional modeling and SCD concepts. - Experience with data pipelines and ingestion frameworks such as Azure Data Factory or equivalent. - Experience with data warehouse and semantic model design. - Strong stakeholder management, communication, and delivery coordination skills. - Experience with Agile delivery methodologies and DevOps backlog management. Preferred Qualifications - Experience supporting HR, People, Workforce, or Human Capital data domains. - Experience with Power BI, including data modeling, DAX, and reporting. - Familiarity with Azure Databricks and the broader Azure Data Platform ecosystem. - Working knowledge of C# or similar programming languages. - Experience with Collibra or other metadata management and data catalog tools. - Experience implementing data ownership, stewardship, and governance operating models. - Master's degree in Information Systems, Computer Science, Data Analytics, or a related discipline.

EST (UTC-5)

Role Description Oxford Client is building a new Planning System and needs a Data Engineer to assist. Qualifications - Strong experience in data engineering and pipeline development in enterprise environments. - Advanced SQL skills, including complex joins, window functions, and performance optimization. - Experience with cloud data platforms, data lakes, and curated analytical layers. - Proficiency in Python or similar scripting languages for data transformation and automation. - Hands-on experience integrating data from SAP or other ERP systems. - Solid understanding of data modeling, including dimensional and time series data. Requirements - Experience implementing data quality checks, validations, and monitoring. - Understanding of data governance, access controls, and regulatory or audit considerations. - Familiarity with supply chain and planning concepts, including demand forecasting, supply planning, inventory management, and capacity planning. - Understanding of planning KPIs such as forecast accuracy and bias, service levels, inventory turns, and cost tradeoffs. - Experience supporting planning, optimization, or decision support systems is highly desirable. Company Description

India