• Design, develop, and maintain scalable, robust data pipelines. • Build data ingestion, transformation, and modeling solutions using Databricks, Spark/PySpark, Cloudera, and Azure Data Factory (ADF). • Ensure the quality, integrity, and usability of data across the entire pipeline. • Implement CI/CD pipelines focused on ETL processes. • Design and maintain Data Lake and Data Warehouse solutions, applying data governance best practices. • Apply FinOps concepts to optimize cloud data processing costs. • Develop and document data workflows, processes, and architectures. • Perform technical troubleshooting of pipelines and data infrastructure. • Work with concepts and structures such as Bronze, Silver, and Gold layers, Star Schema, Delta Tables, Delta Sharing, and analytical tables. • Collaborate with technical and business teams to orchestrate solutions aligned with company objectives. • Manage the technical roadmap for data projects, considering dependencies, trade-offs, and change management.

• Build, scale, and maintain robust data solutions. • Implement and optimize high-performance data pipelines: extraction, loading, transformation, and orchestration – that are designed for scalability, reliability, maintainability, and speed. • Champion modern software engineering practices as CI/CD, infrastructure-as-code, containerization, and cloud-native deployments • Collaborate closely with business stakeholders to transform use cases into production-ready services and solutions, owning the system from concept to production. • Implement rigorous testing and monitoring practices to maintain superior data quality and integrity.

• Architect and implement scalable ETL and data pipelines spanning ClickHouse, Postgres, Athena, and diverse cloud-native sources to support real-time risk management and advanced analytics for AI-driven decisioning. • Design, develop, and optimize distributed data storage solutions to ensure both high performance (low latency, high throughput) and reliability at scale—serving mission-critical models for fraud detection and compliance. • Drive schema evolution, data modeling, and advanced optimizations for analytical and operational databases, including sharding, partitioning, and pipeline orchestration (batch, streaming, CDC frameworks). • Own the end-to-end data flow: integrate multiple internal and external data sources, enforce data validation and lineage, automate and monitor workflow reliability (CI/CD for data, anomaly detection, etc.). • Collaborate cross-functionally with engineers, product managers, and data scientists to deliver secure, scalable solutions that enable fast experimentation and robust operationalization of new ML/AI models. • Champion radical ownership—identify opportunities, propose improvements, and implement innovative technical and process solutions within a fast-moving, remote-first culture. • Mentor and upskill team members, cultivate a learning environment, and contribute to a collaborative, mission-oriented culture.

Role Description We are looking for a Mid/Senior Data Engineer to join our Data Engineering practice and help clients build modern data foundations on Databricks and AWS. - Design and build data pipelines that extract from enterprise ERP systems, transform through medallion architectures, and deliver governed, AI-ready data products. - Work directly with client subject-matter experts to understand business domains, validate data models, and ensure the platform is production-grade from day one. - Current engagements involve regulated manufacturing environments where data governance, quality management, and traceability are essential. - This is a fully remote role with collaboration across distributed teams and daily overlap with the US Eastern Time Zone. Qualifications - 4–7+ years of experience as a Data Engineer or in a closely related role. - Strong programming skills in Python, including PySpark. - Solid SQL skills including complex analytical queries against large enterprise databases. - Hands-on experience with Databricks: Delta Lake, Unity Catalog, Databricks Workflows, and SQL Warehouse. - Working knowledge of AWS core services: S3, IAM, VPC, and networking fundamentals. - Experience building ETL/ELT pipelines that extract from enterprise ERP or transactional systems (Oracle, SAP, Microsoft Dynamics, or similar). - Strong understanding of data modeling, medallion architectures, and dimensional design. - Experience with data quality frameworks: validation rules, anomaly detection, and exception handling. - Experience using AI and LLM tools to accelerate engineering workflows — including deriving data contracts, mapping specifications, and schema documentation from database metadata and limited business context. - Comfortable collaborating directly with business stakeholders and subject-matter experts, not just engineering teams. - Ability to participate in technical discussions, code reviews, and architectural decisions with confidence. - Reliable high-speed internet and ability to work effectively in a remote-first environment. - Daily overlap with US Eastern Time Zone. Requirements - Familiarity with Oracle E-Business Suite table structures and data patterns (INV, PO, BOM, WIP modules). - Exposure to manufacturing domain concepts: bills of material, work orders, production routing, inventory management. - Experience with dbt for data transformation and data product development. - Hands-on experience with data governance and catalog tooling (Unity Catalog, AWS Glue/Datazone, Apache Atlas, or similar). - Multi-system data integration or ERP consolidation experience, reconciling different source schemas into a unified canonical model. - Spec-driven or contract-driven development methodology, YAML specifications, schema validation, data contracts. - Experience in medical device, pharmaceutical, or other regulated manufacturing environments. - Databricks Asset Bundles and CI/CD automation for data platform deployments. - Familiarity with Apache Iceberg or Delta Lake UniForm for open table format interoperability. - Experience supporting AI/ML workflows in production: feature engineering, model serving integration, or AI-ready data product design. Benefits - Building data foundations that power AI, analytics, and operational decision-making for manufacturing enterprises. - Working directly with domain experts to understand how real businesses operate, not just pushing data through pipes. - Solving multi-system integration challenges where no two ERPs store data the same way. - Designing platforms with governance, observability, and data quality built in from the outset. - Contributing to a reusable platform accelerator that will be deployed across multiple client engagements. - Raising the bar for how data engineering is done: spec-driven, tested, version-controlled, and production-grade.