Pioneering AI-first solutions, solving complex business challenges through expertise, cloud, data engineering, and AI.
Architect, Data Engineer
Location
Massachusetts
Posted
3 days ago
Salary
0
Seniority
Lead
Job Description
Architect, Data Engineer
Quantiphi
• Lead the architectural vision for a next-generation data layer designed specifically for Agentic AI. • Design the end-to-end blueprint for a modern data layer that seamlessly integrates structured, unstructured, and relational (Graph) data for AI agents. • Oversee the health, security, and performance optimization of our data clusters (Snowflake/Kinetica), ensuring 99.9% availability for mission-critical AI workflows. • Act as the 'Face of Engineering' for the customer. Lead discovery workshops, manage technical expectations, and align the architectural roadmap with their business objectives. • Establish benchmarks for data latency and retrieval accuracy, ensuring the data layer can keep pace with the real-time demands of agentic execution.
Job Requirements
- 10+ years of experience in system architecture and data engineering
- Proven expertise in architecting for Snowflake (Data Cloud) and Kinetica (Real-time/Vector/OLAP)
- Ability to design Property Graphs or RDF schemas that map enterprise entities into a machine-readable 'World Model'
- Deep knowledge of data orchestration patterns (Change Data Capture, Streaming, and Batch)
- Strong DBA skills—partitioning strategies, indexing, vacuuming, and resource scaling in cloud-native environments.
Benefits
- Make an impact at one of the world’s fastest-growing AI-first digital engineering companies.
- Upskill and discover your potential as you solve complex challenges in cutting-edge areas of technology alongside passionate, talented colleagues.
- Work where innovation happens - work with disruptive innovators in a research-focused organization with 60+ patents filed across various disciplines.
- Stay ahead of the curve—immerse yourself in breakthrough AI, ML, data, and cloud technologies and gain exposure working with Fortune 500 companies.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineering Manager
EYBuilding a #BetterWorkingWorld by providing trust through assurance and helping organizations grow, transform & operate.
• Lead and mentor a team of data engineers • Define and drive the overall data architecture strategy • Oversee the design and implementation of data ingestion frameworks and integration solutions • Develop and manage CI/CD pipelines • Collaborate closely with clients and internal stakeholders • Act as a trusted advisor to clients • Ensure adherence to data governance and security standards • Drive the adoption of DevOps/DataOps principles within the team • Manage project priorities and delivery timelines
• Support scalable data operations through development of ETL processes, SQL-based integrations, Power Platform solutions, and Power BI reporting capabilities. • Design, build, maintain, monitor, and troubleshoot data-processing automations. • Develop and maintain data ingestion pipelines from external sources into SQL databases. • Manage automated flows to trigger Logic Apps and handle lightweight processes. • Perform full-stack BI development including data modeling, DAX development, and report publishing. • Leverage Microsoft Fabric as the unified access platform. • Ensure alignment with security and compliance requirements. • Conduct root-cause analysis to reconcile discrepancies between systems.
Role Description We are looking for a Senior Data Engineer to join the Innovation team as a core member of the PF-LLM programme — our initiative to build a from-scratch multivariate time-series foundation model across a fleet of ~1,000 wind and PV sites. You will be the connective tissue of the entire programme: - Owning the data foundation that makes state-of-the-art model training possible. - Managing the inference service that makes model outputs usable. - Overseeing platform integration that puts those outputs in front of pilot customers. - From production ETL through to shadow-mode validation pipelines, you will be the engineer who keeps every track moving. This role is critical-path from day one. Qualifications - 6+ years of back-end and data engineering experience, with a proven track record of shipping production systems. - Production-grade ETL/ELT pipeline design at scale: idempotency, retry logic, backfill jobs, incremental loading, and cost-controlled warehouse compute. - Schema design and data modelling across heterogeneous sources — experience reconciling signals from disparate systems into a canonical, queryable format. - Data quality engineering: automated quality gates (sparsity, flatline detection, outlier flagging, freshness checks), alerting pipelines, and dataset versioning for ML reproducibility. - API design and development: RESTful inference services with contract testing, latency and throughput budgeting, and structured observability (logs, metrics, traces). - Experience integrating ML model outputs into SaaS product surfaces: auth and authorisation, customer isolation, and feature flag management. - Cloud infrastructure proficiency (AWS preferred), containerisation (Docker, Kubernetes), and CI/CD pipeline ownership. - Python and SQL as core tools; hands-on experience with modern warehouse technologies (Snowflake, BigQuery, or Databricks). - Pipeline orchestration with Airflow, Prefect, Dagster, or equivalent. - Excellent written and verbal communication skills in English. Requirements - Design and build the production ETL pipeline from source systems to warehouse and feature store at fleet scale, covering thousands of wind and PV sites across multiple OEMs. - Own canonical signal schema design across wind and PV asset classes and OEMs — the deepest technical unknown in the programme and the foundation everything else depends on. - Implement automated data quality gates: sparsity and missingness checks, flatline detection, outlier flagging, and freshness validation, with alerting that generates tickets automatically. - Implement dataset versioning sufficient to reproduce every trained model from scratch. - Build and maintain backfill jobs, idempotency guarantees, and retry logic that survive mid-run failure without duplicating data. - Govern storage and compute costs on the warehouse from day one. - Build the batch and on-demand inference API with contract tests, sized for fleet-wide daily runs. - Establish latency and throughput baselines; own the cold-start and model-loading strategy. - Instrument the service with structured logs and metrics from the outset. - Integrate forecasts into the Power Factors product platform: auth and authorisation with customer isolation, observability hooked into the existing stack, and feature flags per customer and per site. - Build and maintain the shadow validation pipeline: run live inference in parallel with the existing forecast path, log predictions and actuals, and produce weekly validation reports broken down by asset class, OEM, and region. - Support the pilot customer rollout: enable the product for friendly customers behind flags and own incoming data and integration tickets throughout the pilot window. - Work closely with the ML Engineer to align on data quality requirements, feature store interfaces, and the handoff between the data platform and training pipeline. - Partner with the Tech Lead and Frontend Engineer during platform integration to ensure a clean, maintainable integration surface. - Contribute to architectural decisions across the programme and document data flows, schemas, and pipeline runbooks to a standard that supports the broader team. Benefits - Comprehensive benefits package including health, dental, and vision coverage, plus dedicated wellness support. - Generous paid vacation policy. - Employer RRSP matching program. - Work-from-abroad opportunities with manager approval. - Exposure to a global team operating across multiple countries and time zones. - A humble cause with a clear purpose — you will help us fight climate change with code every day at work.
Senior Data Engineer – Enterprise B2B Marketplace
Truelogic SoftwarePremium boutique software development company that helps brands with big ideas to make a difference in people’s lives.
• Data Platform Evolution: Guide the foundational architecture, scaling strategies, and long-term roadmap of the enterprise data platform. • Pipeline Engineering: Design and lead the development of highly scalable data pipelines using Airflow, dbt, and Python. • Modern Stack Integration: Build and maintain high-throughput integrations across core modern data stack tools, including Fivetran, Redshift, and Sigma. • Serverless Architecture: Develop and optimize serverless data services and ingestion layers leveraging AWS infrastructure (e.g., AWS Lambda). • Advanced Data Modeling: Partner with cross-functional stakeholders to define reliable, performant data warehouse architectures and analytical datasets. • Observability & Reliability: Implement automated testing, rigorous monitoring frameworks, and tracing to maximize pipeline reliability and minimize operational downtime. • Technical Leadership & Governance: Mentor data engineers and analysts on engineering best practices, while driving continuous improvements in data governance and documentation.



