We make sense of data to drive your business forward. #MakeSenseofData #DriveYourBusinessForward #PartnerYourWay
Senior Data Engineer
Location
Washington
Posted
4 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
EXL
Senior Data Engineer - Job Identification 14365 - Job Role Data Engineering-Big Data Engineering - Experience (In Years)6-9 - Job Location Seattle Job Description: EXL is hiring a Senior Data Engineer to join a strategic AI / ML platform engagement with a leading specialty retailer. This is a hands-on build role embedded with the client''s platform engineering team. The role requires shipping production-grade data pipelines that feed real-time customer event data into machine learning workflows. The right person is comfortable owning the full lifecycle of pipeline design, build, and deployment: from streaming ingestion through event store design to model-ready feature delivery. This is a high-visibility role with growth potential into a larger book of work as the engagement expands. Required Skills & Experience - 6-12 years of experience in data engineering, platform engineering, or a closely related discipline. - Streaming: Production experience with Kafka consumers and Flink stream processing - building, deploying, and operating streaming jobs at meaningful scale. - GCP Data Stack: Strong SQL on BigQuery (or an equivalent cloud warehouse), with demonstrated query optimization, cost management, and partitioning chops. - Python Data Engineering: Hands-on with Polars or Pandas at scale; deep working knowledge of Parquet partitioning and performance on large joins. - ML Pipelines: Hands-on experience building and deploying components on Kubeflow Pipelines (KFP) and/or Vertex AI Pipelines; working proficiency with Docker. - Event Store Design: Demonstrated experience designing event stores - partitioning by customer, time-ordered event assembly across sources, schema strategy for mixed event types (clicks, purchases, returns). - Communication: Strong written and verbal communication; comfortable being the senior IC voice in design conversations with client stakeholders. Nice to Have - Domain experience in Retail or E-commerce - customer journey data, transaction analytics, returns and exchanges modeling. - Exposure to schema registry tooling (e.g., Confluent), Iceberg, or Delta Lake. - Experience working in client-facing or consulting engagements. - Google Cloud certifications (Professional Data Engineer or equivalent). Work Arrangement & Eligibility - This role requires 3-4 days per week onsite in Seattle, WA. Fully remote and out-of-state candidates will not be considered. - EXL is open to sponsoring H1B transfers for qualified candidates. What You''ll Do - Design and operate event-driven data pipelines using Kafka consumers and Flink jobs to process high-volume customer events (clicks, purchases, returns) in near-real time. - Build and optimize large-scale data transformations on Google Cloud Platform - BigQuery SQL, query performance tuning, and partitioning strategy at scale. - Develop Python data engineering workloads using Polars or Pandas at scale, with rigorous attention to Parquet partitioning, join performance on large datasets, and memory efficiency. - Build, deploy, and maintain ML pipeline components on Kubeflow Pipelines (KFP) and Vertex AI; package and deploy services with Docker. - Design event store architecture: partitioning by customer, time-ordered event assembly across heterogeneous sources, and schema management for mixed event types. - Partner with ML engineers, platform engineers, and data scientists to deliver clean, performant, model-ready data products. - Document architecture decisions and contribute to engineering standards across the platform team. What You''ll Do - Design and operate event-driven data pipelines using Kafka consumers and Flink jobs to process high-volume customer events (clicks, purchases, returns) in near-real time. - Build and optimize large-scale data transformations on Google Cloud Platform - BigQuery SQL, query performance tuning, and partitioning strategy at scale. - Develop Python data engineering workloads using Polars or Pandas at scale, with rigorous attention to Parquet partitioning, join performance on large datasets, and memory efficiency. - Build, deploy, and maintain ML pipeline components on Kubeflow Pipelines (KFP) and Vertex AI; package and deploy services with Docker. - Design event store architecture: partitioning by customer, time-ordered event assembly across heterogeneous sources, and schema management for mixed event types. - Partner with ML engineers, platform engineers, and data scientists to deliver clean, performant, model-ready data products. - Document architecture decisions and contribute to engineering standards across the platform team.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Developer
QinetiQ USWe are a world-class team of professionals who deliver next generation technology and products in robotic and autonomous platforms, ground, soldier, and maritime systems in 50+ locations world-wide. Much of our work contributes to innovative research in the fields of sensor science, signal processing, data fusion, artificial intelligence (AI), machine learning (ML), and augmented reality (AR). QinetiQ US’s dedicated experts in defense, aerospace, security, and related fields all work together to explore new ways of protecting the American Warfighter, Security Forces, and Allies. Being a part of QinetiQ US means being central to the safety and security of the world around us. Partnering with our customers, we help save lives; reduce risks to society; and maintain the global infrastructure on which we all depend.
Role Description Join us in our support to the US Space Force (HQ USSF) Commercial Satellite Communication (COMSATCOM) Office (CSCO), where you will help shape the future of assured, resilient SATCOM for our warfighters and mission partners. As a key member of a highly collaborative, mission-focused team, you will apply your expertise to help deliver innovative commercial SATCOM capabilities, integrate a transformative COMSATCOM business model into the broader SATCOM enterprise, and translate complex operational needs into effective, affordable solutions. In this role, you will work directly with COMSATCOM customers and stakeholders, providing trusted guidance, insightful analysis, and clear recommendations that drive real-world impact in a rapidly evolving space domain. Responsibilities - Design, build, and maintain automated data pipelines on the Advana and Financial Accountability System and Transformation (FASTR) platform using Python, PySpark, and SQL to ingest, normalize, and integrate COMSATCOM contract, billing, and operational data from disparate DoD and commercial sources. - Build and maintain Power BI dashboards and interactive reports supporting CSCO program management, executive decision-making, and Working Capital Fund financial reporting. - Support CSCO's operationalization of Advana and FASTR as the organization's primary financial reporting and analytics environment; lead data integration testing and platform configuration activities. - Develop automated database processes, including prepopulated worksheets, change logs, and audit trails, in support of monthly deliverables, data calls, and program performance reporting. - Integrate data flows from DoD enterprise systems — including FAMIS as a Service (FaaS), CORAS, and Electronic Document Access (EDA) — into the Advana/FASTR analytics ecosystem. - Partner with Data Scientists to operationalize predictive model and machine learning outputs within production infrastructure, ensuring pipelines are version-controlled, reproducible, and auditable. - Maintain data quality, governance, and documentation standards; develop training materials and provide platform guidance to program team members. - Respond to ad hoc development requirements from program leadership, including custom reports, database queries, and integration tasks, within program-defined timelines. Qualifications - Minimum of 5 years of experience in data engineering, software development, database administration, or a closely related technical discipline. - Bachelor's Degree in Computer Science, Data Science, Information Systems, Software Engineering, or a related field. - Secret Clearance required. - Demonstrated proficiency in Python and SQL for data engineering, ETL development, and pipeline automation. - Experience with enterprise data platforms, distributed computing frameworks (Apache Spark/PySpark), or cloud-based analytics environments (Azure Government, AWS GovCloud, or Databricks). - Experience developing production-grade Power BI dashboards and data models for operational or executive-level stakeholders. - Strong understanding of data quality, governance, and documentation practices, including version control and audit trail standards. - Excellent communication skills with the ability to document technical work clearly and support non-technical stakeholders. - Deadline-driven, organized, and detail-oriented with the ability to operate effectively in a fast-paced DoD program environment. - Ability to support on-site client meetings within the National Capital Region. Preferred Qualifications - Experience developing on the Advana platform (Databricks/Delta Lake architecture) or the FASTR environment within a DoD financial management context. - Familiarity with DoD financial management and enterprise systems including FAMIS, CORAS, Electronic Document Access (EDA), or Defense Agencies Initiative (DAI). - Experience supporting DoD or USSF space or satellite communications programs in a data engineering capacity. - Familiarity with Working Capital Fund (WCF) financial data structures, ESAG reporting requirements, and enterprise SATCOM program data. - TS/SCI. - Experience with MLOps practices, including model deployment and pipeline integration within enterprise data environments. - Familiarity with Microsoft Power Platform or SharePoint data architecture in a DoD collaboration environment. Pay Transparency The salary range for this role is $120,000- $160,000 USD. The salary range provided is a good faith estimate representative of all experience levels. QinetiQ US considers several factors when extending an offer, including but not limited to, the role, function and associated responsibilities, a candidate’s work experience, location, education/training, and key skills. Company EEO Statement Accessibility/Accommodation: If because of a medical condition or disability you need a reasonable accommodation for any part of the employment process, please send an e-mail to staffing@us.QinetiQ.com or call (540) 658-2720 Opt. 1 and let us know the nature of your request and contact information. QinetiQ US is an Equal Opportunity employer. All Qualified Applicants will receive equal consideration for employment without regard to race, age, color, religion, creed, sex, sexual orientation, gender identity, national origin, disability, or protected Veteran status.
• Architect and build scalable, high-performance ETL/ELT pipelines for large datasets. • Design and implement real-time data processing solutions. • Ensure data integrity, governance, and compliance with industry regulations. • Optimize data storage and query performance in cloud-based environments. • Automate data engineering workflows using Apache Airflow and CI/CD pipelines. • Collaborate with cross-functional teams to enhance data accessibility and reliability.
• Senior Big Data Engineer responsible for designing, building, and delivering scalable big data and ETL solutions. • Support projects at various and unanticipated client worksites throughout the U.S. • Work closely with cross-functional teams to develop, test, and deploy data solutions. • Design and implement highly scalable ETL applications on Hadoop and Big Data ecosystems. • Develop new scripts, tools, and methodologies for ETL workflows. • Deliver big data projects using Spark, Python, Scala, SQL, and Hive. • Prepare technical design documents for solutions and write well-documented code.
• Conduzir o processo completo de migração de dados SAP ECC → S/4HANA; • Realizar levantamento, análise e mapeamento de objetos e definição de regras de migração; • Executar extração, transformação e carga de dados utilizando: SAP Data Services (BODS), SAP Migration Cockpit, IDocs, LSMW, RFC / integrações; • Trabalhar com objetos dos módulos MM, SD, PP, WM, FI e CO; • Coordenar testes de migração, validações, reconciliações e atividades de cutover; • Interagir com o cliente e equipas internacionais para esclarecimentos e validações; • Produzir documentação técnica e funcional em inglês;



