We're making driverless vehicles a safe, reliable, and accessible reality.
Senior Data Engineer
Location
United States
Posted
17 hours ago
Salary
$159K - $207K / year
Seniority
Senior
Job Description
Senior Data Engineer
Motional
Role Description We are seeking a highly skilled and motivated Senior Data Analysis Engineer for our large-scale AI model and software evaluation framework – Ground Truth Regression. The ideal candidate will have a strong background in data engineering, machine learning principles, and statistical analysis. At Motional, large scale orchestration for data analysis plays a critical role in delivering our ML-centered autonomous driving vehicle. Our robo-taxi ML Models and software are developed by hundreds of developers and deployed multiple times a day. The Ground Truth Regression team provides the large scale perception analysis framework for validating all changes to perception which can impact the autonomous vehicle behavior. The GTRegression team validates the end impact of ML and software changes to the autonomous vehicle while those changes are in development, providing the tools for Root Cause Analysis and safety consideration. We monitor for model errors, anomalies, rare objects & long-tail driving scenarios across thousands of driving hours. The team develops the full stack – AWS Kubernetes orchestration framework, ReSimulation, model metrics, regression reports, and deep dive analysis UIs. The team works together with all other perception and ML teams to develop the metrics and analyze results. What You'll Do: - Work with ML Engineers and Autonomy Software Developers to develop new data analysis metrics and KPIs for the validation of the autonomous vehicle performance. - Drive innovation by researching and developing new large scale data analysis systems. - Own large-scale data analysis workflows that surface unexpected impacts of planned and unplanned changes to the ML models and software. - Build high-quality datasets to improve ML products through training & edge case validation. - Provide statistical depth on model performance & generalization through rigorous error and change analysis. Qualifications - Bachelor's or Master's degree in Computer Science, Data Science, or a related field. - Strong programming skills in Python, SciPy, and related data analysis frameworks. - Experience in machine learning, data mining, and statistical analysis. - Experience with large-scale data processing and distributed computing technologies. - Strong communication and software development skills: goal setting, design documentation, test frameworks, collaborative cross team development. Requirements - Ph.D. in Computer Science or a related field (Bonus Points). - Experience with cloud platforms such as AWS, Google Cloud, or Azure (Bonus Points). - Experience with machine learning in the autonomous driving domain (Bonus Points). - Publications or contributions to the AI/ML community (Bonus Points). Benefits - Motional’s benefits include but are not limited to medical, dental, vision, 401k with a company match, health saving accounts, life insurance, pet insurance, and more. Salary Range $159,000 — $207,000 USD Company Description Motional is a driverless technology company making autonomous vehicles a safe, reliable, and accessible reality. We’re driven by something more. - Our journey is always people first. - We aren't just developing driverless cars; we're creating safer roadways, more equitable transportation options, and making our communities better places to live, work, and connect. - Our team is made up of engineers, researchers, innovators, dreamers, and doers, who are creating a technology with the potential to transform the way we move. - We’re creating first-of-its-kind technology that will transform transportation. - To do so successfully, we must design for everyone in our cities and on our roads. - We believe in building a great place to work through a progressive, global culture that is diverse, inclusive, and ensures people feel valued at every level of the organization. - Diversity helps us to see the world differently; it’s not only good for our business, it’s the right thing to do. - Our team is behind some of the industry's largest leaps forward, including the first fully-autonomous cross-country drive in the U.S, the launch of the world's first robotaxi pilot, and operation of the world's longest-standing public robotaxi fleet. - We’re driven to scale; we’re moving towards commercialization of our technology, and we need team members who are ready to embrace change and challenges. - Formed as a joint venture between Hyundai Motor Group and Aptiv, Motional is fundamentally changing how people move through their lives. - Headquartered in Boston, Motional has operations in the U.S and Asia.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Role Description Estamos buscando uma pessoa para fazer parte deste time que vem crescendo a cada dia, tanto em número quanto em desenvolvimento. Se você é essa pessoa, que tem um senso de equipe e que procura por desenvolvimento constante, a BIX é para você! Qualifications - Experiência sólida no provisionamento e uso de serviços AWS como DMS, S3, Glue, Athena, Kinesis, IAM (roles/policies), VPC/networking, etc. - Experiência no desenho de pipelines near-real-time, semantics de processamento (at-least-once / exactly-once) e trade-offs entre latência e consistência. - Experiência com captura contínua de mudanças via redo logs, supplemental logging e replicação incremental. - Experiência sólida e comprovada em produção com Apache Kafka. - Domínio de Spark, Structured Streaming, Delta Lake e arquitetura lakehouse (camadas bronze/prata/ouro), Unity Catalog, lineage e data quality frameworks no Databricks. - SQL avançado e modelagem de dados relacional e dimensional. - Inglês avançado/fluente para atuação internacional. Requirements - Orquestração: Airflow, Data Factory ou equivalente. - Kafka Connect / Debezium: conectores de CDC integrados ao Kafka. - Certificação nas clouds (AWS, GCP, Azure). - Certificação em ferramentas de Engenharia de Dados (Databricks, Clickhouse, Airflow, dentre outras). Responsibilities - Projetar e implementar a migração de pipelines batch legados para uma arquitetura de streaming de dados em tempo real. - Configurar e operar captura de mudanças (CDC) na origem, replicando dados via AWS DMS para as camadas de streaming e lakehouse. - Modelar tópicos, partições e estratégias de serialização no Kafka, garantindo ordering, throughput e tolerância a falhas. - Desenvolver pipelines de ingestão e processamento no Databricks (Structured Streaming, Delta Lake), atendendo também demandas correntes da plataforma. - Garantir qualidade, observabilidade e governança dos dados em trânsito e em repouso. - Documentar arquitetura, decisões técnicas e realizar code review com o time. - Realizar reuniões com clientes, por meio de videoconferência, desenvolvendo os questionamentos estratégicos voltados ao seu negócio/processo, a cada início de projeto ou sprint. - Gerenciar atividades do projeto, utilizando métodos ágeis, de modo a garantir as entregas, a cada novo projeto. Benefits - Trabalho 100% remoto. - Plataforma de terapia e nutricionistas online. - Plataforma de bem estar. - Plano de carreira acelerado. - Contato direto com líderes de grandes empresas. - Ambiente diferenciado (Hora BIX, integrações e Happy Hours, Programa de Mentoria, Reuniões 1on1). - Aulas on-line de inglês (cambly). - Folga no aniversário.
Data Engineer
Fieldwire by HiltiThe all-in-one jobsite management software for field to office communication.
• Design, build, and maintain automated data pipelines that move data from source systems (Salesforce, Xero, Ramp, product databases) into our central data lake and warehouses • Own the end-to-end data architecture, including storage strategy, processing systems, and pipeline orchestration • Implement and maintain ETL/ELT workflows that extract, transform, and load data into clean, analytics-ready formats • Partner with Data Insights Managers and business stakeholders to translate reporting requirements into robust technical data solutions • Build automated validation and quality-check layers into every pipeline to prevent bad data from reaching reporting layers • Monitor pipeline health in real time; triage and resolve failures quickly to meet data availability SLAs • Enforce data standards, naming conventions, schema consistency, and access controls across all systems • Support integration and maintenance of key tools including Salesforce, Xero, Ramp and Greenhouse into the data lake • Maintain auditability of all data flows and support compliance and governance requirements • Collaborate with the DIM TL and Director of Operations on the data roadmap and architectural decisions.
Junior Architect / Senior Data Engineer - Remote
OptumOptum, part of the UnitedHealth Group family of businesses, is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together. At Optum, we support your well-being with an understanding team, extensive benefits and rewarding opportunities. By joining us, you’ll have the resources to drive system transformation while we help you take care of your future. We recognize the power of connection to drive change, improve efficiency and make a difference in health care. Join a team where your skills and ideas can make an impact and where collaboration is key to creating technology that produces healthier outcomes.
Requisition Number: 2374167 Optum Insight is improving the flow of health data and information to create a more connected system. We remove friction and drive alignment between care providers and payers, and ultimately consumers. Our deep expertise in the industry and innovative technology empower us to help organizations reduce costs while improving risk management, quality and revenue growth. Ready to help us deliver results that improve lives? Join us to start Caring. Connecting. Growing together. We are seeking a Senior Azure Data Engineer to help design, build, and operate our next-generation enterprise data platform on Microsoft Azure. You will own end-to-end delivery of data pipelines and data products that power analytics, regulatory reporting, operational dashboards, and emerging AI/ML use cases. You will partner closely with data architects, analytics engineers, data scientists, business stakeholders, and platform engineering teams to deliver reliable, performance, secure, and cost-efficient data solutions. This role is ideal for an engineer with solid hands-on depth in Azure Data Factory, Azure Synapse Analytics and/or Databricks, and modern Lakehouse patterns, who is comfortable leading migration programs (e.g., Informatica-to-ADF, on-prem warehouse-to-cloud), mentoring mid-level engineers, and shaping engineering standards across the team. You'll enjoy the flexibility to work remotely * from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week. Primary Responsibilities: - Pipeline Design & Development - Design and build robust, reusable, parameter-driven ingestion and transformation pipelines using Azure Data Factory, Synapse Pipelines, Data Bricks and/or Microsoft Fabric Data Factory - Implement medallion architecture (Bronze / Silver / Gold) on Azure Data Lake Storage Gen2 using Delta Lake, Parquet, and structured streaming patterns - Build performant ELT workflows that leverage pushdown to source systems (Synapse Dedicated SQL Pool, Azure SQL, Teradata) where appropriate - Develop and optimize PySpark notebooks and jobs on Azure Databricks or Synapse Spark - Data Modeling & Warehousing - Design dimensional models (Kimball star/snowflake) and data vault patterns for analytics consumption - Implement Slowly Changing Dimensions (Type 1/2/3), Change Data Capture, and late-arriving data patterns - Tune distributed SQL workloads in Synapse Dedicated SQL Pool / Fabric Warehouse, including distribution keys, partitioning, and clustered column store indexes - Platform Engineering & DevOps - Implement CI/CD for data pipelines using Azure DevOps (YAML pipelines, ARM/Bicep/Terraform) across Dev / SIT / UAT / Prod environments - Instrument pipelines with robust logging, auditing, and monitoring using Azure Monitor, Log Analytics, and KQL - Define and enforce coding standards, code review practices, branching strategies, and release management - Migration & Modernization - Lead or contribute to legacy-to-cloud migrations - e.g., Informatica PowerCenter to Azure Data Factory, on-premises Teradata / Oracle / SQL Server to Synapse or Fabric - Perform workload assessment, capacity planning, and cost modeling for target-state architectures - production incident response for critical pipelines You'll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in. Required Qualifications: - Demonstrated experience in Data Bricks and PySpark - Production experience with one or more of: Azure Synapse Analytics (Dedicated and Serverless SQL Pools, Spark Pools) OR Azure Databricks (Delta Lake, Unity Catalog) OR Microsoft Fabric (Warehouse, Lakehouse, OneLake) - Experience with Azure Key Vault, Azure AD / Entra ID (including managed identities and service principals), and private networking (VNet integration, private endpoints) - Experience with monitoring and troubleshooting with Azure Monitor, Log Analytics, and KQL - Deep hands-on expertise with Azure Data Factory: pipelines, datasets, linked services, triggers, parameterization, mapping data flows, and all three Integration Runtime types (Azure, Self-hosted, SSIS) - Demonstrated working knowledge of Azure Data Lake Storage Gen2 (hierarchical namespace, RBAC + ACLs, lifecycle management, security) - Proficient in T-SQL; familiarity with Spark SQL, KQL, PowerShell, and Bash shell scripting - Advanced SQL - window functions, CTEs, query optimization, execution plan analysis, performance tuning - Demonstrated Python for data engineering - pandas, PySpark, REST API integration, unit testing (pytest) - Willing/able to travel approximately 10% of the time as business needs dictate (occasional travel to Springfield, IL two to three times per year) Preferred Qualifications: - 5+ years of data warehouse development experience - 5+ years of data modeling experience using ERWIN or similar tools - 2+ years of experience with Azure Data Factory and Snowflake - Medicaid Domain Knowledge *All employees working remotely will be required to adhere to UnitedHealth Group's Telecommuter Policy Pay is based on several factors including but not limited to local labor markets, education, work experience, certifications, etc. In addition to your salary, we offer benefits such as, a comprehensive benefits package, incentive and recognition programs, equity stock purchase and 401k contribution (all benefits are subject to eligibility requirements). No matter where or when you begin a career with us, you'll find a far-reaching choice of benefits and incentives. The salary for this role will range from $91,700 - $163,700 annually based on full-time employment. We comply with all minimum wage laws as applicable. Application Deadline: This will be posted for a minimum of 2 business days or until a sufficient candidate pool has been collected. Job posting may come down early due to volume of applicants. At UnitedHealth Group, our mission is to help people live healthier lives and make the health system work better for everyone. We believe everyone-of every race, gender, sexuality, age, location and income-deserves the opportunity to live their healthiest life. Today, however, there are still far too many barriers to good health which are disproportionately experienced by people of color, historically marginalized groups and those with lower incomes. We are committed to mitigating our impact on the environment and enabling and delivering equitable care that addresses health disparities and improves health outcomes - an enterprise priority reflected in our mission. UnitedHealth Group is an Equal Employment Opportunity employer under applicable law and qualified applicants will receive consideration for employment without regard to race, national origin, religion, age, color, sex, sexual orientation, gender identity, disability, or protected veteran status, or any other characteristic protected by local, state, or federal laws, rules, or regulations. UnitedHealth Group is a drug - free workplace. Candidates are required to pass a drug test before beginning employment.
Senior Data Engineer
HungryrootHungryroot is the online grocery service that makes healthy eating easy and personal.
• Develop pipelines in Spark (Python) in the Databricks Platform • Build cross-functional working relationships with business partners in Food Analytics, Operations, Marketing, and Web/App Development teams to power pipeline development for the business • Ensure system reliability and performance • Deploy and maintain data pipelines in production • Set an example of code quality, data quality, and best practices • Work with Analysts and Data Engineers to enable high quality self-service analytics for all of Hungryroot • Investigate datasets to answer business questions, ensuring data quality and business assumptions are understood before deploying a pipeline



