• Design, build, and operate scheduled and event-driven data pipelines for simulation outputs, telemetry, logs, dashboards, and scenario metadata • Build and operate data storage systems (structured and semi-structured) optimized for scale, versioning, and replay • Support analytics, reporting, and ML workflows by exposing clean, well-documented datasets and APIs • Enable scenario comparison, lineage, and traceability across time, versions, and parameter sets • Work with engineers to define data contracts and schemas between the simulation environment and downstream systems • Monitor and improve data reliability, performance, and availability

• Evaluar el estado actual (“AS IS”) de los sistemas y organización de datos (modelo, linaje, calidad, gobierno). • Definir la hoja de ruta (“TO BE”) para la transformación de datos. • Liderar la definición del Plan de Transformación, incluyendo aspectos regulatorios y técnicos.

• Design and implement complex Spark data logic, focusing on performance optimization, data volume tuning, and robust execution. • Own the architectural design of graph build pipelines, ensuring they are scalable, automated, and highly resilient. • Plan and oversee the strategic re-architecture of data pipelines to meet evolving business needs and scale. • Optimize infrastructure-as-code and schema designs to reduce cloud costs and improve pipeline latency. • Act as a technical consultant for the team, fostering a collaborative and engineer-led approach to design decisions. • Support the development of the engineering team through code reviews, design docs, and architectural best practices. • Ensure the accuracy of mission-critical data outputs.

This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description We’re seeking a skilled Data Engineer to build the next-generation data management and artificial intelligence platform for maritime domain awareness. What you’ll do: - Implement real-time data pipelines with MQTT and Redpanda for stream processing. - Implement offline data pipelines using Dagster for batch processing. - Parse and process binary message formats from various data sources. - Build data warehouses using Postgres, Apache Iceberg, Parquet, and S3. - Design data models that allow for high-performance queries. - Validate and normalize data sources. - Improve local development and CI/CD using modern tooling and GitHub Actions. Qualifications - Expertise in time-series data processing and analysis (windowing, resampling, interpolation, etc.) - Proficiency in Python and Rust for data engineering workflows. - Experience with binary message parsing. - Experience with row-based & columnar-based data formats. - Experience with OLTP & OLAP databases. - Knowledge of distributed systems, streaming architectures, and batch processing patterns. - Hands-on experience with a batch orchestrator such as Dagster/Airflow. - Hands-on experience with a streaming platform such as Redpanda/Kafka. - Hands-on experience with binary message formats such as Protobuf. Requirements - Experience with IoT devices and sensors (nice to have). - Digital signal processing experience (nice to have). - Geospatial analysis and GIS experience (nice to have). - Familiar with working in monorepos (nice to have). Benefits - Unlimited PTO — Take the time you need to recharge and maintain work-life balance. - Dedicated Sick Time — Your health and well-being come first. - Comprehensive Health & Benefits – Medical, dental, and vision coverage to keep you and your family protected. - 11 Paid Holidays — Enjoy time off throughout the year to celebrate and spend time with loved ones. - Professional Development — Educational opportunities and resources to help you grow your skills and advance your career. - Collaborative Environment — Work directly with leadership in our flat organizational structure, where your ideas and contributions matter. - Mission-Driven Work — Contribute to projects that directly support national security and make a real-world impact. - Growth Opportunities — Join us during an exciting expansion phase where you can help shape our future. - 401(k) with company match. - Onsite / Remote / Flexible work arrangements or hybrid options (position dependent). - Relocation assistance (position dependent). - Referral bonuses. - Performance bonuses. - Life insurance and disability coverage. - Technology home office setup stipend. - Professional certification reimbursement (position dependent).