Transportation Simplified
Senior Data Engineer
Location
United States
Posted
5 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Echo Global Logistics
Role Description This is an AI-first engineering role. We expect you to leverage AI tools to design, build, and deliver solutions - the value you bring is knowing what to build, how to solve problems, and how to validate results. As our Senior Data Engineer, you'll lead the design and development of scalable data pipelines and data products on our Azure-based platform, with Databricks at the core. You'll build trusted, production-grade data that powers analytics, operational intelligence, and AI initiatives across the business. What You'll Do - Lead the design and development of scalable data pipelines in Databricks - Build and optimize Lakehouse architecture using Medallion (Bronze/Silver/Gold) design - Develop data solutions using Python, Pyspark, Spark SQL, and Delta Lake - Integrate APIs, enterprise systems, SaaS platforms, and external data sources - Establish data quality, testing, monitoring, and performance standards - Mentor engineers and promote engineering best practices - Drive CI/CD, automation, and AI-first development practices - Partner with analytics engineers, data scientists, and application teams - Evaluate new technologies through proof-of-concept projects Qualifications - AI-first development mindset - 5+ years of data engineering experience, including senior or lead responsibilities - Expert Python and SQL skills - Deep experience with Databricks, Apache Spark, and Pyspark - Experience building production data pipelines - Lakehouse architecture and Medallion modeling experience - Data quality, testing, and monitoring expertise - CI/CD and Git experience - Experience with Azure Data Factory, Airflow, or similar orchestration tools - REST API and enterprise system integrations - Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent experience - Strong communication and collaboration skills Requirements - Azure Databricks - Azure Data Factory - Azure Data Lake Storage - Delta Lake - Apache Airflow - Databricks Asset Bundles - Databricks Apps - MLflow, Mosaic AI, Unity Catalog - Experience supporting AI/ML workloads - Databricks Data Engineer Certification Benefits - This role is eligible for a bonus that is based on a combination of personal and business performance. - For more information about our benefit offerings, please visit our careers page at https://www.echo.com/company/careers .
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer, DBT
DriveTimeDriveTime is a used car dealership and automotive financial network company founded in 2002. The company specializes in helping all individuals finance a reliab
• Owning the design and development of robust dbt Core models that transform raw data into trusted, analytics‑ready datasets in Snowflake • Architecting scalable, high‑performance data models that support enterprise reporting, analytics, and AI use cases • Translating complex business and analytical requirements into efficient, well‑structured ELT solutions through close collaboration with BI, analytics, and business stakeholders • Embedding best practices in data quality, testing, documentation, and lineage to ensure transparency, reliability, and trust in our data ecosystem • Leveraging Python to support automation, data validation, orchestration, and performance monitoring across ELT pipelines • Monitoring, tuning, and optimizing Snowflake query performance and cost efficiency • Leading technical design discussions and contributing hands‑on to critical data initiatives • Serving as a technical lead and mentor, guiding other engineers and elevating standards across the full data transformation lifecycle • Providing thought leadership on modern data transformation patterns, tooling, and architecture to help shape enterprise data strategy • Supporting data governance and metadata enrichment initiatives in alignment with broader enterprise data goals
• You'll be the first person at LawnStarter dedicated to data governance - the owner of whether our data can be trusted. • That means the quality and freshness of our source data, pipelines, and reports; the definitions behind our metrics; the standards behind our Segment event tracking; the health of our Lightdash workspace; the data feeding our machine learning models; and the security of the data itself. • This is a hands-on role. You'll work solo at first, with the Analytics team around you but nobody under you - building automation, writing checks, fixing what's broken, and putting processes in place that scale past you. If the scope grows the way we expect, this becomes the foundation of a team you'd build. • Data quality and freshness - automated monitoring across source data, pipelines, and reports; catching upstream schema and source changes before they break anything downstream; running incidents to resolution when they happen. • Data lineage and impact analysis - a living map from production source to warehouse model to dashboard, and the process that uses it: when a production change is proposed, its downstream impact on pipelines, metrics, and reports gets assessed before it ships, not discovered after. The end-state is data contracts with engineering, so breaking changes get caught in their workflow, not ours. • Lightdash - administration, workspace structure, permissions, and the rollout itself. Your job is to give the company self-serve autonomy while keeping the workspace tidy enough that people can find and trust what's there. Enablement is part of the deal - people follow standards they've been taught - and so is keeping queries fast and warehouse costs sane. • The semantic layer - we just shipped it for our most critical metrics: one governed definition per metric, in code. You'll extend definition and mapping to the rest and guard the layer against uncontrolled growth as it scales. • Event tracking governance - our governed Segment event catalog: reviewing new events against its standards, keeping it matched to what production actually sends, and evolving the guardrails (naming, property dictionary, drift detection) as tracking grows. • AI data readiness - AI agents query our warehouse every day through Brain, our internal AI toolkit. You'll govern what data AI tools can access and keep the warehouse AI-legible: documented, consistent, and safe for an agent to query and get the right answer. • Data security and privacy - access controls, PII handling and retention under US state privacy laws, and periodic reviews of who - and which AI tools - can see what. • The governance system itself - the documentation, ownership models, and review loops that keep all of the above running without heroics.
AWS Snowflake Data Architect, 12+ Years of Experience
3Pillar GlobalBuilding digital businesses, together.
• Design and implement scalable data platforms using Snowflake, Databricks, Delta Lake, and cloud technologies. • Build batch and real-time data pipelines using PySpark, Kafka, and Spark Structured Streaming. • Develop AI-ready data architectures supporting analytics, ML, LLMs, and RAG applications. • Design semantic models, data governance, metadata, and data lineage solutions. • Implement vector databases, embedding pipelines, and retrieval solutions for AI applications. • Build and manage ML/LLMOps pipelines, model deployment, monitoring, and CI/CD. • Ensure data security, RBAC, compliance, and governance across the platform. • Mentor engineering teams and define architecture best practices.
• Assist in developing and maintaining basic data ingestion and transformation pipelines (ETL/ELT) using PySpark and SQL. • Help monitor data pipelines and implement basic checks to ensure data reliability for internal consumers (such as Data Scientists). • Learn and assist in automating pipeline testing and deployment processes. • Work alongside data scientists and software engineers to understand and support integrated data flows. • Assist in documenting data schemas, pipeline architectures, and metadata cataloging.




