Spend is the fuel to help your company deliver performance, profitability, and purpose!
Senior Lead Software Engineer, Data Platform
Location
United States
Posted
68 days ago
Salary
$149K - $193.5K / year
Seniority
Senior
Job Description
Senior Lead Software Engineer, Data Platform
Coupa Software
• Design and implement scalable, high-throughput data ingestion systems that integrate internal and external data across domains • Design and build core data platform components, including ingestion, validation, orchestration, and lineage, with a focus on code quality and reliability • Build and evolve a centralized data lake using Apache Iceberg (or similar table formats) • Work across multi-cloud environments (AWS, GCP, Azure) to design and implement cloud-agnostic data ingestion and processing patterns • Contribute hands-on to the Semantic layer, ensuring data is easy to consume for BI and analytics teams • Partner with Senior Data Engineers, Platform Engineers, and Analytics Engineers to align how data is produced, stored, and consumed • Establish practical engineering standards for testing, observability, and operational excellence • Provide technical leadership through mentorship, code reviews, and design discussions, while remaining hands-on
Job Requirements
- 8-10+ years of experience in software or platform engineering, with a focus on building scalable data and analytics platforms
- Strong understanding of data ingestion patterns at scale, including CDC, and how data should be modeled and stored in a data lake for fast, efficient retrieval
- Proven experience building and operating large-scale data pipelines in production
- Experience working with modern data warehouses such as Databricks, BigQuery, or Snowflake
- Strong proficiency in Python and SQL, with a focus on writing production-quality, maintainable, and testable code
- Hands-on experience working with cloud data services in AWS, GCP, or Azure
- Experience working with query engines such as Presto or Trino to enable fast, reliable analytics over data lakes
- Familiarity with Lakehouse architectures and table formats such as Iceberg or Delta Lake
- Familiarity with data governance, lineage, metadata, cataloging, and data quality practices.
Benefits
- Health insurance
- Retirement plans
- Paid time off
- Flexible work arrangements
- Professional development
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
Territorio HibridoGestar la innovación, integrando tecnologías, esto para potenciar el crecimiento personal y profesional.
• Develop and maintain scalable ETL/ELT data pipelines, performing extraction, transformation, and loading of data from multiple sources, using Pentaho Data Integration as the primary platform. • Integrate new data sources into existing systems, ensuring data quality, consistency, and availability for analytics and business teams (for example, for Power BI dashboards). • Optimize and monitor the performance of data processes, performing tuning and adjustments as needed to ensure efficient, reliable, and low-latency pipelines. • Collaborate with the Automation Architect and BI/Analytics teams to understand data requirements for automation and reporting projects, and deliver solutions aligned to those needs. • Implement data governance, security, and compliance practices, ensuring pipelines adhere to data protection policies and that data quality is continuously verified (monitoring for failures, missing data, etc.). • Document developed data processes and flows, keeping clear records of transformations, pipeline configurations, and architecture to facilitate maintenance and future evolution.
Senior Data Engineer
Apartment ListAt Apartment List, we carefully consider a variety of factors to determine compensation for each position, including the role, level, and work. The US Total Target Compensation (TTC) for this position is: Zone 1: $172,700 – $209,000 TTC + Equity Zone 2: $159,700 – $193,000 TTC + Equity Zone 3: $146,800 – $177,000 TTC + Equity This reflects the compensation target for new hire salaries for the position across all US locations. Please note, the compensation details provided do not include benefits and perks that we offer. We also rely on market indicators along with considering your work location, job related skills, experience and relevant education and training, to determine compensation that is fair and competitive for you. Apartment List will consider paying compensation near the higher of the range in exceptional circumstances, where candidates have the experience, credentials or expertise that would warrant such consideration. It is always our goal to hire exceptional talent and we would be happy to share more about compensation during the hiring process.
Who we are Apartment List is the premier rental matchmaker, connecting ready-to-move renters with compatible city properties. Our renters benefit from cutting-edge technology and a personalized approach when we match them with curated properties that fit their wishlist. Meanwhile, our supply partners harness our generative AI and machine learning to efficiently connect with our qualified renters. With performance-based pricing, our partners’ success is our success as we strive to deliver renters a home they love at the value they deserve. About the Role We are seeking a Senior Data Engineer (L3) to help build and operate the reliable, scalable data systems that power analytics, experimentation, and decision-making across Apartment List. In this role, you will be a strong end-to-end executor responsible for delivering production-grade data pipelines and workflows that meet defined service-level agreements (SLAs) for freshness, quality, and cost. You will work closely with Analytics Engineering, Data Science, Product, and Engineering partners to deliver durable data platform improvements that support company-wide initiatives. The ideal candidate is comfortable owning medium-sized data platform projects end-to-end—from design through deployment and operational support—while working within established platform architecture and engineering patterns. This role emphasizes execution excellence, reliability, and operational ownership within existing platform standards. Platform-level architecture and system design are owned at more senior levels, but this role plays a critical part in ensuring that the platform functions reliably at scale. Responsibilities - Design, build, test, and deploy scalable and reliable data pipelines that power analytics and product decision-making. - Own medium-sized data platform initiatives end-to-end, from initial design through production deployment and operational support. - Design, migrate, and maintain data workflows in Apache Airflow, including supporting migration of legacy ETL systems to modern orchestration patterns. - Ensure pipeline reliability by proactively monitoring workflow SLAs for freshness, quality, and performance, and resolving failures efficiently. - Implement and utilize monitoring systems to detect pipeline failures, schema drift, and data quality anomalies. - Participate in on-call rotations and contribute to incident response and root cause analysis for data incidents. - Apply best practices in warehouse performance and cost optimization, including partitioning, indexing, and efficient data modeling to control BigQuery spend. - Build maintainable, modular data models and pipelines using reusable patterns and shared components across the team. - Partner closely with Analytics Engineering, Data Science, and business stakeholders to deliver durable improvements across the ingestion, transformation, modeling, and serving layers of the data platform. - Contribute to operational excellence through documentation, monitoring improvements, and participation in postmortems and reliability initiatives. Key Qualifications & Competencies - 5+ years of experience in data engineering, with a track record of delivering reliable production data pipelines and systems. - Strong experience designing and maintaining orchestration workflows using Apache Airflow. - Experience building scalable data pipelines using modern cloud data platforms such as BigQuery and tools such as DBT. - Strong understanding of data modeling, schema design, and building maintainable, modular data systems. - Experience implementing CI/CD best practices for data pipelines, including automated testing, validation, and deployment workflows to ensure reliable and repeatable production releases. - Experience monitoring and operating production data systems, including pipeline observability, data quality checks, and incident response. - Ability to identify performance and cost risks in large-scale data systems and implement optimizations. - Strong collaboration skills and experience working with cross-functional partners including analytics engineers, data scientists, and product teams. - Proven ability to independently execute medium-sized projects and deliver reliable, production-grade systems. Nice-to-haves - Familiarity with Kubernetes-based data infrastructure, including deploying and operating containerized data services and workflows in a production environment. - Experience migrating legacy ETL systems to modern orchestration frameworks. - Familiarity with observability and monitoring tools for data pipelines (e.g., Datadog or similar). - Experience operating data systems with strict SLAs for freshness, reliability, and cost efficiency. Here's the Pay Range: At Apartment List, we carefully consider a variety of factors to determine compensation for each position, including the role, level, and work. The US Total Target Compensation (TTC) for this position is: - Zone 1: $148,000 - $180,000 TTC (including $133,000 - $15,8000 base salary) + equity - Zone 2: $137,000 - $167,000 TTC (including $123,000 - $147,000 base salary) + equity - Zone 3: $126,000 - $153,000 TTC (including $113,000 - $135,000 base salary) + equity This reflects the compensation target for new hire salaries for the position across all US locations. Please note, the compensation details provided do not include benefits and perks that we offer. We also rely on market indicators along with considering your work location, job related skills, experience and relevant education and training, to determine compensation that is fair and competitive for you. Apartment List will consider paying compensation near the higher of the range in exceptional circumstances, where candidates have the experience, credentials or expertise that would warrant such consideration. It is always our goal to hire exceptional talent and we would be happy to share more about compensation during the hiring process.
• Deliver end-to-end data solutions, including acquisition, engineering, modelling, analysis, and visualisation. • Lead client workshops, gather requirements, and communicate effectively on business and technical levels. • Design and implement scalable ETL/ELT pipelines and data lakes using Microsoft Azure technologies (Fabric, Databricks, Synapse). • Migrate on-premises SQL Server solutions to the cloud and build robust, future-ready architectures. • Create insightful dashboards and reports with Power BI to help clients unlock data-driven decisions. • Own delivery workstreams, manage resources, and mentor team members to drive knowledge sharing. • Build strong client relationships, manage expectations, and ensure timely, high-quality delivery aligned with best practices.
Senior Data Engineer – Power BI & Azure Databricks
Acuity AnalyticsAscent has recently been acquired by Acuity Analytics. This is both a significant milestone for us and a tremendous opportunity for you. Acuity Analytics is a business with a strong global reputation, an impressive client base and ambitious growth plans. We deliver deep insights and domain-led digital transformation to high-growth and heavily regulated organisations. To our customers, we bring a partnership that provides the talent, technology and capability to enhance performance and operational efficiency.
About Us Ascent is now part of Acuity Analytics, marking an exciting new chapter with strong global reach, an impressive client base, and ambitious growth plans. Together, we deliver cutting-edge data and digital transformation solutions to high-growth and regulated organisations—combining talent, technology, and deep domain expertise to drive real business impact. About the Role We’re looking for a (Senior) BI & Data Engineer to support global claims reporting for a leading reinsurance organisation. You’ll work at the intersection of Power BI and Azure Databricks, building scalable data models and high-impact dashboards that serve as a single source of truth for the business. Key Responsibilities Power BI & Analytics - Build and enhance Power BI dashboards, datasets, and semantic models - Apply best-practice data modelling (star schema, conformed dimensions) - Develop DAX measures for key claims KPIs (incurred, paid, reserves, etc.) - Optimize performance (efficient models, DAX tuning, incremental refresh) - Ensure consistent metrics across all reporting Databricks & Data Engineering - Work with Azure Databricks (Delta Lake) as the core data platform - Support development of curated data layers (Silver / Gold) - Build SQL / PySpark transformations for reporting - Ensure data quality, reconciliation, and traceability - Follow governance standards and collaborate with data teams Collaboration & Delivery - Partner with Product Owners and stakeholders - Turn business needs into scalable BI solutions - Contribute to agile delivery (sprints, backlog, documentation) - Share knowledge and best practices across the team Skills Required - Strong Power BI experience (data modelling, DAX, performance tuning) - Solid experience with Azure Databricks (PySpark and/or Spark SQL) - Good understanding of Delta Lake and medallion architecture - Strong SQL and data warehousing knowledge - Experience working with large, complex datasets - Comfortable in international, stakeholder-heavy environments Nice to Have - Insurance / reinsurance or claims reporting experience - Knowledge of data governance, data products, or data mesh - Exposure to financial or actuarial reporting - Experience with DevOps / CI-CD (Power BI or Databricks) Ways of Working - Agile environment - Collaborative, team-focused culture - Strong focus on quality, ownership, and delivery If you’re excited about building modern data platforms and delivering meaningful insights at scale, we’d love to hear from you. Why join us People are at the Heart of our Business. By investing in people, we achieve exceptional results for our clients and create new opportunities for our teams to thrive. Check out this page for more details. If you have any questions contact our Talent Acquisition team on ta.admin@acuityanalytics. For more details about life at Ascent, check out our Life Page here. For more details about Acuity Analytics please see here: Read here.



