Headquartered in Chicago, Illinois, Cushman & Wakefield is a privately-held, international real estate company serving developers, investors, lenders, and occupiers across a broad
Lead Data Engineer
Location
California
Posted
74 days ago
Salary
$86.6K - $101.9K / year
Seniority
Senior
Job Description
Lead Data Engineer
Cushman & Wakefield
• Helps architect and lead the development of enterprise-scale data platforms • Assist the project delivery team with scoping and planning integration projects • Contribute to the design, implementation, and support of data artifacts in the data lakehouse and databases • Collaborates with cross-functional stakeholders to translate business and technical requirements into scalable data solutions • Manage and mentor a small team of data engineers
Job Requirements
- Extensive hands-on experience (6+ years) in cloud-native data engineering
- 3+ years with Databricks and Delta Lake in Azure
- 2+ years in a technical leadership or architecture role
- Deep practical experience of the Databricks ecosystem including Delta Lake, Unity Catalog, Declarative Pipelines, Spark Structured Streaming
- Expert-level SQL skills
- Proficiency in multiple programming languages (e.g., PySpark, Scala, R, Python)
- Deep understanding of cloud-native data architecture
- Proven ability to design and deliver scalable, production-grade data pipelines
- Strong experience with CI/CD and version control (Git)
- Excellent communication and stakeholder engagement skills
Benefits
- health insurance
- vision insurance
- dental insurance
- flexible spending accounts
- health savings accounts
- retirement savings plans
- life and disability insurance programs
- paid and unpaid time away from work
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Architect
NielsenIQNielsenIQ is an industry leader in data analytics and global measurement. The company delivers information to partners, retailers, and manufacturers through pow
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description We are seeking a Data Architect to own and evolve the end-to-end architecture behind the Consumer Canvas platform. You will work across Snowflake, Databricks, identity systems, ingestion pipelines, and activation layers — ensuring our data is scalable, secure, privacy-compliant, and highly performant. You will partner closely with Data Engineering, Data Science, Machine Learning, and Product teams to build next-gen solutions that power the future of consumer insights and advertising activation. This is not a migration or maintenance role — it is a strategic influence position with the opportunity to architect new systems, components, and methodologies from the ground up. - Own the end-to-end architecture of the Consumer Canvas product. - Design systems across ingestion, identity spine creation, enrichment, governance, and activation workflows. - Collaborate with cross-functional partners (Data Scientists, ML Engineers, Data Engineers, Product Owners). - Define and enforce PII governance, privacy controls, and security best practices. - Translate complex business and technical requirements into scalable architecture. - Support technical teams with architectural guidance and model/structure design. - Participate directly in design, prototyping, and testing cycles within an Agile environment. - Act as a senior technical thought partner, providing mentorship (non-manager role). Qualifications - 5+ years of experience in Data Architecture or related technical leadership roles. - Hands-on experience with Snowflake (including Data Clean Rooms) and/or Databricks. - Strong SQL and experience with Python, PySpark, and Spark. - Proven experience working with PII data, privacy constraints, and governance frameworks (critical & non-negotiable). - Data modeling expertise (conceptual, logical, physical). - Experience designing scalable data pipelines and cloud-native architectures. - Strong communication skills and ability to influence cross-functional stakeholders. Requirements - Experience in Ad-Tech, Martech, Media, Consumer Data, or Identity Ecosystems (Nice-to-Have). - Familiarity with Medallion Architecture (Bronze/Silver/Gold) (Nice-to-Have). - Azure services (ADF, Azure Data Lake, Azure DevOps) (Nice-to-Have). - Containers & orchestration (Docker, Kubernetes) (Nice-to-Have). - Experience with audience activation platforms (e.g., Trade Desk, LiveRamp) (Nice-to-Have). Benefits - Flexible working environment. - Volunteer time off. - LinkedIn Learning. - Employee-Assistance-Program (EAP).
Senior Data Engineer
Kong Inc.The cloud connectivity company. Powering connections to build a reliable digital world.
• Design, build, and maintain ETL/ELT pipelines using Fivetran + Snowflake integrations to ingest data from a variety of sources into our Snowflake data warehouse • Develop and manage robust data models in Snowflake, ensuring data is structured for performance, reliability, and ease of use by analysts and business stakeholders • Use Hightouch to operationalize data by syncing warehouse data to downstream CRM, marketing, and sales tools • Monitor pipeline health, troubleshoot data quality issues, and implement alerting to proactively catch failures • Document data models, pipelines, and lineage to support a culture of data literacy and self-service analytics • Integrate Claude and other LLMs directly with our Snowflake data warehouse, enabling AI-powered querying, summarization, and insight generation on top of live revenue data • Build and maintain data sources, semantic layers, and search services within Snowflake Cortex and connected AI platforms • Design and deploy AI agents that can reason over structured and unstructured revenue data to support go-to-market workflows • Architect and manage multi-step agent workflows, coordinating across tools, APIs, and data sources to automate complex analytical and operational tasks • Evaluate and implement orchestration frameworks (e.g., LangChain, LlamaIndex, or custom solutions) best suited to our use cases • Run rigorous evaluations of AI tools, models, and platforms to determine the best solution for each use case (e.g., Snowflake Cortex vs. Claude vs. Gemini vs. custom fine-tuned models) • Develop evaluation frameworks covering quality, latency, cost, and security to inform build vs. buy decisions and guide our overall AI roadmap • Stay current on the rapidly evolving AI landscape and proactively recommend new tools or approaches as the space matures • Implement and manage row-level security (RLS) in Snowflake to ensure AI tools only surface data that users are authorized to see • Maintain and evolve role-based access controls (RBAC) alongside new RLS policies • Contribute to data governance practices, including access controls, PII handling, and schema management • Partner with Data, Security, and Legal teams to establish AI data governance standards and guardrails • Monitor and manage AI credit consumption across Snowflake Cortex, API usage, and other platforms to keep spending within budget • Identify and implement optimizations — such as caching, prompt tuning, model selection, and query efficiency improvements — to reduce cost without sacrificing quality • Build reporting to give stakeholders visibility into AI spend and usage trends • Partner with Revenue Operations, Finance, and Sales to understand data needs and translate them into scalable engineering solutions • Collaborate across technical and non-technical teams to deliver data and AI solutions that directly influence revenue strategy
Data Engineer (US Remote)
First AdvantageFounded in 2003, First Advantage provides comprehensive background-check insights and solutions, enabling employers and housing providers to make confident choices, diminish risks,
At First Advantage (Nasdaq: FA), people are at the heart of everything we do. From our customers and partners to our greatest advantage — our team members. Operating with empathy and compassion, First Advantage fosters a global inclusive workforce devoted to the diverse voices that make up our talent and products. Our team members empower each other to be their authentic selves and treat all with respect, integrity, and fairness. Say hello to a rewarding career and come join a leading provider of mission-critical background screening solutions to some of the most recognized Fortune 100 and Global 500 brands. What We Do: We are on the frontline of recruitment enabling organizations to Hire Smarter. Onboard Faster™ First Advantage is an HR Tech company delivering innovative solutions and insights to enable our clients to manage risk and hire the best talent. Leveraging an advanced technology platform, First Advantage builds fully scalable, configurable screening programs that meet the unique needs of over 30,000 clients. Headquartered in Atlanta, GA and with an internationally distributed workforce spanning 17 countries with about 5,000 employees, First Advantage performs over 100 million screens in over 200 countries and territories annually. We are seeking a Data Engineer with solid Python/PySpark programming skills to join the Data Engineering Team and help us build the Data Analytics Platform in Azure cloud. Who You Are: You are self-motivated and ready to “roll up your sleeves." While you are an independent contributor, you are also collaborative. You can spearhead a project and see it through from start to completion. As a team player, you navigate cross-functional teams and work well with team members in other business units and departments toward a common goal. An Innovator — you see gaps in current processes or workflows as an opportunity to improve and try something new. A lifelong learner and always seeking out opportunities to learn and upskill, you understand the importance of thorough and secure screenings and are interested in the Human Capital sector and the confluence of people, process, and technology. What You'll Do: - Develop reusable, metadata-driven data pipelines - Automate and optimize any data platform related processes - Build integrations with data sources and data consumers - Add data transformation methods to shared ETL libraries - Write unit tests - Develop solutions for the Databricks data platform monitoring - Proactively resolve any performance or quality issues in ETL processes - Cooperate with infrastructure engineering team to set up cloud resources - Contribute to data platform wiki / documentation - Perform code reviews and ensures code quality - Initiate and implements improvements to the data platform architecture What You Need to be Successful: - Programming: Python/PySpark, SQL - Proficient in building robust data pipelines using Databricks Spark - Experienced in dealing with large and complex datasets - Knowledgeable about building data transformations modules organized as libraries (Python packages) - Familiar with Databricks Delta optimization techniques (partitioning, z-ordering, compaction, etc.) - Experienced in developing CI/CD pipelines - Experienced in leveraging event brokers (Kafka /Event Hubs / Kinesis) to integrate with data sources and data consumers - Understanding of basic networking concepts - Familiar with Agile Software Development methodologies (Scrum) Nice to Have Skills: - Understanding of stream processing challenges and familiarity with Spark Structured Streaming - Experience with IaC (Terraform, Bicep or other) - Experience running containerized applications (Azure Container Apps, Kubernetes) - Experience building event sourcing solutions - Familiarity with platforms for change data capture (e.g. Debezium) - Knowledge of Azure cloud native solutions (e.g. Azure Data Factory, Azure Function App, Azure Container Instances) Why First Advantage is Your Next Big Career Move First Advantage is going through a technology transformation! We are looking for experts who are excited to work with advanced technologies and provide best-in-class user experiences, drive the development and deployment of scalable solutions, and smoothly guide our agile teams and clients through meaningful changes as we continue to expand our impact. Additional benefits offered to our eligible people include: - Competitive benefits package, including health care, life insurance and Multisport, - Challenging projects. - Spacious, modern, and fully equipped office space in the heart of Krakow. - Flexibility and possibility to work remotely. - Superior co-working and personal development experience. What Are You Waiting For? Apply Today! You have learned a little about us today – we want to learn about you! If you think this position and our company are a great fit for your areas of interest and expertise, tell us about you by applying now! The base salary range for this position is approximately $75,000-90,000 annually plus there is additional opportunity for Variable Compensation. This range reflects our good faith estimate to pay fairly as to what our ideal candidates are likely to expect, and we tailor our offers within the range based on the selected candidate’s experience, industry knowledge, technical and communication skills, and other factors that may prove relevant during the interview process. United States Equal Opportunity Employment: First Advantage is proud to be a global leader in removing barriers and supporting our community members to ensure the changing demographics of the workforce are reflected in our hiring and employment practices. We value all of our candidates, employees, and clients, and place great emphasis on hiring and supporting qualified individuals in each role. We are an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, genetic information, or any other area protected by applicable law.
Software Engineer III, Data Engineering
GitHub, Inc.GitHub is the world’s leading AI-powered developer platform with 150 million developers and counting. We’re also home to the biggest open-source community on earth (and 99% of the world’s software has open-source code in its DNA). Many of the apps and programs you use every day are built on GitHub. Our teams are dreamers, doers, and pioneers, leading the way in AI, driving humanitarian efforts around the globe, and even sending open source to Mars (and beyond!). At GitHub, our goal is to create the space you need to do your best work. We’re remote-first and offer competitive pay, generous learning and growth opportunities, and excellent benefits to support you, wherever you are—because we know that people flourish when they can work on their own terms. Join us, and let’s change the world, together.
About GitHub GitHub is the world’s leading platform for agentic software development — powered by Copilot to build, scale, and deliver secure software. Over 180 million developers, including more than 90% of the Fortune 100 companies, use GitHub to collaborate, and more than 77,000 organisations have adopted GitHub Copilot. Locations In this role you can work from Remote, United States Overview As a software engineer at GitHub, you will enhance the collaboration experience at GitHub by working closely with a community of engineers and designers with a distributed, diverse and passionate team delivering the services that millions of developers depend on. In this role you will design, prototype, implement, ship and support highly performant and inspiring user experiences with your team. We are looking for creative problem solvers and diverse thinkers, people who care about culture as well as customers and features. We believe that how we do things is as important as what we do. Big vision, a common purpose, passion for quality, curiosity, dedication, and investment in fun and collaboration are what lead to great results. Great products reflect the teams that build them. Responsibilities - Design, develop, test and ship high-quality technical solutions that scale across multiple GitHub services and become intimately familiar with the systems you build and take pride in writing maintainable code. - Provide technical leadership, mentorship, pairing opportunities, and code reviews to encourage the growth of others; support teams in producing extensible and maintainable code, ensuring integration with downstream dependencies and adherence to quality standards. - Own and advocate for the health and quality of the systems that the team builds, including participating in on-call for first responder rotations and live incidents. - Write architecture briefs and proposals and carry out code experiments. - Design and implement APIs to facilitate seamless integration between software components. - Utilize CI/CD tools to set up automated pipelines for continuous integration and delivery. - Collaborate with cross-functional teams and partner with stakeholders and lead discussions for technical solutions, including design and cost considerations. - Create and guide others in 1) developing clear testing plans to assure solution quality, reliability, and performance; 2) defining success metrics; and 3) integrating customer feedback for continuous improvement - all while ensuring system architecture meets security and compliance standards. - Maintain executional and operational excellence within and potentially across teams/organizations. - Apply debugging tools and telemetry to verify assumptions, proactively resolve issues, and optimize code performance and maintainability. Qualifications Required Qualifications: - 4+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python - OR Associate’s Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 3+ years experience - OR Bachelor's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 2+ years experience in Computer Science, or related technical discipline with proven experience coding in languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python - OR Master's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field - OR equivalent experience. Preferred Qualifications: - Demonstrated experience with large-scale system architecture and design, particularly in cloud-based environments, with a strong understanding of distributed systems and microservices. - Experience working closely with product management, design, and other engineering teams to drive cross-functional projects and deliver high-quality products - Experience in one or more scripting languages (e.g., Bash, Python, or a similar language) - Experience with cloud environments and/or Cloud Native Compute Foundation (CNCF) concepts - Experience working with both relational (e.g. mysql) and most importantly non-relational datastores (e.g. Cosmos) - Experience working with Azure resource such as Azure Storage (blob and table particularly), Azure Redis Cache, Azure Data Explorer Clusters. - Experience operating Cosmos DB clusters at scale. Compensation Range The base salary range for this job is USD $107,700.00 - USD $285,900.00 /Yr. These pay ranges are intended to cover roles based across the United States. An individual's base pay depends on various factors including geographical location and review of experience, knowledge, skills, abilities of the applicant. At GitHub certain roles are eligible for benefits and additional rewards, including annual bonus and stock. These rewards are allocated based on individual impact in role. In addition, certain roles also have the opportunity to earn sales incentives based on revenue or utilization, depending on the terms of the plan and the employee's role. GitHub values - Customer-obsessed - Ship to learn - Growth mindset - Own the outcome - Better together - Diverse and inclusive Manager fundamentals - Model - Coach - Care Leadership principles - Create clarity - Generate energy - Deliver success Who We Are GitHub is the world’s leading AI-powered developer platform with 150 million developers and counting. We’re also home to the biggest open-source community on earth (and 99% of the world’s software has open-source code in its DNA). Many of the apps and programs you use every day are built on GitHub. Our teams are dreamers, doers, and pioneers, leading the way in AI, driving humanitarian efforts around the globe, and even sending open source to Mars (and beyond!). At GitHub, our goal is to create the space you need to do your best work. We’re remote-first and offer competitive pay, generous learning and growth opportunities, and excellent benefits to support you, wherever you are—because we know that people flourish when they can work on their own terms. Join us, and let’s change the world, together. EEO Statement GitHub is made up of people from a wide variety of backgrounds and lifestyles. We embrace diversity and invite applications from people of all walks of life. We don't discriminate against employees or applicants based on gender identity or expression, sexual orientation, race, religion, age, national origin, citizenship, disability, pregnancy status, veteran status, or any other differences. Also, if you have a disability, please let us know if there's any way we can make the interview process better for you; we're happy to accommodate!




