Senior Lead Data Engineer, Content Engineering

Data EngineerData EngineerFull TimeRemoteSeniorTeam 10,001+Since 1912H1B SponsorCompany SiteLinkedIn

Location

New York

Posted

71 days ago

Salary

$156.8K - $235.2K / year

Seniority

Senior

Job Description

Senior Lead Data Engineer, Content Engineering

Paramount

• Build & Operate Large-Scale Feature Pipelines: Design and maintain batch/streaming pipelines (Spark, Flink, Databricks, Airflow) producing ML features for ranking models. • Ensure Point-in-Time Correctness: Develop feature sets that enable unbiased offline training and credible online inference. • Develop Embedding & Content Pipelines: Build scalable workflows for metadata, imagery, and multimodal representations; partner with Science teams to operationalize new models. • Architect Data Foundations: Design Delta/Parquet data models and medallion layers, optimizing storage layout and partitioning for latency and cost. • Real-Time Engineering: Build Kafka-based systems for real-time features and user-activity aggregations, ensuring robust handling of out-of-order events and exactly-once semantics. • Governance & Leadership: Define data quality rules and schema evolution processes while collaborating across ML pods to translate model needs into infrastructure.

Job Requirements

  • 7+ years of experience in large-scale data or software engineering
  • Deep experience with Spark (PySpark/Scala), Databricks, Airflow, and Kafka.
  • Proficiency in feature pipelines, temporal joins, and mitigating training-serving skew.
  • Experience with AWS/Azure/GCP and high-performance engines like Snowflake or Redshift.
  • Proficient programming skills in Python and SQL with a focus on performance optimization.
  • Experience in personalization domains (search, ranking, or recommender systems).
  • Experience supporting petabyte-scale data lakehouses or feature stores.
  • Familiarity with GenAI/RAG systems, multimodal content, or Delta Live Tables.
  • Knowledge of Causal Inference, experimentation signals, or ML evaluation workflows.
  • Experience with Terraform for governed, repeatable deployments.

Benefits

  • medical
  • dental
  • vision
  • 401(k) plan
  • life insurance coverage
  • disability benefits
  • tuition assistance program
  • PTO

Related Categories

Related Job Pages

More Data Engineer Jobs

Analytica logo

Data Engineer, Data Quality & Automation

Analytica

Data-driven consulting and technology services

Data Engineer71 days ago
Full TimeRemoteTeam 51-200H1B No Sponsor

• Design, develop, and maintain ETL/ELT data pipelines supporting enterprise data platforms. • Implement and enhance data ingestion, transformation, and integration processes using SQL and modern programming languages. • Design and implement automated data quality and validation checks (e.g., schema integrity, completeness, freshness, volume). • Embed data validation and testing logic directly into data pipelines and CI/CD workflows. • Refactor and modernize existing data pipelines to improve reliability, performance, and maintainability. • Collaborate with technical leads, architects, and engineering teams to define and promote standard data engineering and quality practices. • Support deployment, monitoring, and troubleshooting of data pipelines across development, test, and production environments. • Contribute to technical documentation, runbooks, and reusable components to improve team efficiency and consistency. • Participate in design and code reviews with a focus on sustainability, automation, and operational excellence.

United States
Job Closed
Full TimeRemoteTeam 10,001+Since 1978H1B No Sponsor

• Design, build, and maintain scalable data pipelines and APIs on Google Cloud Platform • Develop automated workflows and data platforms that support analytics, reporting, and AI/ML use cases • Implement best practices for data security, governance, CI/CD, and automated deployment • Collaborate with data engineers, architects, data scientists, and business stakeholders • Produce high-quality, reusable code and mentor team members on best practices • Support testing, deployment, monitoring, and production troubleshooting

Tennessee
$76.2K - $4,187.7K / year
Job Closed
Hunt St logo

Senior Data Engineer

Hunt St

We help Aussie companies find top 3% remote talent in the Philippines & Nepal for a single finder's fee.

Data Engineer71 days ago
Full TimeRemoteTeam 1-10H1B No Sponsor

• Design, build, and maintain scalable data pipelines in modern lakehouse architectures. • Develop clean, efficient, and production-ready Python and SQL code. • Implement ETL/ELT processes, transformations, and orchestration workflows. • Model data using medallion architecture (Bronze/Silver/Gold), star schemas, and SCDs. • Integrate multiple data sources (APIs, databases, SaaS platforms, flat files). • Deploy pipelines using CI/CD tools and version control best practices. • Leverage AI tools (e.g., agent-based workflows, automation scripts) to improve delivery speed and quality. • Collaborate directly with clients on requirements, architecture, and delivery updates. • Monitor, troubleshoot, and optimise pipelines for performance and reliability. • Ensure data quality, integrity, and production readiness.

Philippines
$2.5K - $3.5K / month
Job Closed
Hunt St logo

Senior Data Engineer

Hunt St

We help Aussie companies find top 3% remote talent in the Philippines & Nepal for a single finder's fee.

Data Engineer71 days ago
Full TimeRemoteTeam 1-10H1B No Sponsor

Role Description We’re looking for a Senior Data Engineer to join a high-performing delivery team and work on end-to-end data solutions across modern lakehouse environments. This is a hands-on role, focused on building and shipping production pipelines—not people management. You’ll work directly with clients and senior engineers, owning delivery from ingestion through to transformation and consumption. - Design, build, and maintain scalable data pipelines in modern lakehouse architectures. - Develop clean, efficient, and production-ready Python and SQL code. - Implement ETL/ELT processes, transformations, and orchestration workflows. - Model data using medallion architecture (Bronze/Silver/Gold), star schemas, and SCDs. - Integrate multiple data sources (APIs, databases, SaaS platforms, flat files). - Deploy pipelines using CI/CD tools and version control best practices. - Leverage AI tools (e.g., agent-based workflows, automation scripts) to improve delivery speed and quality. - Collaborate directly with clients on requirements, architecture, and delivery updates. - Monitor, troubleshoot, and optimise pipelines for performance and reliability. - Ensure data quality, integrity, and production readiness. Qualifications - 5+ years of professional experience in data engineering. - Must have experience with Databricks or Fabric. - Strong Python skills for production environments. - Advanced SQL (CTEs, window functions, performance tuning, complex joins). - Hands-on experience with modern data platforms (lakehouse, pipelines, distributed processing). - Experience with dimensional modelling and analytics-ready data design. - Solid understanding of CI/CD, Git workflows, and deployment practices. - Strong communication skills with the ability to work directly with clients. - Ability to translate business requirements into technical solutions. - AI Proficiency (Required) Requirements - This role requires strong, practical experience using AI in development workflows: - Comfortable working in modern development environments with AI-assisted tooling. - Proven experience building AI-powered tools, automations, or agents with real business impact. - Ability to use AI across the development lifecycle (design, coding, debugging, testing, documentation). - Strong judgment in validating and refining AI-generated outputs. - Experience with prompt design, context handling, and tool integrations. Nice to Have - Experience with integration platforms or middleware tools. - Exposure to dbt or modern data transformation frameworks. - Experience with cloud-based data ecosystems. - Dashboarding or semantic modelling experience. - Familiarity with AI/agent frameworks or orchestration tools. - Previous consulting or client-facing project delivery. Work Arrangement & Expectations This is a remote role that will be set up as an employer of record. To ensure alignment and transparency, successful candidates will be expected to: - Disclose any existing ongoing roles or client work. - Reflect this engagement on their LinkedIn profile.

Philippines
A$2.5K - A$3.5K / month
Job Closed