Job Closed

This listing is no longer active.

Lightcast logo
Lightcast

We unlock new possibilities in the labor market.

Data Engineer

Data EngineerData EngineerFull TimeRemoteMid LevelTeam 501-1,000H1B No SponsorCompany SiteLinkedIn

Location

Idaho

Posted

59 days ago

Salary

0

Seniority

Mid Level

Job Description

Data Engineer

Lightcast

• Collect, manage, and transform large-scale raw data from multiple sources into structured, reliable, and usable formats that meet business and analytical requirements. • Design, develop, and maintain data architectures, pipelines, and applications using appropriate tools and frameworks to support scalable data processing and product development. • Build and maintain robust extract, transform, and load (ETL) processes that ensure data accuracy, efficiency, and availability across systems. • Perform root cause analysis to identify data issues, system inefficiencies, or process gaps, and implement improvements that enhance data quality and operational performance. • Work in an agile Scrum environment by contributing to sprint planning, development tasks, testing, and sprint reviews to deliver reliable and scalable data solutions

Job Requirements

  • Bachelor’s in Computer Science, Mathematics, Statistics, or related field (or equivalent experience)
  • 2+ years in data engineering, data analysis, or similar roles
  • Experience with ETL, data warehousing, and scalable data systems.
  • Solid understanding of statistical methods and data analysis
  • Strong communication skills with ability to work independently and in teams
  • Proficiency in Python and SQL; familiarity with Java, C/C++, or similar
  • Strong software engineering, programming, and QA skills
  • Experience with Linux/Unix (POSIX systems) and Git
  • Experience with Docker, Terraform, and AWS (Batch, EMR/Spark, ecosystem tools)
  • Familiarity with GitLab and Scala

Related Categories

Related Job Pages

More Data Engineer Jobs

PERCEPTIO S.A.S. logo

Consultor Ingeniero de Datos, Azure

PERCEPTIO S.A.S.

Consultoría SAP🔷SAP Cloud ERP🔷Expert Hub🔷Centro de atención en la nube🔷SAP Business IA

Data Engineer59 days ago
Full TimeRemoteTeam 201-500Since 2000H1B No Sponsor

• Consultor Ingeniero de Datos Azure • Dataricks • SQL • Transformación de Datos

Colombia
Job Closed
Leega logo

Data Engineer, GCP, Databricks, Mid-level

Leega

Inteligência, Inovação e Tecnologia.

Data Engineer59 days ago
Full TimeRemoteTeam 201-500Since 2010H1B No Sponsor

• Develop and maintain scalable, efficient data pipelines (ETL/ELT); • Work on data ingestion, transformation, and provisioning in GCP environments; • Build solutions using Databricks (Spark/PySpark); • Design and evolve data architectures (Data Lake, Data Warehouse, Lakehouse); • Integrate multiple data sources (APIs, databases, streaming); • Ensure data quality, governance, and reliability; • Monitor performance and optimize cloud costs; • Collaborate with BI, Analytics, and Data Science teams.

Brazil
Job Closed
Leega logo

Data Engineer – GCP, Databricks

Leega

Inteligência, Inovação e Tecnologia.

Data Engineer59 days ago
Full TimeRemoteTeam 201-500Since 2010H1B No Sponsor

• Company focused on delivering efficient and innovative service to its clients • Innovative professionals who are driven by challenges • Technically lead the evolution of the data platform • Guide the team and ensure the delivery of robust solutions in cloud environments

Brazil
Job Closed
Attentive logo

Senior Software Engineer, Data Centralization & Storage

Attentive

The most comprehensive text message marketing solution.

Data Engineer60 days ago
Full TimeRemoteTeam 1,001-5,000Since 2016H1B Sponsor

Role Description Our engineering department consists of 200+ people across multiple teams, such as application development, infrastructure, data platform, machine learning, and security. We believe our company will win in the long run through product innovation. To get there, we obsess over iteratively delivering customer value through rapid prototyping and data-driven decision-making. We’re looking for a self-motivated, highly driven Software Engineer with a strong understanding of low-level distributed systems concepts - not just tool users, but systems thinkers. We want engineers who are comfortable reasoning about how infrastructure behaves under load, how services communicate over the network, and how data flows through complex architectures. Our team is focused on building the platform to support data driven products and strategic outcomes for our customers. We do this by providing an extensible and flexible compute platform to access highly available, mission critical data sets. What You’ll Accomplish - Architect high-throughput solutions that power our most critical operations, ensuring scalability and efficiency. - Expand and enhance our self-service platform, collaborating with cross-functional teams to fuel our AI, ML, and analytics goals. - Tackle complex distributed data challenges, streamline system integrations, and uphold high standards of quality and governance. - Champion cutting-edge technologies, keeping our platform at the forefront of industry advancements and enabling strategic outcomes. - Unify data from diverse systems, paving the way for experimentation and innovation while empowering teams with intuitive tools and frameworks. Your Expertise - Track record of debugging issues across layers—from application logic to infrastructure bottlenecks—and understand the tradeoffs in system design, not just the settings in a managed UI. - You don’t just know how to run jobs in tools, you understand how they operate under the hood and how connected storage impacts performance like: - How simple storage API semantics (e.g. consistency, eventual visibility, multi-part uploads) affect job execution. - The impact of network I/O and data locality on performance and cost. - The way resource scheduling and JVM tuning influence distributed job behavior. - Proven experience as a Software Engineer with a focus on high throughput scalable systems. - In-depth knowledge of high throughput processing technologies such as Spark, Flink, and/or Kafka. - Proficiency in Java and strong understanding of object-oriented design, data structures, algorithms, and optimization. - You have development experience working with data warehouse tools like Snowflake, Clickhouse, Trino, etc. - You’ve experience with open source data storage formats such as Apache Iceberg, Parquet, Arrow, or Hudi. - You are knowledgeable about data modeling, data access, and data replication techniques, such as CDC. - You have a proven track record of architecting applications at scale and maintaining infrastructure as code via Terraform. - You are excited by new technologies but are conscious of choosing them for the right reasons. What we use - Our backend is Java / Spring Boot microservices, built with Gradle, coupled with things like DynamoDB, Pulsar, Flink, Spark, Postgres, Planetscale, and Redis, hosted via AWS. - Our frontend is built with React and TypeScript, and uses best practices like GraphQL, Storybook, Radix UI, Vite, esbuild, and Playwright. Salary and Benefits - The US base salary range for this full-time position is $178,400 - $205,500 annually + equity + benefits. - Our salary ranges are determined by role, level and location. Company Values - Default to Action - Move swiftly and with purpose. - Be One Unstoppable Team - Rally as each other’s champions. - Champion the Customer - Our success is defined by our customers' success. - Act Like an Owner - Take responsibility for Attentive’s success.

United States
$178.4K - $205.5K / year