Job Closed

This listing is no longer active.

3M Consultancy logo
3M Consultancy

Mark. Make. Match.

Databricks Developer, Java, Apache Spark, IRS MBI Clearance

Backend EngineerSoftware EngineerOtherRemoteLeadTeam 1-10Since 2018H1B No SponsorCompany SiteLinkedIn

Location

United States

Posted

150 days ago

Salary

$100K - $150K / year

Seniority

Lead

Job Description

Databricks Developer, Java, Apache Spark, IRS MBI Clearance

3M Consultancy

• Build and maintain data pipelines using Apache Spark on Databricks • Write clean and efficient Java (Java 8 or higher) following best coding practices • Work with large structured and semi-structured data sets • Tune Spark jobs to improve performance, stability, and cost • Partner with other teams to understand requirements and deliver solutions • Make sure data handling follows security and governance standards • Investigate and fix issues in production environments

Job Requirements

  • Active federal background clearance (client-issued laptop is a plus)
  • Bachelor’s degree in Computer Science or a related field
  • 8+ years of hands-on experience in data engineering or big data development
  • Data & Systems Experience working with large enterprise or government data systems
  • Good understanding of data structures, security, and compliance
  • Strong experience with Java 8 or newer
  • Comfortable with Streams, Lambdas, and object-oriented design
  • Hands-on experience with Spark Core, Spark SQL, and DataFrames
  • Understanding of RDDs and when to use them
  • Experience with batch processing and streaming
  • Strong skills in Spark performance tuning
  • Able to debug jobs using Spark UI and logs
  • Experience with tools like HDFS, Hive, or HBase
  • Familiar with Kafka, S3, or cloud-based data lakes
  • Experience with Parquet, Avro, or ORC formats
  • Experience building batch and real-time ETL pipelines
  • Skilled in data cleansing, transformation, and enrichment
  • Experience running Spark on YARN, Kubernetes, or cloud platforms
  • Familiar with CI/CD tools like Jenkins or GitHub Actions
  • Experience with monitoring tools and production logs
  • Unit testing using JUnit or TestNG
  • Experience with Mockito or similar tools
  • Comfortable validating data and testing Spark jobs

Related Job Pages

More Backend Engineer Jobs

OtherRemoteTeam 201-500Since 2008H1B Sponsor

• Design, build, and maintain backend services that support device connectivity, telemetry ingest, health monitoring, and command/control operations. • Collaborate with firmware, mobile, and product teams to define APIs and workflows for device features. • Deliver scalable, resilient systems in a microservices environment running on AWS (EKS, Lambda, DynamoDB, SQS, etc.). • Build and own data pipelines for high-throughput streams using Kafka or similar streaming tech. • Instrument services for observability, reliability, and SLO compliance. • Participate in on-call rotations and live incident response. • Write clean, testable, performant code and help improve CI/CD and automation.

United States
$148K - $216.5K / year
Job Closed
Kalypso: A Rockwell Automation Business logo

Python Engineer

Kalypso: A Rockwell Automation Business

We're not your average consultants. We are innovators, strategists, data scientists and technologists.

Backend Engineer150 days ago
Full TimeRemoteTeam 1,001-5,000Since 2004H1B No Sponsor

• Develop and refine automation solutions through direct Python development, while also leveraging internal platforms and traditional RPA software as complementary tools where appropriate. • Collaborate with cross-functional teams, including AI/ML experts, to design robust, efficient automation that integrates seamlessly with existing systems. • Architect and build advanced automations and reusable components that form the backbone of our as-a-service offerings for clients. • Engage directly with clients to understand their business processes and tailor solutions to meet their unique needs. • Stay current with emerging technologies, applying advancements in AI, LLMs, and GPT to enhance automation capabilities. • Design and implement API integrations, whether using REST or SOAP protocols, to ensure smooth connectivity between systems and services. • Mentor and support junior developers, fostering a culture of growth, collaboration, and continuous improvement.

Mexico
Job Closed
Affirm logo

Staff Software Engineer, Backend – Growth Platform

Affirm

We create honest financial products that improve lives.

Backend Engineer150 days ago
OtherRemoteTeam 1,001-5,000Since 2012H1B Sponsor

• Responsible for setting technical strategy for your team on a year-long time scale, owning backend platforms that power incentives and growth initiatives. • Collaborate with product management, analytics, and partner teams to shape technical direction for incentive systems. • Act as a force-multiplier by defining platform abstractions, APIs, and architectural patterns. • Take ownership of your team’s operations and availability by ensuring robust monitoring, alerting, playbooks, and on-call practices. • Foster a culture of quality and ownership by setting design and code review standards. • Help develop talent on your team by providing actionable feedback, technical guidance, and leadership.

United States
$200K - $250K / year
Job Closed
Cabify logo

Backend Software Engineer

Cabify

Cabify offers a platform through which people can connect with private drivers. Its aim is to make getting around cities an easier, more enjoyable, and safer process. Founded in 20

Backend Engineer150 days ago

• Collaborate with the principal engineers on developments planning and architecture designs. • Coordinate subprojects, building and maintaining distributed, clean, testable and data oriented applications. • Improve services reliability, scalability and performance. • Own your code all the way to production with a self-service continuous delivery pipeline and production infrastructure. • Dive deep into host-level problem-solving and help to improve detection methods. • Play a vital role in nurturing our engineering culture, emphasizing quality through peer-reviews, monitoring, and on-call best practices. • Mentor junior developers to help them grow. • Collaborate closely with the Data Science team on data oriented feature development.

Spain
€72K / year