Job Closed
This listing is no longer active.
Mark. Make. Match.
Databricks Developer, Java, Apache Spark, IRS MBI Clearance
Location
United States
Posted
150 days ago
Salary
$100K - $150K / year
Seniority
Lead
Job Description
Databricks Developer, Java, Apache Spark, IRS MBI Clearance
3M Consultancy
• Build and maintain data pipelines using Apache Spark on Databricks • Write clean and efficient Java (Java 8 or higher) following best coding practices • Work with large structured and semi-structured data sets • Tune Spark jobs to improve performance, stability, and cost • Partner with other teams to understand requirements and deliver solutions • Make sure data handling follows security and governance standards • Investigate and fix issues in production environments
Job Requirements
- Active federal background clearance (client-issued laptop is a plus)
- Bachelor’s degree in Computer Science or a related field
- 8+ years of hands-on experience in data engineering or big data development
- Data & Systems Experience working with large enterprise or government data systems
- Good understanding of data structures, security, and compliance
- Strong experience with Java 8 or newer
- Comfortable with Streams, Lambdas, and object-oriented design
- Hands-on experience with Spark Core, Spark SQL, and DataFrames
- Understanding of RDDs and when to use them
- Experience with batch processing and streaming
- Strong skills in Spark performance tuning
- Able to debug jobs using Spark UI and logs
- Experience with tools like HDFS, Hive, or HBase
- Familiar with Kafka, S3, or cloud-based data lakes
- Experience with Parquet, Avro, or ORC formats
- Experience building batch and real-time ETL pipelines
- Skilled in data cleansing, transformation, and enrichment
- Experience running Spark on YARN, Kubernetes, or cloud platforms
- Familiar with CI/CD tools like Jenkins or GitHub Actions
- Experience with monitoring tools and production logs
- Unit testing using JUnit or TestNG
- Experience with Mockito or similar tools
- Comfortable validating data and testing Spark jobs
Related Guides
Related Job Pages
More Backend Engineer Jobs
• Design, build, and maintain backend services that support device connectivity, telemetry ingest, health monitoring, and command/control operations. • Collaborate with firmware, mobile, and product teams to define APIs and workflows for device features. • Deliver scalable, resilient systems in a microservices environment running on AWS (EKS, Lambda, DynamoDB, SQS, etc.). • Build and own data pipelines for high-throughput streams using Kafka or similar streaming tech. • Instrument services for observability, reliability, and SLO compliance. • Participate in on-call rotations and live incident response. • Write clean, testable, performant code and help improve CI/CD and automation.
Python Engineer
Kalypso: A Rockwell Automation BusinessWe're not your average consultants. We are innovators, strategists, data scientists and technologists.
• Develop and refine automation solutions through direct Python development, while also leveraging internal platforms and traditional RPA software as complementary tools where appropriate. • Collaborate with cross-functional teams, including AI/ML experts, to design robust, efficient automation that integrates seamlessly with existing systems. • Architect and build advanced automations and reusable components that form the backbone of our as-a-service offerings for clients. • Engage directly with clients to understand their business processes and tailor solutions to meet their unique needs. • Stay current with emerging technologies, applying advancements in AI, LLMs, and GPT to enhance automation capabilities. • Design and implement API integrations, whether using REST or SOAP protocols, to ensure smooth connectivity between systems and services. • Mentor and support junior developers, fostering a culture of growth, collaboration, and continuous improvement.
Staff Software Engineer, Backend – Growth Platform
AffirmWe create honest financial products that improve lives.
• Responsible for setting technical strategy for your team on a year-long time scale, owning backend platforms that power incentives and growth initiatives. • Collaborate with product management, analytics, and partner teams to shape technical direction for incentive systems. • Act as a force-multiplier by defining platform abstractions, APIs, and architectural patterns. • Take ownership of your team’s operations and availability by ensuring robust monitoring, alerting, playbooks, and on-call practices. • Foster a culture of quality and ownership by setting design and code review standards. • Help develop talent on your team by providing actionable feedback, technical guidance, and leadership.
Backend Software Engineer
CabifyCabify offers a platform through which people can connect with private drivers. Its aim is to make getting around cities an easier, more enjoyable, and safer process. Founded in 20
• Collaborate with the principal engineers on developments planning and architecture designs. • Coordinate subprojects, building and maintaining distributed, clean, testable and data oriented applications. • Improve services reliability, scalability and performance. • Own your code all the way to production with a self-service continuous delivery pipeline and production infrastructure. • Dive deep into host-level problem-solving and help to improve detection methods. • Play a vital role in nurturing our engineering culture, emphasizing quality through peer-reviews, monitoring, and on-call best practices. • Mentor junior developers to help them grow. • Collaborate closely with the Data Science team on data oriented feature development.




