Apex Systems logo
Apex Systems

Apex Systems, an IT staffing and workforce solutions firm, provides recruiting and staffing services to large and small companies alike. Founded in 1995 by thre

Databricks Engineer

Location

United States

Posted

14 days ago

Salary

$70 - $100 / hour

Seniority

Senior

Job Description

Databricks Engineer

Apex Systems

Title: Databricks Engineer Location: United States Job Description: Job#: 3033658 Pay Range: $70 - $100 per hour Location: Raleigh, NC preferred – Remote candidate might be considered. 6-month assignment, with high probability of extension and longer term. This position is currently unfunded with award expected soon. Description: Apex is seeking a Data Engineer to support our client with data ingestion, data deduplication and data tagging for migration of a large-scale data environment into Databricks. The ideal candidate will also bring hands-on expertise in end-to-end data pipeline management, including data ingestion from diverse sources, de-duplication of large-scale datasets, and data tagging to support downstream analytics, governance, and machine learning workflows. Roles and Responsibilities (including but not limited to) · Design, develop, and maintain scalable data ingestion pipelines to onboard structured, semi-structured, and unstructured data from batch and streaming sources (e.g., APIs, databases, flat files, message queues) into the Azure/Databricks environment. · Implement de-duplication strategies across large-scale datasets using deterministic and probabilistic matching techniques to ensure data integrity and reduce redundancy within the Data Lake. · Develop and enforce data tagging frameworks to classify, label, and annotate datasets with appropriate metadata (e.g., sensitivity, source, domain, lineage) to support data governance, discoverability, and compliance requirements. · Assist with Operationalizing deployments and support of Cloud services for ETL Operations. This will include standardizing and automating processes and workflows, creating documentation/knowledge articles, and overall assisting Operations staff who have limited experience in Cloud. · Written and oral presentations to high-level CIO management on status of current efforts. · Possesses skills and experience related to business management, systems engineering, operations research, and management engineering. Typically has specialization in a particular technology or business application. Keeps abreast of technological developments and industry trends. · Assist with deployment, configuration, and management of Azure Cloud environment. · Assist with migration efforts of existing ETL jobs into Azure/Databricks cloud environment. · Ability to share optimization and efficiencies with the larger team and management. · Ability to automate solutions to repetitive problems/tasks. Basic Qualifications · Must be eligible for a Position of Public Trust, including U.S. citizenship or permanent residency, five years of U.S. residency, and no more than six months of international travel in the past five years (excluding travel for U.S.-based work). · Bachelor’s degree and 13 years of experience. A degree from an accredited College/University in the applicable field of services is preferred. Four additional years of relevant experience in lieu of a college degree is required. If Degree is not in the applicable field, then four additional years of related experience is required. · 3+ years demonstrated experience designing and implementing data ingestion pipelines using tools such as Azure Data Factory, Apache Kafka, Apache NiFi, Spark Structured Streaming, or equivalent technologies. · 3+ years of experience applying de-duplication techniques at scale, including record linkage, fuzzy matching, and entity resolution across structured and unstructured datasets. · 3+ Hands-on experience with data tagging and metadata management, including the use of tagging schemas, data catalogs (e.g., Azure Purview, Apache Atlas), and automated classification tools to support data governance and lineage tracking. · 3 + Demonstrated experience working with unstructured data. · 2 + years of experience in using Databricks or other Spark-based platforms. · Fluency in at least one scripting language (Python, Perl, Ruby, or equivalent). Desired Skills: · Integration of Git in continuous deployment and experience with DevOps monitoring tools. · Experience with one or more of the following products and technologies: SAS, Python, C++, Hadoop, SQL Database/Coding, Teradata, Oracle, Amazon S3, Apache Spark, Machine Learning, Natural Language Processing, and visualization tools such as Tableau, Strategy and QLIK. · Strong skills and experience in Cloud Operations support in Azure. Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRated's Best of Staffing® in Talent Satisfaction in the United States and Great Place to Work® in the United Kingdom and Mexico. Everforth Apex uses a virtual recruiter as part of the application process. Click here for more details. Everforth Apex Benefits Overview: Everforth Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Everforth Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Everforth Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA. Everforth Apex has a dedicated customer service team for our Consultants that can address questions around benefits and other resources, as well as a certified Career Coach. You can access a full list of our benefits, programs, support teams and resources within our ‘Welcome Packet’ as well, which an Everforth Apex team member can provide.

Related Categories

Related Job Pages

More Engineer Jobs

L3Harris Technologies logo

Lead, Systems Software Engineer

L3Harris Technologies

Headquartered in Melbourne, Florida, L3Harris Technologies is self-described as an agile defense technology and aerospace innovator delivering comprehensive solutions that meet the

Engineer14 days ago

• Conducts technical planning and analysis activities throughout all phases of the software, network, and/or cloud system lifecycle, from concept through to delivery. • Author and conduct Program Gate reviews such as SRR, SDR, PDR, CDR, and TRR. • Overseas system integration activities with SI&T leader to ensure cohesive operation across software modules and cloud infrastructure. • Develops, maintains, and documents Interface Control Documents (ICDs) to clearly define and manage software and cloud subsystem interfaces. • In concert with SI&T Leader manage verification and validation processes to confirm that cloud-based and software-centric systems meet all functional and performance requirements. • Performs cost, risk, supportability, and effectiveness analyses with a focus on scalable and maintainable cloud architectures. • Translates customer and stakeholder requirements into comprehensive system solutions, considering technical, schedule, and cost constraints. • Executes functional decomposition, requirements allocation, and detailed trade studies to define hardware and software specifications for software and cloud systems. • Develops and maintains interface definitions between software services, cloud resources, and any hardware elements. • Collaborates with cross-disciplinary teams during design, implementation, and testing to ensure requirements traceability and system integrity. • Supports installation, operation, and maintenance planning for cloud-hosted software products, ensuring long-term supportability and security. • Identifies opportunities for system improvement and advocates for best practices in software architecture and cloud platform utilization. • Ability to obtain and maintain a FAA Public Trust security clearance.

Florida
$127.5K - $236.5K / year
Job Closed

Senior Coastal And Ports Engineer

BMT Group

BMT Group provides design, engineering, risk management, and scientific services for ocean and land-based operations around the world. The company was establish

Engineer14 days ago

Title: Senior Coastal and Ports Engineer Location: Perth - Australia Job Description: Employment type: Full Time Workplace type: Hybrid BMT is a leading global consultancy in design, engineering, science, and risk management in the maritime sector, known for excellence and innovation. With 1,300+ experts across the UK, Europe, Asia, Australia, and the Americas, we help clients make informed decisions by delivering practical solutions, with the environment and sustainability at their core. As an employee-owned company, we reinvest profits into technology, strategic growth, and profit sharing, driving a safer, more efficient future. Join our Coastal and Maritime team at BMT as a Senior Coastal and Ports Engineer. We are seeking experienced engineers at a Senior level to support our growing program of work. Our Australian team of 50+ coastal engineers and scientists has a strong reputation of delivering high quality work in design of coastal infrastructure, boating facilities and coastal protection. We also work in coastal modelling, adaptation planning, nature-based coastal protection, dredging, port planning and offshore wind. We are looking for coastal and maritime specialists with expertise in either Coastal Structures Engineering, Coastal Modelling, or Port Engineering. Key Responsibilities Depending on your area of expertise, in this role you will: - Contribute to a diverse range of coastal and offshore projects Including cross-shore erosion modelling, supporting coastal hazard planning, and contributing to the port development projects, for private and public sector clients. - Contribute to port development and feasibility studies including navigation risk assessment, traffic risk study, operability assessment, throughput and channel utilisation studies, mooring and berthing assessments, dredging design. - Act as a Project Design Manager for small to large scale projects and lead the engagement with the end client. - Lead the design of coastal infrastructure across Australia including jetties, marinas, wharves, boat ramps, breakwaters, seawalls, revetments and other coastal protection infrastructure. - Be part of a cross-discipline team, working with coastal engineers, mechanical/electrical engineers, oceanographers, environmental scientists, naval architects, GIS and approvals specialists. - Be a technical leader in the maritime structures area, conducting internal reviews and contributing to the development of templates and workflows. - Providing leadership and mentoring to less experienced team members. - Write proposals and support with business development and customer relationship management. About you To be successful in this role, you should have: - A bachelor's degree in engineering or other relevant area of study - Minimum 7 years' experience post-graduation - Experience in either Coastal Structures and Port Engineering, Coastal or Metocean Modelling. - In depth understanding of the coastal processes, waves , currents, sediment transport, siltation, metocean data acquisition, instrument set up, post- processing. - Working knowledge of metocean and coastal modelling software, including SWAN, Tuflow FV (or similar, e.g. DELFT-3D, MIKE21), XBeach and their application for solving coastal issues. - Working knowledge of Australian and/or international standards and guidelines relevant to the design of coastal structures such as seawalls, revetments, breakwaters, groynes etc. - Experience working within the engineering/consulting sector including experience with proposals and directly managing customers or external stakeholders - Experience with numerical data analysis software (e.g. MATLAB, Python) and mooring analysis software (e.g. Optimoor, OrcaFlex), as well as structural modelling tools (e.g. STAAD.Pro, SAP, SPACE GASS, Mathcad), will be considered advantageous) - Relevant registration with recognised engineering/environmental body including chartered ship with Engineers Australia, and a RPEV/RPEQ or the ability to obtain one. - Unrestricted and permanent work rights in Australia Why Work with Us? At BMT, we value flexibility, career growth, and an inclusive culture where diverse perspectives thrive. - We offer remote, hybrid, and part-time work options, with offices across Australia. We're proudly 'flexified' with Flexa! - Our industry competitive benefits include a competitive salary offering, performance bonuses, and companywide bonuses as part of being an employee benefits trust - Defence Reserve Leave enabling defence reservist employees to undertake ADF obligations without financial disadvantage. - 24/7 Employee Assistance Program (EAP). - We invest in learning and development to help you grow. Diversity, equity, and inclusion are at our core. We foster a workplace where everyone feels valued and supported, promoting innovation, collaboration, and respect. Our values-Trust, Innovation, Passion, Respect, and Collaboration-define who we are and how we work. Notice to Recruitment Agency & Related Services - Please note BMT receive job applications exclusively via our ATS system only. BMT do not warrant or agree to any unsolicited CVs submitted via any contact method including physical hard copies left with any employee at BMT or within our operational teams. We will not progress CVs shared on a speculative basis by email, and you accept our right to pursue such candidates with no obligation to third-party terms and conditions or liability to a fee.

Australia

Title: Senior Ecosystem Engineer Location: United States, San Mateo, CA Work Type: Remote Job Description: Why Sony Interactive Entertainment? Sony Interactive Entertainment isn't just the Best Place to Play - it's also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As a subsidiary of Sony Group Corporation, we're part of a proud legacy of innovation and excellence. SIE is a dynamic technology company, delivering cutting-edge hardware and network services to more than 100 million people and an entertainment leader, home to some of the most beloved and recognizable intellectual properties (IP) in the world. Our role at SIE is to create and nurture the experiences under the PlayStation brand, a name synonymous with entertainment excellence and creativity. About the team: PlayStation Product Development is looking for a versatile and motivated Ecosystem Engineer to drive the development of PlayStation's growing hardware product ecosystem. Our PD team is working on future hardware products and gaming experiences around and beyond the console. The successful applicant is: - A self-motivator with initiative and a track record of getting things done and working through ambiguity. - Quick to understand and own the integration of new features and technologies into ecosystem products. - Used to working with remote and international engineering teams. - A strong troubleshooter and hands-on fixer. - Not necessarily always right, but always committed to make things right. - Curious and eager to learn new skills. - An engineer in mind, but a gamer at heart! Responsibilities: - Write and maintain PlayStation ecosystem product specifications and validation plans. - Lead product validation activities at external test labs. - Travel occasionally to work with test labs and internal/external engineering teams. Qualifications: - Proven track record with validation and qualification of consumer hardware products with software components. - Strong technical writing, analytical and problem-solving skills - Deep understanding of hardware/software system integration testing - Experience with CI/CD tools such as Jenkins, Jira - Leadership experience with small to medium projects - Able to work both independently and as part of a team - Bachelor's degree in Computer Science, Computer Engineering, Mechanical Engineering, Electrical Engineering, or equivalent. Working Conditions: - Agile team; feels like a start-up within a large organization - Hiring Location: San Mateo, remote considered - Travel (domestic and international) required. Estimated 10%-15% - Individual Contributor: Does not have direct supervisory responsibilities nor budgetary or hiring accountabilities Equal Opportunity Statement: Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to gender (including gender identity, gender expression and gender reassignment), race (including color, nationality, ethnic or national origin), religion or belief, marital or civil partnership status, disability, age, sexual orientation, pregnancy, birthing parent or parental status, trade union membership or membership in any other legally protected category. We strive to build an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond. PlayStation is a Fair Chance employer and qualified applicants with arrest and conviction records will be considered for employment. #LI-AT1 Please refer to our Candidate Privacy Notice for more information about how we process your personal information, and your data protection rights. At SIE, we consider several factors when setting each role's base pay range, including the competitive benchmarking data for the market and geographic location. Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location. In addition, this role is eligible for SIE's top-tier benefits package that includes medical, dental, vision, matching 401(k), paid time off, wellness program and coveted employee discounts for Sony products. This role also may be eligible for a bonus package. Click here to learn more. The estimated base pay range for this role is listed below. $193,300 - $289,900 USD Please note, Sony Interactive Entertainment conducts background checks at the offer stage for all new employees (which may include criminal background checks for some roles) and will need to process personal information to support these checks.

California
$193.3K - $289.9K / year

Lead QA Engineer - Bangor, All UK Bangor, United Kingdom · Permanent · Remote Company Description: Trust Payments is an MFSA-regulated in Malta; and an FCA-regulated company in the UK. We provide a range of payments and commerce solutions, with market-leading technology and data insights. We offer innovative payment methods and cutting-edge technologies, with a truly global presence. Our global offices cater to the most demanding business sectors, including retail, travel, hospitality, forex, and financial services. Driving value for our clients and demonstrating genuine care for their success, is a core value of ours. We also believe in striving to build a better, more sustainable tomorrow and believe in conducting our business ethically, driving social and environmental change. We have a passionate, collaborative, and diverse culture that recognizes that every employee contributes to our business success. Trust Payments has an exciting opportunity for a Lead QA Engineer to join their team. Location: Remote - Bangor / All UK Salary: Competitive + Benefits How will you make an impact in this role? We have an exciting opportunity for a Lead QA Engineer to join our team. You will play a key role in shaping QA automation architecture, guiding a team of engineers, and working collaboratively across the business to deliver high-quality software. The Role: - Developing and maintaining automated tests for Services & UI using tools such as Selenium - Writing automated tests using a BDD framework with tools such as Cucumber - Applying design principles to write clean, maintainable test code - Developing test solutions in Java and/or Python - Working effectively within a CI/CD pipeline environment - Contributing to the continuous improvement of processes, tools, methodologies, and overall product quality - Leading a team of mid and senior-level QA engineers - Designing and implementing QA automation architecture following best practices to ensure efficient code quality - Working closely with developers, product teams, and BAs to define and develop effective test solutions - Defining and implementing test automation processes and best practices - Ensuring defects are raised in accordance with the defect management process using tools such as Jira - Raising test automation compliance issues to the test manager - Developing and maintaining automated functional and regression tests in line with defined scripting standards - Applying a thorough understanding of the software product development lifecycle and test processes What We Expect of You, Day To Day: - Mentoring team members and guiding them towards best practices - Providing effective QA solutions and supporting their implementation - Maintaining and improving the automated test framework - Creating new automated tests and investigating test failures - Triaging discovered defects with the product owner and team - Writing automated tests following the BDD framework and contributing to unit-level test guidance for developers - Investigating defects reported by stakeholders and end users, supporting root cause analysis and resolution - Performing manual and exploratory testing on new features - Collaborating with the team to sign off on the quality of new releases following the Agile release pattern - Attending retrospectives and workshops to support continual improvement - Defining and adding new test cases to automated test packs, capturing results and defects - Understanding business requirements to define test plans with continuous review of scope and testability - Providing test estimations and testing approaches to the product owner and team Qualifications - Highly motivated and committed, with a professional and positive approach - Able to work independently and collaboratively, with a commitment to supporting and mentoring colleagues - Experience with API testing frameworks such as REST Assured - Proficient in Java and/or Python - Experience using Cucumber to write test cases in feature files using Gherkin language - Experience with CI tools (Jenkins, GitLab Pipelines) from a testing configuration perspective - Effective problem-solving skills with attention to thoroughness - Able to manage multiple priorities and remain organised in a dynamic environment - Enthusiastic about software test development, with a genuine curiosity about technology and an interest in emerging testing tools and methodologies We offer comprehensive benefits to support our team members and their families' well-being. Our robust benefits package includes (subject to local office benefits policy): - Be part of a dynamic, market-leading Fintech experiencing rapid growth - Flexible work arrangements tailored to role requirements and business needs - Comprehensive wellness initiatives, including mental health resources supported by internally qualified mental health first aiders - Extensive leave provisions, encompassing annual, volunteering, and birthday allowances - Progressive family-oriented policies and benefits, including Maternity, Paternity and Adoption leave - Robust compensation package, including pension scheme, healthcare plans, and life assurance - Regular corporate events fostering team cohesion and company culture - Diverse company culture and global working environment - Continuous professional development and career advancement opportunities

United Kingdom