Apex Systems logo
Apex Systems

Apex Systems, an IT staffing and workforce solutions firm, provides recruiting and staffing services to large and small companies alike. Founded in 1995 by three Virginia Tech clas

Senior Data Engineer - Databricks

Location

North Carolina

Posted

23 hours ago

Salary

$65 - $75 / hour

Seniority

Senior

Job Description

Senior Data Engineer - Databricks

Apex Systems

Senior Data Engineer - Databricks Location: Charlotte, NC, US Employee Type: Contract Job Type: Pay Range: $65 - $75 per hour Job Description: Job#: 3035030 Job Description: Sr. Data Engineer - Databricks Location: Charlotte, North Carolina- 3X A WEEK HYBRID Employment Type: Contract Role Overview We are seeking a Sr. Data Engineer with expertise in Databricks. This role involves the design, development, and maintenance of scalable data pipelines and solutions. The position requires a focus on workflow orchestration, performance optimization, and data governance within the Databricks ecosystem. Key Responsibilities - Build and schedule jobs using Databricks Workflows, Asset Bundles, or orchestrators like Airflow. - Optimize Spark jobs for performance, including cluster sizing, caching, AQE, and broadcast joins, reduce costs, and troubleshoot via the Spark UI. - Design and implement dimensional models, slowly changing dimensions (SCDs), and curated datasets for BI and Databricks SQL consumption. - Implement data governance and security using Unity Catalog, including schemas, lineage, access controls, and row/column-level security, and manage secrets. - Manage CI/CD pipelines and infrastructure using version control (Git) and deployment tools like Asset Bundles or Terraform. - Establish data quality frameworks using tools such as DLT or Great Expectations, along with monitoring and alerting systems. Required Qualifications Technical Skills: Databricks Compensation & Benefits The anticipated pay range for this position is $65.00 to $75.00 per hour. Please note that the final pay rate will be determined by a variety of factors, including the candidate's experience and qualifications. Benefits may be available to eligible employees. This employer is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, or protected veteran status and will not be discriminated against on the basis of disability. Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Everforth Apex Benefits Overview: Everforth Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Everforth Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Everforth Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA.

Related Categories

Related Job Pages

More Data Engineer Jobs

Appfire logo

Senior Data Analytics Engineer - Data Insights

Appfire

Appfire is a computer software company that has been recognized as a “global authority” on Atlassian products and software. As an employer, the company aims to cultivate a cult

Data Engineer1 day ago

Title: Senior Data Analytics Engineer - Data Insights Location: Spain United States Job Description: At Appfire, we believe that great work happens when people get to choose how they work. After 20 years of creating software that empowers teams to break silos and collaborate seamlessly, we've learned that one size does not fit all. That's why at Appfire, you choose. Choose to work where you thrive: Whether from home, in one of our offices, or while exploring the world, you decide where you're most productive. From hardware and software to access to two decades of collective wisdom on working effectively in a remote-first company, we've got you covered. Choose to balance your life without compromise: Plan your workday around your needs and what matters most to you. Enjoy flexible time off without the hassle of complicated approvals. From day one, we trust you to deliver quality work without sacrificing your personal life, hobbies, loved ones, and well-being. Choose to grow on your terms: Take charge of your professional growth with access to online learning platforms, facilitated training, leadership programs, and internal hackathons. Collaborate with a global team to gain fresh perspectives. When you're ready for a new challenge, our internal mobility program is here to support your journey. No one knows you better than you do. So join us and customize your experience. Choose how you want to work. Choose Appfire. About the role: Appfire is seeking an experienced full-time Senior Analytics Engineer! In this role you will build, define and activate Appfire's company-wide data asset (Snowflake) from ground up data sources. You will work with stakeholders across Appfire to integrate, test, connect, and build brand new data and metrics within our Snowflake metrics layers, achieving a trusted, accessible and well-defined view of information across the company. Our Analytics Engineers build metric foundations for reporting, analysis and experimentation, pushing the boundaries of decisions across the company under an aligned view of the world. They leverage their expertise in metric development, DBT, data modeling, SQL and Python. Using these skills you will transform data from raw data warehouse into data artifacts powering impactful analytic use cases (e.g. metrics, dashboards, adhoc analysis and self-service solutions) that enable and push the boundaries of decisions across the company. As an Analytics Engineer you will sit at the intersection of business and data, working collaboratively to enable and achieve meaningful outcomes by evolving our ecosystems to scale impact. We're growing quickly and have a lot to accomplish. What You Will Do: - Understand decision needs by interfacing with fellow Analytics Engineers, Data Engineers, Data Analysts and Business Partners - Architect, build, and launch efficient & reliable data models and pipelines in partnership with Data Engineering, leveraging DBT and Snowflake - Design and implement metrics and dimensions enabling self-service analysis and structured views of Appfire's KPI's that scale trust, accountability and decision impact - Define and share best practices on metric, dimension, and data model development and definitions for analytics use cases What We Would Like to See: - 5+ years of relevant industry experience - Passion for high data quality and impactful analytics engineering work - Expert SQL skills for data modeling and query optimization. - Extensive experience with self service, advanced analytics, semantic and metrics layers - 3+ Years of Experience with Python and DBT - Strong understanding of data warehousing concepts and best practices. - Efficient problem-solving and analytical skills - Proven ability to succeed in both collaborative and independent work environments at a remote-work company - Proven experience managing concurrent complex data projects from start to finish - Continuous learner with an eye on industry trends and new technologies. - Excellent communication and documentation skills with the ability to explain technical concepts to a non-technical audience. - Ability to effectively leverage AI tools and techniques to enhance dbt development, increase efficiency, and strengthen documentation. Preferred Qualifications: - Experience with dbt (Data Build Tool) and its implementation for data transformation and modeling. - Proven experience with data wrangling and designing/developing experience owning end to end SDLC and implementation, with full ownership of stakeholder management and expectation setting, prioritization, development and delivery. - Proficiency in source code management tools such as GIT, BitBucket, etc. - Experience working with Cloud Technologies (AWS/GCP/Azure) - Experience working with various semi structured data formats (ex. JSON) - Experience building data solutions for future scale and analytics use-cases - 3+ years of development experience on industry standard BI tools - ML analytics engineering experience a plus - Marketing Analytics experience a plus Benefits & Perks Equity Every Appfire team member is eligible for company equity, fostering a true sense of ownership and connection to our growth. Time Off & Flexibility - 25 days of annual leave per calendar year (January-December) - Up to 10 unused vacation days may be carried over to the following year and must be used by December 31 - Reduced summer hours to support better work-life balance - Flexible bank holidays, allowing employees to exchange one public holiday for another - This role is fully remote within Spain, with the option to work from our Bilbao office whenever preferred Learning & Development Grow with Appfire University - our custom, on-demand learning platform designed to support continuous learning and professional development. Health & Wellbeing - Private health insurance - Family members may also be added at a discounted rate through payroll deduction - €400 gross annual sport allowance for gym memberships, outdoor activities, sports equipment, running gear, and other wellness-related expenses Work-from-Home Support Appfire provides €50 per month to help cover home office and remote work expenses. Community & Volunteering Employees receive 3 fully paid volunteering days each year through Appfire Town, our Corporate Social Responsibility (CSR) program supporting local communities. About Appfire A people-first approach to business Since its inception, Appfire has been a remote-first company. With 850+ employees (who we call fireflies) across 28 countries, we foster an environment where everyone is respected. We invest in team members by ensuring they grow professionally and personally. Making an impact At Appfire, CSR means embedding purpose, responsibility, and impact into everything we do. We use our people, products, and partnerships to make a meaningful difference in the world, we act responsibly as a business, and we empower communities while strengthening our skills and culture and fostering belonging across Appfire. In 2015, Appfire joined the Pledge 1% network of organizations committed to philanthropy. Appfire has since grown our Pledge 1% program to include all four pledge types - product, profit, equity, and employee time. We were among the first to do this, and we're proud that Pledge 1% is part of our evolution. Our business Appfire mission to equip and connect every team so they can plan and deliver their best work. We are committed to building a durable, multi-generational business, and to evolving, innovating, and scaling in a way that ensures stability and opportunity for years to come What's our secret sauce? - We follow teams. We do our research and build software that solves real-life collaboration challenges while being easy to implement and a joy to use. We're proud to support over 20,000 customers and growing, including 55% of the Fortune 500. From the entertainment delivered by Netflix to the devices crafted by Samsung, and Dell Technologies and the financial transactions handled by Visa, Edward Jones, and US Bank, Appfire's technology is indispensable. Our products also play a pivotal role in streamlining operations and fostering innovation at companies like Tesla and significant institutions such as NASA, Boeing, and many more. - We enhance. Our software is designed to give developers, knowledge workers, and teams the ability to extend and get greater value from the platforms they've invested in and enjoy. So far, our solutions extend and enhance the capabilities offered by Atlassian, Microsoft, monday.com, and Salesforce. - We build bridges and invest in our partners. Appfire's success is underscored by its channel program as its primary path to market. Today, Appfire has a dedicated Channel team supporting 800+ channel partners. - We make security and privacy a priority, but we also keep it simple for our customers. We've achieved International Organization for Standardization (ISO) 27001 and ISO 27017 and System and Organization Controls (SOC) SOC 2, Type I and SOC 2, Type II certifications. Our award-winning Appfire Trust Center, offers our customers, partners, and prospects the latest security, privacy, and compliance information, including pre-completed questionnaires (CAIQ, SIG, and VSA) with an accelerated NDA process and just one EULA to cover it all. Market recognition Appfire has been consistently recognized for company growth, culture, corporate social responsibility, and product excellence and has been included among the Deloitte Technology Fast 500, Inc. Best Workplaces, BuiltIn Best Places to Work, and Inc. 5000. Learn more about our accomplishments, which would not be possible without our team members, partners, and customers: https://appfire.com/awards. Equal Employer Opportunity (EEO) Appfire is an equal opportunity employer and does not discriminate based on race, color, religion, sex (including pregnancy), sexual orientation, gender identity or expression, national origin, age, disability, genetic information, marital status, veteran status, or any other protected characteristic as defined by applicable law. Our commitment extends to all employment practices, including recruitment, hiring, training, promotion, compensation, benefits, and termination.

Spain
Sleek logo

Data Engineer

Sleek

One-stop-shop for entrepreneurs to start & grow their business in Australia, Hong Kong, Singapore and the UK.

Data Engineer1 day ago
Full TimeRemoteTeam 51-200Since 2017H1B No Sponsor

Role Description We are looking for a skilled and passionate Data Engineer to join our growing Data Platform team. Mission: - Design, build, and maintain robust data pipelines on Databricks and AWS infrastructure that power analytics and reporting capabilities across the organization. Key responsibilities: - Design and implement scalable ETL/ELT pipelines using both batch and streaming patterns. - Build and maintain ingestion workflows from diverse sources (databases, APIs, event streams). - Implement Change Data Capture (CDC), full-load, and incremental ingestion strategies. - Develop and manage data workflows using Apache Airflow for orchestration. - Configure and manage data ingestion connectors using Airbyte. - Work with Databricks to build and optimize data engineering workloads on the Lakehouse platform. - Write and optimize complex SQL queries. - Solid hands-on experience on dbt with Databricks, build modular, testable dbt models for data transformation. - Develop and maintain data models in staging, intermediate, and mart layers following data warehousing best practices. - Working knowledge of AWS services like S3, Lambda, EC2, IAM, etc. - Containerize data services and applications using Docker & EKS. - Ensure data quality, observability, and reliability across the data platform. - Document pipelines, models, and data dictionaries to maintain platform knowledge. Qualifications - 3+ years of professional experience in data engineering. - Strong understanding of data platform architecture: Lakehouse, Data Warehouse, Data Lake patterns. - Hands-on experience with ETL/ELT design patterns including batch processing and stream processing. - Familiarity with ingestion patterns: full load, incremental, CDC, event-driven. - 5+ years of hands-on experience as a Database Administrator in production environments. - Strong understanding of database internals — storage engines, transactions, isolation levels, locking, MVCC, query planners. - Proven experience supporting mission-critical OLTP workloads with high availability requirements. - Solid scripting skills in Bash and/or Python for automation. Requirements - Experience building data pipelines on Databricks (Delta Live Tables, Jobs, Notebooks). - Proficiency with PySpark or Spark SQL for large-scale data processing. - Familiarity with Delta Lake concepts: ACID transactions, time travel, schema evolution. - Proficiency with Apache Airflow — authoring, scheduling, and monitoring DAGs. - Experience with Airbyte for managing source-to-destination data connectors. - Hands-on experience administering MongoDB (self-managed and/or Atlas). - Replica set configuration, sharding, indexing strategies, and aggregation pipeline tuning. - Backup, restore, and disaster recovery using mongodump/mongorestore or Ops Manager. - Strong SQL skills — query optimization, window functions, CTEs, and complex joins. - Experience with dbt (data build tool) for transformation, testing, and documentation. - Practical experience with AWS services (S3, Lambda, IAM, CloudWatch, etc). Benefits - Humility and kindness: A culture of not taking ourselves too seriously and being able to laugh. - Flexibility: Work from home and the option to work fully remote from anywhere in the world for 1 month each year. - Financial benefits: Competitive market salaries, generous paid time off, and access to a flexi benefits scheme. - Personal growth: Responsibility and autonomy with a range of internal and external training programs. Company Description Sleek launched in 2017 and now has around 15,000 customers across our offices in Singapore, Hong Kong, Australia, and the UK. We have around 500 staff with an intact startup mindset. We have recently raised Series B financing off the back of >70% compound annual growth in Revenue over the last 5 years. Sleek has been recognised by The Financial Times, The Straits Times, Forbes, and LinkedIn as one of the fastest-growing companies in Asia. Backed by world-class investors, we are on track to be one of the few cash flow positive, tech-enabled unicorns based out of Asia Pacific.

Worldwide
Blue Cross and Blue Shield of Kansas City logo

Lead Data Engineer

Blue Cross and Blue Shield of Kansas City

Blue Cross and Blue Shield of Kansas City is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, status as a protected veteran, or disability.

Data Engineer1 day ago
Full TimeRemoteTeam 1,001-5,000

Role Description The Lead Data Engineer analyzes, develops, and maintains complex logical data designs in support of corporate and customer information system requirements. Through proper data design and governance, this role maintains and monitors the use of standards and procedures and the integration of systems. This individual contributes to the enterprise architecture as a representative of data design and governance. Our ideal candidate uses expert specialist skills and knowledge to define and deliver data solutions and to contribute to the development and implementation of IT plans in line with the overall strategy of IT and the business. - Provides vision, leadership, and strategic direction on data management and enterprise information management and leads & coaches junior Data Engineers. - Defines, develops, and delivers consistent information and data standards, methodologies, guidelines, best practices, and approved modeling techniques on data/information management, data quality, and data governance. - Designs enterprise data architecture, including conceptual architectures for solution strategies, for data stores supporting transactional processing, data warehouses, operational data stores, data marts, and analytics. - Provides assistance and guidance to the development team regarding data model, SQL, and Stored procedures, including code review of complex SQL and Stored procedures and performance tuning as needed by development teams. - Develops data models as required by project teams. - Collects, maintains, and publishes corporate metadata related to the modeling in an enterprise asset repository. - Simplifies the existing data architecture and data architecture SDLC process, delivering reusable services and cost-saving opportunities in line with the policies and standards of the group. - Utilizes skills with the appropriate tools and business requirements to model entity relationships, multi-dimensionality, subject areas, state transitions, data flows and integrations, and reference data. - Leads the design, development, implementation, and maintenance of complex data systems and solutions ensuring technology solutions are in alignment with data architecture principles and target state. - Creates and maintains accurate, complete, and consistent governed conceptual, logical, and physical data models with lineage and implementation traceability metadata. - Responsible for source to target mapping where business rules are clearly documented and defined. Qualifications - Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field. - 7 years’ experience with data warehousing, including 5 years’ experience with relational modeling. - 7+ years of SQL development. - Experience with DLM and Enterprise Platforms for managing data movement. - Thorough understanding of the insurance industry (including claims processes, how attribution works, and the core business concepts of insurance). - Understanding of Scrum and Agile development practices. - Proven data governance and information management experience. - Strong interpersonal communication and follow-through skills and demonstrated initiative in autonomous environments. - Experience with database technologies (e.g., SQL, NoSQL, Snowflake, and Data Lake design). - Experience with ETL and other business intelligence tools. - Experience analyzing, organizing, and preparing data for use through the development of proprietary systems. - Experience designing multi-dimensional data models using Kimball Methodology. - Very effective interpersonal skills. Requirements - Master’s degree in Business Administration, Computer Science, Information Systems, or a related field of study. - 3 years’ experience in the healthcare industry. - 7+ years in IT. - Ability to lead, guide, motivate, and direct the work of others. - Experience with source to target mapping where business rules are clearly documented and defined. - Highly technical with hands-on experience using Python, or JVM language (Java, Scala, Clojure, or Kotlin). - Experience implementing, managing, and developing in Apache Airflow or other orchestration tools. - Experience implementing data solutions in a major cloud provider (Azure, AWS, or GCP). - Experience with semi-structured formats (Avro, Parquet, ORC, YAML, JSON, or XML). - Experience with MDM Implementation. - Experience with MDM Tools (Informatica, Profisee, or similar). - Experience implementing Kafka, Schema Registry, and Kafka Streaming and/or Spark Streaming. - Experience with Microsoft Azure. - Experience with Apache Airflow and Celery. - Experience with Docker, Kubernetes, and Terraform. - Experience with Snowflake or Synapse. - Experience with Informatica IDQ, EDC, IICS, and Axon. - Experience with Scala, Python, and Java. - Experience with NodeJS and React. - Experience with Azure Data Lake Storage Gen2. - Experience with cloud platforms such as Azure, AWS. Benefits - Highly competitive total rewards package, including comprehensive medical, dental, and vision benefits as well as a 401(k) plan that both the employee and employer contribute. - Annual incentive bonus plan based on company achievement of goals. - Time away from work including paid holidays, paid time off, and volunteer time off. - Professional development courses, mentorship opportunities, and tuition reimbursement program. - Paid parental leave and adoption leave with adoption financial assistance. - Employee discount program. Company Description Blue Cross and Blue Shield of Kansas City is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to, among other things, race, color, religion, sex, sexual orientation, gender identity, national origin, age, status as a protected veteran, or disability.

United States

Forward Deployed Data Engineer

Datafold

Datafold is the data engineering automation platform. A major part of our business is delivering large-scale data engineering projects such as data platform migrations with AI, at fixed price and guaranteed timeline. We are not a typical services provider or SI – we are a venture-backed software company that reimagined automation from the ground up and has been delivering projects up to 6x faster than alternatives. We partner heavily with leading data platforms, including Databricks and Snowflake, to deliver large-scale complex migrations.

Data Engineer1 day ago

Role Description Datafold is hiring a Forward Deployed Data Engineer to own the delivery of our AI-automated data platform migration projects. This is a high-ownership, customer-facing role at the intersection of data engineering, AI, and project leadership. As a Forward Deployed Data Engineer leveraging Datafold's AI technology, you will own migration projects end-to-end, gain deep understanding of the customer's data platform, manage execution of the project, and remove any blockers on the way for a timely migration delivery. Using our proprietary AI tooling to plan and deliver the migration projects, you will be doing the work equivalent to a full team of consultants. What You'll Do - Own 1–4 concurrent migration projects end-to-end: scoping, planning, execution, and customer handoff - Be the primary customer contact: run weekly check-ins, manage stakeholder expectations, and escalate risks early before they compound - Configure Datafold's Migration Agent and oversee the migration execution - Partner with Datafold's engineering team to execute migrations - Help refine and scale our product and delivery playbook as the team grows Qualifications - 3–6 years in data consulting, professional services, or a customer-facing data engineering role - Excellent communication skills — equally comfortable in an exec check-in and a technical design session - Strong grasp of the modern data stack: dbt, Snowflake, Databricks, orchestration tools, and major patterns (stored procedures, streaming, incremental processing) - Extreme ownership mentality — you identify, surface, and fix problems and rally the team to help without being told - AI power user — using AI every day and always learning and improving on how to use it more effectively - Exposure to legacy data stack and patterns (ETL, stored procedures, etc.) and data platform migration projects is a strong plus Benefits - Competitive salary + equity Company Description Datafold is the data engineering automation platform. A major part of our business is delivering large-scale data engineering projects such as data platform migrations with AI, at fixed price and guaranteed timeline. We are not a typical services provider or SI – we are a venture-backed software company that reimagined automation from the ground up and has been delivering projects up to 6x faster than alternatives. We partner heavily with leading data platforms, including Databricks and Snowflake, to deliver large-scale complex migrations.

United States
$155K - $185K / year