Better health, easier.
Senior Databricks Engineer – Platform, Data Engineering
Location
Pennsylvania
Posted
48 days ago
Salary
0
Seniority
Senior
Job Description
Senior Databricks Engineer – Platform, Data Engineering
Geisinger
• Manage workspaces, clusters, jobs, security configurations, libraries, and overall platform health. • Build and optimize data pipelines and transformations using PySpark and T‑SQL . • Create automation and tooling using Python , Bash , and PowerShell . • Implement and automate data governance, access controls, auditability, and compliance best practices. • Translate conversational ideas into clear solution architectures; break work into tasks in Planner or Azure DevOps for team execution. • Estimate level of effort and forecast delivery dates with reasonable accuracy. • Produce high-quality documentation including workflows, gap analyses, design specs, build docs, test plans, and transition documentation. • Oversee unit, scenario-based, integrated, capacity, and parallel testing. • Participate in a 24×7×365 on-call rotation to support Databricks , SQL Server , and other relevant applications supported by the team. • Support system upgrades, patches, performance tuning, observability, and proactive monitoring. • Maintain accountability for data integrity across applications and pipelines. • Complete system build and documentation in adherence to departmental guidelines and change management policy. • Review teammates’ work and provide constructive feedback. • Communicate effectively with peers, stakeholders, and leadership; provide regular status updates and escalate issues appropriately. • Drive discussions, mentor accountable leaders, and coordinate resources across initiatives. • Provide feedback on training materials; conduct training sessions; develop and mentor other team member.
Job Requirements
- Solid experience with Databricks administration , including workspace management, cluster configuration, security, jobs, and libraries.
- Strong proficiency in Python and scripting with Bash and PowerShell .
- Experience with PySpark and/or T‑SQL for data processing and transformation.
- Excellent communication skills with the ability to articulate technical concepts clearly to varied audiences.
- Demonstrated critical thinking , problem-solving , and troubleshooting skills.
- Ability to estimate effort and deliver on time .
- Strong understanding of data governance principles and best practices.
- Commitment to information security best practices.
Benefits
- healthcare benefits for full time and part time positions from day one
- vision insurance
- dental insurance
- domestic partner benefits
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
AMS Data Platform Support Specialist
ReplyReply designs and implements innovative solutions in the areas: Digital Services, Technology and Consulting.
• Support and maintain data platforms, ensuring operational continuity and incident resolution. • Support workloads on Azure Databricks (notebooks, jobs, clusters, Delta tables, Unity Catalog). • Maintain and evolve CI/CD pipelines using Azure DevOps. • Troubleshoot pipelines in Azure Data Factory. • Support Power BI environments (workspaces, refresh, access management, and report availability). • Manage and maintain data structures in Azure Data Lake Storage Gen2. • Support Infrastructure as Code practices (Terraform/Azure Blueprints). • Provide operational support for Azure virtual machines. • Troubleshoot Azure networking (VNets, NSGs, DNS, connectivity). • Contribute to technical documentation, runbooks, and the knowledge base. • Participate in continuous improvement, automation, and platform standardization initiatives.
• Забезпечувати коректність, узгодженість і доступність ключових бізнес-метрик для прийняття рішень на основі надійних даних. • Координувати роботу суміжних спеціалістів та формувати технічні вимоги до розвитку всієї data-інфраструктури компанії. • Перетворювати складні бізнес-вимоги та гіпотези у фінальні, структуровані data marts. • Постановка задач та контроль виконання робіт контрактними спеціалістами (Data Engineering / DevOps). • Впроваджувати практики контролю якості даних та управління ризиками для забезпечення їхньої цілісності. • Документувати основні бізнес-метрики (логіка розрахунку, джерела) та впроваджувати процеси пріоритезації задач.
Lead Data Engineer
Social Discovery GroupTop world’s largest social discovery company uniting 70+ brands with 500M+ users
• Development of various services in Python: integration with marketing partners, obtaining data from various sources • Creation and support of processes on Airflow • Development/refinement of reports on SSRS / SuperSet dashboards • Creation/support of SQL DWH, creation/support of stored procedures • Supporting the migration of marketing data pipelines and DWH components from MS SQL to Google Cloud Platform (including BigQuery), contributing to architecture decisions and best practices.
Principal Data Engineer
CedarCedar is the AI-powered healthcare financial experience platform, built for the rising cost and complexity of healthcare payments. We help millions of people every year understand and resolve their medical bills with clarity and compassion, while helping healthcare organizations operate more efficiently. We’re combining AI, smart design, and empathy to fix one of healthcare’s most urgent crises.
Role Description Cedar is at a pivotal moment in its technical evolution. We are evolving towards a brand-new, enterprise data platform and model, architected specifically to power the future of Cedar’s product offerings. As the Principal Data Architect, you will be the senior technical authority leading this transition, ensuring our new data foundation is clean, scalable, and cleanly separated from legacy debt. This is a hands-on technical leadership role for an engineer who loves building production systems at scale and views AI as a massive accelerator for both personal and team productivity. What You’ll Do - Architect the Future-State Data Model & Storage: Lead the design and execution of a new, high-scale data model and storage architecture that lives outside our legacy monolith. - Ensure Metrics Reliability: Own the accuracy and reliability of Cedar’s core business metrics including collection rate, days in AR, and invoice balance tracking. - Power Intelligent Agents: Build the real-time data foundations that power Cedar’s AI-driven products. - Design for the Future: Design data models for new product lines that expand Cedar’s presence across the full revenue cycle. - Build Production Code: Write, ship, and maintain high-quality production code alongside our team of Data Engineers. - Guide and Mentor Engineers: Act as the technical role model for a team of Data Engineers. - Collaborate: Drive technical alignment across data engineering, platform, and product teams. - Steward Data Integrity: Define and enforce standards for Cedar's enterprise Data Dictionary and metadata strategy. Qualifications - 10+ years of experience in data engineering and backend systems at scale. - Experience leading large technical projects with a high accountability mentality. - Deep, hands-on proficiency with a wide range of data and engineering technologies and patterns. - Experience building "source of truth" systems and a passion for data accuracy. - Experience building data APIs, context systems, or feature stores that power downstream ML/AI applications. - Ability to spot over-engineering and under-engineering with equal confidence. - Clear communication skills to translate complex technical risks and trade-offs into business-relevant terms. - Avid learner and adopter of AI engineering tools. - Demonstrable understanding of healthcare concepts, including revenue cycle management (RCM), billing, and insurance adjudication (highly preferred). Requirements - Experience with technologies such as Snowflake, dbt, Liquibase, Fivetran, Airflow, OpenMetadata, Kafka, SQL, Python, and real-time data processing. - Experience with building production-grade ELT/ETL pipelines and managing complex data transformations in cloud-first environments. - Experience with data validation and reconciliation, comparing pipeline output against source systems. Benefits - Salary/Hourly Rate Range: $246,500 - $311,750. - This role is equity eligible. - This role offers a competitive benefits and wellness package. - Unlimited PTO for vacation, sick, and mental health days. - 16 weeks paid parental leave with health benefits for all parents. - Diversity initiatives that encourage Cedarians to bring their whole selves to work. - Competitive pay, equity (for qualifying roles), and health benefits. - Cedar matches 100% of your 401(k) contributions, up to 3% of your annual compensation. - Access to hands-on mentorship, employee and management coaching, and a team discretionary budget for learning and development resources.




