Better health, easier.
Senior Databricks Engineer – Platform, Data Engineering
Location
Pennsylvania
Posted
2 days ago
Salary
0
Seniority
Senior
Job Description
Senior Databricks Engineer – Platform, Data Engineering
Geisinger
• Manage workspaces, clusters, jobs, security configurations, libraries, and overall platform health. • Build and optimize data pipelines and transformations using PySpark and T‑SQL . • Create automation and tooling using Python , Bash , and PowerShell . • Implement and automate data governance, access controls, auditability, and compliance best practices. • Translate conversational ideas into clear solution architectures; break work into tasks in Planner or Azure DevOps for team execution. • Estimate level of effort and forecast delivery dates with reasonable accuracy. • Produce high-quality documentation including workflows, gap analyses, design specs, build docs, test plans, and transition documentation. • Oversee unit, scenario-based, integrated, capacity, and parallel testing. • Participate in a 24×7×365 on-call rotation to support Databricks , SQL Server , and other relevant applications supported by the team. • Support system upgrades, patches, performance tuning, observability, and proactive monitoring. • Maintain accountability for data integrity across applications and pipelines. • Complete system build and documentation in adherence to departmental guidelines and change management policy. • Review teammates’ work and provide constructive feedback. • Communicate effectively with peers, stakeholders, and leadership; provide regular status updates and escalate issues appropriately. • Drive discussions, mentor accountable leaders, and coordinate resources across initiatives. • Provide feedback on training materials; conduct training sessions; develop and mentor other team member.
Job Requirements
- Solid experience with Databricks administration , including workspace management, cluster configuration, security, jobs, and libraries.
- Strong proficiency in Python and scripting with Bash and PowerShell .
- Experience with PySpark and/or T‑SQL for data processing and transformation.
- Excellent communication skills with the ability to articulate technical concepts clearly to varied audiences.
- Demonstrated critical thinking , problem-solving , and troubleshooting skills.
- Ability to estimate effort and deliver on time .
- Strong understanding of data governance principles and best practices.
- Commitment to information security best practices.
Benefits
- healthcare benefits for full time and part time positions from day one
- vision insurance
- dental insurance
- domestic partner benefits
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
AMS Data Platform Support Specialist
ReplyReply designs and implements innovative solutions in the areas: Digital Services, Technology and Consulting.
• Support and maintain data platforms, ensuring operational continuity and incident resolution. • Support workloads on Azure Databricks (notebooks, jobs, clusters, Delta tables, Unity Catalog). • Maintain and evolve CI/CD pipelines using Azure DevOps. • Troubleshoot pipelines in Azure Data Factory. • Support Power BI environments (workspaces, refresh, access management, and report availability). • Manage and maintain data structures in Azure Data Lake Storage Gen2. • Support Infrastructure as Code practices (Terraform/Azure Blueprints). • Provide operational support for Azure virtual machines. • Troubleshoot Azure networking (VNets, NSGs, DNS, connectivity). • Contribute to technical documentation, runbooks, and the knowledge base. • Participate in continuous improvement, automation, and platform standardization initiatives.
• Забезпечувати коректність, узгодженість і доступність ключових бізнес-метрик для прийняття рішень на основі надійних даних. • Координувати роботу суміжних спеціалістів та формувати технічні вимоги до розвитку всієї data-інфраструктури компанії. • Перетворювати складні бізнес-вимоги та гіпотези у фінальні, структуровані data marts. • Постановка задач та контроль виконання робіт контрактними спеціалістами (Data Engineering / DevOps). • Впроваджувати практики контролю якості даних та управління ризиками для забезпечення їхньої цілісності. • Документувати основні бізнес-метрики (логіка розрахунку, джерела) та впроваджувати процеси пріоритезації задач.
Data Engineer – Contract
AnattaTurnkey flexible digital product teams for fast growing e-commerce brands with annual revenue of $25MM-$500
• Design, build, and maintain scalable data pipelines using BigQuery and dbt • Architect and optimize warehouse-first data models to support analytics, marketing, and operational reporting • Develop and maintain Looker dashboards and semantic layers • Integrate and transform data from Shopify, Klaviyo, Loop (subscriptions/returns) and 3PL systems (e.g., ShipHero, ShipBob, etc.) • Build automated workflows for data ingestion, validation, and monitoring • Implement best practices for data quality, governance, and documentation • Leverage AI tools (LLMs, automation frameworks) to: Accelerate data transformation workflows Refactor and optimize SQL/dbt models Automate anomaly detection and QA processes • Collaborate with analytics, product, and marketing teams to translate business requirements into scalable data solutions • Troubleshoot data discrepancies and provide root-cause analysis • Recommend architectural improvements to improve performance, reliability, and scalability
Lead Data Engineer
Social Discovery GroupTop world’s largest social discovery company uniting 70+ brands with 500M+ users
• Development of various services in Python: integration with marketing partners, obtaining data from various sources • Creation and support of processes on Airflow • Development/refinement of reports on SSRS / SuperSet dashboards • Creation/support of SQL DWH, creation/support of stored procedures • Supporting the migration of marketing data pipelines and DWH components from MS SQL to Google Cloud Platform (including BigQuery), contributing to architecture decisions and best practices.
