Job Closed
This listing is no longer active.
Reach new heights
Senior Data Engineer – Databricks
Location
United States
Posted
95 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer – Databricks
Risepoint
• Build reliable, scalable data pipelines on Databricks • Turn requirements into production-grade ELT/ETL jobs, Delta Lake tables, and reusable components • Implement and improve reference patterns, optimize Spark for performance and cost • Apply best practices with Unity Catalog and workflow orchestration • Ship high-quality code others can build on • Partner directly with the customer’s data engineering team to design and deliver scalable, cloud-based data solutions • Execute complex ad-hoc queries using Databricks SQL • Develop robust data transformation workflows using PySpark and SQL • Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks • Build ETL/ELT workflows with AWS and Azure Services • Optimize Spark jobs for performance and cost • Collaborate with data scientists, ML engineers, and business analysts • Contribute to the development of AI-powered solutions • Implement CI/CD pipelines for Databricks jobs
Job Requirements
- Bachelor's degree in computer science, Engineering, or a related field
- 5+ years of experience in software/data engineering, including at least 2 years working with Databricks and Apache Spark
- Strong proficiency in Python, SQL, and PySpark
- Deep understanding of AWS and Azure Cloud service
- Experience with Databricks Data LakeHouse, Databricks Workflows, and Databricks SQL, dbt
- Solid grasp of data Lakehouse and warehousing architecture
- Prior experience supporting AI/ML workflows, including training data pipelines and model deployment support
- Familiarity with infrastructure-as-code tools like Terraform or CloudFormation
- Strong analytical and troubleshooting skills in a fast-paced, agile environment
- Excellent collaboration skills for interfacing with both technical and non-technical customer stakeholders
- Clear communicator with strong documentation habits
- Comfortable leading discussions, offering strategic input, and mentoring others
Benefits
- Health insurance
- Flexible work arrangements
- Professional development opportunities
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Senior Data Engineer
Odisea CulturaFeeding today's most promising missions by reengineering global partnership standards
• Architect and sustain self-healing pipelines using Astronomer/Airflow to ensure 24/7 data availability • Design and optimize event-driven API ingestion frameworks leveraging AWS Lambda and DLT (Data Load Tool) • Manage high-performance modeling within AWS Redshift, utilizing DBT to transform raw transactional data into high-fidelity business intelligence • Implement advanced monitoring and alerting (CloudWatch) that predicts failures • Lead the ingestion of complex, third-party data streams (Ad-tech, CRM, and high-volume Sales systems) into a unified, scalable source of truth
• Diseñar, construir y mantener **pipelines de datos ETL** robustos y escalables. • Desarrollar soluciones de datos que habiliten **analítica avanzada y explotación eficiente de información**. • Implementar y operar arquitecturas de datos en **Google Cloud Platform (GCP)**. • Garantizar el correcto manejo de **datos sensibles y cumplimiento de estándares de data compliance**. • Colaborar con equipos técnicos y de negocio para transformar requerimientos en **soluciones de datos confiables y eficientes**.
Millennium Health LLC is an accredited specialty laboratory with more than a decade of experience in medication monitoring and drug testing services, helping clinicians monitor the use and misuse of prescription medications and illicit drugs. The testing is used by healthcare professionals to obtain objective information about patients’ recent use of prescription medications and/or illicit drugs and helps monitor the effectiveness of treatment plans. Data Management is tasked with implementing and maintaining revenue cycle configurations within our practice management system. The role maintains system integrity, performs analytics and by directive implements system updates. The following are intended to be examples of the accountabilities for which the person in this position is responsible. This position description is not intended to be complete or all-inclusive and does not preclude management from assigning other or related functions for which the individual has demonstrated competency through performance. - Responsible for performing, tracking, communicating, and monitoring system changes/updates - to the Billing Department staff. - Responsible for system analytics, reporting and conducting quality assurance audits. - Reviews problematic system errors and makes recommendations to the dept manager of how to - resolve. - Participates in educational activities and attends staff meetings. - Must be well organized and detail oriented. - Facilitate incoming cases for billing support via Salesforce within turnaround time - Communicates effectively and professionally, clear, and concise (verbal and written) information - Maintains confidentiality and privacy standards of protected health information and other - information in accordance with employer policies. Maintains strictest confidentiality; adheres to - all HIPAA guidelines/regulations - Upholds the standards of Millennium Health as mandated by its mission, purpose, guiding - principles and standards of quality - Ability to ensure HIPAA, confidentiality and compliance policies, procedures, and standards are always adhered to. - Ability to ensure administrative, physical and technical cybersecurity controls are always adhered to. - Other duties as assigned
• Architect Scalable Systems: Design, develop, and maintain highly scalable, fault-tolerant, and performant ELT/ETL data pipelines • Technical Leadership: Define data engineering standards, code review processes, and architectural best practices • Data Modeling & Optimization: Lead the design and implementation of optimized data models in our cloud data warehouse • Infrastructure as Code (IaC): Implement and manage data infrastructure using Terraform or CloudFormation • Ensure Data Quality & Governance: Establish advanced monitoring, alerting, and automated testing frameworks • Cross-Functional Collaboration: Partner closely with Product Managers, Data Scientists, and Software Engineering teams




