Accelerate Your Cloud Development Efforts
Lead Data Engineer – Databricks
Location
United States
Posted
110 days ago
Salary
0
Seniority
Senior
Job Description
Lead Data Engineer – Databricks
Rearc
• Establish and maintain technical excellence within the data engineering team • Design and implement robust data solutions aligning with business objectives • Drive data-driven initiatives and ensure successful delivery • Lead by example, combining strong technical execution with mentorship • Engage with stakeholders to understand data needs and technical constraints • Implement with a DataOps mindset, building reliable and efficient data pipelines • Mentor and develop data engineers, contributing to knowledge sharing and thought leadership
Job Requirements
- 10+ years of experience in data engineering, data architecture, or related technical fields
- Proven ability to design, build, and optimize large-scale data ecosystems
- Strong track record of leading complex data engineering initiatives
- Deep hands-on expertise in ETL/ELT design, data warehousing, and data modeling
- Extensive experience with data integration frameworks and best practices
- Advanced knowledge of cloud-based data services and architectures
- Proficiency with modern data engineering frameworks, including Databricks, Spark, and lakehouse technologies
Benefits
- Comprehensive health benefits
- Generous time away and flexible PTO
- Maternity and paternity leave
- Access to educational resources with reimbursement for continued learning
- 401(k) plan with company contribution
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Support the development and maintenance of secure, scalable data systems that enable clinical, operational, and financial analytics in a HIPAA‑regulated environment. • Contribute to building and operating data pipelines that ingest, transform, and standardize data from internal and external healthcare sources such as EHRs, claims vendors, and SDOH APIs. • Work closely with analysts and business stakeholders to understand requirements and help create reliable, analytics‑ready datasets and basic data models. • Assist in documenting data processes and governance practices, including data lineage, access controls, and handling procedures for sensitive health information. • Participate in data quality efforts by implementing validation checks, monitoring data flows, and helping identify anomalies that could impact downstream analytics. • Help develop and maintain reusable dbt models and shared data transformations, with exposure to open healthcare data models like Tuva, OMOP, or FHIR. • Collaborate with senior engineers to improve infrastructure and team processes, taking on increasing responsibility as skills grow and the data engineering function expands.
• Build and maintain automated data workflows and orchestrations using Apache Airflow • Build automation processes using Copilot to generate Airflow DAGs • Implement at least two major end-to-end data pipeline projects using Airflow • Design and optimize complex DAGs for scalability, maintainability, and reliability • Create reusable, parameterized, and modular Airflow components (operators, sensors, hooks) to streamline workflow development • Ensure effective monitoring, alerting, and logging of Airflow DAGs for quick issue resolution • Document workflows, solutions, and processes for team knowledge sharing and training • Mentor and support other team members in Airflow usage and adoption • Explain best practices, identify pros and cons, and communicate technical decisions to team members • Develop reusable frameworks, leveraging reusable concepts for efficiency and scalability • Implement and utilize reusable ecosystem components, including Python & Apache Airflow, DynamoDB, Amazon RDS • Develop reusable frameworks to enforce data governance and data quality standards • CI/CD pipeline development using re-usable frameworks and Jenkins
• DecisionPoint seeks a Data Architect to lead the design, governance, and validation of enterprise data architectures for a Department of Defense (DoD) modernization program supporting Casualty and Mortuary Affairs (CMA) operations across all Service components. • The Data Architect will define and maintain logical and physical data models across Impact Level (IL) 2, 4, and 5 environments, ensuring high availability, data integrity, and secure data exchange within a cloud-hosted ecosystem. • The Data Architect will work closely with system engineers, database administrators, and developers to standardize schemas, improve data quality, and align data governance with DoD enterprise architecture standards.
Senior Data Engineer
General DynamicsA business unit of General Dynamics, General Dynamics Information Technology (GDIT) supports some of the United States' most complex government, defense, and in
• Provide advanced analytical, machine learning, and data engineering support. • Design, develop, and deploy machine learning models for classification, regression, time series forecasting, and natural language processing applications. • Build and optimize automated, scalable ETL/ELT pipelines using Python, SQL, and cloud-based tools. • Develop and maintain production ML systems including model deployment, monitoring, versioning, and performance tracking. • Design, develop, and deploy interactive dashboards and data visualizations. • Perform end-to-end model development including exploratory data analysis, feature engineering, hyperparameter tuning, model validation, and documentation. • Develop and maintain data pipelines and workflows using tools such as AWS services, Databricks, and GitLab CI/CD. • Conduct data mining, cleaning, and manipulation using SQL, Python (Pandas, NumPy), or R.




