Job Closed
This listing is no longer active.
SBA Certified 8(a), EDWOSB/WOSB and CMMI L3, ISO 9001:2015 Certified QMS
Data Engineer – Level III
Location
Washington
Posted
44 days ago
Salary
$150K - $170K / year
Seniority
Lead
Job Description
Data Engineer – Level III
AttainX, Inc.
• Architect, lead and optimize large-scale ETL/ELT pipelines using PySpark, Python, Spark SQL and Databricks to support advanced analytics, fraud detection and inter-agency data sharing. • Implement enterprise-grade monitoring, observability and data quality frameworks, including automated validation, anomaly detection and SLA-driven alerting. • Develop and tune review complex SQL for large datasets, distributed compute environments and mission-critical analytical workloads. • Build automated ingestion frameworks for APIs, flat files, RDBMS sources and streaming feeds, ensuring lineage, auditability and compliance with Department of Homeland Security (DHS) and federal data governance standards. • Design and govern data lakehouse architectures, including Delta Lake, medallion (bronze/silver/gold) patterns, schema evolution and performance optimization. • Lead exploratory data analysis to identify operational trends, anomalies and mission-impacting insights across large, federated datasets. • Build and maintain analytics products, dashboards and reporting solutions using Databricks SQL, Tableau, Power BI and SSRS. • Support machine learning and entity resolution initiatives through feature engineering, data preparation and scalable data services. • Translate ambiguous business problems into structured analytical frameworks, delivering executive-ready visualizations, technical briefs and data-driven recommendations. • Serve as a technical subject matter expert (SME) and engineering lead across multi-disciplinary teams including data scientists, program analysts, cybersecurity and IT operations. • Provide mentorship, code reviews and architectural guidance to engineering teams across multiple programs. • Lead requirements analysis, system design, integration and modernization efforts for enterprise systems supporting operations and high-security missions.
Job Requirements
- Active DHS or public trust clearance
- U.S. citizen
- Bachelor’s degree in computer science, information systems, data science, engineering, mathematics or a related technical field
- 10+ years of experience in data engineering, analytics engineering or large-scale data platform development
- Demonstrated experience supporting federal government programs, preferably within DHS or similar high-security environments
- Expert-level proficiency in SQL, including advanced query tuning, distributed query optimization and large dataset performance engineering
- Extensive hands-on experience with PySpark, distributed data processing and performance optimization techniques
- Strong proficiency in Python for automation, data manipulation and pipeline orchestration
- Deep experience with Databricks, including clusters, jobs, notebooks, Unity Catalog and Delta Lake
- Strong understanding of lakehouse architecture, medallion design patterns and enterprise data governance
- Experience with Git-based development workflows, code review processes and CI/CD integration for data engineering
Benefits
- Competitive compensation and benefits packages including paid vacation, paid holidays and sick pay
- medical, dental and vision plans plus HSA and FSA accounts
- matching 401K plan
- tuition, training, certification, professional development programs and long and short-term disability
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Lead Data Engineer
HumanaHumana Inc. (NYSE: HUM) is a leading U.S. healthcare company. Through our Humana insurance services and our CenterWell healthcare services, we make it easier for the millions of people we serve to achieve their best health – delivering the care and service they need, when they need it. These efforts are leading to a better quality of life for people with Medicare and Medicaid, families, individuals, military service personnel, and communities at large.
Become a part of our caring community We are seeking an experienced and strategic Lead Data Engineer to join our ITSM Applied AI and Analytics team. This individual will lead the development and evolution of our enterprise-facing data store built on Azure Databricks, driving architectural decisions, mentoring team members, and ensuring scalable, high-quality data solutions across the organization. Key Responsibilities: - Lead the design, development, and optimization of ETL/ELT pipelines using Databricks, Apache Spark, Unity Catalog, and Delta Lake. - Architect data workflows following Medallion Architecture (Bronze, Silver, Gold layers). - Guide the integration of external APIs and systems, including ServiceNow and Kafka for real-time data streaming. - Establish and enforce best practices in CI/CD, GitHub workflows, and code quality. - Manage compute appropriately to ensure cost control. - Collaborate with stakeholders to define data strategy, governance, and platform enhancements. - Provide technical leadership and mentorship to data engineers and analysts. - Conduct code reviews and ensure adherence to engineering standards. - Communicate effectively with technical and non-technical stakeholders. You will report to an Associate Director, Data Engineering. Use your skills to make an impact Required Qualifications: - 5+ years of experience in data engineering, with at least 2 years in a lead or senior role. - Deep expertise in Databricks, Apache Spark, Unity Catalog, and Delta Lake. - Strong understanding of Medallion Architecture and scalable data design. - Strong proficiency in Python for data engineering tasks (APIs, Spark data frames, OAuth, etc.). - Familiarity with Databricks Materialized views. - Experience with Kafka for real-time data ingestion and processing. - Proficiency in GitHub, CI/CD pipelines, and DevOps practices. - Experience building and integrating REST APIs. - Excellent communication, leadership, and documentation skills. Preferred Qualifications: - Databricks certifications (e.g., Data Engineer Professional). - Familiarity with ServiceNow data structures and integration. - Experience with enterprise analytics platforms like Power BI or Power Apps. - Background in enterprise environments with data governance and compliance requirements. - Background in enterprise data architecture and governance. Additional Information Candidates will be asked to present a GitHub-hosted project as part of the interview process. Why Humana? At Humana, we know your well-being is important to you, and it’s important to us too. That’s why we’re committed to making resources available to you that will enable you to become happier, healthier, and more productive in all areas of your life. Just to name a few: - Work-Life Balance - Generous PTO package - Health benefits effective day 1 - Annual Incentive Plan - 401K - Excellent company match - Well-being program - Paid Volunteer Time Off If you share our passion for helping people, we likely have the right place for you at Humana. Work at Home Guidance To ensure Home or Hybrid Home/Office associates’ ability to work effectively, the self-provided internet service of Home or Hybrid Home/Office associates must meet the following criteria: - At minimum, a download speed of 25 Mbps and an upload speed of 10 Mbps is recommended; wireless, wired cable or DSL connection is suggested - Satellite, cellular and microwave connection can be used only if approved by leadership - Associates who live and work from Home in the state of California, Illinois, Montana, or South Dakota will be provided a bi-weekly payment for their internet expense. - Humana will provide Home or Hybrid Home/Office associates with telephone equipment appropriate to meet the business requirements for their position/job. - Work from a dedicated space lacking ongoing interruptions to protect member PHI / HIPAA information SSN Alert Statement Humana values personal identity protection. Please be aware that applicants may be asked to provide their Social Security Number, if it is not already on file. When required, an email will be sent from Humana@myworkday.com with instructions on how to add the information into your official application on Humana’s secure website. Travel: While this is a remote position, occasional travel to Humana's offices for training or meetings may be required. Scheduled Weekly Hours 40 Pay Range The compensation range below reflects a good faith estimate of starting base pay for full time (40 hours per week) employment at the time of posting. The pay range may be higher or lower based on geographic location and individual pay will vary based on demonstrated job related skills, knowledge, experience, education, certifications, etc. $142,300 - $195,700 per year This job is eligible for a bonus incentive plan. This incentive opportunity is based upon company and/or individual performance. Description of Benefits Humana, Inc. and its affiliated subsidiaries (collectively, “Humana”) offers competitive benefits that support whole-person well-being. Associate benefits are designed to encourage personal wellness and smart healthcare decisions for you and your family while also knowing your life extends outside of work. Among our benefits, Humana provides medical, dental and vision benefits, 401(k) retirement savings plan, time off (including paid time off, company and personal holidays, volunteer time off, paid parental and caregiver leave), short-term and long-term disability, life insurance and many other opportunities. Application Deadline: 04-24-2026 About us About Humana: Humana Inc. (NYSE: HUM) is a leading U.S. healthcare company. Through our Humana insurance services and our CenterWell healthcare services, we make it easier for the millions of people we serve to achieve their best health – delivering the care and service they need, when they need it. These efforts are leading to a better quality of life for people with Medicare and Medicaid, families, individuals, military service personnel, and communities at large. Learn more about what we offer at Humana.com and at CenterWell.com. Equal Opportunity Employer It is the policy of Humana not to discriminate against any employee or applicant for employment because of race, color, religion, sex, sexual orientation, gender identity, national origin, age, marital status, genetic information, disability or protected veteran status. It is also the policy of Humana to take affirmative action, in compliance with Section 503 of the Rehabilitation Act and VEVRAA, to employ and to advance in employment individuals with disability or protected veteran status, and to base all employment decisions only on valid job requirements. This policy shall apply to all employment actions, including but not limited to recruitment, hiring, upgrading, promotion, transfer, demotion, layoff, recall, termination, rates of pay or other forms of compensation and selection for training, including apprenticeship, at all levels of employment.
Manager, Data Engineering
OmnidianThe Nation’s Leading Residential and Commercial Solar Performance Plan And Cash-Back Energy Guarantee
• Lead, mentor and coach a team of six Data Engineering professionals • Own workload management for the Data Engineering team, including intake, prioritization, resource allocation • Ensure team has measurable SMART goals and understands their impact on company objectives • Collaborate cross functionally to drive initiatives • Establish and provide visibility for performance metrics
• Data Collection: Collect data from various internal and external sources, following established procedures and proposing process improvements when necessary. • Data Preparation: Perform data cleaning, transformation, and integration tasks to ensure data quality and consistency with minimal supervision. • Project Support: Collaborate with team members on the development and maintenance of data pipelines, taking on larger responsibilities and leading small subprojects. • Process Documentation: Make significant contributions to data engineering process documentation by recording procedures and workflows and ensuring they remain up to date. • Exploratory Data Analysis: Conduct initial data analyses to identify patterns, trends, and business-relevant insights with greater autonomy. • Query Support: Execute more complex SQL queries and extract information from relational databases, providing technical support to the team. • Quality Monitoring: Proactively monitor data quality, identify and resolve issues or inconsistencies, and suggest improvements to quality control practices. • Continuous Learning: Participate in and contribute to training and development programs, and mentor junior team members. • Cross-Functional Collaboration: Work effectively with other departments to ensure proper data integration into solutions and systems, contributing specialized knowledge. • Meeting Deadlines: Manage tasks and ensure completion within established deadlines, proactively communicating any challenges or obstacles to team leadership.
Data Engineer – Web Scraping, ETL
TechBiz GlobalTechBiz Global is a leading IT recruitment and software development company
• Monitor and manage overnight scraper and ingestion runs, triaging failures and applying fixes in real time to minimize data gaps before US market open • Verify data completeness and quality across all automated feeds, flagging anomalies and coordinating with the Houston team on persistent issues • Maintain run logs, error documentation, and escalation notes for seamless async handoffs • Build and maintain scrapers, parsers, and ingestion pipelines across a growing set of energy market data domains • Contribute to the design and build-out of our broader ETL infrastructure, including scheduling, orchestration, and error handling • Write transformation logic to clean, normalize, and load raw data into PostgreSQL staging and production tables • Optimize existing pipelines for performance, reliability, and cost efficiency • Help build monitoring dashboards and alerting for pipeline health and data freshness • Document data lineage, schema changes, and pipeline dependencies




