Job Closed

This listing is no longer active.

Lead Data Engineer

Location

United States

Posted

91 days ago

Salary

$1.2K - $1.3K / month

Seniority

Lead

Job Description

Lead Data Engineer

CommIT Enterprises

This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description CommIT Enterprises, Inc. is seeking a highly skilled Lead Data Engineer to join a team in Charleston, SC, but this role can be REMOTE, to design, develop, and maintain advanced data ingestion pipelines and models that support the United States Marine Corps (USMC) logistics situational awareness and decision-making. This role requires expertise in ETL/ELT processes, CI/CD pipeline development, and data modeling to ensure reliable, scalable, and secure data solutions. This role directly supports the USMC by enabling data-driven situational awareness, ensuring leaders have timely, accurate, and actionable insights to inform operational decisions. Responsibilities - Lead the design, development, and maintenance of CI/CD pipelines that enable ingestion teams to seamlessly move ETL/ELT processes through Development, Testing, and Production environments. - Build and optimize ETL/ELT workflows to ingest, validate, and transform data from diverse sources. - Apply data modeling techniques to create Silver and Gold Tier data assets that enhance situational awareness and support decision-making. - Collaborate with cross-functional teams to ensure data pipelines and models align with operational requirements and reporting needs. - Drive best practices in data engineering, automation, and pipeline reliability to support mission-critical analytics. Qualifications - Master’s degree with 6 years of experience (or Bachelors with 8 years of experience) in Computer Science, Software Engineering, Computer Engineering, Mathematics or relevant field. Degree may be substituted with additional relevant industry experience and/or industry accepted training and certification. - 5+ years of experience in data engineering with a focus on ETL/ELT and CI/CD. - Hands-on experience with Databricks, Delta Lake, and GitLab Enterprise. - Strong proficiency in Python and SQL. - Experience supporting logistics, defense, or mission-critical environments is a plus. - Proven experience with ETL and ELT processes for ingesting structured, semi-structured, and unstructured data sources. - Deep understanding of CI/CD practices, with the ability to design and implement pipelines using Databricks and GitLab Enterprise for Data Ingestion teams. - Strong background in data modeling, with the ability to build Silver and Gold Tier Tables and Views to support operational dashboards and reporting. Requirements - ETL and ELT methods - Databricks and Delta Tables - Python and SQL programming - CI/CD pipeline development and automation - GitLab Enterprise for version control and deployment Security Requirements - Secret Clearance - Security+ Certification Equal Opportunity Employer CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.

Job Requirements

  • Master’s degree with 6 years of experience (or Bachelors with 8 years of experience) in Computer Science, Software Engineering, Computer Engineering, Mathematics or relevant field. Degree may be substituted with additional relevant industry experience and/or industry accepted training and certification.
  • 5+ years of experience in data engineering with a focus on ETL/ELT and CI/CD.
  • Hands-on experience with Databricks, Delta Lake, and GitLab Enterprise.
  • Strong proficiency in Python and SQL.
  • Experience supporting logistics, defense, or mission-critical environments is a plus.
  • Proven experience with ETL and ELT processes for ingesting structured, semi-structured, and unstructured data sources.
  • Deep understanding of CI/CD practices, with the ability to design and implement pipelines using Databricks and GitLab Enterprise for Data Ingestion teams.
  • Strong background in data modeling, with the ability to build Silver and Gold Tier Tables and Views to support operational dashboards and reporting.
  • ETL and ELT methods
  • Databricks and Delta Tables
  • Python and SQL programming
  • CI/CD pipeline development and automation
  • GitLab Enterprise for version control and deployment
  • Security Requirements
  • Secret Clearance
  • Security+ Certification
  • Equal Opportunity Employer
  • CommIT Enterprises, Inc. is an Equal Opportunity Employer. Employment decisions are made without regard to race, color, religion, national origin, gender, sexual orientation, gender identity, age, physical or mental disability, genetic factors, military/veteran status or other characteristics protected by law.

Related Categories

Related Job Pages

More Data Engineer Jobs

Senior AWS Data Engineer (Scientific Data Platforms)

GAMA-1 Technologies

Founded in 2004, GAMA-1 Technologies is a certified HUBZone and minority-owned small business that provides advanced IT solutions and mission support services t

Data Engineer91 days ago

We are seeking a Senior AWS Data Engineer to design and build cloud-native data platforms supporting scientific research at the NOAA. This role focuses on developing scalable AWS-based data pipelines, API-driven data platforms, and machine learning workflows that enable researchers to ingest, process, analyze, and visualize complex scientific and geospatial datasets. You will work within an Agile development team to architect and implement secure, scalable data solutions used by research laboratories and programs across the country. This is a hands-on engineering role focused on building production-ready cloud data systems in AWS. What You’ll Do - Design and build AWS-based data pipelines for ingesting and processing large scientific datasets - Develop API-driven data platforms to host and distribute datasets internally and externally - Build and support machine learning pipelines for research and analytics workloads - Architect scalable data lakes and cloud-native data platforms in AWS - Implement infrastructure using Infrastructure as Code (Terraform preferred) - Work with researchers and data scientists to support scientific and geospatial data workflows - Develop architecture diagrams, technical documentation, and deployment plans - Collaborate with cloud engineers, security teams, and program stakeholders in an Agile environment Required Experience - 4–8 years of experience building data platforms, data pipelines, or cloud data infrastructure - Hands-on experience with AWS data services - Experience building end-to-end data pipelines and analytics platforms - Experience developing API-driven data platforms - Experience with Infrastructure as Code (Terraform or CloudFormation) - Strong experience with Python or similar data engineering languages - Experience working in cloud environments supporting analytics or machine learning - Ability to work with both technical and non-technical stakeholders Nice to Have - Experience working with scientific, research, or geospatial datasets - Experience supporting federal or government environments - Experience with machine learning workflows - AWS certifications such as Solutions Architect or Data Specialty - Experience with serverless data processing (Lambda, event-driven pipelines) Location Remote (U.S.) Security Ability to obtain a security clearance. GAMA-1 also offers a variety of benefits, including health insurance coverage, life and disability insurance, 401(k) savings plan, training and career development opportunities, paid holidays and paid time off (PTO - to cover vacation, illness or disability, appointments, emergencies or other situations that require time off from work). For more information click here. ABOUT GAMA-1 GAMA-1 is a rapidly growing technology business that is based in Greenbelt, Maryland. GAMA-1 Technologies provides strategic information assurance, information security, and business enterprise and networking solutions to the Federal Government. Our success is based on the utilization of industry and agency standards, establishment of standardized processes, and IT Services expertise. At GAMA-1, we believe employees should grow, achieve, and develop just as the company grows, achieves, and develops. GAMA-1 is committed to providing our employees with opportunities for career advancement throughout their employment. For more information, visit www.gama1tech.com GAMA-1 is an Equal Opportunity Employer and all qualified applicants will receive consideration for employment without regard to: veteran status, uniformed servicemember status, race, color, religion, sex, sexual orientation, gender identity, age, pregnancy (including childbirth, lactation and related medical conditions), national origin or ancestry, citizenship or immigration status, physical or mental disability, genetic information (including testing and characteristics), domestic violence victims, political orientation, status as a smoker or tobacco user, hairstyle, use of a service animal, education status, familial status, HIV/AIDS status, height, weight, reproductive healthcare decisions or any other category protected by federal, state or local law.

United States
Job Closed
HHAeXchange logo

Data Engineer

HHAeXchange

Better Homecare, Better Health

Data Engineer91 days ago
OtherRemoteTeam 501-1,000Since 2008H1B Sponsor

• Design, build, and own scalable data platforms that power analytics, machine learning, regulatory reporting, and mission-critical operational systems. • Develop robust ELT workflows, data models, and distributed processing jobs using modern cloud data technologies. • Implement modern data engineering solutions using technologies such as Python, SQL, dbt, Airflow, Snowflake, Tableau, Power BI, AWS data services, containerization (Docker/Kubernetes), and related cloud-native tools. • Leverage best practices to ensure data quality, integrity, lineage, governance, performance, and reliability across all pipelines, data platforms, and analytics applications. • Establish and maintain SLAs, monitoring, and alerting for mission-critical data services with observability design and implementation. • Identify and resolve performance bottlenecks to ensure scalable and cost-efficient data processing.

United States
$105K - $115K / year
Stellus Rx logo

Senior Data Architect

Stellus Rx

Trusted, pharmacist-led health support in every moment that matters.

Data Engineer91 days ago
Full TimeRemoteTeam 201-500Since 2022H1B No Sponsor

• Define and maintain enterprise data architecture standards across structured, semi-structured, and unstructured data domains — with deliberate design for AI/ML workloads, including feature stores, vector databases, and embedding pipelines. • Use AI-assisted modeling tools to accelerate data model design, evaluate architectural trade-offs, and validate designs against business requirements before committing to implementation. • Design and govern the organization's cloud data lake, data warehouse, and lakehouse architectures on AWS — ensuring they are optimized for both analytical and AI/ML consumption patterns. • Establish data ontology, taxonomy, and semantic layer standards that enable AI systems to reason over organizational data accurately and consistently. • Evaluate emerging data architecture patterns — including retrieval-augmented generation (RAG), real-time feature serving, and vector search — and build a roadmap for their adoption across Stellus Rx. • Design scalable data models and ELT/ETL pipeline architectures that support both traditional analytics and AI/ML model training and inference workloads. • Use AI code generation tools to accelerate the authoring and validation of data models, transformation logic, and pipeline configurations — replacing manual, repetitive design work with intelligent, AI-assisted development. • Define standards for data partitioning, indexing, caching, and storage optimization; use AI-driven performance analysis to continuously validate and improve architectural decisions. • Partner with Data Engineers to translate architectural blueprints into production-ready pipelines, providing hands-on guidance and AI-augmented design reviews. • Define and enforce data governance frameworks, data quality standards, and data contracts across the enterprise — using AI-powered data observability tools to automate quality monitoring and surface issues proactively rather than through manual review. • Ensure data architecture meets compliance requirements relevant to healthcare (HIPAA, SOC 2, NIST); use AI-assisted compliance tooling to continuously monitor for policy drift and streamline audit evidence generation. • Develop and maintain a master data management (MDM) strategy that ensures consistency, accuracy, and trustworthiness of critical data assets across systems. • Champion data privacy and security principles in architectural design, including data lineage tracking, access controls, and anonymization strategies for sensitive healthcare data. • Design data infrastructure that serves as the foundation for AI/ML initiatives — ensuring data is accessible, well-labeled, versioned, and structured to support model training, validation, and ongoing inference at scale. • Collaborate with data scientists and ML engineers to understand modeling requirements and translate them into data architecture decisions that reduce friction in the AI development lifecycle. • Use AI-assisted analysis to identify high-value data assets that are underutilized, and develop strategies to unlock their potential for analytics and AI-driven decision-making. • Partner with Business Intelligence and Product teams to ensure the data architecture supports self-service analytics, real-time dashboards, and AI-powered reporting capabilities. • Define and maintain data architecture standards, patterns, and best practices across the organization; use AI tools to generate, review, and keep documentation current with minimal manual overhead. • Mentor Data Engineers and Analysts, guiding them in applying architectural standards and AI-augmented data development practices. • Communicate architectural decisions, trade-offs, and roadmap recommendations clearly to both technical teams and executive leadership. • Stay current on emerging data technologies, AI/ML data infrastructure trends, and industry best practices; provide recommendations on adoption timing and implementation approach.

Peru
$52K / year
Full TimeRemoteTeam 10,001+Since 1978H1B No Sponsor

• Design and maintain data ingestion and transformation pipelines. • Integrate multiple data sources in various formats. • Ensure data quality, integrity, and reliability. • Optimize performance for high-volume data extraction processes. • Collaborate with technical teams using event-driven architectures.

Mexico
Job Closed