Mid Data Engineer

Location

United States

Posted

3 days ago

Salary

$130K - $155K / year

Seniority

Mid Level

Job Description

Mid Data Engineer

Forge Group, LLC

Role Description Forge requires a Mid Data Engineer to support legacy-to-modern data transformation in a secure AWS environment for a DoW customer. The role will develop batch and event-driven pipelines, automate data quality and testing, integrate with application services, and provide observable, recoverable, high-quality data flows across mission and external interfaces. Key Responsibilities - Build secure Python and AWS ETL/ELT pipelines for ingestion, transformation, reconciliation, and delivery. - Develop and evolve relational data models, schemas, indexes, constraints, views, and access patterns for MariaDB, PostgreSQL, or comparable platforms. - Develop data workflows and interfaces using Python on AWS Lambda and PySpark for event-driven, batch, and distributed transformation workloads. - Create automated data-quality checks for accuracy, completeness, consistency, timeliness, uniqueness, and business-rule conformance. - Implement source-to-target mapping, lineage, auditability, restartability, exception handling, and controlled replay. - Develop parity tests that compare legacy and modern processing outcomes and document the disposition of intentional differences. - Tune SQL and pipeline performance for high-volume batch and near-real-time workloads while protecting transactional integrity. - Implement monitoring, logging, alerting, and operational dashboards for pipeline health, latency, failures, and data quality. - Automate CI/CD, version-controlled data changes, deployments, rollback, and operational recovery controls. - Collaborate with architects, mission SMEs, Appian developers, testers, security personnel, and interface partners. Qualifications - Ability to think strategically, act tactically, and demonstrate strong analytical and critical-thinking skills. - Build strong cross-group working relationships and demonstrate exceptional organizational skills and attention to detail. - Thrive and succeed in an entrepreneurial environment and not be hindered by ambiguity or competing priorities. - Self-managing candidates who enjoy working collaboratively in a fast-paced environment and with dynamic teams. Requirements - U.S. Citizen (Authorization to Work in the U.S. will not suffice); previous professional experience supporting the U.S. Federal Government, either as a federal employee or contractor, is required. - 4+ years of professional experience in data engineering, database development, or data-platform delivery. - Bachelor's degree in Computer Science, Information Systems, Data Engineering, or equivalent, OR 4 additional years of relevant professional experience in lieu of a degree. - Advanced Python software-engineering skills and experience building AWS Lambda functions and PySpark data-transformation pipelines. - Experience building, testing, and operating production ETL/ELT pipelines with automated data-quality controls. - Experience with data modeling, schema migration, source-to-target mapping, lineage, reconciliation, and data-quality automation. - Experience integrating data platforms with REST APIs, application services, file exchanges, and event-driven interfaces. - Experience with Git, CI/CD, automated testing, logging, monitoring, performance tuning, and production support. - Active CompTIA Security+ or equivalent DoW-approved baseline cybersecurity certification, or ability to obtain within the first 30 days of starting. - Active Tier 2 background investigation or higher, completed or favorably adjudicated within the previous 18 months. Highly Desired Qualifications - Experience using Palantir Foundry for data integration, transformation, lineage, governance, and operational workflows. - Active Secret security clearance preferred. - Previous professional experience supporting a DoW organization, mission, or customer is strongly preferred; experience in modernizing COBOL flat files or legacy relational data into a modern relational architecture. - Experience integrating with Appian, Python microservices, financial transactions, logistics workflows, or high-volume external interfaces. - Experience serving as a technical team lead, mentoring junior engineers, or assisting teammates across delivery tasks. - Experience with BI, analytics, archival, records-retention, or NARA-aligned data lifecycle requirements. Benefits - Complete Flextime - 401k With Employer Matching - Healthcare, Including Medical, Dental, and Vision - Health Savings Account (HSA) And Pre-Tax Premium Options - Supplementary healthcare and family support - Extended Short-Term Disability and Long-Term Disability - Healthcare Insurance Deductible Paydown - Health and Wellness Programs - Tuition Reimbursement, Student Loan Repayment, and Education & Training Stipends - Cell Phone / Internet Stipends - College Saving Plans with Employer Contributions - Alternative Work Locations and Tele-Commuting - Employee Referral Awards - Retention, Signing & Performance Bonuses - Commuter Benefits - Paid Sabbatical

Related Categories

Related Job Pages

More Data Engineer Jobs

Weekday (YC W21) logo

Senior Data Engineer, Microsoft Fabric Engineer

Weekday (YC W21)

We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent

Data Engineer3 days ago
Full TimeRemoteTeam 11-50Since 2021H1B No Sponsor

• Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. • Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. • Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. • Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. • Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. • Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. • Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. • Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. • Implement best practices for data engineering, governance, documentation, testing, and operational support. • Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes.

India
Rs1,100K - Rs5,000K / year
Weekday (YC W21) logo

Senior Data Engineer / Microsoft Fabric Engineer

Weekday (YC W21)

We are a Y-Combinator-backed startup building your AI-powered Recruiter Agent

Data Engineer3 days ago
Full TimeRemoteTeam 11-50Since 2021H1B No Sponsor

Role Description We are looking for an experienced Data Engineer to design, develop, and optimize scalable data platforms and modern cloud-based data solutions. This role is ideal for professionals with strong expertise in Microsoft Azure Fabric, Databricks, Azure data services, and enterprise data engineering, along with a passion for building reliable data pipelines and leveraging AI-assisted development to improve engineering productivity. As a Data Engineer, you will take ownership of the complete data engineering lifecycle—from solution design and implementation to production support and continuous optimization. You will collaborate closely with business stakeholders, architects, analysts, and development teams to transform complex data requirements into scalable, high-performance data solutions that enable analytics and informed business decision-making. Key Responsibilities - Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services. - Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments. - Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions. - Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability. - Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability. - Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality. - Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization. - Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions. - Implement best practices for data engineering, governance, documentation, testing, and operational support. - Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes. Qualifications - 5+ years of hands-on experience as a Data Engineer with expertise in enterprise-scale data engineering solutions. - Strong experience with Microsoft Azure Fabric, Databricks, and Azure data services in production environments. - Solid understanding of SQL, data modeling, data warehousing concepts, and cloud-based data platforms. - Proven experience designing and implementing scalable ETL/ELT pipelines and enterprise data integration solutions. - Proficiency in Python, Apache Spark, or similar technologies for large-scale data processing. - Experience optimizing cloud-based data workflows for performance, scalability, reliability, and cost efficiency. - Strong analytical and problem-solving skills with the ability to troubleshoot complex production issues effectively. - Comfortable using AI-assisted development tools and automation techniques to improve engineering productivity and solution quality. - Excellent communication and stakeholder management skills, with the ability to collaborate effectively across global, cross-functional teams. - A proactive, ownership-driven mindset with the flexibility to adapt to evolving priorities while consistently delivering high-quality, business-critical data solutions.

India
₹1,100K - ₹5,000K / year

Role Description We’re looking for a Data Engineer to join our Data & Analytics team and take ownership of the data infrastructure that powers internal analytics, reporting, and operational insights. This is a hands‑on role where you’ll manage and evolve our ELT pipelines, Snowflake environment, and Python‑based integrations while partnering closely with analysts and systems administrators who rely on high‑quality, reliable data. - Own the day‑to‑day operation, maintenance, and enhancement of ELT pipelines and the Snowflake data warehouse - Troubleshoot and resolve pipeline failures, data quality issues, and performance bottlenecks - Design, build, and document new data models and transformations as analytical needs evolve - Ensure data reliability, consistency, and availability for downstream consumers - Support and extend API integration workflows built with Boomi and Python - Maintain and contribute to custom Python applications used across the team - Collaborate with internal stakeholders to scope and deliver new data integration projects - Participate in evaluating modern data tooling as we consider improvements to our architecture - Document systems, processes, and decisions to build long‑term institutional knowledge Qualifications - Bachelor’s degree in Computer Science, Information Systems, Statistics, or a related field (preferred) - 3–5 years of hands‑on experience in a data engineering role - Strong SQL skills and a solid understanding of data modeling for analytics - Practical experience with Snowflake or a similar cloud data warehouse - Familiarity with ELT/ETL concepts and tools (WhereScape RED, Matillion, Airbyte, dbt, etc.) - Working knowledge of Python for data processing and scripting - Experience with REST APIs and integrating external data sources - Comfort digging into unfamiliar systems to diagnose root‑cause issues - Experience using GitHub for version control Nice to Have - Exposure to enterprise integration platforms (Boomi or similar iPaaS tools) - Experience with Oracle EBS or Salesforce - Familiarity with data pipeline orchestration concepts - Experience partnering closely with BI or analytics teams

United States
Satalia logo

Data Engineer – Mid Level

Satalia

We use AI to solve exponentially hard efficiency problems.

Data Engineer3 days ago
Full TimeRemoteTeam 51-200Since 2010H1B No Sponsor

• Design, build, and maintain data pipelines that serve models, data, and AI workflows to internal and client-facing applications. • Work across database types — relational, vector, and graph — to model and store data appropriately for each access pattern, in partnership with data scientists. • Build and maintain dbt models — writing transformation logic, tests, and documentation that ensure data quality and traceability. • Operate what you build : instrument pipelines with logging, metrics, and tracing, and help diagnose and resolve production data issues. • Write clean, tested, production-quality code and contribute to CI/CD pipelines and infrastructure-as-code. • Participate in code reviews, design discussions, and retrospectives.

Greece