1900 Campus Commons Drive Reston, VA 20191
Data Engineer
Location
United States
Posted
118 days ago
Salary
0
Seniority
Lead
Job Description
Data Engineer
Cybermedia Technologies, LLC (CTEC)
• Design, develop, and maintain scalable ETL pipelines and data workflows. • Build, optimize, and maintain data processing solutions using Azure Databricks. • Support phased data migration from legacy databases and ETL tools. • Implement layered lakehouse data architectures in Databricks. • Develop data processing notebooks, workflows, and distributed data transformations using Python and PySpark. • Develop data validation, reconciliation, and testing processes. • Integrate Databricks data platforms with analytics and reporting tools. • Support data governance initiatives including metadata management. • Maintain source control and CI/CD pipelines for Databricks workflows. • Provide ongoing support for Databricks workflows and troubleshoot complex issues. • Provide guidance to junior data engineers and contribute to documentation and team enablement.
Job Requirements
- At least seven (7–9+) years of experience in data engineering, ETL development, or large-scale data integration environments.
- Strong experience designing and developing ETL pipelines and data transformations in Azure Databricks environments.
- Strong proficiency in SQL and Python, with hands-on experience using PySpark for distributed data processing.
- Experience working with cloud-based data platforms, data lakes, and lakehouse environments, preferably on Microsoft Azure.
- Experience implementing layered lakehouse data architectures (bronze, silver, gold) for enterprise analytics.
- Familiarity with Spark-based big data processing frameworks.
- Experience supporting data migration from legacy databases and ETL tools to Databricks-based platforms.
- Experience integrating Databricks platforms with business intelligence and reporting tools such as Power BI.
- Familiarity with data governance, metadata management, and data security best practices.
- Experience with source control and CI/CD pipelines for data engineering and Databricks workflows.
- Working knowledge of SDLC and Agile delivery methodologies.
- Excellent organizational, communication, and collaboration skills.
Benefits
- Paid vacation & Sick leave
- Health insurance coverage
- Career training
- Performance bonus programs
- 401K contribution & Employer Match
- 11 Federal Holidays
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Data Engineering Internship
AffirmedRx, a Public Benefit CorporationPatients over Profits - We are dedicated to designing a PBM solution that is ideal for your members and for you.
• Develop and maintain ETL/ELT pipelines to ingest, transform, and load data from various sources including pharmacy claims, eligibility files, and client data feeds • Support automation of file processing workflows, including format standardization and validation • Create and maintain documentation for data models, pipeline architecture, and transformation logic • Implement data validation and quality checks to ensure accuracy and completeness of source data • Troubleshoot and resolve data pipeline failures and anomalies • Support the design and maintenance of dimensional models in our cloud data warehouse • Partner with analytics and reporting teams to understand data requirements and deliver reliable, well-structured datasets • Ensure adherence to compliance and data security standards when handling sensitive information (PHI and PII), including completion of HIPAA training • Participate in code reviews and follow version control best practices
• Responsible for designing, building, and maintaining data pipelines, data integration processes, and data infrastructure. • Collaborate closely with data scientists, analysts, and other stakeholders to ensure efficient data flow and support data-driven decision making across the organization.
Data Engineer – Messaging
NetflixDescribed as the world's top internet television network, Netflix is a publicly-traded entertainment company offering video-on-demand and streaming media. As an
• Work collaboratively with data engineers and other partners in the messaging space • Build reliable, scalable data pipelines using Apache Spark/Flink • Power analytic dashboards, custom viz applications, A/B experiments, and algo features • See the problem through a platform lens • Build scalable, flexible data products
Snowflake Data Engineer
AffinitivFully integrated data-driven solutions for the automotive customer lifecycle
• Monitor and support ETL processes moving data from on-premise or hosted servers into Snowflake • Troubleshoot data pipeline issues, including failed loads, data discrepancies, and performance bottlenecks • Design and maintain aggregate and reporting tables optimized for Tableau and Power BI dashboards • Create and maintain Snowflake stored procedures (GET procs) used by APIs to support Angular reporting applications • Optimize Snowflake performance and cost, including warehouse usage, query tuning, and table design • Support ad-hoc data requests from product, engineering, and analytics teams • Integrate third-party vendor data by reviewing technical documentation, designing ingestion processes, and validating data accuracy • Collaborate with application engineers to ensure data structures align with frontend and API requirements • Maintain clear documentation for data pipelines, stored procedures, and integrations
