Data-driven consulting and technology services
Data Engineer
Location
Virginia
Posted
211 days ago
Salary
0
Seniority
Senior
Job Description
Data Engineer
Analytica
• Translate business requirements to data engineering solutions to support an enterprise scale AWS and/or Databricks based data environments. • Support designing, maintaining, automating and optimizing ETL operations ensuring data quality and efficient data management. • Design, build, and optimize scalable data solutions using a Medallion Architecture. • Manage ingestion routines for processing multi-terabyte datasets efficiently for multiple projects simultaneously. • Integrate data from various structured and unstructured sources to enable high-quality business insights. • Implement effective data management strategies to ensure data integrity, availability, and accessibility. • Identify opportunities for cost optimization in data storage, processing, and analytics operations. • Monitor and support user requests, addressing platform or performance issues, cluster stability, Spark optimization, and configuration management. • Collaborate with the team to enable advanced AI-driven analytics and data science workflows.
Job Requirements
- 5+ years of hands-on Data Integration experience creating and maintaining efficient scripts/data pipelines to clean, transform and ingest data from a variety of formats into database tables, data warehouses or data lake repositories.
- 3+ years of experience using SQL and Python to manipulate, profile and build data products
- Strong Experience building data pipelines using Databricks and/or AWS serverless components; using AWS Glue to build, maintain and monitor ETL jobs; using Python to implement ETL scripts and AWS Secrets Manager to manage credentials.
- Databricks and/or AWS Data Engineer Certification is strongly preferred.
- Proficient written, verbal, and client communication and presentation skills
- Must be US Citizen and be able to obtain and maintain a Public Trust security clearance
Benefits
- Our compensation packages include not only base salary but also a range of incentive bonus programs to reward outstanding performance and contributions
