A Digital Automation Company - Enabling Frictionless Transactions with Digital Engagement & Intelligent Automation
Senior Data Engineer
Location
District Of Columbia + 1 moreAll locations: District Of Columbia | Washington
Posted
3 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Node.Digital
• Provide authoritative expertise on data engineering methods and best practices, including code first development approaches and modern pipeline design patterns. • Design, implement, and maintain the data architecture that supports products and end users, with all assets managed under source control. • Design, implement, and maintain ELT and ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning, using both SDK V1 and SDK V2. • Migrate source data identified by SBA OIG into Azure Data Lake Storage. • Normalize entity attributes such as addresses, phone numbers, and other common fields. • Review, maintain, and improve existing architecture and pipelines, including periodic audits addressing bottlenecks, deprecated dependencies, and architecture drift. • Establish quality controls across all pipelines and introduce error handling, logging mechanisms, and validation checks. • Incorporate source control across all pipelines and analytics codebases so code can evolve iteratively without destabilizing the architecture. • Optimize ingestion, processing, and storage across a wide variety of datasets and data types, including modern columnar formats such as Parquet. • Develop self service capabilities that let SBA OIG analysts query and export data for investigations and audits. • Author robust standard operating procedures governing the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in the Azure environment. • Produce detailed documentation of the data architecture, including data dictionaries, entity relationship diagrams, and pipeline process maps. • Maintain and expand the environment with additional datasets and services on request, following a defined intake and testing process before production deployment. • Stay current with emerging AI tooling relevant to data engineering and contribute to exploratory work evaluating automation and language model assisted capabilities.
Job Requirements
- Bachelor's degree in data engineering, computer science, data science, machine learning, mathematics, or a related field. Alternatively, five years of applied work experience in any of the same fields.
- 5 years - Maintaining SQL databases and conducting advanced operations in SQL and T-SQL.
- 5 years - Designing, implementing, and maintaining ELT and ETL processes in cloud based data analytics environments.
- 3 years - Working in Azure Synapse and Azure Machine Learning with the modern data stack. Certifications preferred, DP-203 or equivalent.
- 3 years - Manipulating data in Python. Pandas is required. PySpark and Polars preferred. Experience developing reusable, modular code preferred.
- DP-203, Microsoft Certified Azure Data Engineer Associate, or an equivalent current certification preferred.
- Implementing pipelines and infrastructure using code first approaches: Python SDK, CLI, REST APIs, or infrastructure as code tooling such as Terraform or Bicep.
- Implementing source control and continuous integration and delivery workflows for data assets.
- Demonstrated familiarity with AI coding assistants and large language model integration patterns.
- PySpark or Polars at production scale.
- Entity resolution and attribute normalization across records with inconsistent addresses, names, and identifiers.
- Building self service analytic access for non-engineering users.
Benefits
- Medical
- Dental
- Vision
- Basic Life
- Health Saving Account
- 401K matching
- Three weeks of PTO/Sick
- 11 Paid Holidays
- Pre-Approved Online Training
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Enterprise Data Engineer
BJC HealthCareBJC HealthCare is one of the largest healthcare organizations in the U.S. focused on delivering "the world's best medicine," made better by its 30,000+ clinical
Role Description The Data Engineer is responsible for transforming data into a format that can be easily analyzed. This position is also responsible for developing, maintaining, and testing infrastructures for data generation. This position works closely with data scientists and is largely accountable for architecture and creation of solutions that support the work performed by the data scientists. - Assembles large, complex data sets that meet functional / non-functional business requirements. - Optimizes extraction, transformation, and loading (ETL) of data from a wide variety of data sources. - Creates data tools for analytics and data scientists that assist in the building and optimization of reporting and analytic products. - Works with data and analytics experts towards building greater functionality within our data systems. Qualifications - Bachelor's Degree - 5-10 years of experience Requirements - Master's Degree (preferred) Benefits - Comprehensive medical, dental, vision, life insurance, and legal services available first day of the month after hire date - Disability insurance* paid for by BJC - Annual 4% BJC Automatic Retirement Contribution - 401(k) plan with BJC match - Tuition Assistance available on first day - BJC Institute for Learning and Development - Health Care and Dependent Care Flexible Spending Accounts - Paid Time Off benefit combines vacation, sick days, holidays and personal time - Adoption assistance
Senior Data Engineer
Node.DigitalA Digital Automation Company - Enabling Frictionless Transactions with Digital Engagement & Intelligent Automation
Role Description - Provide authoritative expertise on data engineering methods and best practices, including code first development approaches and modern pipeline design patterns. - Design, implement, and maintain the data architecture that supports products and end users, with all assets managed under source control. - Design, implement, and maintain ELT and ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning, using both SDK V1 and SDK V2. - Migrate source data identified by SBA OIG into Azure Data Lake Storage. - Normalize entity attributes such as addresses, phone numbers, and other common fields. - Review, maintain, and improve existing architecture and pipelines, including periodic audits addressing bottlenecks, deprecated dependencies, and architecture drift. - Establish quality controls across all pipelines and introduce error handling, logging mechanisms, and validation checks. - Incorporate source control across all pipelines and analytics codebases so code can evolve iteratively without destabilizing the architecture. - Optimize ingestion, processing, and storage across a wide variety of datasets and data types, including modern columnar formats such as Parquet. - Develop self service capabilities that let SBA OIG analysts query and export data for investigations and audits. - Author robust standard operating procedures governing the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in the Azure environment. - Produce detailed documentation of the data architecture, including data dictionaries, entity relationship diagrams, and pipeline process maps. - Maintain and expand the environment with additional datasets and services on request, following a defined intake and testing process before production deployment. - Stay current with emerging AI tooling relevant to data engineering and contribute to exploratory work evaluating automation and language model assisted capabilities. Qualifications - Bachelor's degree in data engineering, computer science, data science, machine learning, mathematics, or a related field. Alternatively, five years of applied work experience in any of the same fields. - 5 years - Maintaining SQL databases and conducting advanced operations in SQL and T-SQL. - 5 years - Designing, implementing, and maintaining ELT and ETL processes in cloud based data analytics environments. - 3 years - Working in Azure Synapse and Azure Machine Learning with the modern data stack. Certifications preferred, DP-203 or equivalent. - 3 years - Manipulating data in Python. Pandas is required. PySpark and Polars preferred. Experience developing reusable, modular code preferred. Requirements - DP-203, Microsoft Certified Azure Data Engineer Associate, or an equivalent current certification. - Implementing pipelines and infrastructure using code first approaches: Python SDK, CLI, REST APIs, or infrastructure as code tooling such as Terraform or Bicep. - Implementing source control and continuous integration and delivery workflows for data assets. - Demonstrated familiarity with AI coding assistants and large language model integration patterns. - PySpark or Polars at production scale. - Entity resolution and attribute normalization across records with inconsistent addresses, names, and identifiers. - Building self service analytic access for non engineering users. Benefits - Medical - Dental - Vision - Basic Life - Health Saving Account - 401K matching - Three weeks of PTO/Sick - 11 Paid Holidays - Pre-Approved Online Training
DATA QUALITY SPECIALIST
AuthenticomAuthenticom is a remote-first company with headquarters in La Crosse, Wisconsin, and team members across the US and Canada. We believe in the importance of cultivating teams with diverse backgrounds and offering equal opportunities to all. We strive to create a welcoming, inclusive environment where every team member feels valued, and diversity is celebrated.
Role Description Authenticom is seeking a Data Quality Specialist responsible for verifying the integrity and completeness of customer orders and data files. This role serves as a key checkpoint in our data delivery process, ensuring accuracy, identifying issues, and providing actionable feedback to Client Services associates for resolution prior to vendor delivery. The ideal candidate possesses strong analytical skills, a keen eye for detail, and a commitment to maintaining high-quality standards. You will work closely with internal teams, including Implementation and Client Services, to validate data, track common errors, and continuously improve quality control processes. What You'll Do - Quality Control & Data Validation: - Review new orders and associated data files for completeness and accuracy prior to delivering data to vendors. - Follow established Quality guidelines to determine if data and order configurations are correct. - Inspect case comments to ensure required fields have been updated and verify that comments are entered in Salesforce, DealerVault, and QC reporting systems as required. - Complete follow-up checks on corrected data or order issues to ensure required changes have been properly executed. - Error Resolution & Communication: - Provide clear, constructive feedback on orders and data files requiring corrections or additional information. - Escalate complex issues to Implementation Leads or Leadership when necessary while maintaining case ownership. - Track common issues and recurring errors to support coaching, training, and operational improvements. - Operational & Process Support: - Maintain a deep understanding of internal Implementation services processes and software tools required to complete customer orders. - Review case comments to ensure they are thorough, clear, concise, and consistent across systems. - Participate in the evaluation and testing of tools and techniques aimed at driving overall quality improvements. - Assist with verifying and maintaining the accuracy of account information within Salesforce CRM. - Serve as back-up support for the Integration Specialist and Reconnection Specialist roles as needed. - Internal Collaboration: - Establish and maintain strong working relationships with peers across departments. - Work closely with Client Services, Implementations, and Technical Support to resolve data discrepancies. - Perform other related duties and special projects as assigned. Qualifications - Minimum 1 year of experience in a data quality, data validation, quality control, or related data-focused role. - Minimum 1 year of successful remote work experience. - High school diploma or equivalent. - High standards for accuracy and strong attention to detail. - Strong written and verbal communication skills, including high command of professional messaging. - Analytical thinking and problem-solving skills with the ability to work effectively in a fast-paced environment. - Proven ability to multitask, prioritize work, and maintain confidentiality. Requirements - Experience with Salesforce CRM, DealerVault, or internal ticketing and reporting platforms. - Prior experience in software support, data processing, or automotive retail technology platforms. - Equivalent combination of education, training, and experience. Benefits - Medical, dental, and vision insurance - Paid holidays - Paid time off - Volunteer time off - Wellness activities - Birthday time off - Remote-first work environment - Professional development opportunities Company Description At Authenticom, we are the business behind the business in the automotive industry. We help dealerships and technology providers securely connect and exchange critical business data that powers the automotive ecosystem. We're a remote-first company headquartered in La Crosse, Wisconsin, with team members located throughout the United States and Canada. We value collaboration, accountability, continuous learning, and delivering exceptional service to our customers.
Principal Engineer – CPS Data Validation Platform
NielsenIQNielsenIQ is an industry leader in data analytics and global measurement. The company delivers information to partners, retailers, and manufacturers through pow
Role Description We are seeking a highly experienced Principal Engineer to provide technical leadership and operational ownership for the CPS DataValidation platform supporting NielsenIQ's OmniShopper and Homescan solutions. This role serves as the primary technical point of contact for production support, platform reliability, and strategic OmniShopper initiatives. The successful candidate will drive modernization across a hybrid ecosystem spanning on-premise and Azure platforms while partnering closely with Data Science, PMO, Operations, Product Leadership, and Engineering teams located primarily in the US Eastern Time zone. Job Description Platform Ownership & Production Leadership - Own the overall health, reliability, performance, and scalability of the DataValidation platform. - Serve as the primary engineering escalation point for production incidents and business-critical issues. - Lead incident management, root cause analysis, and post-incident remediation activities. - Drive improvements in observability, monitoring, alerting, and operational excellence. - Establish and maintain operational KPIs, SLAs, and platform health metrics. Technical Leadership - Define and execute the technical strategy and roadmap for the DataValidation platform. - Participate in architecture decisions across both on-premise and Azure-based systems. - Ensure solutions are scalable, maintainable, secure, and cost-effective. - Review system designs and provide technical guidance to development teams. - Balance technical debt reduction with business delivery commitments. - Act as the engineering lead for strategic OmniShopper initiatives. - Collaborate with Product Leadership and PMO to define technical solutions supporting business objectives. - Coordinate execution across multiple engineering and business teams. - Ensure DataValidation capabilities align with future OmniShopper and Homescan requirements. Cross-Functional Collaboration - Partner closely with Data Science teams. - PMO. - Operations teams. - Product Leadership. - Global Engineering teams. - Translating business requirements into technical solutions. - Managing technical risks and dependencies. - Driving prioritization discussions. - Communicating platform health and roadmap updates to stakeholders. - Influencing decision-making without direct authority. Qualifications - 10+ years of software engineering experience. - 5+ years in Staff Engineer, Principal Engineer, Architect, or equivalent technical leadership roles. - Experience owning mission-critical production systems. - Experience supporting enterprise-scale data platforms. - Proven success influencing multiple organizations and stakeholders. Technical Skills - Strong hands-on experience with Python. - Oracle Database. - Linux environments. - SQL performance tuning. - Azure Data Services. - PySpark. - Parquet datasets. - ETL and data engineering frameworks. - Data quality and validation platforms. Leadership Skills - Strong stakeholder management skills. - Excellent communication with technical and non-technical audiences. - Ability to work effectively across global teams. - Strong ownership mindset and bias for action. - Experience leading complex incident resolution efforts. Preferred Qualifications - Retail measurement, consumer panel, or syndicated data domain experience. - Experience working with OmniShopper, Homescan, or similar consumer analytics platforms. - Experience working with Data Science and advanced analytics teams. - Knowledge of modern data quality, governance, and observability frameworks. Benefits - Flexible working environment. - Volunteer time off. - LinkedIn Learning. - Employee-Assistance-Program (EAP).


