Job Closed
This listing is no longer active.
Senior Data Engineer
Location
United States
Posted
149 days ago
Salary
0
Seniority
Senior
Job Description
Senior Data Engineer
Knowledge Services
• Develop efficient and scalable data extraction methodologies to retrieve data from diverse sources, such as databases, APIs, web scraping, flat files, and streaming platforms. • Design and implement robust data loading processes to efficiently ingest and integrate data into the latest data warehousing technology, ensuring data quality and consistency. • Develop and maintain staging processes to facilitate the organization and transformation of raw data into structured formats, preparing it for downstream analysis and reporting. • Implement data quality checks and validation processes to identify and address data anomalies, inconsistencies, and integrity issues. • Identify and resolve performance bottlenecks in data extraction and loading processes, optimizing overall system performance and data availability. • Ensure adherence to data security and privacy standards throughout the data extraction and warehousing processes, implementing appropriate access controls and encryption mechanisms. • Create and maintain comprehensive documentation of data extraction and warehousing processes, including data flow diagrams, data dictionaries, and process workflows. • Mentor and support junior data engineers, providing guidance on best practices, technical design, and professional development to elevate overall team capability and performance. • Collaborate with cross-functional teams, including data scientists, data analysts, software engineers, and business stakeholders, to understand their data requirements and provide efficient data engineering solutions. • Stay updated with the latest advancements in data engineering, data warehousing, and cloud technologies, and proactively propose innovative solutions to enhance data extraction and warehousing capabilities.
Job Requirements
- Minimum of 5 years’ experience in data engineering, with a strong focus on data extraction and cloud-based warehousing; a combination of years of experience and relevant advanced technology proficiency will also be considered.
- Proficiency with Snowflake and data integration tools like Fivetran.
- Advanced SQL skills and experience with ETL/ELT frameworks.
- Experience with scripting languages such as Python for data processing and automation.
- Solid understanding of data modeling and relational database design.
- Strong communication skills and the ability to collaborate with technical and non-technical stakeholders.
- Strong analytical and problem-solving skills, with the ability to identify and resolve complex data engineering challenges.
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Design, build, and maintain scalable data pipelines and architectures • Collaborate with self-organized teams to develop tailor-made technological solutions • Engage in continuous learning and promote a people-first culture
• Designing, building, and maintaining scalable data pipelines and architectures • Collaborating with cross-functional teams to deliver high-quality data solutions
Senior Data Engineer, Data Platform Operations
Scratch FinancialScratch Financial is the world's simplest patient financing solution.
• Define partner onboarding and clean room architecture patterns across Snowflake, LiveRamp, and Databricks that are secure, scalable, and repeatable. • Configure and manage partner-specific clean room environments; deploy and manage Python-based libraries within the platform ecosystem. • Establish and maintain MLOps practices, including model serving, monitoring, and pipeline orchestration for AI/ML features deployed within the platform ecosystem. • Own design and enforcement of granular RBAC policies and least-privilege service accounts. • Serve as the technical lead for onboarding new partners, implementing privacy-preserving controls (e.g., aggregation thresholds and anonymization techniques). • Design, build, and operate scalable ELT pipelines using Snowpark and/or PySpark and advanced SQL to provision Gold datasets. • Implement and evolve identity resolution logic mapping internal data to 3P identifiers (including LUIDs, RampIDs, TransUnion IDs), ensuring privacy-safe practices. • Design and operate scalable data architectures across Snowflake and Databricks supporting batch and near real-time processing patterns. • Build robust automated checks (e.g., Great Expectations or custom SQL assertions) and define quality standards to detect schema drift, null rate spikes, and volume anomalies. • Lead performance optimization across platforms (query tuning, caching, incremental processing) and define and implement query tagging and chargeback models for accurate cost attribution. • Establish monitoring, alerting, runbooks, and standard operating procedures to improve platform reliability and reduce incident time-to-resolution. • Validate that output data adheres to privacy and business requirements, and define test strategies for partner-facing releases. • Serve as the escalation point for diagnosing connection failures, data discrepancies, or latency issues with partner technical teams. • Design and build internal AI agents (using frameworks like LangChain, Snowflake Cortex) and mentor other engineers through code reviews, design discussions, and operational best practices.
Senior Data Engineer
Roo VeterinaryRoo Veterinary is a service platform that gives veterinarians, hospitals, and vet techs complete control over where and how they work. The company aims to solve
• Design, develop, and maintain reliable end-to-end data pipelines (both batch and streaming) that connect internal and external systems in ways that best support marketplace growth, customer experience, and operational efficiency. • Contribute to the performance, scalability, and reliability of our entire data ecosystem. • Work with analysts and other data stakeholders to engineer data structures and orchestrate workflows that encode core business logic. • Implement observability, testing, monitoring, validation, and documentation to ensure accuracy, stability, and consistency throughout the data stack. • Join cross-functional squads and tiger teams to rapidly translate evolving data needs into scalable and extensible data models, metrics, and analytical frameworks. • Mentor data stakeholders throughout the organization, share best practices, and meaningfully contribute to architectural and tooling decisions as the data stack evolves.



