Job Closed
This listing is no longer active.
Scratch Financial is the world's simplest patient financing solution.
Senior Data Engineer, Engineering & Operations
Location
New York
Posted
152 days ago
Salary
$140K - $180K / year
Seniority
Senior
Job Description
Senior Data Engineer, Engineering & Operations
Scratch Financial
• Define partner onboarding and clean room architecture patterns across Snowflake, LiveRamp, and Databricks that are secure, scalable, and repeatable • Configure and manage partner-specific clean room environments; deploy and manage Python-based libraries within the platform ecosystem • Establish and maintain MLOps practices, including model serving, monitoring, and pipeline orchestration for AI/ML features deployed within the platform ecosystem • Own design and enforcement of granular RBAC policies and least-privilege service accounts • Serve as the technical lead for onboarding new partners, implementing privacy-preserving controls (e.g., aggregation thresholds and anonymization techniques) • Design, build, and operate scalable ELT pipelines using Snowpark and/or PySpark and advanced SQL to provision Gold datasets • Implement and evolve identity resolution logic mapping internal data to 3P identifiers (including LUIDs, RampIDs, TransUnion IDs), ensuring privacy-safe practices • Design and operate scalable data architectures across Snowflake and Databricks supporting batch and near real-time processing patterns • Build robust automated checks (e.g., Great Expectations or custom SQL assertions) and define quality standards to detect schema drift, null rate spikes, and volume anomalies • Lead performance optimization across platforms (query tuning, caching, incremental processing) and define and implement query tagging and chargeback models for accurate cost attribution • Establish monitoring, alerting, runbooks, and standard operating procedures to improve platform reliability and reduce incident time-to-resolution • Validate that output data adheres to privacy and business requirements, and define test strategies for partner-facing releases • Serve as the escalation point for diagnosing connection failures, data discrepancies, or latency issues with partner technical teams • Design and build internal AI agents (using frameworks like LangChain, Snowflake Cortex) and mentor other engineers through code reviews, design discussions, and operational best practices
Job Requirements
- Bachelor’s degree or higher in Computer Science, Information Systems, Software, Electrical or Electronics Engineering
- 5+ years of Data Engineering experience, with deep proficiency in advanced SQL and Python
- 3+ years of hands-on experience with cloud data platforms, specifically Snowflake or Databricks
- Proven experience building and operating scalable ELT pipelines using orchestration tools (e.g., Airflow, dbt)
- Strong track record designing production-grade systems (observability, reliability, performance tuning, incident response)
- Clean Room Knowledge: Exposure to Data Clean Room concepts and Clean Room platforms like LiveRamp, Snowflake or Databricks
- AI/LLM Experience: Experience building applications with LLMs, RAG, Vector Databases, or frameworks like LangChain/LlamaIndex
- Ability to mentor other engineers through code reviews, design discussions, and operational best practices
- Certifications Preferred: SnowPro Core Certification OR Databricks Certified Data Engineer Associate
- Certifications Highly Preferred: SnowPro Advanced: Data Engineer OR Databricks Certified Data Engineer Professional
Benefits
- medical, dental and vision insurance
- 401(k)
- paid leave
- tuition reimbursement
- a variety of other discounts and perks
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
• Title: Senior Data Engineer • Location: Remote • Duration: C2H • Salary: 1-2Lkhs Per Month
Manager, Data Engineering
Henry Schein OneDentrix Enterprise. Dentrix. Dentrix Ascend. Jarvis Analytics. Lighthouse 360.
• Lead teams of software engineers to improve the lives of Henry Schein One users and their patients through the creation of market-leading solutions • Help recruit, develop, and retain engineers to create a high-performing team • Oversee team leaders and support them in fulfilling their mission and delivery commitments • Enable engineering teams to practice continuous delivery at scale by identifying and removing workflow roadblocks • Inspire, motivate and coach to enable continuous development of team leaders and their teams • Ensure products are secure, compliant and reliable for Henry Schein One customers • Collaborate with senior leaders on the vision and strategy for workstreams and products • Partner closely with Product leadership to influence and understand the strategic commitments of your teams • Build strong cross-functional relationships and influence others to pursue the best possible user experience • Enable process and practice maturity to deliver on global growth opportunities • Positively and collaboratively influence decision-making to improve practices across your teams, department, and stakeholder network • Travel typically up to 10% of time • Office environment with no special physical demands required
Head of Data Engineering
HolaflyOur mission is to enable all travellers to enjoy Internet connectivity wherever they are.
• Define the architectural backbone and scalability of Holafly’s entire data ecosystem. • Ensure global operations are fueled by high-quality, real-time insights. • Bridge the gap between infrastructure and analytics, enabling data-driven decision-making. • Design and evolve a world-class data architecture that scales seamlessly with rapid global expansion. • Establish engineering excellence by implementing robust CI/CD practices and version control for all data pipelines. • Ensure data reliability and security across all platforms, supporting mission-critical ML models and business analytics. • Drive strategic technology choices, selecting the best tools and cloud platforms to maintain a competitive edge. • Cultivate high-performing squads, mentoring data engineers to deliver efficient, automated ETL/ELT solutions. • Partner with cross-functional leaders to translate complex business requirements into scalable technical realities.
• Architect and maintain complex marketing databases, datasets, pipelines and Samsara’s Customer Data Platform (CDP) to enable advanced segmentation, targeting, automation and analytics. • Design and implement data infrastructure for AI/ML initiatives, including building pipelines for Generative AI applications, feature stores, and vector database integrations to support predictive modeling and personalization. • Support the execution of expanding conversational BI and Ambient AI within the marketing organization in partnership with the BI team. • Support in the automation of many manual tasks through the use of AI leading to efficiency gains for the whole marketing organization. • Manage critical, high-volume data pipelines to enable our growth initiatives and advanced analytics. Manage the SLAs for those data pipelines and constantly improve efficiency and data quality. • Facilitate sophisticated data integration and transformation requirements for moving data between applications; ensuring interoperability of applications with data mart, AI models, and CDP environments. • Autonomously partner directly with non-technical stakeholders (Marketing, Sales, Ops) to translate ambiguous business questions into technical requirements and scalable data solutions without needing constant supervision. • Write sophisticated yet optimized data transformations in Python/SQL to generate data products consumed by customer systems and Analytics, Marketing Operations, and Sales Operations teams. • Mentor junior engineers, conduct code reviews, and help define best practices for the data engineering team.




