• Partner with teams across the business and external partners to understand data needs and deliver reliable pipelines and models that solve real problems. • Build and maintain scalable ingestion and egress pipelines in Airflow and dbt Cloud, ensuring high quality, automated data flows across cloud environments. • Implement unit tests and monitoring to guarantee data integrity and reproducibility. • Model and transform and structure healthcare datasets into usable formats that power data science models, and other reporting marts. • Enhance and scale data models with SQL and dbt, ensuring precision and adaptability for new partnerships. • Write Python code for Apache Airflow DAGs, components and utilities that orchestrate and monitor data workflows. Build complex pipelines that enable flexible scheduling, conditional logic, and smooth integration across multiple data sources.

We are recruiting a Sr. Data Engineer for a client in the AI / health & wellness space Original Job Post link https://www.notion.so/garden3d/Senior-Data-Engineer-2e7131fea2c7800094e4d9340a5df499?source=copy_link About garden3d We are worker owned creative collective, innovating on everything from brands and IRL communities to IoT devices and cross platform apps. We share profit, open source everything, spin out new businesses, and invest in exciting ideas through financial and/or in-kind contributions. Our client roster includes Google, Stripe, Figma, Hinge, Black Socialists in America, ACLU, Pratt, Parsons, Mozilla, The Nobel Prize, MIT, Gnosis, Etsy & Gagosian. We’re the software team behind innovative products like The Light Phone & Mill, and a global, decentralized community space collective called Index Space. We think of our garden3d as collective for creative people, prioritizing a happy, talented, and diverse studio culture. We work on projects that bring value to our world, and we balance deep care for the work we do with a genuine curiosity about life outside of our jobs. About the client Our client is an early-stage AI startup based in NYC (but open to remote team members). The founders have experience building and scaling successful ventures including a 9-figure exit. Who we’re looking for: We’re looking for a Senior Data Engineer with deep expertise in designing and owning data pipelines, workflow orchestration, and complex data integrations. You’ll play a key role in evolving our data ingestion architecture, from an existing in-house, code-defined workflow system backed by queues, to a more scalable and observable orchestration layer using Prefect. In this role, you’ll lead the development and optimization of pipelines handling both structured and unstructured data from a wide range of sources, including web crawls and scrapers. You’ll be expected to make architectural decisions, ensure reliability and scalability, and establish best practices for workflow design, monitoring, and performance as our data platform grows. In this role, you’ll work across a variety of initiatives to find cost-effective, high-quality, pragmatic solutions to complex problems. Responsibilities will include: - Monitoring and maintaining data pipelines, troubleshooting new errors, and addressing format drift - Extracting and enriching additional data elements from diverse sources - Reprocessing and validating large datasets in batch workflows - Designing and integrating new data sources into existing pipelines - Aligning and integrating extracted data with the core application data model to ensure consistency and usability - Participating in code reviews, providing constructive feedback to teammates and ensuring adherence to best practices - Contributing to project success by keeping a close eye on team velocity, project scope, budget, and timeline - Negotiating with clients to align project scope with budget and timeline, if needed Who you are The person we’re looking for is happy, relaxed and easy to get along with. They’re flexible on anything except conceits that will lower their usually outstanding work quality. They work “smart”, by carefully managing their workflow and staggering features that have dependencies intelligently — they prefer deep work but are OK coming up to the surface now and then for top level / strategic conversations. We believe people with backgrounds or interests in design, art, music, food or fashion tend to have a well rounded sense of design & quality — so a variety of hobbies or side projects is a big nice to have! Must Have Competencies: - Senior-level Python expertise - Experience with data/workflow orchestration tools (e.g., Prefect, Airflow, Dagster) - A thorough understanding ETL & data transformation for the ingestion of industry standard LLMs (OpenAI, Claude, etc) - Familiarity with Large Language Models (LLMs) - Skilled in interfacing with APIs (OpenAI, Google Gemini/Vertex, etc.) using wrapper libraries such as Instructor, LiteLLM, etc. - Practical experience in prompt engineering - Ability to work with structured outputs and potentially tool calling - 5+ years general experience in backend (Ruby on Rails, Elixir Phoenix, Python Django, or Node Express) and/or native app development (React Native, Flutter, Android, AOSP, Kotlin/Java). Nice to Have Competencies: We’re always pitching for new and exciting technology niches. Some of the areas below are relevant to us! - Experience with Google Cloud Platform (GCP), particularly Cloud Run and Cloud Tasks - Knowledge of search technologies, including embeddings and vector databases for semantic search, as well as keyword-based search (BM25) - Familiarity with PySpark for batch data processing - Experience working with LLMs, Vector Databases, and other generalist AI-enabled application patterns - Client-facing experience: working directly with customers to gather requirements and provide technical solutions - Product management experience: defining product roadmaps and collaborating closely with stakeholders - Engineering management experience: leading teams, setting technical direction, and mentoring developers - Recent experience working in a startup environment - NYC-based preferred for collaboration, but not a strict requirement. Compensation The pay scale ranges from $125 p/hr to $175 p/hr, $150-200k/year based on experience. In addition to cash compensation, equity may be offered for candidates with the right level of experience, commitment, and long-term alignment. How we interview: Our interview process starts with a call where you get to meet a few members of our team. From there we’ll ask appropriate candidates to take part in a technical exercise which helps illustrate skill level and comfort. Direct application link here: https://garden3d.notion.site/1f1131fea2c78095922ec7e09bd96101 (Tell us a bit about your interest in the role and share your information by filling out the questions.) Quick tip! Adding a Loom recording to your profile in our form to showcase your skillset can really make your application stand out!

• Design, develop, and implement scalable and reliable data solutions on the Microsoft Azure platform. • Collaborate with cross-functional teams to gather and analyze data requirements. • Design and implement data ingestion pipelines to collect data from various sources, ensuring data integrity and reliability. • Perform data integration and transformation activities, ensuring data quality and consistency. • Implement data storage and retrieval mechanisms, utilizing Azure services such as Azure SQL Database, Azure Data Lake, and Azure Blob Storage. • Monitor data pipelines and troubleshoot issues to ensure smooth data flow and availability. • Implement data quality measures and data governance practices to ensure data accuracy, consistency, and privacy. • Collaborate with data scientists and analysts to support their data needs and enable data-driven insights.

• Architect, develop, and maintain scalable data pipelines and ETL workflows that support the ingestion, transformation, and storage of large datasets from diverse sources. • Implement automated data quality checks and validation processes to ensure the accuracy, consistency, and reliability of data across systems. • Work closely with product managers, data analysts, and business stakeholders to gather and understand data requirements, translating them into technical specifications and actionable engineering tasks. • Continuously monitor and optimize data systems for performance, scalability, and cost-efficiency, ensuring that data infrastructure meets evolving business needs. • Diagnose and resolve data-related issues promptly, providing root cause analysis and implementing preventive measures. • Maintain comprehensive documentation of data pipelines, ETL processes, and system architecture. Participate in design and code reviews to uphold high engineering standards. • Stay abreast of emerging data engineering technologies, tools, and best practices to drive innovation and continuous improvement within the team. • Provide guidance and mentorship to junior data engineers, fostering a culture of knowledge sharing and technical excellence.