Personalized, evidence-based autism therapy—accessible everywhere.
Data Scientist
Location
United States
Posted
3 days ago
Salary
$144K - $168K / year
Seniority
Senior
Job Description
Data Scientist
AnswersNow
• Architect and continuously improve the RAG pipeline that retrieves client-specific clinical context — session notes, treatment plan goals, historical performance data — and injects it into inference-time prompts • Design the retrieval layer: chunking strategies, embedding models, vector store configuration, and retrieval ranking — optimizing for clinical relevance, not just semantic similarity • Build a context assembly system that selects and structures the most relevant clinical information for each model invocation, given token constraints and clinical priority • Evaluate retrieval quality rigorously: build test sets, measure recall and precision, and iterate on the pipeline based on where retrieval fails • Design evaluation frameworks that assess AI recommendation quality beyond standard NLP metrics — working with clinical stakeholders to define what 'good' means for each use case • Build automated evaluation pipelines that can test AI outputs at scale: LLM-as-judge evaluators, human review workflows, and clinical validity checks • Maintain evaluation datasets that reflect the real distribution of clinical scenarios the model encounters in production • Systematically identify where foundation model capabilities fall short for AnswersNow's care model: what clinical reasoning the model gets wrong, what it hallucinates, what it doesn't know how to handle • For each identified gap, recommend and implement the appropriate mitigation — improved retrieval, prompt engineering, output validation, or escalation to human review • Monitor production AI outputs for quality, drift, and failure modes using the evaluation infrastructure you've built • Define alerting thresholds and escalation paths for when AI quality falls below acceptable clinical standards • Work closely with clinical leadership and BCBAs to understand the care model deeply enough to design AI systems that support it accurately • Translate clinical domain knowledge into technical requirements: what context does the model need, what outputs are clinically acceptable, where does the model need to defer to the clinician
Job Requirements
- 4+ years of experience in applied data science, ML engineering, or AI engineering in a production environment
- Deep understanding of RAG architectures: retrieval systems, embedding models, vector databases (Pinecone, Weaviate, pgvector, or similar), chunking strategies, and context assembly
- Experience designing and running evaluation frameworks for AI systems — you've thought hard about how to measure quality in domains where ground truth is ambiguous
- Strong Python skills; experience with LLM orchestration frameworks (LangChain, LlamaIndex, or similar)
- Clinical NLP experience or healthcare AI background is strongly preferred — you understand why clinical data is different from general text and what that means for AI system design
- You think like an engineer and a scientist: you build systems that can be measured, iterated on, and trusted — not black boxes
- Strong written communication: you can explain RAG pipeline design to a clinician and explain clinical requirements to an engineer
- Genuine interest in the clinical domain — you want to understand Applied Behavior Analysis well enough to build AI that actually helps BCBAs do their jobs
Benefits
- Fully remote – work from anywhere in the U.S.
- Flexible hours with an async-friendly team culture
Related Guides
Related Categories
Related Job Pages
More Data Scientist Jobs
Role Description In our Data Science team, you'll have the chance to work as a part of our product squads delivering features that enrich our product and a first-class experience that we are known for. You'll work alongside brilliant minds, with different skills: Product Managers, Designers, UX Writers, Data Scientists, Machine Learning Engineers, Software Engineers, etc. Be prepared for the job of your life - hard challenges, high expectations, intelligent life at work, meaningful conversations, outstanding productivity. - Architect, develop and deploy machine learning systems to production. The main use cases are related to chatbots and search. - Design and execute experiments. - Analyze a wide variety of data (structured and unstructured, observational and experimental) to improve current models or create new ones. - Apply the scientific method to develop LLM-powered applications, including testing different prompts, exploring and understanding LLM capabilities, evaluating agents and prompts, assessing AI application guardrails, and building PoCs to validate the feasibility of generative AI for product features. - Participate actively in discussions and decisions regarding the whole Data Science chapter (Guilds, Design Reviews, Demos, etc.). Qualifications - People that are seeking to learn and deliver real impact through Data Science. - Expert knowledge of machine learning concepts: regression and classification, clustering, neural networks, feature selection, cross-validation, curse of dimensionality, bias-variance tradeoff, model explainability, etc. - Good understanding of the engineering challenges to deploy machine learning systems to production. - Proficiency in Python. - Some knowledge or experience with Deep Learning. - Experience with technical advice for other data scientists (technical leadership). - Excellent written and verbal technical communication skills. - Good English skills (verbal and written) is mandatory. Requirements - Have a MSc. (or Undergrad + intense experience) in machine learning, data science, information retrieval, ranking systems, recommender systems, natural language processing or other relevant fields. - Have experience with applied generative AI: AI agents, prompt engineering, LLMs finetuning (lora, qlora, peft), LLM routing, LLM monitoring, LLM guardrails. - Have experience working at fast-growing startups. Benefits - Competitive salary - Profit sharing - Meal allowance - Health insurance - Dental plan - Life insurance - Childcare subsidy and Atypical Parenthood subsidy - Wellhub - Home office allowance - Employee assistance program (mental health, social, legal, and financial support) - Extended parental leave - Day off on birthday, Mother’s Day, and Father’s Day - Benefits Club (discounts on everyday services) - Discounts at educational institutions - Reading kit for children – PlayKids
• Work on the latest applications of data science to solve business problems. • Work directly with client stakeholders to translate business problems into high level analytics solution designs. • Present analytic solutions to business audiences highlighting robustness of the solution and how it could help generate business value. • Develop end-to-end solutions based on in-depth understanding of business problems to ensure analytics solutions are delivered efficiently, predictably, and sustainably. • Design and develop machine learning and Generative AI solutions using RAG. • Build LLM-powered applications leveraging Azure OpenAI and orchestrate workflows using LangGraph. • Develop agentic AI workflows for automation, insights generation, and decision support. • Implement Document Intelligence solutions for extracting insights from unstructured data. • Participate in discussions with team members to select and apply relevant analytic techniques and create actionable business insights. • Responsible for making presentations to senior management, communicating results to business teams, and develop plans to help operationalize analytic solution.
Product Data Scientist – AI Evaluation, Quality
FinomFinancial solutions for entrepreneurs and freelancers - combining business account benefits with multiple services
• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics • Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency • Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts • Harden methodology: judge stability, non-determinism handling • Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake
Product Data Scientist – AI Evaluation, Quality
FinomFinancial solutions for entrepreneurs and freelancers - combining business account benefits with multiple services
• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics • Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency • Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts • Harden methodology: judge stability, non-determinism handling • Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake
