Job Closed
This listing is no longer active.
Headquartered in Pleasanton, California, Veeva is a leading provider of cloud-based software and services for the life sciences industry. As an employer, Veeva
AI Data Engineer
Location
Massachusetts
Posted
173 days ago
Salary
$85K - $225K / year
Seniority
Senior
Job Description
AI Data Engineer
Veeva
• Define and establish comprehensive evaluation strategies for new AI Agents. Prioritize the integrity and coverage of test data sets to reflect real-world usage and potential failure modes • Programmatically and manually evaluate the quality of LLM-generated content against predefined metrics (e.g., factual accuracy, contextual relevance, coherence, and safety standards) • Design, curate, and generate diverse, high-quality test data sets, including challenging prompts and scenarios. Evaluate LLM outputs to proactively identify system biases, unsafe content, hallucinations, and critical edge cases • Develop, implement, and maintain scalable automated evaluations to ensure efficient, continuous validation of agent behavior and prevent regressions with new features and model updates • Understand model behaviors and assist in the trace and root-cause analysis of identified defects or performance degradations • Clearly document, track, and communicate performance metrics, validation results, and bug status to the broader development and product teams
Job Requirements
- A strong, specialized understanding of data quality principles, including methods for validating datasets against bias, integrity concerns, and quality standards. Ability to craft diverse and adversarial test data to uncover AI edge cases
- Demonstrated skill in advanced prompt engineering techniques to create evaluation scenarios that test the AI's reasoning, action planning, and adherence to system instructions. Deep knowledge of LLM common failure modes (hallucination, incoherence, jailbreaking)
- Proficiency in designing and deploying automated evaluation pipelines to assess complex, agentic AI behaviors. Familiarity with quality metrics such as task success rate, semantic similarity, and sentiment analysis for output measurement
- Must be comfortable with the specific challenges of debugging agentic systems, including tracing and interpreting an agent's internal reasoning, tool use, and action sequence to pinpoint failure points
- Proficiency in Python for developing custom evaluation frameworks, writing scripts, and integrating pipelines with CI/CD systems. Familiarity with standard test automation tools (e.g., Pytest, modern web automation tools)
- Bachelor's degree in Data Science, Machine Learning, Computer Science, or a related field, with experience in Gen AI / LLMs
- High work ethic. Veeva is a hard-working company
- High integrity and honesty. Veeva is a PBC and a “do the right thing” company. We expect that from all employees
- Applicants must have the unrestricted right to work in the United States or Canada. Veeva will not provide sponsorship at this time.
Benefits
- Medical, dental, vision, and basic life insurance
- Flexible PTO and company paid holidays
- Retirement programs
- 1% charitable giving program
Related Guides
Related Categories
Related Job Pages
More Data Engineer Jobs
Mid-Level Data Engineer
NOUS LATAMConnect with LATAM-based tech talent for your most challenging projects!
• Design and implement efficient data pipelines using Python and AWS services (Lambda, S3, Glue, etc.) • Ensure data quality, reliability, and scalability across multiple sources and formats • Collaborate closely with data analysts, software engineers, and product teams to deliver actionable insights • Contribute to continuous improvement of our data infrastructure and best practices
Mid-Level Data Engineer
NOUS LATAMConnect with LATAM-based tech talent for your most challenging projects!
• Design and implement efficient data pipelines using Python and AWS services (Lambda, S3, Glue, etc.) • Ensure data quality, reliability, and scalability across multiple sources and formats • Collaborate closely with data analysts, software engineers, and product teams to deliver actionable insights • Contribute to continuous improvement of our data infrastructure and best practices
Mid-Level Data Engineer
NOUS LATAMConnect with LATAM-based tech talent for your most challenging projects!
• Design and implement efficient data pipelines using Python and AWS services (Lambda, S3, Glue, etc.) • Ensure data quality, reliability, and scalability across multiple sources and formats • Collaborate closely with data analysts, software engineers, and product teams to deliver actionable insights • Contribute to continuous improvement of our data infrastructure and best practices
ETL Data Engineer
ComPsychThe World’s Largest Provider of Mental Health Services and GuidanceResources® for Life.
• Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading (ETL) activities • Gather requirements and business process knowledge to transform data in ways that meet end-user needs • Create new SDLC processes as needed and maintain and improve existing processes • Ensure data architecture is scalable and maintainable • Collaborate with business stakeholders and colleagues to design and deliver accurate, high-quality data • Investigate data to identify potential issues within ETL pipelines, notify end users, and propose effective solutions • Prepare and maintain documentation for future reference • Additional duties as assigned
