AI Validator
Location
Mexico
Posted
13 days ago
Salary
0
Seniority
Mid Level
Job Description
AI Validator
Nexaminds
Role Description Nexaminds is looking for an AI Validation Engineer to lead the validation and quality strategy for AI-powered systems, including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) applications, and agent-based workflows. The ideal candidate has deep experience in Quality Engineering, AI evaluation, and automated validation frameworks, with a strong understanding of how to measure, monitor, and improve the reliability of non-deterministic AI systems. This role focuses on: - Building enterprise-grade AI validation capabilities - Designing automated evaluation pipelines - Implementing AI guardrails - Partnering closely with engineering, AI/ML, and data governance teams to ensure AI-powered solutions are accurate, safe, and production-ready Qualifications - 7+ years of experience in Quality Engineering, Software Development, or a related technical discipline, including recent hands-on experience with AI/ML or LLM-based systems - Experience designing validation or evaluation frameworks for non-deterministic or probabilistic systems such as Large Language Models (LLMs) or Machine Learning applications - Strong understanding of Retrieval-Augmented Generation (RAG), agentic workflows, and LLM evaluation concepts, including hallucination detection, groundedness, relevance, and consistency - Hands-on experience with Large Language Models (LLMs) and prompt engineering, including prompt design, optimization, and regression validation - Practical experience using Claude Code, Claude SDK, or similar AI-assisted development platforms and agentic coding tools - Proficiency with Node.js for building automation tools, validation pipelines, and engineering utilities - Experience working with MongoDB to store, manage, and analyze evaluation results, validation data, or AI output logs - Experience integrating automated validation processes into CI/CD pipelines - Experience designing semantic validation approaches that evaluate meaning, relevance, and contextual accuracy beyond traditional software testing - Strong understanding of AI quality metrics, confidence scoring, output consistency, and production monitoring - Solid knowledge of data validation principles, including schema validation, business rule compliance, and data consistency - Excellent cross-functional collaboration skills, with experience partnering across Quality Engineering, AI/ML Engineering, Software Engineering, and Data Governance teams Requirements - Experience with prompt regression testing tools or frameworks (nice to have) - Familiarity with AI governance, fairness, or bias-detection practices (nice to have) - Experience with tools such as Playwright, RestSharp, or similar UI/API test automation (nice to have) - Prior experience standing up a validation or evaluation function from scratch (nice to have) - Exposure to confidence scoring or guardrail systems for production AI (nice to have) - Experience designing or orchestrating multi-agent systems in production environments (nice to have) Job Duties - Define and lead the AI validation strategy for LLMs, RAG systems, and AI agents - Build automated validation pipelines integrated into CI/CD - Establish evaluation frameworks covering correctness, relevance, groundedness, consistency, and hallucination rate - Design and maintain prompt regression testing to catch silent quality degradation after model or prompt changes - Introduce semantic validation techniques that go beyond traditional QA (evaluating meaning, not just structure) - Lead production monitoring of AI output quality and define acceptable output ranges for non-deterministic systems - Partner with engineering teams to validate AI-generated code, test cases, and AI-assisted decision workflows - Introduce AI guardrails and confidence scoring to flag unsafe or low-confidence outputs - Build validation checkpoints across multi-step agent pipelines - Define and track quality metrics/KPIs (defect reduction, output consistency, prompt regression stability, adoption confidence) Benefits - Stock options 📈 - Remote work options 🏠 - Flexible working hours 🕜 - Benefits above the law
Related Guides
Related Categories
Related Job Pages
More Blockchain Engineer Jobs
Blockchain Legal Support
Cardano FoundationEmpowering enterprises with blockchain solutions for traceability, authenticity, and sustainability.
Role Description Make a real difference in the blockchain industry! Build the change you want to see. We are looking for a motivated and analytical Blockchain Legal Support (Working Student) to join the legal team of the Cardano Foundation in Zurich. This role offers a unique opportunity to gain hands-on experience within the work of a leading blockchain foundation. You will work at the intersection of technology, regulation, and governance in a dynamic, international environment. - A unique chance to step into the Blockchain space and see how global blockchain infrastructure in a multi-stakeholder environment is shaped. - A 30-40% average workload with flexible hours, allowing you to easily balance your work and academic commitments; possibility to work full-time during breaks. - Direct mentorship from experienced legal professionals in the blockchain space, offering a strong foundation for your future legal career. Qualifications - Currently enrolled in a Swiss law degree programme (BLaw, MLaw, or equivalent), having successfully completed at least the first three semesters of study. - A strong interest in contract law, financial services regulation, data protection, intellectual property, or related areas of law. Prior experience is valued. - Curious about blockchain technology and public decentralized infrastructure, with a willingness to learn. - Reliable, organized, and detail-oriented, with strong analytical skills and the ability to manage multiple tasks in a structured manner. - Fluent in English (our primary corporate language) and German. Requirements - Conduct legal research and support legal assessments on global blockchain regulation, financial services law, data protection, and other relevant legal and regulatory developments. - Assist with the drafting, review, administration, and organization of various agreements, such as non-disclosure agreements (NDAs) and provider contracts. - Support the preparation of responses to general inquiries, maintain compliance documentation, and assist with internal governance processes. Benefits - Enjoy the benefits of remote work with the flexibility to tailor your hours that best supports your productivity, while staying aligned with the team. - We foster a collaborative remote environment where ideas flow freely and everyone feels empowered to contribute. - We believe in continuous growth and development. Every employee receives an annual learning budget to invest in their professional journey. - We provide opportunities for growth and encourage you to continuously evolve your skills and expertise within a supportive environment. - Your work will directly contribute to the development and adoption of Cardano, shaping the future of blockchain technology and its real-world applications. - Connect with your colleagues and strengthen team bonds through a dedicated annual team building event, as well as a departmental team-building day. Enjoy an extra day off for your birthday and additional days off to celebrate milestone work anniversaries. - Surround yourself with a team of dedicated individuals who are driven by a shared passion for blockchain technology and its potential to change the world.
Senior Applications Developer – Supply Chain
The J.M. Smucker Co.We're an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, genetic information, age, national origin, disability status or protected veteran status.
• Participating in planning, design, development and support activities • Developing scalable solutions to integrate internal Supply Chain applications • Better communications with external cloud vendors
Blockchain Investigator – Law Enforcement Liaison
Tether.toBringing real world currency to the blockchain.
• Employ state-of-the-art blockchain analytics tools to analyze virtual currency transactions. • Act as a primary point of contact for law enforcement agencies. • Provide detailed, understandable reports and insights on blockchain activities to assist in criminal investigations. • Efficiently respond to and action incoming requests from law enforcement. • Enhancing law enforcement's understanding of patterns and indicators of illicit activity on the blockchain. • Stay abreast of the latest developments in blockchain technology, virtual currency trends, and regulatory changes. • Work closely with other departments, such as legal and compliance, to ensure adherence to regulatory requirements and company policies.
• Lead the design and delivery of complex systems and multi-service projects in our trading platform. • Take system-level ownership of correctness, reliability, and operability in production. • Anticipate and mitigate architectural, operational, and product risks before they result in incidents or user impact. • Act as a technical co-lead with Product, Risk, Operations, and other stakeholders to drive clarity, alignment, and execution. • Provide high-quality code reviews that surface risks, clarify assumptions, and raise engineering standards. • Coach and mentor engineers through design discussions, feedback, and day-to-day collaboration. • Drive technical leverage by introducing abstractions, patterns, or platforms that structurally improve correctness, reliability, and long-term velocity. • Participate actively in incident response and ensure durable improvements through root-cause analysis and follow-up work. • Communicate complex technical decisions clearly and effectively to both technical and non-technical audiences.
