• Build and maintain scalable data pipelines and workflows. • Develop evaluation frameworks for GenAI and LLM applications. • Create and maintain golden datasets for model testing. • Evaluate and benchmark LLM performance. • Build ML-assisted tools to improve delivery and automation. • Deploy production-ready AI solutions. • Work closely with delivery teams to solve technical challenges. • Continuously improve data quality, automation, and operational efficiency.

• GenAI Solutions Development • Implement RAG (Retrieval-Augmented Generation) pipelines; • Develop agents using frameworks such as LangGraph; • Build chatbots and copilots based on LLMs; • Integrate multi-model orchestration solutions using LiteLLM; • Implement features such as: Function Calling; Agent memory; Agent orchestration; Integration with multiple language models. • Software Engineering • Develop full-stack solutions using: Python; .NET/C#; React/TypeScript; • Build robust APIs with support for: Streaming; Asynchronous processing; System integrations; • Apply resilience patterns such as: Circuit Breaker; Retry; Timeout; • Work on distributed and scalable solutions. • Quality and Operations • Develop and run automated tests; • Implement LLM quality evaluations (LLM Evals); • Support regression testing for models and prompts; • Implement observability for AI solutions; • Monitor metrics such as: Token consumption; Latency; Inference costs; • Identify and resolve production issues. • Technical Collaboration • Participate in code reviews; • Contribute to ADRs (Architecture Decision Records); • Support the creation of POCs and MVPs; • Prepare technical documentation; • Work closely with the GenAI Solution Architect and other technical teams.

• Design, develop, and maintain the LLM-powered backend services using Python and FastAPI. • Implement retrieval-augmented generation (RAG) to fetch structured and unstructured data from OXRED. • Use frameworks like LangChain, LlamaIndex, or Haystack to manage context retrieval, query routing, and summarization. • Integrate the chatbot logic with the existing OXRED AskOX frontend. • Design, implement, and validate support for diverse fleet analytics use cases including fleet summaries, vehicle diagnostics, predictive insights, and performance metrics. • Ensure secure and efficient data flow between OXRED APIs and the AskOX backend. • Evaluate and improve retrieval accuracy, latency, and hallucination rates. • Maintain technical documentation and API specifications.

Role Description We are looking for a detail-oriented LLM QA Contractor to join an AI validation project focused on improving the quality of Large Language Model (LLM) outputs. You will review photos and delivery instructions, compare them with AI-generated responses, and ensure outputs are accurate, complete, and aligned with established quality guidelines. Your feedback will directly contribute to developing safer and more reliable AI experiences. - Review AI-generated outputs against source images and delivery instructions. - Verify factual accuracy, completeness, and adherence to annotation guidelines. - Identify and classify issues such as hallucinations, missing information, incorrect interpretations, or policy violations. - Provide concise and structured feedback to support continuous model improvement. - Follow detailed Standard Operating Procedures (SOPs) and participate in calibration sessions. - Track your work using annotation tools and spreadsheets. - Escalate ambiguous cases or guideline gaps when needed. - Consistently meet quality, accuracy, and productivity targets. Qualifications - Excellent attention to detail and a structured, analytical mindset. - Previous experience in Content QA, Data Labeling, Trust & Safety, Content Moderation, or similar quality assurance roles. - Strong visual analysis and reading comprehension skills. - Comfortable working with web-based tools, spreadsheets, and annotation platforms. - Reliable, organized, and able to handle repetitive tasks while maintaining a high level of accuracy. - Strong written communication skills in English. - Able to handle confidential and user-generated content responsibly. Requirements - Remote: Work from anywhere. - Contract: Freelance / Service Agreement. - Initial duration: 90-day contract with the possibility of extension. - Schedule: Part-time (4 hours per day). - Weekly workload: The project has two main work peaks each week, on Mondays and Wednesdays. - Critical requirement: Every batch of photos must be reviewed within 36 hours. Meeting this turnaround time is essential. - Availability: No need to work during Portugal business hours. Only 1 hour of overlap is required for a brief team status update on Monday afternoon (Vietnam time) and Wednesday afternoon (Vietnam time). - Rate: USD 11/hour.