• Take operational ownership of the configuration, reliability, and standards of Storyteller’s AI tools and AI platforms. • Support the AI Operations team’s response when AI tools, platforms, models, or underlying capabilities change. • Set, maintain, and improve operational standards for how AI tooling is configured, accessed, used, and monitored. • Improve visibility around AI tool configuration, platform usage, access administration, permissions, dependencies, and operational performance. • Identify weak, unstable, or high-friction areas in our AI tooling setup and help develop practical solutions that are durable and scalable. • Work with internal teams to understand where AI tooling is helping, where it is creating friction, and what needs to improve. • Give leadership clearer visibility on where the AI tooling stack is performing well, where it is fragile, and what platform or workflow changes are needed next. • Help ensure Storyteller’s AI tooling remains useful, controlled, reliable, and aligned with how the business actually works.

Role Description This is a deeply hands-on role for an experienced engineer who wants to spend their time building. A big part of the job is growing our fleet of AI sub-agents — designing new specialized agents and wiring them into complete, end-to-end multi-agent agentic workflows and architecture. You'll design, implement, and harden our RAG, agent, and LLM features end-to-end — the retrieval pipelines, the agent orchestration, the model-gateway routing and fallback logic, the AI automation pipelines, and the evaluations that keep quality high — all inside a GDPR-first, EU-only data boundary. Your work ships to production and you own it through to it working reliably. What you'll do - Grow our fleet of AI sub-agents — design and build more specialized agents, and wire each into a complete multi-agent, agentic AI-driven workflow and architecture. - Build and improve RAG pipelines — chunking, embeddings, vector storage and retrieval, re-ranking, grounding and citation over proprietary data. - Build LLM features on cloud-hosted large language models — classification, extraction, summarisation, structured/JSON output, prioritisation, and multimodal reasoning. - Extend the internal Model Gateway — task/tier model routing, retries, fallback, cost estimation, and per-tenant usage limits. - Build adversarial validation agents — AI agents that critique and stress-test model outputs, plans, and designs. - Design and build AI automation pipelines — orchestrated, repeatable AI workflows. - Automate the SDLC with AI — integrate AI into planning, generating, reviewing, testing, and shipping code. - Integrate and extend an automated testing framework — including AI-assisted test generation. - Build the evaluations — datasets and harnesses that measure accuracy, faithfulness/hallucination, latency, and cost. - Do the prompt engineering — schema-validated outputs, tool/function calling, and tuning to reduce hallucination. - Enforce data-protection by design — tenant-scoped retrieval, PII handling, and erasure/cascade in the retrieval layer. - Ship to production on AWS (Python, ECS) with proper observability, and own your features through to reliable operation. Qualifications - 5–8+ years of software engineering, with deep, production-grade Python. - Substantial hands-on experience building and shipping LLM-powered systems to production. - Designing and composing multi-agent systems. - Deep RAG expertise — embeddings, vector stores, semantic search, chunking strategies. - Embeddings at scale and multilingual retrieval. - Advanced prompt engineering and structured output. - Deep experience with cloud-hosted commercial LLM APIs. - Agentic / multi-agent frameworks experience. - AI-driven SDLC automation experience. - Integrating automated testing frameworks and AI-assisted test automation. - Proven ability to build evaluations for AI systems. - Real command of cost/latency trade-offs at production scale. - Strong AWS background — building, deploying, and operating services in production. - Streaming, latency optimisation, and prompt caching. - Vector-DB operations, observability, and IaC. - GDPR / data-residency / responsible-AI experience. - Experience with AI systems running at production scale. - Excellent testing discipline, code quality, and engineering judgement. Beware of Recruitment Scams We have been made aware of individuals fraudulently posing as members of our Talent Acquisition team and extending fake job offers. These scams may involve requests for personal information or payment for equipment. Protect yourself by following these steps: - Verify that all communications from our recruiting team come from an @banyansoftware.com email address. - Remember, employers will never request payment or banking information during the hiring process. - If you receive a suspicious message, do not respond — instead, forward it to careers@banyansoftware.com and/or report it to the platform where you received it. Your safety and security are important to us. Thank you for staying vigilant.

• Write prompts in your native language designed to test an AI model. • Evaluate the AI's responses using a structured rating guide. • Identify areas where the AI's responses can be improved. • Translate your prompts and evaluations into English.

• Write prompts in your native language designed to test an AI model. • Evaluate the AI's responses using a structured rating guide. • Identify areas where the AI's responses can be improved. • Translate your prompts and evaluations into English.