Are you looking for that perfect real estate, at that perfect location and at that perfect price? Look no further.
AI/LLM Safety Engineer
Location
Kansas
Posted
15 hours ago
Salary
0
Seniority
Senior
Job Description
AI/LLM Safety Engineer
Propio Aruba Realty
• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships. • Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure. • Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting. • Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users. • Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls. • Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves. • Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production. • Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards. • Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.
Job Requirements
- Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, or a related technical field—or equivalent practical experience.
- 4+ years building production software, with direct experience working on—or securing—ML/LLM systems.
- Strong software engineering skills with the ability to write production-grade code (primarily Python), beyond scripting or notebook prototyping.
- Solid understanding of LLMs and ML: how models work, prompt engineering, and the safety implications of fine-tuning and RAG (e.g., unsafe retrieval, tool misuse, and data exfiltration).
- A security mindset with demonstrated threat-modeling ability; able to threat-model AI workflows and familiar with the fundamentals of access control, data retention, and incident response.
- Familiarity with the LLM attack surface—prompt injection, jailbreaks, data poisoning, and supply-chain risk—and working knowledge of the OWASP LLM Top 10.
- Hands-on experience with at least one of safety evaluation or red teaming, with the ability to walk through a real finding and how it was remediated.
Benefits
- Health insurance
- Paid time off
- Flexible work arrangements
- Professional development
- Stock options
Related Guides
Related Job Pages
More LLM Engineer Jobs
LLM Engineer
NagarroNagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
• Join an existing development team to build and ship LLM-powered features in a complex, large-scale production application • This is a hands-on, full-stack role spanning backend services, APIs, and the LLM systems (retrieval, agents, and evaluation) that power them • Work as an agentic engineer, leveraging AI coding tools and autonomous agents to write code, automate workflows, and optimize delivery • Collaborate effectively with Product Owners and stakeholders to solve complex problems • Partner cross-functionally to deliver impactful solutions across teams • Continuously expand your technical expertise and stay current with emerging technologies • Demonstrate curiosity, initiative, and a commitment to continuous learning • Apply a data-driven approach to technical decision-making and problem-solving • Use systems thinking to connect data science and engineering principles • Take full ownership of features and projects, delivering high-quality solutions with minimal oversight.
Senior Staff Engineer, LLM
NagarroNagarro (Frankfurt: NA9) is a leader in digital product engineering and drives technology-led business breakthroughs.
• Hands-on, daily use of AI-assisted and agentic coding tools (e.g., Claude Code, Cursor, GitHub Copilot, autonomous coding agents) to write and refactor code, automate workflows, and optimize engineering processes • Proficiency with server-side events, event-driven architectures, and messaging systems • Strong critical thinking and systems thinking skills, with experience debugging, optimizing, and making sound engineering decisions across complex backend systems, not just solving isolated problems • Solid understanding of security best practices for backend systems, including authentication and data protection
AI Engineer – LLM Products
dexter healthKI für die Pflege. Von Sprachdokumentation bis zu KI-Dienstplanung. Wir sorgen dafür, dass Pflegekräfte mehr Zeit haben.
• Build new AI-powered product features from idea to production • Improve existing AI workflows for quality, reliability, latency, and user value • Design and implement LLM-based workflows, structured outputs, validation logic, and fallback behavior • Build evaluation loops, tests, and quality checks for AI-generated outputs • Integrate AI capabilities into existing product and backend systems • Work with commercial and open-source LLMs without being tied to one specific provider • Support self-hosted model workflows where they make sense for quality, speed, cost, or control • Debug AI feature failures across inputs, outputs, data, backend logic, and user flows • Use AI development tools as a core part of your daily workflow • Ship quickly while keeping production quality high
AI Engineer – LLM
dexter healthKI für die Pflege. Von Sprachdokumentation bis zu KI-Dienstplanung. Wir sorgen dafür, dass Pflegekräfte mehr Zeit haben.
• Build new AI-powered product features from idea to production • Improve existing AI workflows for quality, reliability, latency, and user value • Design and implement LLM-based workflows, structured outputs, validation logic, and fallback behavior • Build evaluation loops, tests, and quality checks for AI-generated outputs • Integrate AI capabilities into existing product and backend systems • Work with commercial and open-source LLMs without being tied to one specific provider • Support self-hosted model workflows where they make sense for quality, speed, cost, or control • Debug AI feature failures across inputs, outputs, data, backend logic, and user flows • Use AI development tools as a core part of your daily workflow • Ship quickly while keeping production quality high
