• As a Senior AI Engineer focused on Agentic Evaluation and Verification and Validation (V&V), you will join the AI and Data Science team within Slingshot’s Research and Development organization. • You will contribute to advancing how intelligent systems are evaluated and validated for mission-critical space operations. • This role focuses on building and scaling evaluation frameworks, benchmarks, and simulation-backed validation systems for agentic AI systems, including multi-step, tool-using, and autonomous decision-making workflows powered by LLMs and reinforcement learning. • Your work will directly support the development of reliable and trustworthy autonomous mission planning systems. • You will partner closely with AI researchers and domain experts to translate real-world mission concepts into structured, testable evaluation systems.

Role Description We’re looking for AI and Machine Learning Engineers to join our Expert Network to help train and evaluate the next generation of LLMs using deep technical expertise. If you have the necessary experience, we’ll send you a quick 10- to 15-minute test to assess your skills and suitability for AI tasks. If successful, you’ll be invited to join Prolific as a participant, where you’ll get paid to train and evaluate powerful AI models. Researchers looking for your skills tend to pay up to $80 per hour. You must be prepared to complete paid tasks that require one hour of uninterrupted work, though many are shorter. Qualifications - Education: a BS, MS, or PhD in Computer Science, Artificial Intelligence, Robotics, or a related quantitative field with a focus on Machine Learning. - Professional Experience: experience building, deploying, or fine-tuning ML models in a production environment. - Deep Learning Mastery: professional-level understanding of neural network architectures (Transformers, CNNs, RNNs) and optimization techniques. - LLM Specialization: hands-on experience with Prompt Engineering, RLHF (Reinforcement Learning from Human Feedback), or RAG (Retrieval-Augmented Generation) workflows. - Technical Rigor: the ability to audit complex model logic, identify training data contamination, and evaluate mathematical proofs behind ML algorithms. - Analytical Critique: high attention to detail in spotting "hallucinations," biased outputs, or logical failures in AI-generated technical content. Requirements - Evaluate LLM Architecture Logic: review AI-generated explanations of model architectures, loss functions, and backpropagation for technical accuracy. - Audit Code & Notebooks: validate ML-specific code (e.g., training loops, data preprocessing scripts, or model evaluations) for efficiency and correctness. - Refine RLHF Frameworks: provide the high-quality human feedback necessary to align models with human intent, safety, and helpfulness. - Analyze Model Reasoning: critically assess how an AI model navigates complex chain-of-thought (CoT) prompts and identify where the reasoning breaks down. - Benchmark Performance: conduct comparative testing between different model outputs based on specific technical taxonomies and performance metrics. Key Technologies - Frameworks: expert proficiency in PyTorch or TensorFlow/Keras. - Language & Data: advanced Python (NumPy, Pandas, Scikit-learn) and experience with Hugging Face Transformers. - Cloud & MLOps: experience with AWS (SageMaker), Google Cloud (Vertex AI), or specialized tools like Weights & Biases and LangChain. - Vector Databases: familiarity with Pinecone, Milvus, or Weaviate for RAG evaluation. Benefits - Competitive pay rates. - Flexible hours. - Ability to work from home.

• Translate ambiguous stakeholder requirements into structured solutions, user flows, and deliverables • Design system architecture, making decisions around scalability, maintainability, and security • Build and deliver end-to-end features across frontend and backend with full ownership • Rapidly prototype AI-enabled solutions and evolve them into production-ready systems • Improve and scale an existing Python-based application, addressing backlog, performance, and UX gaps • Define and implement UX flows and usability improvements (not visual design, but user journeys) • Integrate with APIs, including AI-driven services, and design robust data flows • Apply secure development practices and ensure systems are reliable and maintainable • Balance speed of delivery with long-term engineering quality and structure • Contribute to and elevate engineering practices (architecture, SDLC, testing) through hands-on work • Collaborate with stakeholders and team members while operating with a high degree of independence

• Translate ambiguous stakeholder requirements into structured solutions, user flows, and deliverables • Design system architecture, making decisions around scalability, maintainability, and security • Build and deliver end-to-end features across frontend and backend with full ownership • Rapidly prototype AI-enabled solutions and evolve them into production-ready systems • Improve and scale an existing Python-based application, addressing backlog, performance, and UX gaps • Define and implement UX flows and usability improvements (not visual design, but user journeys) • Integrate with APIs, including AI-driven services, and design robust data flows • Apply secure development practices and ensure systems are reliable and maintainable • Balance speed of delivery with long-term engineering quality and structure • Contribute to and elevate engineering practices (architecture, SDLC, testing) through hands-on work • Collaborate with stakeholders and team members while operating with a high degree of independence