• Design, refine, and optimize prompts and prompt pipelines for LLM-based applications. • Build structured prompt frameworks (few-shot, chain-of-thought, role prompting, tool prompting). • Evaluate AI outputs for accuracy, bias, hallucination, and reliability. • Work with engineering teams to integrate prompts into APIs and AI products. • Develop prompt libraries and reusable templates for various use cases. • Run experiments to benchmark prompt strategies across different LLMs. • Create guardrails and evaluation methods for safe and consistent AI responses. • Collaborate with product, data science, and engineering teams to improve AI performance.

• Build and maintain agentic systems that continuously collect, process, and surface signals from social platforms, on-chain data, and ads ecosystems. Turn raw, multi-source data into structured intelligence that informs real account and campaign decisions. • Design and maintain the pipelines that power those systems: ingesting from multiple external APIs, enriching and modeling data, and building semantic layers that both humans and AI agents can actually use. • Build multi-step AI agents that automate research, synthesis, monitoring, and reporting. Not single-call LLM wrappers, but orchestrated systems with memory, defined tools, and structured output schemas that teams can depend on. • Build full-stack applications and dashboards that make intelligence accessible across client accounts. Live tools, not decks. • Spend real time with internal teams across marketing, PR, content, growth, and social before writing a line of code. Map how work actually moves, identify where signal gaps create friction, and build systems that solve for multiple teams from a single architecture. • Stay ahead of the AI tooling landscape. Know when a no-code connector is the right call and when you need to build. Define how impact gets measured, and build the reporting to prove it.

Role Description We are looking for an LLM Engineer to design, execute, and operationalize fine-tuning workflows for large language models across supervised, preference-based, and reinforcement learning approaches. The role requires deep practical experience with modern training stacks, careful dataset construction, rigorous evaluation methodology, and the engineering discipline to operate complex training pipelines reliably. The ideal candidate combines strong ML intuition with production-grade engineering practices, and is comfortable navigating the trade-offs between data quality, compute budget, evaluation rigor, and shipping velocity. In this role you will work closely with cross-functional partners — product, design, engineering, operations, and business stakeholders — to translate ambiguous requirements into well-engineered solutions, and will be expected to raise the bar through code review, design review, and mentorship of more junior engineers. The successful candidate brings strong engineering discipline, a clear communication style, and a track record of shipping meaningful work that holds up well in production. Key Responsibilities - Design and execute fine-tuning experiments for large language models using supervised, DPO, RLHF, and related techniques. - Lead dataset construction, curation, and quality assurance processes for instruction tuning and preference data. - Build scalable training pipelines on top of modern distributed training frameworks. - Tune hyperparameters, optimizer configurations, and training stability strategies for large-model fine-tuning. - Implement parameter-efficient fine-tuning techniques such as LoRA, QLoRA, and adapter-based methods. - Design rigorous evaluation suites including automated benchmarks, human evaluation, and capability-specific probes. - Implement safety, refusal, and policy evaluations to track model behavior across releases. - Operate large-scale training jobs on GPU clusters, diagnosing failures and recovering training state reliably. - Optimize training throughput using mixed precision, sequence packing, and efficient attention implementations. - Manage model artifacts, lineage tracking, and reproducibility across many concurrent experiments. - Collaborate with product, research, and platform teams to align fine-tuning roadmaps with business needs. - Document training methodology, results, and decisions clearly for technical and non-technical audiences. - Mentor engineers on fine-tuning best practices, evaluation rigor, and responsible deployment. - Stay current with LLM research and translate advances into production-ready fine-tuning recipes. Qualifications - Master’s or PhD in Computer Science, Machine Learning, or a related field; or equivalent experience. - Six or more years of combined ML research and engineering experience, with significant LLM exposure. - Strong proficiency in Python and modern deep learning frameworks, especially PyTorch. - Hands-on experience fine-tuning transformer-based language models at non-trivial scale. - Familiarity with distributed training strategies including FSDP, ZeRO, and pipeline parallelism. - Experience with RLHF, DPO, or other preference optimization techniques. - Strong understanding of evaluation methodology, benchmarks, and human evaluation design. - Experience operating training jobs on GPU clusters and recovering from failures. - Strong written and verbal communication skills. - Track record of shipping or publishing impactful LLM work. Preferred Qualifications - Publications at top-tier ML venues. - Experience with multimodal model fine-tuning. - Familiarity with synthetic data generation and dataset distillation. - Open-source contributions to LLM training libraries. - Exposure to responsible AI evaluation and red-teaming practices. How to Apply Would you like to know more about this opportunity? For immediate consideration, please send your resume to [email protected] or contact us at (908) 505-3899. Learn more about Bright Vision Technologies at www.bvteck.com .

• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships. • Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure. • Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting. • Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users. • Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls. • Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves. • Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production. • Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards. • Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.