Soluzioni tecnologiche
Technical Solutions Architect, Evals & Fine-Tuning
Location
Canada
Posted
8 days ago
Salary
$190K - $225K / year
Seniority
Lead
Job Description
Technical Solutions Architect, Evals & Fine-Tuning
InnoData
• Lead technical discovery with prospective and existing customers — foundation model labs, frontier AI teams, and large enterprises — to understand model objectives, gaps, and constraints. • Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context). • Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains. • Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance. • Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly. • Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent. • Feed customer signal back into Innodata’s R&D and product roadmap — what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction. • Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation). • Represent Innodata externally — at customer reviews, conferences, and in technical content.
Job Requirements
- 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training.
- Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
- Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design.
- Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents).
- Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement.
- A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation.
- Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field — or equivalent demonstrated experience.
Related Guides
Related Categories
Related Job Pages
More Solutions Engineer Jobs
• Partner with customers in presales to understand their model objectives, current data gaps, and technical constraints. • Shape requirements: define acoustic specs, language/accent coverage, speaker demographics, emotional/paralinguistic range, transcript and metadata schema, and QA targets (WER/DER, LUFS, etc.). • Translate requirements into scoped execution plans — volumes, timelines, methodology, pricing inputs — in partnership with delivery. • Serve as the credible technical voice in the room: explain tradeoffs (studio vs. real-world vs. telephonic, scripted vs. spontaneous, single vs. multi-speaker) and defend methodology choices. • Build reusable solutioning assets: scoping frameworks, spec templates, reference architectures for common audio data use cases. • Stay at the tip of the spear on speech-AI developments (TTS, ASR, speech-to-speech) and what data the next generation of models will need. • Produce go-to-market material: technical blog posts, white papers, benchmark reports, and reference content that demonstrates Innodata's depth. • Represent Innodata externally: speak at and work conferences (Interspeech, ICASSP, industry events), engage the speech-AI community, and build our public technical profile. • Feed market intelligence back into strategy — advise on emerging data categories and where to invest ahead of demand.
• Integrating and leading the deployment of the API Security Platform and its components across diverse enterprise ecosystems. • Resolving deployment problems on Kubernetes platforms, Virtual Appliances, Cloud infrastructure, Networking, API gateways, Authentication flows, Hybrid architectures, Distributed applications. • Acting as a partner to customer engineering, DevOps, cloud, networking, or security teams to deploy and operationalize Akamai API Security Platform. • Analyzing API traffic to identify misconfigurations, authentication gaps, performance bottlenecks. • Supporting configuration of in/out-bound traffic sources for major API gateways and platforms (such as F5, Citrix, Apigee, Kong, NGINX, or MuleSoft). • Developing enterprise architecture diagrams, integration patterns, deployment plans, and operational runbooks. • Leading customer workshops, technical enablement sessions, and architectural reviews.
Solutions Architect – Technical Team Lead, AWS, MCP
OneSeven Tech (OST)Digital Product Studio for Startups and Enterprises.
• Set and own the cloud-native architecture on AWS for this MCP project, ensuring scalability and security from the ground up • Lead the technical team (engineers, AI specialists) building this project, staying hands-on with code and architecture decisions • Be the primary technical point of contact with the client, translating complex architecture decisions into clear business value • Define strategy for agentic workflows, RAG implementations, and MCP-based integrations • Conduct code reviews, enforce engineering best practices, and lead technical interviews to benchmark talent as the team scales
• Work closely with clients to understand their business goals around implementing AI technologies, present sustainable and achievable technical solutions, and adjust strategies based on feedback. • Guide and mentor engineering teams with your AI expertise, both internally to Livefront and externally with the client, by setting technical standards, conducting code reviews, and contributing to product vision. • Engage in technical spikes to explore new technologies and methods. • Lead the design of robust architectures tailored to client needs and implement these solutions hands-on, ensuring scalability and maintainability. • Design, develop, and integrate APIs, data pipelines, and end-to-end infrastructure that ensure seamless communication between systems. Help optimize performance and enable AI models to be useful, scalable, and maintainable. • Adapt your skills across diverse projects, including API design, real-time technical support, and DevOps, demonstrating versatility and responsiveness to varied client needs. • Collaborate cross-functionally, streamlining communication between backend teams, frontend teams, designers, and other team members.
