AI Testing

Location

Canada

Posted

19 days ago

Salary

0

Seniority

Mid Level

No structured requirement data.

Job Description

AI Testing

BCE GLOBAL TECHNOLOGY CENTRE PRIVATE LIMITED

Role Description BCE Global Tech's Global Quality Engineering (GQE) function is building one of Canada's most ambitious AI quality programs — certifying every AI and agentic system deployed across Bell Canada before it reaches production. As a QA AI Specialist, you sit at the intersection of artificial intelligence, software engineering, and quality assurance: a hybrid role that does not yet have a textbook, because the discipline is being written in real time. You will do two things simultaneously: - Bring AI into GQE's existing testing practice — embedding AI-powered capabilities into the test automation tooling, pipelines, and frameworks that 250 QA engineers already use every day. - Build and operate the evaluation frameworks that test the AI systems being created by other Bell engineering teams — agents, orchestration pipelines, RAG applications, Salesforce AgentForce workflows, and ServiceNow Now Assist integrations. Qualifications - 5+ years of software quality engineering experience, with at least 2 years working directly with AI/ML systems, LLMs, or AI-powered applications. - Hands-on experience building or evaluating LLM-based applications — including prompt engineering, RAG pipelines, or agentic workflows. - Proficiency in Python: test framework development, API integration, data processing, and evaluation scripting. - Experience with modern test automation frameworks (Playwright, Selenium, Pytest, RestAssured, Postman/Newman) and CI/CD platforms (GitHub Actions, Google Cloud Build, Jenkins). - Working knowledge of at least one major AI/ML platform — Google Vertex AI, Azure OpenAI, or AWS Bedrock — with hands-on API usage. - Strong conceptual understanding of how LLMs work: tokenization, temperature and sampling, context windows, grounding, hallucination mechanics, and fine-tuning. - Demonstrated ability to design test strategies for non-deterministic systems — moving beyond assertion-based testing to probabilistic, rubric-based evaluation. Requirements - AI-Enhanced QA Tooling - Modernize GQE’s QA stack by embedding AI to improve speed, coverage, and intelligence: - Integrate AI-driven test generation into Selenium, Playwright, and Postman frameworks. - Use predictive models to prioritize tests based on code changes and defect history. - Enable self-healing automation for UI/API changes. - Automate defect triage and root-cause analysis using failure clustering. - Support natural-language test authoring (English/French) for non-technical QA. - Continuously pilot emerging AI testing tools via a technology radar. - AI Evaluation & Quality Pipelines - Build scalable evaluation systems tailored for AI behavior, not rule-based logic: - Implement LLM-as-Judge pipelines on Vertex AI (Gemini) across key quality dimensions. - Generate large, diverse, and adversarial test corpora from seed intents. - Evaluate RAG systems using metrics like faithfulness, relevance, and recall (RAGAS). - Validate multi-step agent workflows, tool usage, and escalation behavior. - Embed AI evaluations into CI/CD as mandatory release gates. - AI Safety & Adversarial Testing - Operate a dedicated AI red-teaming capability to uncover AI-specific risks: - Execute prompt injection and poisoned-context attacks on RAG systems. - Run automated jailbreak and constraint-bypass probes (e.g., Garak). - Systematically test hallucination, numerical accuracy, and domain knowledge. - Assess toxicity, bias, and fairness across English and French interactions. - Stress-test agentic systems for runaway actions and scope violations. - Continuous Quality Evolution - Ensure the quality framework evolves as models and systems change: - Monitor production AI outputs for quality drift and trigger re-certification. - Feed real production failures back into the test corpus. - Track model/version changes and generate quality delta reports. - Maintain a living benchmark of Bell-specific AI quality standards. - Continuously adopt new evaluation research and industry best practices. - Partner early with AI/ML teams to embed quality by design. - AI Quality Certification Operations - Lead technical execution of the AIQC program: - Own Tier 2 & 3 certification testing from corpus design to red-teaming. - Calibrate LLM-as-Judge rubrics using human-labeled golden datasets. - Produce clear AI Quality Certificates with scores, risks, and conditions. - Advise teams on AI testability, prompts, and evaluation instrumentation. - Contribute to AIQC playbooks, documentation, and knowledge sharing. Benefits - Competitive salaries and comprehensive health benefits. - Flexible work hours and remote work options. - Professional development and training opportunities. - A supportive and inclusive work environment. - Access to cutting-edge technology and tools.

Related Job Pages

More Artificial Intelligence Jobs

Evidence Action logo

Chief Program Officer

Evidence Action

We aim to be a world leader in scaling evidence-based and cost-effective programs to reduce the burden of poverty.

Full TimeRemoteTeam 501-1,000Since 2013H1B No Sponsor

• Serve as a core member of the executive leadership team, shaping organizational strategy, culture, and enterprise-wide priorities. • Hire and mentor a high-performing leadership team and staff, creating a highly mission-driven culture focused on moving with the scale, thoughtfulness, and urgency required. • Ensure every program has a strong learning architecture: clear hypotheses, rigorous measurement, rapid iteration cycles, and honest reporting. Use cost-effectiveness evidence to guide resource allocation — scaling what earns it, cutting what doesn't. • Develop and steward 2AI's open-sourced playbooks and insights — building durable field knowledge, not just running programs. • Set and achieve ambitious program strategies for 2AI's Agriculture and Health portfolios — vision, priorities, delivery models, and expansion roadmap — in close collaboration with the CEO and regional and program leads. Achieve high-caliber reach and impact for hundreds of millions of people. • Design and run fast, rigorous pilots. This includes strong measurement and feedback loops from day one, rapid iteration on what isn't working, and clear-eyed calls on what deserves to scale. • Stay at the frontier of AI-enabled solutions for LMICs; actively tracking, testing, and integrating emerging tools and approaches. Bring intellectual curiosity and the ability to learn rapidly in a fast-moving space, translating what's technically possible into what's practically scalable for underserved populations. • Partner with technical and research teams to ensure AI-powered approaches are grounded in rigorous evidence, adapted to local realities, and designed for sustainable delivery. • Serve as 2AI's programmatic bridge among AI labs, global development actors, leading researchers, frontline implementers, and regional governments — translating credibly across all of these worlds. • Shape technical offerings in partnership with leading AI labs, including adapting AI tools for underrepresented languages, low-bandwidth environments, and last-mile contexts. • Support regional leaders — including our India Country Director — in navigating government relationships, adoption pathways, and policy enablement, providing strategic input and senior representation where needed. • Lead efforts on fundraising — contributing to compelling narratives for major funders, leading grant development, and making the case for ambitious investment in the portfolio. • Represent 2AI at senior levels in public and private forums, building 2AI's profile as the leading ecosystem coordinator for AI deployment in LMICs.

United States
$210K - $250K / year
3Cloud logo

Principal Architect – Data & AI Solutions

3Cloud

Delivering the ultimate Microsoft Azure experience.

Full TimeRemoteTeam 501-1,000H1B No Sponsor

• Lead as a technical evangelist and trusted advisor for clients, shaping early-stage Data & AI conversations into compelling solution visions built on the Microsoft platform. • Partner closely with sales and account teams to drive opportunity qualification, solution strategy, and deal closure for Data & AI engagements. • Lead technical discoveries with clients to understand business priorities, decision criteria, and value drivers, translating them into winning Data & AI architectures. • Lead design and present Data & AI solutions that clearly articulate business outcomes, technical feasibility, and ROI. • Provide executive-level technical depth in Azure with depth in data engineering, data architect, machine learning, or AI to influence client stakeholders from practitioner to C-level. • Lead technical credibility throughout the sales cycle, including whiteboarding sessions, architecture reviews, proposals, and executive presentations. • Guide and mentor architects and data scientists to ensure solution consistency, technical excellence, and alignment with go-to-market strategy. • Identify and lead opportunities to expand pipeline through innovation workshops, POVs, and targeted thought leadership aligned to Microsoft Data & AI offerings. • Ensure proposed solutions align with Microsoft best practices, security standards, and regulatory requirements. • Stay current on Microsoft Data & AI roadmaps and industry trends to continuously sharpen competitive positioning and client impact.

United States
$168.4K - $217.8K / year
Steel Partners logo

AI Intern

Steel Partners

We are a diversified global holding company with operations in diversified industrial products, energy, defense, supply chain management and logistics, banking and youth sports. We work with our companies to increase long term corporate value for all our stakeholders and shareholders utilizing Steel Partners Operational Excellence Programs, the Steel Partners Purchasing Council, Steel Partners Corporate Services, balance sheet improvements, capital allocation policies and growth initiatives. Steel Partners Operational Excellence Programs include Lean Manufacturing, Design for Six Sigma, Six Sigma and Strategy Deployment to reduce and eliminate waste. Please find out more at www.steelpartners.com

Role Description The Artificial Intelligence (AI) Intern will be embedded across Steel Partners' corporate functions — including Finance, Operations, HR, and Strategy — to help design, build, and deploy AI-powered tools, automations, and intelligent bots. The intern will work closely with functional leaders and the investment team to identify inefficiency, prototype solutions, and implement AI strategies that drive measurable impact. Given the lean structure of our corporate team, the AI Intern will take on meaningful ownership of projects from day one and have direct exposure to senior leadership. Responsibilities - AI Strategy & Implementation: Partner with functional teams to assess processes, identify automation opportunities, and develop AI-driven solutions that reduce manual work and improve decision-making. - Bot Development: Design, build, and deploy intelligent bots (e.g., workflow automation bots, data retrieval bots, reporting assistants) tailored to specific business functions such as finance, HR, supply chain, and operations. - Tool Integration: Integrate large language models (LLMs) and AI APIs into existing workflows and internal systems to enhance productivity across corporate functions. - Data & Analytics: Analyze internal data sets to surface insights, build dashboards, and support data-driven reporting for portfolio companies and the investment team. - Process Documentation: Document AI solutions, technical workflows, and user guides to enable adoption and sustainability of new tools. - Cross-Functional Collaboration: Work across departments to understand pain points, gather requirements, and ensure solutions are practical and user-friendly. - Research & Emerging Tech: Monitor advancements in AI/ML tools, platforms, and best practices, and bring forward relevant innovations for evaluation. Qualifications - Currently pursuing or recently completed (within 1–2 years) a Bachelor's or Master's degree in Computer Science, Data Science, Engineering, Information Systems, or a related field. - Hands-on experience with AI/ML frameworks, LLM APIs (e.g., OpenAI, Anthropic, Google), or automation platforms. - Proficiency in Python and/or JavaScript; experience with REST APIs and cloud services a plus. - Strong analytical and problem-solving skills with a bias toward action and building. - Excellent communication skills — able to translate technical concepts for non-technical business stakeholders. - Strong organizational skills and ability to manage multiple projects simultaneously across different functional teams. - Experience or coursework in industrial, manufacturing, defense, finance, or supply chain domains considered a plus. - A self-starter mentality and comfort working in a fast-paced, lean environment. Benefits - Networking opportunities with key business and technology community members. - Opportunity to earn class or internship credits pursuant to your attending college or university's credentials. - Real-world experience building AI solutions deployed across a global holding company. - Great mentors who care about your professional and technical growth. - Available on site or remote. New York City or Miami location available for corporate internships. Company Description Steel Partners Holdings L.P. is a global diversified holding company with operations spanning diversified industrial products, energy, defense, supply chain management and logistics, banking, and youth sports. Founded in 1990 by Warren G. Lichtenstein, Steel Partners has grown from a small private investment partnership into a $2 billion enterprise with approximately 5,200 employees across 90 locations in 14 countries. In January 2026, Steel Partners returned to being a privately held, family-owned company, continuing its long-term, values-driven approach to ownership and operations.

United States

Role Description The AI Practitioner leads AI enablement and implementation initiatives for pharma, biotech, and regulated clients. This role bridges strategy and hands-on execution—driving AI adoption through workshops, advisory, and development of AI workflows, automations, and agents. Key Responsibilities - AI Strategy & Advisory - Conduct AI maturity assessments and develop enablement roadmaps - Advise clients on AI capabilities, limitations, and automation opportunities - Guide AI agent strategy and identify high-value use cases - AI Enablement & Training - Deliver hands-on workshops (e.g., ChatGPT, Claude, AI tools) - Run office hours and 1:1 coaching sessions - Tailor enablement programs based on user maturity levels - AI Development & Implementation - Build and deploy AI workflows, automations, and agents - Support transition from experimentation to production use - Integrate AI solutions into business processes - Client Engagement & Delivery - Lead end-to-end AI engagements with ownership from day one - Collaborate with stakeholders to refine solutions and outcomes - Continuously improve delivery approach based on feedback - Internal Enablement & Innovation - Support internal AI adoption and knowledge sharing - Experiment with emerging AI tools and approaches - Drive continuous improvement across AI offerings Qualifications - Experience in life sciences, pharma, or regulated industries - Hands-on experience building AI solutions (workflows, agents, integrations) - Strong understanding of AI capabilities and limitations - Experience leading workshops or client-facing engagements - Ability to communicate complex AI concepts to varied audiences - Strong written communication skills Preferred Qualifications - Experience in consulting, MSP, or professional services - Familiarity with enterprise AI platforms and workflow tools Core Skills & Attributes - Strong AI fluency and experimentation mindset - Ability to bridge business needs with technical execution - Excellent facilitation and stakeholder engagement skills - Adaptability across varying user skill levels - Curiosity and continuous learning approach - Ability to deliver value quickly in fast-paced environments Benefits - Competitive salary - 100% Remote (U.S.) - Must work Eastern Time Zone hours - Mid–Senior Level (AI-focused, client-facing delivery experience) This is a remote position.

United States