Mindrift logo

Mindrift

Remote Jobs

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

9,648 open rolesLatest: Jul 17, 2026, 11:13 AM UTCCompany Site
Post Date
Minimum Salary
Experience

9648 Jobs

Mindrift logo

Ex-MBB Strategy Consultant - AI Training

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Strategy1 day ago

Role Description Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners. Mindrift, powered by Toloka, connects top domain experts with cutting-edge AI initiatives. We are launching a Management Consulting domain focused on translating real-world consulting engagements into structured learning environments for advanced AI systems. You will join a growing team of consultants from leading strategy firms shaping how AI learns high-level business reasoning. Important: This role is exclusively for consultants with direct experience at a top-tier strategy consulting firm. If you do not have hands-on project experience at one of the firms listed below, please do not apply. - Eligible firms: McKinsey & Company - Boston Consulting Group (BCG) - Bain & Company - Oliver Wyman - Roland Berger - Monitor Deloitte (Deloitte S&C) - EY-Parthenon - Kearney - Strategy& (PwC) Qualifications - Consultants with 3+ years of experience at one of the firms listed above - Hands-on project experience in: - Structuring ambiguous client problems into workable analytical plans - Building financial models, market analyses, or synthesized findings from messy inputs - Producing client-ready deliverables under time pressure - Forming and defending recommendations under uncertainty - No deep technical background is required — onboarding will be provided on the lightweight tools involved Requirements - 3+ years at McKinsey, BCG, Bain, Oliver Wyman, Roland Berger, Monitor Deloitte, EY-Parthenon, Kearney, or Strategy& - Strong structured problem-solving and hypothesis-driven thinking - Ability to translate vague problems into clear analytical steps and deliverables - High attention to logical consistency and output quality - Independent, self-directed working style - Clear written English (B2+) Benefits - On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution - Compensation varies across projects depending on scope, complexity, and required expertise - Please note that other projects on the platform may offer different earning levels based on their requirements

France
$60 / hour
Mindrift logo

Freelance Web Designer

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Web Designer1 day ago

Role Description Mindrift is looking for a highly skilled Web Designer to join the Tendem project (https://tendem.ai/) and drive specialized data scraping workflows within our hybrid AI + human system. In this role, you'll collaborate with Tendem Agents that handle repetitive tasks, while you provide the critical thinking, domain expertise, and layout skills necessary to turn briefs and rough drafts into clean, on-brand, and easy-to-ship web-ready designs. This part-time remote opportunity is ideal for creative professionals with hands-on experience in UI/web design, marketing pages, and digital visual communications. Key Responsibilities - Design high-converting one-pagers, pitch-style landing pages, and simple marketing sites based on textual copy, outlines, or existing drafts. - Translate brand assets and social media visuals into responsive web layouts, structured components, and sections that can be easily implemented by developers or no-code tools. - Prepare and securely package production-ready files (organizing Figma frames, exporting assets, and detailing specs) for seamless handoff. - Ensure absolute visual consistency between interactive web pages, static PDFs, and social materials. - Enforce layout quality standards through systematic verification of responsiveness, proper spacing, clarity, and adherence to modern web trends prior to delivery. Qualifications - At least 2 years of relevant experience in web design, UI design, digital marketing design, or visual communications is desirable. - Bachelor’s or Master’s Degree in Web Design, Graphic Design, Human-Computer Interaction (HCI), or related creative/technical fields is a plus. - Deep proficiency in Figma for creating scalable, responsive web layouts, auto-layout components, and comprehensive design systems. - Expertise in structuring high-converting, pitch-style landing pages with a strong eye for clarity, spacing, and modern design trends. - Solid understanding of how to prepare, spec, and package digital assets for efficient implementation via developers or no-code platforms (e.g., Webflow, Framer). - Ability to seamlessly translate core brand aesthetics across varying touchpoints, ensuring web pages match existing social materials and pitch decks. - Upper-intermediate (B2) or above English proficiency (required). - A structured, user-centric, and conversion-minded approach to design. - Strong eye for clarity, white space, and modern web layout trends. - Self-directed work ethic with the ability to troubleshoot independently and deliver production-ready assets fast. - Your CV must include a link to your portfolio with web design examples. Applications without a portfolio link, or without relevant web design cases, will not be considered. Project Time Expectations For this project, tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Compensation On this project, contributors can earn up to $50 per hour equivalent, depending on their level and pace of contribution. Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements.

Worldwide
$50 / hour
Mindrift logo

Senior Python Data Scraping Engineer

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift is looking for highly skilled Senior Python Data Scraping Engineers to join the Tendem project and drive specialized data scraping workflows within our hybrid AI + human system. In this role, as an AI Pilot – that’s how we refer to this role at Mindrift – you’ll collaborate with Tendem Agents that handle repetitive tasks, while you provide critical thinking, domain expertise, and quality control to deliver accurate and actionable results. This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction and processing. This is a freelance role for a Tendem project. As a Senior Python Data Scraping Engineer, you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing various tools such as our provided Apify and OpenRouter alongside your own resourceful approaches. Key Responsibilities - Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets. - Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements. - Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior. - Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery. - Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes. Qualifications - At least 5+ years of relevant experience in data engineering, web scraping, automation, or software development (required). - Bachelor’s or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus. - Strong technical foundation and practical experience with scripting, automation, and AI-assisted workflows. - Ability to solve non-trivial problems and work confidently with LLMs. - Methodical, detail-oriented approach and the ability to work independently are essential. - Strong experience in Python web scraping (BeautifulSoup, Selenium or similar), including dynamic content (JS, AJAX, infinite scroll) and APIs via proxies. - Proven ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML). - Solid background in data cleaning, normalization, and validation, delivering structured datasets (CSV, JSON, Google Sheets). - Demonstrated experience handling anti-bot mechanisms and dynamic site structures at scale. - Experience with cloud infrastructure (AWS or equivalent) and containerization (Docker) as part of real workflows. - Hands-on experience with LLM frameworks (LangChain, OpenRouter, or similar) applied to automation tasks. - Strong attention to detail and commitment to data accuracy. - Self-directed work ethic with ability to troubleshoot independently. - A link to GitHub is a plus. - English proficiency: Upper-intermediate (B2) or above (required). Requirements - Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements. This is an estimate, not a guaranteed workload, and applies only while the project is active. Compensation - On this project, contributors can earn up to $30 per hour equivalent, depending on their level and pace of contribution. - Compensation varies across projects depending on scope, complexity, and required expertise. - Please note that other projects on the platform may offer different earning levels based on their requirements.

Worldwide
$30 / hour
Mindrift logo

Freelance Data Scraping Engineer

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Data Engineer2 days ago

Role Description Mindrift is looking for highly skilled Python Data Scraping Engineers to join the Tendem project and drive specialized data scraping workflows within our hybrid AI + human system. In this role, as an AI Pilot – that’s how we refer to this role at Mindrift – you’ll collaborate with Tendem Agents that handle repetitive tasks, while you provide critical thinking, domain expertise, and quality control to deliver accurate and actionable results. This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction and processing. Key Responsibilities - Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets. - Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements. - Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior. - Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery. - Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes. Qualifications - At least 3 years of relevant experience in data engineering, web scraping, automation, or software development (required). - Bachelor's or Master’s Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus. - Strong experience in Python web scraping (BeautifulSoup, Selenium or similar), including dynamic content (JS, AJAX, infinite scroll) and APIs via proxies. - Proven ability to extract data from complex structures (hierarchies, archived pages, inconsistent HTML). - Solid background in data cleaning, normalization, and validation, delivering structured datasets (CSV, JSON, Google Sheets). - Hands-on experience with LLMs and AI frameworks to enhance automation and problem-solving. - Strong attention to detail and commitment to data accuracy. - Self-directed work ethic with ability to troubleshoot independently. - A link to GitHub is a plus. - English proficiency: Upper-intermediate (B2) or above (required). Compensation On this project, contributors can earn up to $37 per hour equivalent, depending on their level and pace of contribution. Compensation varies across projects depending on scope, complexity, and required expertise. Please note that other projects on the platform may offer different earning levels based on their requirements. Benefits - Work fully remote on your own schedule with just a laptop and stable internet connection. - Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly — a distinctive skill set in a rapidly growing field. - Participate in performance-based bonus programs that reward high-quality work and consistent delivery.

Worldwide
$37 / hour
Mindrift logo

Senior Python Engineer - AI Coding Agent Evaluation

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

AI Engineer2 days ago

Role Description Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves: - Building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. - Creating challenging tasks and evaluation criteria within realistic simulated environments: - Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. - Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent. - Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. What this is NOT: - Not data labeling. - Not prompt engineering. - Not writing code from scratch - the agent writes most of the code; you guide and evaluate. Qualifications - 8+ years in software development. - Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis. - Experience writing tests (functional, integration). - English proficiency - B2+. Requirements - Deep understanding of where models fail and what scenarios reveal the difference between a good and a bad solution. - Ability to create tasks that genuinely challenge the best models. - Writing tests that accept all correct solutions and reject incorrect ones. Benefits - Compensation: Up to $200/hr equivalent, depending on level and pace. - Tasks are estimated at ~30 hours each; you set your own schedule. How it works - Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. - Tasks for this project are estimated to take 30 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. - Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Worldwide
$200 / hour
Mindrift logo

Chatbot Developer

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift is looking for skilled Bot Developers (WhatsApp Business API, Telegram Bot API, Discord API) to join the Tendem project (https://tendem.ai/) and build conversational bots and messaging-platform integrations within our hybrid AI + human environment. In this role, as an AI Pilot – that's how we refer to this position at Mindrift – you'll collaborate with Tendem Agents that handle repetitive tasks, while you provide bot engineering expertise, conversational design judgment, and quality control to ensure bots are reliable, useful, and ready for real users. This part-time remote opportunity is ideal for professionals with hands-on experience building messaging bots, working with platform APIs and webhooks, and implementing conversational logic. Key Responsibilities - Build bots for one or more messaging platforms, such as WhatsApp (Business API / Cloud API), Telegram (Bot API), Discord, Slack and similar messaging platforms. - Design and implement conversational flows, dialogue state, and fallback handling. - Integrate bots with LLMs (OpenAI, Anthropic, or similar) for natural language responses where appropriate. - Connect bots to backend services, databases, CRMs, and third-party APIs (booking systems, payment, content sources). - Handle webhooks, rate limits, and platform-specific message formats (interactive messages, buttons, media, templates). - Evaluate AI-generated bot code and refactor it for correctness, reliability, and graceful error handling. - Implement logging, monitoring, and recovery so bots stay healthy in production. Qualifications - At least 3 years of relevant experience in backend, integration, automation, or bot development (required). - Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related technical fields is a plus. - Strong foundation in bot development, messaging platform integrations, and building reliable conversational workflows. - Ability to design and maintain production-ready bots, work confidently with APIs, webhooks, and backend services. - Strong problem-solving skills, attention to detail, and the ability to work independently are essential. Requirements - At least 1 year of hands-on experience building bots for at least one major messaging platform (WhatsApp, Telegram, Discord, Slack, or similar) is required. - Strong command of Python or Node.js for backend bot logic. - Solid experience with REST APIs, webhooks, OAuth, and async request handling. - Experience with relational or NoSQL databases for storing conversation state and user data. - Familiarity with LLM APIs (OpenAI, Anthropic) and prompt design for conversational use is a strong plus. - Understanding of platform-specific limits, message templates, and approval flows (e.g., WhatsApp template messages). - Experience with hosting and deployment (Docker, serverless, VPS, or PaaS). - Strong attention to detail and commitment to bot reliability — no silent failures, no broken flows. - Self-directed work ethic with the ability to design and ship complete bots independently. - Portfolio or examples of bots you've built (required). - English proficiency: Upper-intermediate (B2) or above (required). Compensation - On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution. - Compensation varies across projects depending on scope, complexity, and required expertise. Benefits - Work fully remote on your own schedule with just a laptop and stable internet connection. - Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly — a distinctive skill set in a rapidly growing field. - Participate in performance-based bonus programs that reward high-quality work and consistent delivery.

Worldwide
$60 / hour
Mindrift logo

Chatbot Developer (WhatsApp, Telegram, Discord) - Freelance

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift is looking for skilled Bot Developers (WhatsApp Business API, Telegram Bot API, Discord API) to join the Tendem project (https://tendem.ai/) and build conversational bots and messaging-platform integrations within our hybrid AI + human environment. In this role, as an AI Pilot, you'll collaborate with Tendem Agents that handle repetitive tasks, while you provide bot engineering expertise, conversational design judgment, and quality control to ensure bots are reliable, useful, and ready for real users. This part-time remote opportunity is ideal for professionals with hands-on experience building messaging bots, working with platform APIs and webhooks, and implementing conversational logic. Key Responsibilities - Build bots for one or more messaging platforms, such as WhatsApp (Business API / Cloud API), Telegram (Bot API), Discord, Slack, and similar messaging platforms. - Design and implement conversational flows, dialogue state, and fallback handling. - Integrate bots with LLMs (OpenAI, Anthropic, or similar) for natural language responses where appropriate. - Connect bots to backend services, databases, CRMs, and third-party APIs (booking systems, payment, content sources). - Handle webhooks, rate limits, and platform-specific message formats (interactive messages, buttons, media, templates). - Evaluate AI-generated bot code and refactor it for correctness, reliability, and graceful error handling. - Implement logging, monitoring, and recovery so bots stay healthy in production. Qualifications - At least 3 years of relevant experience in backend, integration, automation, or bot development (required). - Bachelor's or Master's Degree in Computer Science, Engineering, Information Technology, or related technical fields is a plus. - Strong foundation in bot development, messaging platform integrations, and building reliable conversational workflows. - Ability to design and maintain production-ready bots, work confidently with APIs, webhooks, and backend services. - Strong problem-solving skills, attention to detail, and the ability to work independently are essential. Requirements - At least 1 year of hands-on experience building bots for at least one major messaging platform (WhatsApp, Telegram, Discord, Slack, or similar) is required. - Strong command of Python or Node.js for backend bot logic. - Solid experience with REST APIs, webhooks, OAuth, and async request handling. - Experience with relational or NoSQL databases for storing conversation state and user data. - Familiarity with LLM APIs (OpenAI, Anthropic) and prompt design for conversational use is a strong plus. - Understanding of platform-specific limits, message templates, and approval flows (e.g., WhatsApp template messages). - Experience with hosting and deployment (Docker, serverless, VPS, or PaaS). - Strong attention to detail and commitment to bot reliability — no silent failures, no broken flows. - Self-directed work ethic with the ability to design and ship complete bots independently. - Portfolio or examples of bots you've built (required). - English proficiency: Upper-intermediate (B2) or above (required). Compensation - On this project, contributors can earn up to $60 per hour equivalent, depending on their level and pace of contribution. - Compensation varies across projects depending on scope, complexity, and required expertise. Benefits - Work fully remote on your own schedule with just a laptop and stable internet connection. - Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly — a distinctive skill set in a rapidly growing field. - Participate in performance-based bonus programs that reward high-quality work and consistent delivery.

Worldwide
$60 / hour
Mindrift logo

Senior Software Engineer - AI Code Testing & Evaluation

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based. What this opportunity involves: - Building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. - Creating challenging tasks and evaluation criteria within realistic simulated environments: - Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. - Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent. - Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. What this is NOT: - Not data labeling. - Not prompt engineering. - Not writing code from scratch - the agent writes most of the code; you guide and evaluate. Qualifications - 5+ years in software development. - Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis. - Experience writing tests (functional, integration). - English proficiency - B2+. Requirements - Deep understanding of where models fail and what scenarios reveal the difference between a good and a bad solution. - Ability to create tasks that genuinely challenge the best models. - Writing tests that accept all correct solutions and reject incorrect ones. Benefits - Tasks for this project are estimated to take 20 hours to complete, depending on complexity. - You choose when and how to work. - Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. - Compensation: Up to $40/hr equivalent, depending on level and pace. - Tasks are estimated at ~20 hours each; you set your own schedule.

Worldwide
$40 / hour
Mindrift logo

Senior Software Engineer - AI Coding Agents

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based. What this opportunity involves: - Building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. - Creating challenging tasks and evaluation criteria within realistic simulated environments: - Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. - Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent. - Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. What this is NOT: - Not data labeling. - Not prompt engineering. - Not writing code from scratch - the agent writes most of the code; you guide and evaluate. Qualifications - 5+ years in software development. - Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis. - Experience writing tests (functional, integration). - English proficiency - B2+. Requirements Why this is hard: - Frontier models are already good at coding. Creating a task that genuinely challenges the best models is non-trivial. - You need to deeply understand where models fail and what scenarios reveal the difference between a good and a bad solution. - Tasks have many valid solutions - writing tests that accept all correct solutions and reject incorrect ones is harder than it sounds. Benefits - Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. - Tasks for this project are estimated to take 20 hours to complete, depending on complexity. - This is an estimate and not a schedule requirement; you choose when and how to work. - Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. Compensation - Up to $40/hr equivalent, depending on level and pace. - Tasks are estimated at ~20 hours each; you set your own schedule.

Worldwide
$40 / hour
Mindrift logo

Senior Software Engineer - Testing AI Coding Agents

Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Role Description Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based. What this opportunity involves: - Building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. - Creating challenging tasks and evaluation criteria within realistic simulated environments: - Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. - Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent. - Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. What this is NOT: - Not data labeling. - Not prompt engineering. - Not writing code from scratch - the agent writes most of the code; you guide and evaluate. Qualifications - 5+ years in software development. - Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis. - Experience writing tests (functional, integration). - English proficiency - B2+. Requirements - Deep understanding of where models fail and what scenarios reveal the difference between a good and a bad solution. - Ability to create tasks that genuinely challenge the best models. - Writing tests that accept all correct solutions and reject incorrect ones. Benefits - Compensation up to $50/hr equivalent, depending on level and pace. - Tasks are estimated at ~20 hours each; you set your own schedule. Effort Estimate Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.

Worldwide
$50 / hour
Job Closed

9,638more opportunities are still waiting for you.Log in now and take your next shot before someone else does.