Job Closed
This listing is no longer active.
Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
Freelance Agent Evaluation Engineer
Location
Alabama
Posted
100 days ago
Salary
0
Seniority
Mid Level
Job Description
Freelance Agent Evaluation Engineer
Mindrift
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation isproject-based, not permanent employment. What this opportunity involves You’ll create challenging coding test cases that push AI coding systems to their limits: - Review and refine realistic coding tasks based on provided production codebases with realistic scope, requirements and information sources - Write comprehensive functional tests that validate actual end-to-end behavior and edge-cases, not just superficial checks - Craft “fair but hard” challenges where the AI has all the context it needs, but has to work for it (information scattered across files and external sources, complex reasoning required) - Analyze AI failures to understand what the model struggles with vs. what it masters - Iterate based on feedback from expert QA reviewers who score your work on 7 quality criteria What we look for This opportunity is a good fit for experienced developers, software engineers, and/or test automation specialists open to part-time, non-permanent projects. Ideally, contributors will have: - Degree in Computer Science, Software Engineering or related fields - 5+ years in software development, primarily Python (pytest, async/await, subprocess, file operations) - Background in Full-Stack development, with an equal focus on building React-based interfaces and robust Back-end systems - Experience writing tests (functional, integration – not just running them) - Docker containers (running evaluations locally in containers) - CI/CD understanding (GitHub Actions as a user: triggers, labels, reading results) - English proficiency - B2 How it works Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid Effort estimate Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. Payment - Paid contributions, with rates up to $80/hour* - Fixed project rate or individual rates, depending on the project - Some projects include incentive payments *Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
Related Guides
Related Job Pages
More Software Engineer Jobs
Freelance Agent Evaluation Engineer
MindriftApply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation isproject-based, not permanent employment. What this opportunity involves You’ll create challenging coding test cases that push AI coding systems to their limits: - Review and refine realistic coding tasks based on provided production codebases with realistic scope, requirements and information sources - Write comprehensive functional tests that validate actual end-to-end behavior and edge-cases, not just superficial checks - Craft “fair but hard” challenges where the AI has all the context it needs, but has to work for it (information scattered across files and external sources, complex reasoning required) - Analyze AI failures to understand what the model struggles with vs. what it masters - Iterate based on feedback from expert QA reviewers who score your work on 7 quality criteria What we look for This opportunity is a good fit for experienced developers, software engineers, and/or test automation specialists open to part-time, non-permanent projects. Ideally, contributors will have: - Degree in Computer Science, Software Engineering or related fields - 5+ years in software development, primarily Python (pytest, async/await, subprocess, file operations) - Background in Full-Stack development, with an equal focus on building React-based interfaces and robust Back-end systems - Experience writing tests (functional, integration – not just running them) - Docker containers (running evaluations locally in containers) - CI/CD understanding (GitHub Actions as a user: triggers, labels, reading results) - English proficiency - B2 How it works Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid Effort estimate Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. Payment - Paid contributions, with rates up to $80/hour* - Fixed project rate or individual rates, depending on the project - Some projects include incentive payments *Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.
• Crestron Programming (Primary Craft) • Serve as the subject matter expert for Crestron control system programming — design, develop, and modify control code in SIMPL, SIMPL+, and/or C#/VC-4 • Configure and customize system functionality to meet engineering specifications and customer requirements • Develop, modify, and troubleshoot Extron control system code (Global Configurator, Global Scripter, or similar) as projects require • Build and maintain reusable code modules and standards that improve consistency across PEAKE projects • System Design & Documentation • Design, read, and interpret system schematics, cable riser diagrams, and architectural drawings • Translate engineering designs into working control systems and validate signal flow across AV, control, and network components • Contribute to design reviews and flag programming or integration risks early • Installation & On-Site Execution • Work alongside AV Engineers and field technicians to install AV systems from start to finish in customer environments • Oversee equipment rack buildouts and staging — wire lacing, grounding, power distribution, and hardware integration • Terminate and test audio, video, network, control, and power cabling • Mount and configure displays, video walls, microphones, speakers, and control interfaces as needed • Maintain a clean, organized, and professional work environment on every job site • Commissioning & Troubleshooting • Lead system commissioning — testing, performance validation, and final turnover to the customer • Troubleshoot complex issues across control, AV, and network systems, both remotely and on-site • Validate that systems meet design and operational requirements before sign-off • Project Coordination & Communication • Serve as the primary point of contact for customers and stakeholders • Coordinate daily with AV Engineers and Program Managers to keep projects aligned with milestones • Communicate project status, technical decisions, blockers, and timelines clearly to internal teams and external stakeholders • Represent PEAKE professionally in customer-facing government and military environments • Guide and support the field engineers on installation best practices and issue resolution
• Supervise and advise the scrum team to meet software expectations • Manage the product development team to create a strong end product • Nurture ideas and solutions to existing customer problems • Communicate effectively with team members to achieve project goals • Extract and retrieve information and data sets to improve upon software • Determine roadmaps for products in the creation phase • Work closely with the scrum team throughout the development process • Schedule and lead meetings to identify issues and fixes for projects • Own, mentor and guide the team for overall project delivery • Creating business Mockups in Lucid chart - as and when required • Support in release management activities and should lead from front
Founding Forward Deployed Engineer
AscertainAgentic AI for healthcare operations. Ascertain what matters, automate the rest.
This description is a summary of our understanding of the job description. Click on 'Apply' button to find out more. Role Description As a Forward Deployed Engineer at Ascertain, you sit at the intersection of product, engineering, and real-world customer operations. Your job is to understand the complex, messy systems healthcare organizations run on and help shape Ascertain’s product so it works in practice, not just in theory. - Work closely with customers to translate their workflows and edge cases into production-grade solutions. - Integrate systems, extend workflows, and build features. - Identify patterns and feed them back into the core product to solve problems once and scale. - Define the playbook with clear opportunities to grow into pod leadership and, over time, people management. This role is US-based remote, requiring close product collaboration and occasional customer travel (roughly 10–20%). The role reports to the Chief Product Officer. Qualifications - Strong communication skills and experience in a customer-facing role. - Several years of experience as a software engineer, with a track record of shipping production systems and owning work from ambiguous problem definition through delivery. - 4+ years of experience in full stack development using Python, FastAPI, and TypeScript. - Strong programming skills with experience building or integrating with external APIs, third-party systems, or imperfect data sources. - Comfort working at the intersection of product, engineering, and customers. - Proven ability to manage complexity and context-switch across multiple workstreams without losing momentum or detail. - Startup experience preferred. Requirements - A strong sense of ownership: you take responsibility for outcomes, not just tasks, and you follow problems through to resolution. - Comfort working in ambiguous, evolving environments, with the ability to create structure where none exists and adapt as priorities shift. - Thoughtful technical judgment, including knowing when to move quickly and when to invest in durability, reuse, or productization. - A collaborative mindset and low ego: you’re comfortable partnering across Product, Engineering, Sales, and Customer teams. - A customer-oriented lens grounded in curiosity and empathy. - Interest in helping build something bigger than your individual contributions. Benefits - Meaningful work: Your work will directly impact healthcare organizations by reducing administrative burden and improving how care teams operate day to day. - Ownership and growth: You’ll join Ascertain at an early stage, with meaningful equity and the opportunity to shape our product, technical direction, and how this function evolves over time. - Competitive compensation: We offer a competitive salary, meaningful equity, comprehensive health benefits, and other standard benefits appropriate for a growing company. - Flexible work model: This role is remote-friendly. For NYC-based team members, we support a hybrid model with regular in-office collaboration. - Collaborative team: You’ll work alongside a thoughtful, mission-driven team with backgrounds spanning healthcare operations, product, and engineering.

