Mindrift logo
Mindrift

Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid. Project time expectations: Tasks are estimated to require around 10–20 hours per week during active phases, based on project requirements; This is an estimate, not a guaranteed workload, and applies only while the project is active. Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project.

Senior Software Engineer - AI Code Testing & Evaluation

Location

Worldwide

Posted

20 hours ago

Salary

$40 / hour

Seniority

Senior

Job Description

Senior Software Engineer - AI Code Testing & Evaluation

Mindrift

Role Description Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based. What this opportunity involves: - Building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. - Creating challenging tasks and evaluation criteria within realistic simulated environments: - Build realistic developer environments - a virtual company with codebase, infrastructure, and context (tickets, docs, conversations) that forms a believable development history. - Design tasks from intermediate states of these environments - craft the prompt, define what "solved" means, and ensure the task is solvable by an AI agent. - Write tests that verify agent solutions - accept all valid approaches and reject incorrect ones, neither too strict nor too lenient. - Iterate on tasks and tests based on QA feedback - review agent solutions, analyze failures, and refine until the evaluation is fair and robust. What this is NOT: - Not data labeling. - Not prompt engineering. - Not writing code from scratch - the agent writes most of the code; you guide and evaluate. Qualifications - 5+ years in software development. - Core stack: Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, Redis. - Experience writing tests (functional, integration). - English proficiency - B2+. Requirements - Deep understanding of where models fail and what scenarios reveal the difference between a good and a bad solution. - Ability to create tasks that genuinely challenge the best models. - Writing tests that accept all correct solutions and reject incorrect ones. Benefits - Tasks for this project are estimated to take 20 hours to complete, depending on complexity. - You choose when and how to work. - Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. - Compensation: Up to $40/hr equivalent, depending on level and pace. - Tasks are estimated at ~20 hours each; you set your own schedule.

Related Job Pages

More Software Engineer Jobs

Title: Senior Oracle WebCenter Developer Category Information Technology Employment Type Full Time Job Description: Responsibilities for this Position Location: Any Location / Remote Full Part/Time: Full time Job Req: RQ223328 Type of Requisition: Regular Clearance Level Must Currently Possess: None Clearance Level Must Be Able to Obtain: None Public Trust/Other Required: NACI (T1) Job Family: Software Engineering Job Qualifications: Skills: Java (Programming Language), Oracle JDeveloper, Web Applications, Web Services Certifications: None Experience: 7 + years of related experience US Citizenship Required: Yes Job Description: PRINCIPAL BACK END ENGINEER Transform technology into opportunity as a Principal Back End Engineer. Here, you'll see the bigger picture on mission initiatives and where your program management career can go at GDIT. MEANINGFUL WORK AND PERSONAL IMPACT - Install, configure, harden, and maintain Oracle WebCenter and related middleware environments, including addressing and remediating Cyber Security findings and vulnerabilities. - Design, develop, maintain, and deploy REST APIs built with Spring Boot and Java 17 in Oracle WebCenter 14.x environments, ensuring secure, scalable integrations. - Design, develop, and build Oracle Portlets and portlet applications using JDeveloper, including customization and integration with other enterprise systems. - Manage and maintain code repositories using GitHub, including branching, pull requests, code reviews, and release management to support collaborative development. - Design, develop, integrate, and customize Oracle WebCenter solutions using: - Front-end technologies: ADF, HTML/CSS/JavaScript - Back-end languages: Java, Python (and related Oracle technologies) - Assess DOE application modules, integrations, and source data across multiple lines of business, and architect solutions using Java, SQL, scripting, and Oracle technologies to meet functional, performance, and security requirements. - Partner with customers, business users, and internal teams to elicit and refine requirements for new portal and document-sharing capabilities, and translate them into technical designs and implementation plans. - Serve as a core member of the team supporting day-to-day WebCenter/Middleware operations, including incident response, defect resolution, and enhancement deployment. - Collaborate closely with other technical professionals-such as DBAs, other DOE application teams, network engineers, and Oracle support/technicians-to investigate, troubleshoot, and resolve complex system issues. - Develop and maintain test harnesses, test plans, and test scenarios to validate functionality, performance, and security for WebCenter and integration components. - Lead and participate in workgroup meetings and requirements sessions, and produce high-quality functional and technical design documents. - Participate in and conduct peer reviews of requirements, designs, and code to ensure quality, consistency, and adherence to standards and best practices. - Document team workflows, build and deployment steps, configuration baselines, and operational procedures to support long-term maintainability, auditability, and knowledge transfer. WHAT YOU'LL NEED TO SUCCEED Bring your engineering expertise along with a drive for innovation to GDIT. The Back End Engineer Principal must have: Education: Bachelor of Arts/Bachelor of Science Experience: 8+ years of related experience Technical skills: - Senior-level experience with Oracle Database and the following Oracle WebCenter-related frameworks (or similar enterprise platforms): - Oracle WebCenter Content - Oracle WebCenter Portal - JDeveloper IDE - Oracle Internet Directory - Strong hands-on experience in backend Java development using Spring Boot and Java 17, including design and implementation of complex, enterprise-scale services that integrate with WebCenter. - senior-level experience with Oracle Database and the following Oracle WebCenter-related frameworks (or similar enterprise platforms): - Oracle WebCenter Content - Oracle WebCenter Portal - JDeveloper IDE - Oracle Internet Directory - Demonstrated experience leading or owning deployment, configuration, and optimization activities for Oracle WebCenter 14.x, with solid working knowledge of Oracle WebCenter 12.x. - Advanced proficiency in Oracle Portlets design and development, including managing compilation, packaging, deployment, and operational builds within JDeveloper IDE. - Practical experience using GitHub (or similar enterprise version control platforms) for branching strategies, pull requests/code reviews, and release management. - Proven ability to author, review, and maintain technical documentation, including solution designs, configuration run books, deployment guides, and operational procedures for WebCenter-based solutions. - WebCenter environments, including performance tuning and root-cause analysis. - Hands-on experience with: - JDBC, PL/SQL, ADF - HTML/CSS/JavaScript - Shell scripting - Backend programming technologies - Experience leading efforts to identify, analyze, and translate business requirements for WebCenter and related applications into scalable technical solutions, serving as a subject matter expert and trusted advisor to stakeholders. - Experience designing, developing, consuming, and debugging SOAP and REST web services that integrate with Oracle WebCenter and related Oracle technologies. - Proven track record of driving continuous improvement of business processes and technical operations (e.g., automation of deployments, standardization of WebCenter configurations, implementation of best practices). Security clearance level: Ability to obtain and maintain a DOE Clearance and successfully pass a comprehensive government background screening, including detailed forms and fingerprinting US citizenship required GDIT IS YOUR PLACE At GDIT, the mission is our purpose, and our people are at the center of everything we do. Growth: AI-powered career tool that identifies career steps and learning opportunities Support: An internal mobility team focused on helping you achieve your career goals Rewards: Comprehensive benefits and wellness packages, 401K with company match, and competitive pay and paid time off Flexibility: Full-flex work week to own your priorities at work and at home Community: Award-winning culture of innovation and a military-friendly workplace OWN YOUR OPPORTUNITY Explore a career in software development at GDIT and you'll find endless opportunities to grow alongside colleagues who share your dedication to advancing innovation. The likely salary range for this position is $129,813 - $143,750. This is not, however, a guarantee of compensation or salary. Rather, salary will be set based on experience, geographic location and possibly contractual requirements and could fall outside of this range. Scheduled Weekly Hours: 40 Travel Required: None Telecommuting Options: Remote Work Location: Any Location / Remote Additional Work Locations: Total Rewards at GDIT: Our benefits package for all US-based employees includes a variety of medical plan options, some with Health Savings Accounts, dental plan options, a vision plan, and a 401(k) plan offering the ability to contribute both pre and post-tax dollars up to the IRS annual limits and receive a company match. To encourage work/life balance, GDIT offers employees full flex work weeks where possible and a variety of paid time off plans, including vacation, sick and personal time, holidays, paid parental, military, bereavement and jury duty leave. GDIT typically provides new employees with 15 days of paid leave per calendar year to be used for vacations, personal business, and illness and an additional 10 paid holidays per year. Paid leave and paid holidays are prorated based on the employee's date of hire. The GDIT Paid Family Leave program provides a total of up to 160 hours of paid leave in a rolling 12 month period for eligible employees. To ensure our employees are able to protect their income, other offerings such as short and long-term disability benefits, life, accidental death and dismemberment, personal accident, critical illness and business travel and accident insurance are provided or available. We regularly review our Total Rewards package to ensure our offerings are competitive and reflect what our employees have told us they value most. About Our Work: We are GDIT. A global technology and professional services company that delivers consulting, technology and mission services to every major agency across the U.S. government, defense and intelligence community. Our 26,000 experts extract the power of technology to create immediate value and deliver solutions at the edge of innovation. We operate across 50 countries worldwide, offering leading capabilities in digital modernization, AI/ML, Cloud, Cyber and application development. Together with our clients, we strive to create a safer, smarter world by harnessing the power of deep expertise and advanced technology.

United States
$129.8K - $143.8K / year
Upland Software logo

Software Engineer II

Upland Software

Name a challenge. We’ve got a product to solve it.

Software Engineer21 hours ago
Full TimeRemoteTeam 1,001-5,000Since 2010H1B Sponsor

Role Description We are looking for an enthusiastic and dynamic individual to join Upland India as a Software Engineer II (Full stack) for our Panviva product. The individual will work with our global development team. - Develop, review, test, and maintain application code - Collaborate with other developers and product to fulfill objectives - Troubleshoot and diagnose issues - Take lead on tasks as needed - Jump in and help the team deliver features when required Qualifications - A Bachelor’s degree in Computer Science or equivalent Requirements - 3-6 years of experience - Experience with front-end React/JQuery/JavaScript, HTML, and CSS Frameworks - Solid experience in .NET Core, SQL Server - Strong understanding of OOPS concepts and SOLID principles - Unit testing tools and debugging techniques - SQL or other RDBMS - Real-world problem solving - Solid understanding of Agile development processes, including sprint planning, daily stand-ups, retrospectives, and backlog grooming - Familiarity with Agile tools such as JIRA for tracking tasks, managing workflows, and collaborating across teams - Experience working in cross-functional Agile teams and contributing to iterative development cycles - Experience building modern, scalable, reliable applications on the MS Azure cloud - API Design (REST) - Frontend frameworks (React and/or Angular) - Pure JavaScript, jQuery - Bootstrap, HTML5, CSS, SASS - Worked with any cloud-based version control mainly GitHub or Bitbucket - Cloud native deployment tools such as AKS Benefits - Flexibility to manage your working hours to help in your work-life balance

India
Huntington National Bank logo

Software Development Engineer

Huntington National Bank

Sine 1866, Huntington National Bank has served midwestern communities with banking and financial services for consumers and businesses of all sizes. The regiona

Software Engineer21 hours ago

Title: Software Development Engineer (Lead) Location: Austin United States Job Description: Job Profile Summary The Lead Software Development Engineer provides technical and consultative support on highly complex technology initiatives. This role, designs, develops, supports cloud-native applications, APIs, event-driven systems, and enterprise integration platforms. The individual serves as a senior technical contributor, helping evaluate and implement technology solutions that are scalable, secure, resilient, and aligned with enterprise standards. Duties and Responsibilities - Analyzes, designs, develops, and supports enterprise applications and platform services. - Evaluates architectural approaches and recommends appropriate technologies based on business requirements, cost, scalability, reliability, and operational supportability. - Develops cloud-native solutions using APIs, event-driven patterns, and distributed systems principles. - Provides technical leadership and guidance during design reviews and implementation activities. - Troubleshoots complex production issues and drives root cause analysis and remediation. - Maintains knowledge of industry trends, cloud technologies, and enterprise integration patterns. - Mentors less experienced engineers and serve as technical lead, leading by example on projects. - Performs other duties as assigned. Basic Qualifications - Bachelor's Degree. - 7+ years of software engineering, platform engineering, or cloud engineering experience. - Experience developing enterprise applications using Java and/or Python. - Experience designing and supporting distributed systems and APIs. Preferred Qualifications - Experience with cloud-native architecture and Google Cloud Platform (GCP). - Strong understanding of API, messaging, event-driven, and microservices architectures. - Experience evaluating technology tradeoffs and selecting appropriate solutions for business and technical requirements. - Experience with Pub/Sub, Dataflow, Apigee, and CI/CD practices. - Strong analytical, troubleshooting, and communication skills. - Ability to influence technical direction across multiple teams. Preferred Technical Skills - Python, Java, SQL - GCP - Pub/Sub - Spring Boot, REST APIs - Dataflow, BigQuery - CI/CD and Automation - Observability and Monitoring Key differentiator: Demonstrated ability to evaluate and choose between cloud services, APIs, messaging platforms, event-driven solutions, and data processing technologies based on scalability, reliability, operational supportability, security, and cost considerations. Exempt Status: (Yes = not eligible for overtime pay) (No = eligible for overtime pay) Yes Workplace Type: Office Our Approach to Office Workplace Type Certain positions outside our branch network may be eligible for a flexible work arrangement. We're combining the best of both worlds: in-office and work from home. Our approach enables our teams to deepen connections, maintain a strong community, and do their best work. Remote roles will also have the opportunity to come together in our offices for moments that matter. Specific work arrangements will be provided by the hiring team. Compensation Range: 70,000.00 - 140,000.00 USD Annual The compensation range represents the anticipated low and high end of the base compensation range for this position. Actual compensation will vary based on various factors including but not limited to location, experience, and education.  Colleagues in this position are also eligible to participate in an applicable incentive compensation plan.  In addition, Huntington provides a variety of benefits to colleagues, including health insurance coverage, wellness program, life and disability insurance, retirement savings plan, paid leave programs, paid holidays and paid time off (PTO).

Texas
$70K - $140K / year
SAIC - Science Applications International Corporation logo

React Developer Senior

SAIC - Science Applications International Corporation

SAIC - Science Applications International Corporation was founded in 1969 to create scientific and technical products used in industries such as national securi

Software Engineer21 hours ago

Title: React Developer Senior Job ID: 2614259 Location: Ashburn, VA, United States Date Posted: Jul 7, 2026 Category: Software Subcategory: SW Engineer Schedule: Full-Time Shift: Day Job Travel: No Minimum Clearance Required: None Clearance Level Must Be Able to Obtain: Public Trust Potential for Remote Work: Hybrid Benefits: Click here Share: mail Apply Now > Apply Now > Job Description Description SAIC is currently seeking a Mid-Level React UI Developer to join our team. This position will work closely with the Lead Architect and other developers participating in the system architecture design, systems engineering, development, integration testing and validation, and all aspects of the software life cycle (requirements to design to implementation to test) for a browser-based application implemented using React interfacing with backend microservice APIs. The candidate will be responsible for developing code as dictated by project requirement. This position is hybrid-remote and requires 2 days on-site per week in Ashburn, VA (Thursdays and Fridays). Job Responsibilities - Design, develop, unit test UI interfaces using React. - Use Integrated Development Environment Tools such as Visual Studio Code. - Work with users in an Agile development approach to understand requirements, then elicit feedback and refine solution through demonstrations. - Provide application support during testing and production phase and work with the lead to deploy code fixes as needed. Qualifications Required Skills and Experience - BS Degree in Computer Science or Engineering and 5+ years of software development experience including designing, development, and documenting UI code developed a part of a large-scale software system. - Demonstrated 5+ years of experience with React development. - Expertise with: - React. - Typescript / JavaScript. - MVC/MVVM design patterns. - JSON. - HTML4/5. - CSS, SCSS. - Jest, Enzyme, React Testing library. - Strong understanding of browser-based debug tools (eg., chrome devtools.) - Experience utilizing various debugging tools and methodologies for debugging UI and components. - Strong working experience interfacing with complex backend APIs. - Demonstrated ability to think critically and creatively to develop innovative solutions to software problems. Desired Skills and Experience - Developing unit tests. - Agile development approach. - Strong initiative: must be able to take a high-level requirement, ask questions/seek clarifications, and get the job done. - Superior written and oral communication skills are essential. - Must work well in a team environment as well as independently. - Excellent verbal/written communication and problem solving skills; ability to communicate information to a variety of groups at different technical skill levels. Customer requirements - Clearance - Ability to obtain and hold a public trust position and favorable suitability based on a CBP Background Investigation. - Citizenship - Must be a US Citizen. - Location - Hybrid with on-site (Ashburn, VA) visits 2 days per week and if a mission need. All candidates must be within the GWA/commutable area. Target salary range: $120,001 - $160,000. The estimate displayed represents the typical salary range for this position based on experience and other factors. Overview SAIC accepts applications on an ongoing basis and there is no deadline. SAIC is a premier mission integrator focused on advancing the power of technology and innovation to serve and protect our world. Our robust portfolio of offerings across the defense, space, intelligence, and civilian markets includes secure high-end solutions in mission IT, enterprise IT, engineering services, and professional services. We integrate emerging technology, rapidly and securely, into mission critical operations that modernize and enable critical national imperatives. We are approximately 23,000 strong; driven by mission, united by purpose, and inspired by opportunities. SAIC is an Equal Opportunity Employer. Headquartered in Reston, Virginia, SAIC has annual revenues of approximately $7.3 billion. For more information, visit saic.com. For ongoing news, please visit our newsroom. Share: mail Apply Now >

Virginia
$120.0K - $160K / year