Mercor logo
Mercor

Cincinnatus is an enterprise staffing company that partners with leading technology companies to source and employ highly skilled professionals for full-time and long-term contingent roles. Cincinnatus serves as the employer of record for these engagements, providing W-2 employment, payroll, benefits, and compliance, while placing employees directly within client teams to work on high-impact initiatives. Roles hired through Cincinnatus are not project-based or freelance engagements. They are structured, role-based positions that typically involve full-time or fixed-term commitments, close collaboration with a client's internal teams, and integration into standard enterprise workflows. Cincinnatus is a legal entity separate from Mercor. While opportunities may be discovered through Mercor's platform, employment, onboarding, payroll, and benefits for these roles are administered by Cincinnatus. Equal Employment Opportunity Cincinnatus is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or any other legally protected characteristic. Cincinnatus is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans throughout the job application process.

Software Engineer - Evaluation Author

Software EngineerSoftware EngineerPart TimeRemoteMid LevelH1B No Sponsor

Location

Worldwide

Posted

4 days ago

Salary

$35 - $120 / hour

Seniority

Mid Level

Job Description

Software Engineer - Evaluation Author

Mercor

Role Description - Author non-trivial coding tasks with golden solutions and automated verifiers. - Design rubrics and grade agent trajectories and model outputs. - Improve task and rubric quality through structured review. - Evaluate the accuracy and depth of AI-generated content to strengthen reasoning and rigor in model outputs. - Work independently and asynchronously to meet deadlines while improving AI model performance. Qualifications - Must-Have: 5+ years of software engineering at a real product organization (big tech or venture-backed startup). - Strong code quality, systems design, debugging, and testing discipline. - Clear written communication (you write instructions others follow). - Preferred: Familiarity with AI coding tools and evals. Requirements - Short Mercor Technical Screen. - Live Code Review Session. - Domain Expert Interview. - You're paid $200 for completing all three, regardless of outcome. Application Process - Upload resume. - AI interview based on your resume. - Submit form. - Application takes 20–30 mins to complete. Resources & Support - For details about the interview process and platform information, please check: Interview Process Details . - For any help or support, reach out to: support@mercor.com . - Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Related Job Pages

More Software Engineer Jobs

Full TimeRemoteTeam 51-200Since 2001H1B No Sponsor

• Desenvolver, manter e evoluir sistemas com foco em qualidade, legibilidade e performance • Analisar código para identificar falhas, oportunidades de melhoria e riscos técnicos • Participar de entregas de projetos, operações assistidas e treinamentos a usuários • Apoiar a configuração de ambientes produtivos dos clientes (Cutover) • Contribuir na elaboração de planos de ação em situações de crise ou redução de filas de atendimento • Atender chamados e suportes, incluindo incidentes, problemas e garantias • Elaborar especificações técnicas e estimar horas de desenvolvimento com precisão • Codificar funcionalidades conforme especificações, com baixo índice de retrabalho • Desenvolver scripts complexos, estruturas de banco de dados, rotinas e funções • Construir escopos e orçamentos para melhorias e projetos • Apoiar o PO ou Consultor em diagnósticos e direcionamento de soluções • Levantar requisitos, definir escopo e alinhar estimativas com as necessidades do cliente • Produzir documentação clara, objetiva e útil para o time e para o cliente

Brazil
Full TimeRemoteTeam 5,001-10,000Since 1995H1B No Sponsor

• Boa comunicação com o time e com o cliente, dando clareza do dia a dia; • Refinamento técnico; • Realizar a manutenção e desenvolvimento de aplicações em Cobol realizando integração com banco de Dados; • Antecipar-se a oportunidades e problemas, agindo com rapidez e eficácia, desenvolvendo soluções de forma preventiva.

Brazil

C++ Programmer

Keywords Studios

Smoking Gun Interactive is a game development studio founded by industry veterans and known for delivering high-quality titles in partnership with some of the w

Produce high-quality, modular code while creatively solving game technology issues. Meet production deadlines and proactively address tasks, ensuring feedback on code quality is provided and accepted throughout the development process.

Canada
DBSync logo

Engineering Architect

DBSync

Harness the Power of Simplified Application Integration and Data Replication.

Full TimeRemoteTeam 51-200Since 2009H1B No Sponsor

• Work on latest edge Cloud computing to solve difficult, manual and repeatable tasks for users adopting Cloud technologies like AWS, Salesforce, Microsoft and more • Use the latest trends on building SaaS applications using multitenancy, scalability large data volumes / Big Data and more • Develop and expand our portfolio to support most popular Cloud apps (200+) • Develop understanding of not just software development, but also how to design, develop and launch a product from concept to high customer use. • Manage multiple engineering projects and ensure successful delivery of products on time and within budget • Lead a team of engineers and provide guidance on technical issues • Collaborate with other teams such as product, design, and customer support to ensure smooth product development and delivery • Ensure coding standards, best practices, and security protocols are followed by the team • Hire, train, and mentor engineers and support staff • Evaluate and recommend new technologies and methodologies to improve the engineering process • Build and maintain strong relationships with vendors and partners to ensure projects are completed successfully • Must have successfully delivered at least two projects end-to-end. • Oversight of the full software development lifecycle required for a group of developers and testers in an agile environment. • Leading staff to implement clients in the most efficient, time driven manner. The manager is responsible for the total quality of the technical deliverables in their domain, making sure that they are secure, defect-free. • Mentoring technical staff during projects to ensure continuous improvement. Includes working with each resource to define and act upon career paths and obtain appropriate training. Is responsible for the hiring, training, staff development, performance appraisals, corrective action and pay review of technical personnel. • Developing and establishing department standards and procedures. • Recommends the most efficient ways to ensure best implementation practices of new upgraded products. • Evaluates and reports progress and results.

India