This opportunity is available through a leading AI-driven work platform.
Senior Software Engineering Evaluator
Location
United States + 1 moreAll locations: United States | Canada
Posted
2 days ago
Salary
$45 - $70 / hour
Seniority
Senior
No structured requirement data.
Job Description
Senior Software Engineering Evaluator
24-MAG
Role Description We are sharing a specialised part-time consulting opportunity for senior software engineering professionals experienced in production codebases, open-source repositories, code review, software architecture, and structured evaluation of AI-generated solutions. This role supports a remote consulting project focused on evaluating how advanced AI systems understand, reason about, and generate production-quality software. Selected professionals will apply their engineering judgment to review AI-generated code, assess technical quality and maintainability, analyze software artifacts, and provide clear, evidence-based evaluations across modern development environments. Key Responsibilities - AI-Generated Code Evaluation - Evaluate AI-generated software solutions for correctness, quality, maintainability, and engineering best practices - Review implementation choices, software design decisions, and technical reasoning - Identify functional issues, architectural weaknesses, and areas for improvement - Provide structured evaluations supported by clear technical explanations - Repository & Software Architecture Review - Become familiar with assigned open-source codebases and their engineering practices - Navigate unfamiliar repositories and understand existing software architectures efficiently - Review code changes, technical discussions, issues, pull requests, and related software artifacts - Assess readability, maintainability, testing quality, and consistency with repository standards - Technical Collaboration & Quality Calibration - Collaborate with fellow technical reviewers to support consistent and high-quality evaluations - Apply established project criteria when assessing software solutions and engineering decisions - Compare evaluation outcomes against benchmarks, reviewer guidance, and project-specific standards - Communicate technical findings clearly in written English Qualifications - At least 3 years of professional software engineering experience or equivalent open-source development experience - Experience working with large production codebases - Strong engineering judgment and practical knowledge of software engineering best practices - Ability to quickly understand unfamiliar repositories, architectures, and technical discussions - Experience reviewing code for correctness, readability, maintainability, testing, and software design - Strong understanding of Git and GitHub workflows, including branching, issues, pull requests, and code reviews - Availability to contribute at least 10 hours during the project period - Current location in the United States or Canada to support real-time project collaboration Educational Background - A degree in computer science, software engineering, computer engineering, information technology, or a related technical field is helpful - Professional experience in backend, frontend, infrastructure, distributed systems, or open-source software development is highly relevant - Equivalent hands-on experience contributing to production systems or established open-source projects is also highly valuable - Advanced technical training or graduate-level study may be beneficial depending on the project focus Nice to Have - JavaScript or TypeScript experience with Node.js, React, TypeScript, or modern web application development - Go experience involving backend services, distributed systems, REST or gRPC APIs, and cloud-native development - Python experience across backend development, open-source libraries, DevOps, infrastructure automation, or testing frameworks - Familiarity with Kubernetes or related cloud infrastructure environments - Experience navigating and contributing to open-source repositories through GitHub - Familiarity with SWE-bench, SWE-bench Pro, or comparable software engineering evaluation benchmarks - Previous experience reviewing AI-generated code or technical model outputs Why This Opportunity - Apply senior software engineering expertise to structured remote evaluation work - Contribute to the improvement of advanced coding and software reasoning systems - Work with real-world repositories, technical artifacts, and production-oriented engineering standards - Use your experience in code review, architecture, and software quality to support high-impact evaluation workflows - Participate in a focused short-term project with competitive hourly compensation Contract Details - Independent contractor role - Fully remote with flexible scheduling - Candidates must be based in the United States or Canada - Approximate project duration of one week - Current project window scheduled for July 8–14 - Expected commitment of at least 10 hours during the project period - Competitive rates between $45–$70 per hour depending on expertise and project scope - Weekly payments via Stripe or Wise - Projects may be extended, shortened, or adjusted depending on scope and performance - The work centers on software evaluation and technical review rather than hands-on product development - Work will not involve access to confidential or proprietary information from any employer, client, or institution About the Platform This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy .
Related Guides
Related Categories
Related Job Pages
More DevOps Engineer Jobs
DevOps Engineer
MerativeA data and software partner for health and government social services, with tech and expertise to drive real progress.
• Perform product development in accordance with Merge’s methodology and practices; • Perform software module requirement analysis, design, and test design • Prepare software module specifications, designs, test descriptions and estimates • Implement software modules and unit tests as assigned and conduct appropriate module testing • Conduct appropriate verification and validation activities • Detect, report, investigate, and fix product defects as appropriate; • Participate in the review of product engineering artifacts (e.g., specifications, designs, test descriptions, implementations, tests, records, reports, etc) • Design, develop, and recommend next generation platforms for cloud and on-premise deployments • Develop deployment systems for a variety of interdependent applications running on Linux and Kubernetes • Establish and maintain continuous integration and continuous delivery (CI/CD) pipelines to automate the build, test, and deployment of applications • Assist in the preparation and review of product, engineering, and management plans and reports (e.g., program and project plans) as required • Prepare updates to system specifications, designs, test descriptions, hazard analysis, verification and validation test procedures, and related documentation supporting software development to known standards • Mentor team members and peer review product artifacts including requirement specifications, design specifications, code, etc. • Engage actively in design and code reviews of work items completed by the team in conjunction with the Team Lead and/or architect • Interface with customer(s) and Merge staff to gather product requirements, present technical aspects of the product, and understand problems with the product
Senior Site Reliability Engineer
DiscogsThe world's leading music discovery and record collecting platform empowered by a community of obsessed music fans.
• Owning tasks and larger projects from planning to production rollout • Learning new technologies and building expertise with the goal of teaching and mentoring others; mentoring with the goal of force-multiplying through docs and tools • Maintaining organization cloud presence in AWS • Automating and deploying infrastructure configurations using Infrastructure as Code (IAC) • Mentoring engineering squads on Platform best practices for Kubernetes, MySQL, Kafka, and other software development lifecycle areas • Assisting engineering squads with capacity planning, on-call preparation, and production readiness • Writing documentation and runbooks that contribute to the engineering organization’s knowledge base • Implementing monitoring and alerting systems with Discogs observability tools • Working in a containerized, orchestrated environment • Participating in on-call rotation, responding to incidents, and troubleshooting data and other operations issues • Contributing to the reliability and design patterns of our Kafka CDC and event workflows • Contributing to agentic AI best practices and tooling, including skills, agents, and safety
Staff Software Engineer, Dev Ops
CongaConga delivers the most scalable revenue lifecycle management solution to help companies crush operational complexity.
• Serve as a senior technical leader responsible for designing, building, and evolving the infrastructure, automation, and cloud platform capabilities that power Conga’s revenue lifecycle management solutions • Partner closely with Engineering, Product, Security, and Operations teams to establish scalable DevOps practices, accelerate software delivery, and improve platform reliability across our global SaaS ecosystem • Influence architectural decisions, drive modernization initiatives, and champion best practices that improve developer productivity, system scalability, and operational excellence • Help shape the future of our engineering platform by designing highly available cloud infrastructure, advancing automation capabilities, and enabling best-in-class development experiences. • Design, implement, and optimize enterprise-scale CI/CD pipelines that accelerate software delivery • Establish automation strategies that improve deployment consistency, quality, and release velocity • Architect, build, and maintain highly available cloud environments across AWS and Microsoft Azure. • Develop Infrastructure-as-Code solutions using technologies such as Terraform and CloudFormation • Champion Site Reliability Engineering (SRE) principles and reliability-focused engineering practices.
Senior Software Engineer, Dev Ops
CongaConga delivers the most scalable revenue lifecycle management solution to help companies crush operational complexity.
• Design, implement, and maintain scalable CI/CD pipelines that automate software build, testing, deployment, and release processes. • Drive continuous improvement initiatives that increase deployment efficiency, reliability, and developer productivity. • Partner with engineering teams to establish DevOps best practices across the software development lifecycle. • Champion automation-first approaches that reduce manual effort and improve operational consistency. • Design and support highly available, secure, and scalable cloud infrastructure across AWS and Microsoft Azure environments. • Develop and maintain Infrastructure-as-Code (IaC) solutions using tools such as Terraform and CloudFormation. • Optimize platform performance, cost efficiency, and operational resilience across production and non-production environments. • Partner with engineering teams to build and maintain cloud-native architectures that support growth and scale. • Design, deploy, and manage containerized workloads utilizing Docker and Kubernetes. • Establish deployment standards and operational best practices for Kubernetes-based environments. • Support container orchestration, monitoring, scaling, and lifecycle management initiatives. • Drive platform modernization efforts through the adoption of cloud-native technologies. • Implement and maintain infrastructure and application security best practices, including encryption, access controls, network security, and vulnerability management. • Design solutions that support fault tolerance, disaster recovery, and business continuity objectives. • Collaborate with teams to improve observability, monitoring, alerting, and incident response capabilities. • Proactively identify risks and implement solutions that improve system reliability and uptime. • Collaborate with software engineers, architects, and product teams to support scalable application development and deployment strategies. • Provide technical guidance on infrastructure architecture, cloud adoption, and operational best practices. • Participate in architectural reviews and contribute to long-term platform strategy discussions. • Foster a culture of continuous learning, innovation, and engineering excellence.


