Job Closed

This listing is no longer active.

Gramian Consulting Group logo
Gramian Consulting Group

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

AI Evaluation Engineer - Software Engineering Domain

Location

India + 9 moreAll locations: India | Brazil | Colombia | Egypt | Pakistan | Indonesia | Bangladesh | Ghana | Kenya | Nigeria

Posted

61 days ago

Salary

0

Seniority

Mid Level

Job Description

AI Evaluation Engineer - Software Engineering Domain

Gramian Consulting Group

Role Description We are looking for highly analytical engineers and technical domain experts to contribute to advanced AI evaluation and benchmarking projects focused on realistic terminal-based and infrastructure-heavy workflows. In this role, you will design technically challenging tasks that evaluate how AI systems reason through debugging, operational failures, complex workflows, and multi-step problem-solving scenarios. The ideal candidate has strong experience working with production systems, debugging, automation, or large-scale engineering workflows, and can design realistic technical challenges that simulate real-world engineering environments. This role is particularly well suited for professionals with backgrounds in backend engineering, infrastructure, DevOps, data systems, MLOps, cybersecurity, or platform engineering. - CONTRACT: Contractor assignment (5 weeks) - COMMITMENT: Full-time (40h/week) or Part-time (20h/week) with minimum 4h PST overlap - LOCATION: Remote — Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Pakistan, Indonesia, Kenya, Nigeria, Turkey, Vietnam - PROCESS: One technical assessment/interview (~45 min) Responsibilities - Design realistic terminal-based benchmark tasks for AI evaluation systems - Create technically deep debugging and investigation scenarios - Develop task specifications involving infrastructure, workflows, pipelines, or operational failures - Write clear solution approaches and deterministic evaluation criteria - Identify realistic edge cases, failure modes, and system constraints - Design multi-step reasoning challenges across complex technical environments - Contribute expertise across one or more engineering or operational domains - Review and refine benchmark quality, difficulty, and validation logic - Collaborate with reviewers and researchers on AI evaluation workflows Qualifications - 3–10 years of experience in software engineering or related technical domains - Strong debugging, analytical, and systems reasoning skills - Good understanding of system architecture, dependencies, and operational processes - Experience with terminal, CLI, automation, or developer tooling workflows - Exposure to AI systems, LLMs, benchmarking, or evaluation frameworks is preferred - Ability to design technically rigorous and realistic engineering scenarios

Related Job Pages

More Full-stack Engineer Jobs

The Hello Team logo

Software Developer

The Hello Team

Managed global staffing across 30 plus countries with enterprise recruiting, oversight, training, and performance manage

Full TimeRemoteTeam 1,001-5,000Since 2021H1B No Sponsor

• Develop and maintain scalable software features and applications • Assist in building and improving management software systems • Test, debug, and troubleshoot applications • Monitor and maintain system stability and performance • Collaborate with the team on product enhancements • Participate in code reviews and contribute to development best practices • Integrate and maintain APIs and frontend/backend components • Document technical processes and workflows

New York
Castillians logo

Senior Fullstack Engineer

Castillians

The world's trusted engineering network

ContractRemoteTeam 51-200Since 2006H1B No Sponsor

• Develop responsive web applications. • Create intuitive interfaces for laptops, monitors, tablets, and mobile devices. • Build real-time monitoring dashboards displaying vital signs and activity levels. • Implement alert systems and notification interfaces. • Design user-friendly interfaces for care task management and compliance tracking. • Develop record digitalization interfaces. • Design and implement RESTful APIs for system integration. • Develop microservices architecture for data processing. • Integrate with AI computer vision systems for fall detection and behavior analysis. • Build secure data processing pipelines for wearable device data. • Implement real-time alert generation and routing systems. • Develop integration layers for camera systems. • Create APIs for BLE hub integration and smart band data collection. • Integrate with existing facility platforms and systems. • Implement SDK integrations for camera systems and wearable devices. • Develop secure APIs for multi-device access across the network. • Build data synchronization systems between facilities. • Implement GDPR-compliant data handling and privacy protection. • Implement trend analysis and early warning indicators. • Build comprehensive reporting systems for care documentation. • IoT device integration and BLE connectivity. • Camera system integration and computer vision APIs. • Wearable device data processing and analysis. • Real-time data streaming and processing.

Ireland
Capstone Integrated Solutions logo

Senior Full Stack Developer

Capstone Integrated Solutions

A full-service software and services company

ContractRemoteTeam 51-200H1B No Sponsor

• Design and develop the application architecture, UI/UX, and technical requirements during the Discovery & Design phase. • Build and deploy a document upload portal enabling subcontractors to submit insurance certificates, bonding letters, and financial statements. • Develop RESTful and serverless APIs using AWS Lambda and Amazon API Gateway to support frontend/backend integration. • Integrate AI-powered document validation and exception management workflows driven by Amazon Bedrock. • Develop and host frontend applications using AWS Amplify and Amazon S3. • Participate in CI/CD pipeline setup, automated testing, and deployment to production environments. • Collaborate with cross-functional team members including AI/ML, DevOps, and Data Engineering resources. • Conduct integration testing, support user acceptance testing (UAT), and develop user training materials. • Contribute to knowledge transfer documentation and post-deployment operations guides.

New York

Role Description We are seeking a skilled Full Stack Developer with expertise in gRPC. The ideal candidate will have a passion for creating seamless user experiences and robust backend solutions, and will work closely with our talented team of engineers and scientists to develop our web and mobile applications. - Design, develop, and maintain web applications using React and TypeScript. - Develop and manage server-side logic using Node.js. - Create and consume APIs, ensuring robust and scalable solutions. - Collaborate with cross-functional teams to define, design, and ship new features. - Write unit and integration tests to ensure code quality and reliability. - Troubleshoot and debug applications to optimize performance and user experience. - Continuously discover, evaluate, and implement new technologies to maximize development efficiency. Qualifications - Must have completed or be enrolled in an accredited degree program (BS/MS/PhD) in Computer Science, Mathematics, or a related field at an academic institution in the United States. - Proven experience as a Full Stack Developer or similar role. - Proficient in front-end technologies, specifically React and TypeScript. - Strong experience with back-end technologies, specifically Node.js. - Experience with gRPC for communication between services. - Hands-on experience with building APIs. - Familiarity with database technologies (e.g., PostgreSQL). - Knowledge of version control systems (e.g., Git). - Understanding of the entire web development process, including design, development, and deployment. - Excellent problem-solving skills and attention to detail. - Strong communication and teamwork skills. Requirements - Equity-based compensation. - We are only considering US citizens and Permanent Residents for the role.

United States