Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
AI Evaluation Engineer
Location
India + 9 moreAll locations: India | Brazil | Colombia | Egypt | Turkey | Indonesia | Bangladesh | Ghana | Kenya | Nigeria
Posted
32 days ago
Salary
0
Seniority
Mid Level
Job Description
AI Evaluation Engineer
Gramian Consulting Group
Role Description We are looking for an AI Evaluation Engineer specialized in planning and operations to design and build benchmark tasks that simulate real-world scenarios such as scheduling, logistics, and resource allocation. This role focuses on planning, scheduling, and operational optimization problems, where multiple agents must collaborate to solve constraint-rich scenarios involving resources, timelines, and dependencies. Commitments Required: 8 hours per day with an overlap of 4 hours with PST. Employment type: Contractor assignment (no medical/paid leave) Duration of contract: 4 weeks+ Location: Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Kenya, Nigeria, Turkey, Vietnam Interview: take home assessment (60min) + short interview Responsibilities - Design and build multi-agent benchmark tasks involving: - Planning, scheduling, and resource allocation - Operational decision-making (logistics, project planning, incident response, capacity planning) - Create constraint-rich problem statements with multiple interacting variables - Develop verification scripts to evaluate: - Feasibility (all constraints satisfied) - Completeness (all requirements met) - Optimality (efficiency of solutions) - Define task decomposition strategies across specialized sub-agents (e.g., resource allocation, constraint resolution, optimization) - Model realistic operational systems with dependencies, timelines, and constraints - Implement validation logic and evaluation pipelines using Python - Work with Docker environments for reproducibility and execution - Collaborate with internal teams to improve task quality, coverage, and evaluation rigor Qualifications - 5+ years of experience in operations, project management, logistics, or supply chain - Strong ability to formalize constraints, dependencies, and scheduling logic - Proficiency in Python for building validation and verification scripts - Experience with optimization techniques (linear programming, constraint satisfaction, scheduling algorithms) - Strong structured problem-solving and decomposition skills - Experience with AI benchmarks or evaluation frameworks (e.g., SWE-bench or similar) - Hands-on experience with Docker (Dockerfiles, image builds, debugging) Nice to Have - Background in operations research or optimization-heavy domains - Experience with simulation or modeling tools - Familiarity with AI planning systems or automated reasoning - Project management experience or certifications (PMP, Agile, etc.)
Related Guides
Related Job Pages
More Software Engineer Jobs
Senior Software Engineer
Unity TechnologiesUnity [NYSE: U] is the world’s leading game engine, powering play for more than 3 billion consumers each month. The top mobile games in the world, the most played PC indie titles, the most innovative console games, and virtually all of the top XR and Web Games are developed, deployed, and grown in Unity. Unity also enables teams across industries like automotive, manufacturing, and healthcare to design, simulate, and collaborate in 3D — closing the gap between ideas and reality. Unity is a proud equal opportunity employer. We are committed to fostering an inclusive, innovative environment and celebrate our employees across age, race, color, ancestry, national origin, religion, disability, sex, gender identity or expression, sexual orientation, or any other protected status in accordance with applicable law.
Role Description Senior Software Engineer to lead design and implementation of our business-critical data platform. - High autonomy role, embedded in a team of senior software and data engineers, collaborating on the data platform problem and solution space. - Translating complex, ambiguous requirements into resilient software engineering solutions. - Effective use of data is a core focus for the broader company, delivering high-quality solutions that solve real business problems and customer problems across thousands of Unity titles. What you'll be doing - Partnering with stakeholders, defining the problem space, pitching technical solutions, and owning them through to production. - Identifying reliability concerns and opportunities for improvement. - Pitching and delivering architectural improvements. - Influencing the roadmap towards technical excellence in software and data engineering. Qualifications - Strong software engineering fundamentals, and experience applying them within data engineering. - Expertise working with data streaming technologies such as Kafka and Flink. - Autonomous and effective in a team setting. - Experience working with non-technical stakeholders and translating their needs to technical requirements for team-wide delivery. Requirements - Relocation support is not available for this position. - Work visa/immigration sponsorship is not available for this position. Benefits - Comprehensive health, life, and disability insurance. - Commute subsidy. - Employee stock ownership. - Competitive retirement/pension plans. - Generous vacation and personal days. - Support for new parents through leave and family-care programs. - Office food snacks. - Mental Health and Wellbeing programs and support. - Employee Resource Groups. - Global Employee Assistance Program. - Training and development programs. - Volunteering and donation matching program.
Reporting Developer, SQL, Apache Superset
NatuvionNatuvion supports its customers in moving business-critical data and processes from one technology platform to another.
• Development and optimization of complex SQL queries • Building reporting and dashboard solutions with Apache Superset (datasets, SQL Lab, visualizations) • Performance tuning related to processing large volumes of data • Integration and analysis of heterogeneous data sources (especially SAP) • Acting as an interface between Product Development, SAP Consulting and Delivery • Participating in projects and producing technical documentation
Role Description As a Senior Software Engineer, you will play a key technical leadership role in architecting, designing, and developing our next-generation payment platform. This position requires deep expertise in distributed systems, Java-based microservices, and high-volume transaction processing. You will be responsible for ensuring the platform meets the highest standards of scalability, security, and reliability while working closely with product, infrastructure, and security teams. This position offers tremendous career growth and the opportunity to make a direct impact in a rapidly expanding international company. - Leading the design and architecture of complex distributed systems that handle real-time financial transactions at scale. - Spearheading the integration of payment gateways, banks, card networks, and alternative payment methods. - Taking a lead role in our transition to a microservices-based architecture for payments. - Creating and maintaining high-quality, optimized code with robust unit tests and appropriate test coverage. - Providing technical leadership, mentoring junior developers, and guiding the team in best practices and efficient coding techniques. - Collaborating with cross-functional teams to deliver highly scalable, performant solutions. - Driving continuous improvement initiatives, identifying bottlenecks, and optimizing the software development lifecycle. - Collaborating with DevOps to optimize CI/CD pipelines, and monitoring strategies for production systems. Qualifications - Excellent academic background: Bachelor’s or Master’s degree in Computer Engineering or a related field. - Proven experience (7+ years) in backend software development, with at least 3 years leading teams and architecting payment systems. - Proven working experience in fintech/payments. - Strong understanding of payment flows, settlement, reconciliation, and fraud detection mechanisms. - Deep knowledge of Java 11+, Spring Boot. - Strong experience with AWS cloud services, including IAM, EC2, S3, Lambda, RDS, DynamoDB, and API Gateway. - Hands-on experience with transaction management, database tuning (PostgreSQL, MySQL, or NoSQL stores), and high-availability strategies. - Extensive experience with Event-Driven Software Design Patterns and complex systems architecture. - Expertise in microservices architecture, specifically with Java 11+, Spring Boot, Spring Cloud (Netflix OSS), OAuth2 Security, and JPA ORM. - Expertise in designing secure RESTful APIs and working with OAuth2, JWT, and SSO mechanisms. - Advanced proficiency in Git for source control and versioning. - Strong technical writing skills, with the ability to produce clear and concise technical requirements, design documents, and specifications. - A proven ability to communicate complex technical concepts effectively in both English and Spanish. Benefits - 💰 Competitive Compensation - 📈 Career Growth - 🎓 Continuous Learning - 🌱 Inclusive Environment - 🏠 Work-from-home - 🎂 Birthday leave
Director of Engineering – AI
CoLab SoftwareSetting the standard in engineering collaboration. Simplified design review that lets teams build the future—faster.
• Own outcomes across multiple AI/ML product teams: execution, quality, reliability, and customer impact • Help shape technical direction for AI/ML systems, including architecture and tooling decisions • Improve how teams operate via effective planning, delivery, and use of AI tools in development • Drive alignment across Product, Architecture, Cloud, and Security teams • Lead and develop Engineering Managers and senior ICs • Design team structure, hiring plans, and onboarding as the org scales • Manage performance with clarity: goals, feedback, and growth


