AI Evaluator

About Us

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are looking for strong generalists with excellent critical-thinking and communication skills to contribute to an AI training and evaluation project covering a broad range of topics.

The work involves analyzing and annotating data, reviewing AI-generated responses, identifying issues with accuracy or reasoning, and providing clear, structured written feedback. No previous AI experience is required. This opportunity is particularly suited to people who are comfortable learning new topics quickly, following detailed guidelines, and producing consistently high-quality work.

CONTRACT: Part-time/hourly contractor assignment

LOCATIONS: Remote, GLOBAL

HOURLY RATE: $50-$90/h

Responsibilities

• Analyze and evaluate a variety of data and AI-generated content.
• Annotate information according to project-specific guidelines.
• Review AI outputs for accuracy, relevance, clarity, and quality.
• Identify errors, inconsistencies, weak reasoning, and missing information.
• Provide clear written feedback and recommendations.
• Complete problem-solving and reasoning tasks across different subject areas.
• Follow detailed project guidelines and evaluation criteria consistently.
• Participate in quality-assurance and review processes.
• Communicate findings and issues clearly with the project team.
• Maintain high attention to detail across assigned tasks.

Requirements

• Excellent written and spoken English.
• Strong critical-thinking and analytical skills.
• Ability to evaluate information and identify logical or factual problems.