AI Evaluator | $80 Remote
Crossing Hurdles | Lebanon (EMEA regional default)
Contract | Remote
Posted
Salary: $30 - $80/hour
Job description
Position: AI Evaluator | $80 RemoteType: ContractCompensation: $30 - $80/hourLocation: RemoteCommitment: 10-40 hrs/week Role ResponsibilitiesDevelop and map analytic cases to defined taxonomies across multiple evaluation areas.Author complex test cases to evaluate model performance on source handling and analytic rigor.Assess and grade AI-generated responses based on reasoning, source use, and practical utility.Identify and flag instances of over-refusal in analytic exploration by the model.Document edge cases and analytic disagreements to support evaluation methodology improvement.Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.Contribute insights to calibrate analytic standards and enhance AI system benchmarks. RequirementsDemonstrated all-source analytic experience or deep expertise in a specific analytic discipline.Strong proficiency in source evaluation and confidence-level assessment.Experience in teaching, mentoring, or leading analytic reviews with a focus on analytic rigor.Exceptional written and verbal communication skills with clarity and precision.Ability to identify and articulate nuanced model limitations and over-refusal scenarios.Familiarity with structured analytic techniques and intelligence tradecraft standards. Application ProcessEasy Apply on LinkedInCheck email for next stepsParticipate in resume evaluation & interview stage
Apply Now