How Our Scoring System Works
Learn how HeyMilo's AI scores candidates objectively and consistently at scale.
Structured Questions vs Scored Questions
HeyMilo supports two types of interview questions with different evaluation approaches:
Scored Questions (covered in this document):
Use 1-5 scoring scale
Require explicit criteria for scores 1 and 5
AI infers intermediate scores (2, 3, 4)
Best for open-ended, nuanced evaluation
Structured Questions (learn more):
Use binary pass/fail evaluation
Include Yes/No, Minimum Amount, and Multiple Choice types
Can be marked as dealbreakers to automatically disqualify candidates
Best for objective, verifiable requirements
The rest of this document focuses on the 1-5 scoring system for regular interview questions.
Overview of Scoring Mechanism:
The scoring system is explicitly set by your recruiting team.
Recruiters define exactly what constitutes a score of 1 out of 5 (lowest performance) and 5 out of 5 (highest performance) per interview question, using plain-English descriptions (mimicking how a recruiter currently would be scoring a candidate).
Role of the AI in Determining Scores
The candidate's responses (audio/video) are transcribed into text.
This transcript, combined with the explicitly defined scoring benchmarks, is evaluated by our fine tuned Large Language Model (LLM).
The LLM leverages extensive natural language processing (NLP) training on vast, diverse text datasets. This allows it to interpret adjectives and qualitative descriptors (e.g., "excellent," "vague," "significant") in context, similar to human evaluators. This ensures the interpretation aligns closely with general human judgement and understanding.
Furthermore, the AI assigns scores by interpreting the candidate’s response against the clearly defined criteria reviewed by your recruiting team.
🎯 How Intermediate Scores (2, 3, and 4) are Assigned:
Intermediate scores are inferred by the AI based on the continuum explicitly set by recruiters for scores 1 and 5.
The AI evaluates how closely a candidate’s response aligns with either end of this explicitly-defined scoring spectrum.
To avoid overly rigid scoring (overfitting), intermediate levels (2–4) are not explicitly defined by recruiters, which ensures a balanced and flexible evaluation of candidate responses.
🎯 Human Oversight and Score Override:
Recruiters have full visibility into the rationale behind each AI-generated score, including access to exact transcripts.
Recruiters always have the option to manually override any AI-generated scores if they disagree or identify additional context not captured by the AI.
The final decision on candidate progression always remains with your recruiters, not the AI.
🎯 Mitigation of Bias:
The scoring system is fully transparent and auditable. Your team clearly defines evaluation criteria in plain language, reducing ambiguity and subjective interpretations.
HeyMilo routinely undergoes third-party bias audits to ensure the scoring system remains fair and unbiased.
Ongoing monitoring and the option for human overrides help further ensure fairness.
Data Use & Confidentiality
Candidate data and scoring information remain strictly confidential.
HeyMilo complies with both GDPR and SOC2 requirements. View our compliance dashboard.
Additional resources
For additional information not covered here, reach out to [email protected].
Last updated