# Configuring Scoring Criteria

> This is where you decide what qualifies a candidate and how HeyMilo evaluates them across every stage of your end-to-end AI recruiting flow. By setting up clear, weighted criteria, you ensure every candidate is evaluated fairly and consistently, making it easy to identify your top talent.

## Understanding HeyMilo Scoring

{% tabs %}
{% tab title="Automatic Analysis" %}
Your AI agent automatically:

* **Evaluates responses** against your defined criteria
* **Assigns scores** based on response quality and relevance
* **Provides reasoning** for each score given
* **Ranks candidates** by overall performance
* **Highlights strengths** and areas of concern
  {% endtab %}

{% tab title="Scoring Components" %}
Each candidate profile details:

* **Overall Score** (0-100): Composite score across all criteria
* **Question Scores** (1-5): Individual question performance
* **Summary:** Key highlights and lowlights from the interview
* **Speech Score:** Assessment of clarity, delivery, and tone
* **Transcript and Recording:** Full interview transcript and audio recording available for review

{% hint style="info" %}
Question scoring by HeyMilo is automatically weighted from 1-5. Recruiters also have the option to configure weight from 1-10.
{% endhint %}
{% endtab %}
{% endtabs %}

Some employers are happy using the AI-generated questions as they are, while others prefer to edit the evaluation criteria or tweak the questions themselves to better fit their needs. In some cases, employers might not use the AI-generated questions at all and start from scratch.

{% hint style="success" %}
Reviewing the AI-generated questions and customizing them to meet your exact needs ensures the interview reflects your real hiring standards, and that you can gather the most relevant information.
{% endhint %}

Every interview stage in HeyMilo includes a **Questions & Scoring** section:

<figure><img src="/files/gt1xHp7yVh6Yfsd2siua" alt="" width="233"><figcaption></figcaption></figure>

* Resume Screening
* Voice / Video Interview
* SMS Screening
* Application Form

Each stage is fully customizable and transparent.

<figure><img src="/files/veThLNeA7Z3IL0YPbhoF" alt=""><figcaption></figcaption></figure>

<details>

<summary>HeyMilo automatically:</summary>

* generates questions + criteria based on the role for each agent layered in
* proposes scoring logic
* suggests dealbreakers where appropriate

</details>

You decide what stays, what changes, and what matters.

## Customization Is the Point

Some employers:

* use the AI-generated questions exactly as they are
* make light edits to wording or scoring
* generate evaluation criteria + tune with AI
* replace everything with their own rubric and design every detail

All four are supported, and you are never locked into the AI’s first draft.

## What You Can Customize (Across All Stages)

<details>

<summary>For every question or criterion, you can:</summary>

* edit the wording
* choose how it’s evaluated
* assign importance or weight
* decide if it’s a dealbreaker
* control follow-up behavior
* exclude it from the final score if needed
* **reorder questions or criteria to control interview flow**

</details>

This applies to **every screening agent**.

## Reordering Questions and Criteria

You can **drag and drop** questions or criteria to change their order.

<figure><img src="/files/AwnnSb6y28q3rko0nZf6" alt=""><figcaption></figcaption></figure>

<details>

<summary>Reordering affects:</summary>

* the **sequence** candidates experience questions in
* which topics are covered earlier vs later
* how structured or conversational the interview feels

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Put dealbreakers first to identify unqualified candidates early
* Start voice interviews with an open-ended warm-up question
* Group related topics together (e.g. tools → collaboration → outcomes)

</details>

## Understanding Weights (What “Importance” Really Means)

Weighting tells HeyMilo **how much a question should influence the final score**.

<figure><img src="/files/2D4H73YMfe5GmPcpgHsP" alt=""><figcaption></figcaption></figure>

* Higher weight = matters more in the final score
* Lower weight = still evaluated, but less critical

You do **not** need to make everything high weight. Many teams often stick with the default settings.

<details>

<summary>How scores + weighting works</summary>

* Scored questions use a 1–5 answer scale. You can fully customize what a bad/non-ideal response looks like (1), and what a good answer looks like (5).

<figure><img src="/files/5hmip3rt4pg1rbrHWbvj" alt=""><figcaption></figcaption></figure>

* Each question has an importance level using a 1-10 scale (low → average → critical)
* HeyMilo combines score + weight to calculate the overall result

This allows strong candidates to stand out **for the right reasons**, not just because they did well on easy questions.

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Sales role
  * Objection handling = high weight
  * Background story = low weight
* Engineering role
  * Problem-solving approach = high weight
  * Tool familiarity = medium weight
* Support role
  * Communication clarity = high weight
  * Years of experience = medium weight

</details>

Weighting helps you avoid accidental bias, where less critical questions dominate results.

## Resume Screening: Early Signal, Clean Filters

Contextual Resume Screening works best when you want to filter fast before other screeners and interviews.

You can configure:

<details>

<summary>Required Qualifications</summary>

Auto-disqualifiers like certifications, portfolio links, licenses, or minimum experience

<figure><img src="/files/qwganaoMBNAYI49U0hA2" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Evaluation Criteria</summary>

Scored items such as relevance, depth of experience, or role alignment

<figure><img src="/files/wUwamuTftEvwTFW4HB5G" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary>Minimum Score</summary>

Candidates must meet your threshold to move forward

<figure><img src="/files/ZwzrxZN22qE7b59EF7uH" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Design role: portfolio required, score impact and craft
* Support role: customer-facing experience required
* Warehouse role: certifications required, no scoring needed

</details>

Use resume screening to remove noise early while keeping standards consistent.

## Application Forms: Structured Intake

Form questions are ideal when you need structured data, and want to qualify fit before interviews.

<figure><img src="/files/N1YOykhabqYqhB6qnHnv" alt=""><figcaption></figcaption></figure>

<details>

<summary>Supported question types include:</summary>

* short answer
* numeric (minimum or slider)
* multiple choice
* date
* file upload

<figure><img src="/files/OKeMWV7tJ4Hu3AKbazYI" alt=""><figcaption></figcaption></figure>

You can also enforce criteria so candidates who don’t meet requirements don’t advance.

<figure><img src="/files/PFUWuvOw9unYh7gn8o1H" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Healthcare: license upload required
* Contract roles: start date required
* Product roles: portfolio link + explanation

</details>

Forms reduce back-and-forth and clean up your pipeline.

## SMS Screening: Speed and Eligibility

SMS Screening is optimized for quick engagement and qualification.

<figure><img src="/files/1UQY6qv6r3IIifRvjoZC" alt=""><figcaption></figcaption></figure>

<details>

<summary>Questions types supported:</summary>

* yes/no
* numeric
* pass/fail

<figure><img src="/files/UPH0Icad5yAO0ldLFIwm" alt=""><figcaption></figcaption></figure>

You decide:

* which questions are dealbreakers
* minimum acceptable values

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Work authorization
* Years of experience
* Shift availability
* Portfolio link confirmation

</details>

SMS works best when you want fast answers with minimal friction, especially for on-the-go candidates.

## Voice / Video Interviews: Judgment and Depth

Voice interviews are where nuance matters. HeyMilo starts you off with a balanced set of open-ended questions that you can edit or replace.

<figure><img src="/files/7LWnz0ocnVAPaiI3bjzn" alt=""><figcaption></figcaption></figure>

<details>

<summary>You define:</summary>

* evaluation criteria (what you’re looking for)
* what a strong answer looks like (score of 5)
* what signals a weak response (score of 1)
* how much each question matters
* how many follow-ups are allowed to probe for more detail

You can additionally generate evaluation criteria using AI, or get AI to tune the question to be more concise, more open-ended, or use a custom prompt.

</details>

<details>

<summary>Questions types you can mix:</summary>

* **Scored questions** (1–5)

<figure><img src="/files/LxUaKXybMmL51t4STxgd" alt=""><figcaption></figcaption></figure>

* Structured questions (pass/fail, multiple choice, numeric)

<figure><img src="/files/DWAuak6lC97sttEqLFAv" alt=""><figcaption></figcaption></figure>

* Informational questions (no score impact)

<figure><img src="/files/HwT4JFEyPxEsIvCNLLqH" alt=""><figcaption></figcaption></figure>

</details>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* Sales: weight objection handling higher than background
* Engineering: prioritize problem framing and tradeoffs
* Leadership: score decision-making and reflection

</details>

This is where the AI recruiter starts thinking and interviewing like you with adaptive, and conversational voice AI.

#### Hide scoring criteria from candidates (voice / video)

Some teams do not want candidates to hear or see scoring detail during the interview. HeyMilo can still score answers using your rubric; only the live rubric is hidden.

<figure><img src="/files/8cx4Bh4MLaievm0ClHRT" alt="" width="375"><figcaption></figcaption></figure>

Where: **Interviewer** → **Questions & Scoring** (voice/video) → Edit the question.

**Steps**

1. Set context and criteria as usual (used for scoring).
2. Leave Don’t include in overall score unchecked so Hide criteria from interviewer appears.
3. Turn on Hide criteria from interviewer.

<figure><img src="/files/dgqRBnIgWNW7K9oSL64l" alt="" width="375"><figcaption></figcaption></figure>

**Candidate experience**: The interviewer stops surfacing that question’s scoring criteria in the live session (with a short in-product notice). Reports and workflow still use your criteria.

**Follow-ups**: Default follow-ups tied to the hidden rubric may be limited. Use the question instructions field, and Advanced criteria at the bottom, to script follow-ups in your exact wording (including fixed probes that do not depend on the prior answer). Save when done.

## Dealbreakers vs Scoring

A simple way to think about it:

* **Dealbreakers** = eligibility
* **Scoring** = quality

<details>

<summary>Use dealbreakers for:</summary>

* work authorization
* certifications
* availability
* hard minimums

</details>

<details>

<summary>Use scoring for:</summary>

* judgment
* communication
* depth of experience
* decision-making

</details>

This keeps interviews fair, efficient, and predictable.

## Tags (Optional, Informational)

Tags extract additional useful details from candidate responses (transcript) without affecting scores.

<figure><img src="/files/rnZi64V3H89cpbwvqLxK" alt=""><figcaption></figcaption></figure>

<details>

<summary><mark style="color:blue;">Examples:</mark></summary>

* years of experience
* primary tools
* location
* certifications

</details>

Tags appear in candidate profiles + reports to help with sorting and review.

## Test Before You Launch

Every Questions & Scoring section includes **Test Now** before you activate your agent.

<figure><img src="/files/CpiJSjM9PWMGs4cuTZzc" alt=""><figcaption></figcaption></figure>

Testing lets you experience the interview as a candidate, see how follow-ups trigger, validate scoring logic, and catch unclear wording early.

Always test before activating!

## Scoring Best Practices

<details>

<summary>✅ Start Conservative</summary>

* Begin with broader scoring ranges
* Refine criteria based on actual candidate performance
* Avoid being too restrictive initially

</details>

<details>

<summary>✅ Be Specific</summary>

* Define exactly what you're looking for
* Use concrete examples in your rubrics
* Avoid subjective or vague criteria

</details>

<details>

<summary>✅ Test Your Scoring</summary>

* Review initial candidate scores carefully
* Adjust weightings if needed
* Ensure scores align with your expectations

</details>

<details>

<summary>✅ Regular Calibration</summary>

* Review scoring accuracy monthly
* Compare AI scores with hiring outcomes
* Adjust criteria based on performance data

</details>

## Common Scoring Mistakes

<details>

<summary>❌ Over-Weighting Single Factors</summary>

* Balance technical skills and soft skills appropriately
* Consider the full candidate profile

</details>

<details>

<summary>❌ Setting Unrealistic Standards</summary>

* Avoid requiring perfection in all areas
* Consider growth potential, not just current skills
* Allow for different paths to success

</details>

<details>

<summary>❌ Ignoring Cultural Fit</summary>

* Don't focus solely on technical qualifications
* Consider long-term retention factors
* Evaluate team integration potential

</details>

<details>

<summary>❌ Not Updating Criteria</summary>

* Review and iterate on scoring for future opportunities
* Adapt to changing role requirements
* Incorporate lessons learned from hiring

</details>

## Interpreting Overall Candidate Scores

<details>

<summary>Score Ranges</summary>

* **90-100**: Exceptional candidate, likely top 5%
* **80-89**: Strong candidate, definitely worth interviewing
* **70-79**: Good candidate, consider for next round
* **60-69**: Average candidate, may need development
* **Below 50**: Likely not a good fit for the role

</details>

<details>

<summary>Beyond the Numbers</summary>

Consider additional factors:

* **Growth Trajectory**: Is the candidate improving?
* **Potential**: Could they excel with training?
* **Unique Strengths**: Do they bring something special?
* **Team Needs**: Do they fill a specific gap?

</details>

<details>

<summary>Score Trends</summary>

Monitor patterns across candidates:

* Are scores too high/low overall?
* Which questions differentiate best?
* Are knockout criteria too strict?
* Do scores predict hiring success?

</details>

## Continuous Improvement

<details>

<summary>Track Hiring Outcomes</summary>

* Monitor which scored candidates get hired
* Measure performance of hired candidates
* Identify scoring criteria that predict success

</details>

<details>

<summary>Gather Feedback</summary>

* Ask hiring managers/team members about score accuracy
* Get input from successful hires
* Review candidate feedback on the process

</details>

<details>

<summary>Refine As Needed</summary>

* Update scoring criteria
* Adjust weightings based on results
* Add new criteria as roles evolve

</details>

{% hint style="success" %}
**Pro Tip**: The best scoring system is one that consistently identifies candidates who succeed in the role. Focus on criteria that predict actual job performance, not just interview performance.
{% endhint %}

## Ready to Invite Candidates?

With compelling questions and intelligent scoring configured, your interview agent is ready to start evaluating candidates. The next step is activating your agent and learning how to share it with candidates to drive high completion rates.

Remember: Scoring is an iterative process. Start with a solid foundation and refine based on real candidate data and hiring outcomes!


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.admin.heymilo.ai/creating-your-interview-agents/how-our-scoring-system-works/scoring.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.