Re-evaluate an existing run
Apply current metrics to stored call evidence without placing another call, then review what changed.
Re-evaluation applies today's selected metrics to evidence already captured by an existing call. Use it when a new metric becomes available, a metric's evaluation behavior changes, or a metric implementation is corrected. You keep the original run and avoid repeating the conversation.
The recorded conversation does not change. The evaluation does. A higher or lower result reflects the current evaluator's interpretation of the same stored evidence; it does not by itself mean the agent improved or regressed.
Decide whether re-evaluation is the right action
Re-evaluation is useful when:
- A newly supported metric was not available when the call was first evaluated.
- A metric's rules, calibration, or evaluator behavior changed.
- A corrected metric implementation needs to be applied to an earlier result.
- You want to recompute a selected metric without creating another call or run.
Create a fresh comparable run instead when you changed the agent, prompt, scenario, integration, or conversation behavior. The fresh run provides new evidence; re-evaluating the old run does not.
For an uploaded recording whose vertical, call driver, or AUT role is wrong, use Correct uploaded-call context. That workflow updates the saved classification before recomputing applicable metrics.
Check the evidence first
The Re-evaluate action is offered for completed and failed runs. The action can be unavailable if the metric catalog or recent job history could not be loaded. Artifact sufficiency is checked during preflight rather than before the action is shown.
Before submitting it:
- Open the run from its suite.
- Inspect its current Metrics Breakdown and note the result you want to refresh or the newly available metric you want to add.
- Review the recording, transcript, and event evidence the metric will need.
- Check the metric catalog for required audio, transcript, scenario, workflow, policy, or ground-truth context.
A historical call is not guaranteed to contain enough evidence for a metric introduced later. Preflight reports that limitation as Needs setup or Not scored instead of inventing a score.
Apply current metrics to the run
- Open the run from its suite and select Re-evaluate.
- Leave selected only the newly added or changed metrics you need to refresh.
- Submit the job.
- Keep the run open to follow its progress.
The job moves through preparation, metric computation, and publishing, then finishes as completed, failed, or conflict. When publication completes, the run refreshes to the current result.
Worked example: add a metric to a historical call
Suppose an appointment-booking call was completed before Ambiguity Handling was available for evaluation. The team wants to learn whether the agent clarified an ambiguous appointment request without placing the call again.
First, a reviewer opens the existing run and checks that its stored transcript contains both the ambiguous request and an opportunity for the agent to clarify it. They select Re-evaluate, leave Ambiguity Handling selected, and submit the job. When processing completes, the same run contains the current result for that metric. Because the metric had no earlier result, View changes shows a newly added outcome rather than a meaningful score delta.
Review what changed
The Recent re-evaluations section shows up to the latest 20 jobs. Select View changes to compare the previous and current overall and metric outcomes.
- A newly added metric can have a current result with no previous value.
- An updated evaluator can produce a higher, lower, or unchanged result from the same evidence.
- An unchanged result confirms that the current evaluator reached the same outcome.
- A missing result means the available evidence or setup did not support a score; it is not a zero.
The refreshed run remains the current published result, while the history preserves the previous outcome for that re-evaluation.
Handle an unavailable result
| Outcome | What it means | What to do |
|---|---|---|
| Completed | The selected evaluation was published onto the existing run. | Open View changes and inspect the current evidence. |
| No score for a new metric | The stored evidence or context could not support the metric. | Review the metric's prerequisites; do not treat the result as zero. |
| Pending context, skipped, or invalid | A required input was missing or unsuitable. | Check the recording, transcript, scenario context, and metric requirements. |
| Failed | Evaluation processing did not complete. | Read the displayed reason and retry only after correcting the stated problem or a transient outage. |
| Another re-evaluation is already in progress | Submission found an active job for this run and follows that job instead of creating a duplicate. | Follow the adopted active job. |
| Conflict | The run changed before this candidate could be published, so the candidate remains unpublished. | Refresh and review the current run before deciding whether to submit again. |
If tracking is unavailable or the outcome is unknown, refresh before trying again.
Understand the ASR boundary
Re-evaluation uses the run's stored canonical transcript and other existing artifacts. It does not regenerate the transcript or let you switch the ASR engine used for metric computation.
The transcript selector on the run page changes the transcript artifact you are viewing. Switching between OpenAI Whisper and NVIDIA Parakeet there does not recompute persisted metrics. A new or regenerated transcript is a separate artifact-generation task, not part of this workflow.
Success looks like this
In the job-based flow, re-evaluation is complete when the run ID is unchanged, no call was repeated, the selected metrics show their current outcomes, and Recent re-evaluations retains the comparison for review. Environments still using the compatibility endpoint can complete synchronously without phased progress or a history entry.
Next, review how metrics and missing results are presented or create a fresh scenario when you need new conversation evidence.
Review an ASR benchmark replay
Compare frozen provider results, inspect transcript and passage-matching evidence, and understand what can and cannot be recalculated.
Compare agents and export results
Choose a comparable scope, interpret the Analytics view responsibly, and export a portable review snapshot.