Voice Agent
Run tests

Robustness and bias testing

Add controlled variants to see whether results hold across noise and accents.

Audio Robustness and Language Bias add comparable variants to eligible logical test definitions. Use them when you need to know whether an outcome holds beyond the base interaction.

Both controls are in the Real-World Reliability section of Evaluation Metrics.

Availability by run mode

Run modeAudio RobustnessLanguage Bias
Real CallAvailable when eligible metrics and a noise type and level are available.Available when at least two accents are configured and the bias requirements below are met.
Direct simulationAvailable when eligible metrics and a noise type and level are available.Available when at least two accents are configured and the bias requirements below are met.
WebSocket simulationAvailable when eligible metrics and a noise type and level are available.Not exposed. The Real-World Reliability heading describes Audio Robustness only.
Direct-channel agentNot available.Not available.
Transcript simulationAvailable when eligible metrics and a noise type and level are available.The current UI uses the local/direct configuration for this mode, so it is exposed when the bias requirements below are met.

Audio Robustness is not restricted by the Language Bias mode predicate. It is active only after you select at least one available robustness-eligible metric; it also needs a usable noise type and non-None noise level.

Audio Robustness

Enable Audio Robustness to compare a clean interaction with a noisy one. Choose the Noise Type, Intensity Level, and Robustness Metrics that should be evaluated.

For each eligible definition, robustness produces two physical runs:

  • one clean run;
  • one noisy run.

Language Bias

Enable Language Bias to compare Accent A and Accent B for the selected Bias Metrics. It is available for phone/Real Call and local/direct configurations, not WebSocket. The control appears only when at least two accents are configured; to activate it, choose two distinct non-neutral accents and at least one available eligible metric. For each eligible definition, bias produces two physical runs, one for each accent.

How variants change run count

For each eligible logical definition:

  • Base test: 1 run
  • Audio Robustness only: 2 runs
  • Language Bias only: 2 runs
  • Audio Robustness and Language Bias: 4 runs

The expansion applies only when planned metrics overlap the robustness or bias metrics you selected. Available metrics and accents are configuration-dependent; the UI may indicate when no eligible metrics are configured.

Review Test runs needed before submitting. Afterward, use the suite’s Robustness Breakdown and Bias Breakdown to compare outcomes.

On this page