Guide
ADK Live Voice Eval: Turning “Sounds Good” into a Testable CI Gate
Google introduced native live evaluation for Agent Development Kit on August 24. Developers can now drive a live voice agent with an LLM-controlled simulated user that speaks through synthesized audio, score the resulting multi-turn trajectory with natural-language rubrics and per-turn metrics, and run the same evaluation loop programmatically in CI/CD. Google’s example uses three `gemini-live-2.5-flash-native-audio` agents in a graph workflow, `gemini-3.7-flash` for simulated-user turn logic, and `gemini-3.1-flash-tts-preview` for audio generation. This moves voice-agent testing beyond manual demos into repeatable regression engineering.
# ADK Live Voice Eval: Turning “Sounds Good” into a Testable CI Gate
## Article Summary
Google introduced native live evaluation for Agent Development Kit on August 24. Developers can now drive a live voice agent with an LLM-controlled simulated user that speaks through synthesized audio, score the resulting multi-turn trajectory with natural-language rubrics and per-turn metrics, and run the same evaluation loop programmatically in CI/CD. Google’s example uses three `gemini-live-2.5-flash-native-audio` agents in a graph workflow, `gemini-3.7-flash` for simulated-user turn logic, and `gemini-3.1-flash-tts-preview` for audio generation. This moves voice-agent testing beyond manual demos into repeatable regression engineering.
---
The most dangerous sentence in voice-agent development is:
> “I called it once and it sounded good.”
That is not a regression test.
A small prompt or model change can cause:
- missing tool calls;
- forgotten context;
- broken handoffs;
- ignored interruptions;
- premature disclosure;
- incorrect conversation endings.
Voice agents must be evaluated across trajectories, not only final answers.
## Google’s example is a real workflow
The reference workflow contains three live agents:
```text
greeter
→ date-of-birth verifier
→ appointment/goals agent
```
A tool call sits in the middle.
Every stage runs on:
```text
gemini-live-2.5-flash-native-audio
```
The audio stream remains open while ADK preserves session state and conversation history across agent handoffs.
To the user, it feels like one continuous conversation.
## Final-answer accuracy is not enough
A healthcare agent can say the correct appointment time and still fail if it revealed that information before verifying identity.
Google’s example therefore evaluates an end-to-end trajectory with a rubric such as:
> confirm the caller’s name and validate date of birth before disclosing appointment details.
This tests:
```text
sequence
+ permission
+ tool use
+ disclosure
+ final result
```
not just one text string.
## Two evaluation styles
ADK supports both dynamic scenarios and fixed conversations.
### Conversation scenario
Define:
- starting prompt;
- goal;
- conversation plan;
- user persona.
The simulator improvises each turn.
This is useful for testing whether the agent can actively drive the interaction.
### Fixed conversation
Script every user turn.
This is valuable for:
- deterministic regression;
- known bugs;
- compliance cases.
Production suites should usually contain both.
## Dynamic users are closer to reality
Real people express the same information in many ways.
A date of birth can appear as:
```text
July twelfth, 1985
7/12/85
I was born on the twelfth of July in eighty-five
```
A fixed transcript can pass while real users fail.
An LLM-driven simulator creates variation around the same goal.
## Persona is a powerful testing dimension
Google’s built-in `NOVICE` persona gives only high-level information and waits for the agent to ask for details.
Teams can build additional prompt-driven personas such as:
```text
IMPATIENT
INTERRUPTING
NON_NATIVE
CONFUSED
ANGRY
FAST_SPEAKER
```
The persona is not merely cosmetic.
It changes whether the agent must recover, clarify, and lead the conversation.
## Conversation logic and speech synthesis are separate
The simulator configuration distinguishes:
```text
model
```
for deciding what the user says next, and:
```text
audio_model
```
for synthesizing that turn into audio.
This makes it possible to vary:
- reasoning behavior;
- voice;
- language;
- accent;
independently.
## Minimal configuration pattern
A simplified setup looks like:
```json
{
"criteria": {
"rubric_based_multi_turn_trajectory_quality_v1": {
"threshold": 0.7,
"judge_model_options": {
"judge_model": "gemini-3.7-flash"
}
}
},
"live_model_config": {
"timeout_seconds": 300
},
"user_simulator_config": {
"type": "llm_audio",
"model": "gemini-3.7-flash",
"max_allowed_invocations": 10,
"audio_model": "gemini-3.1-flash-tts-preview"
}
}
```
The exact values should be adapted to the workload.
## Turn caps matter
Dynamic conversations can run away.
If an agent never reaches the closing condition, the simulator may continue generating turns.
`max_allowed_invocations` provides a hard upper bound.
Production systems should also enforce:
- max turns;
- max duration;
- max tool calls;
- max cost.
## Five evaluation layers
A mature live-agent suite should score at least:
### Content correctness
Was the information correct?
### Workflow correctness
Were required steps completed in the right order?
### Tool correctness
Were the correct tools called with valid arguments?
### Conversation quality
Did the agent ask clear questions, avoid repetition, and recover from ambiguity?
### Realtime behavior
Did interruption, latency, silence, and handoff behavior remain acceptable?
## Text passing does not guarantee audio passing
ADK can run the same cases in text mode when live configuration is omitted.
That enables a useful gate:
```text
text eval
→ live audio eval
→ release
```
If text passes but live audio fails, the problem may be in:
- timing;
- speech synthesis;
- turn-taking;
- barge-in;
- realtime session state.
That narrows debugging significantly.
## CI/CD integration
Google notes that the same pipeline can be called programmatically through `AgentEvaluator`.
A practical CI flow can be:
```text
pull request
→ unit tests
→ text agent eval
→ core live eval
→ threshold gate
→ merge
```
Voice evaluation is more expensive, so use tiering.
## Recommended CI tiers
### Every pull request
Run a small critical suite.
### Nightly
Run broader persona and scenario coverage.
### Release candidate
Run full voice regression including:
- accents;
- interruptions;
- tool failures;
- long conversations;
- multiple languages.
## ADK Web closes the debugging loop
Automated scores are not enough for speech systems.
ADK Web reconstructs the live run into transcripts plus playable audio clips for each turn.
A developer can inspect not only what the agent said but how it sounded.
That matters for failures such as:
- awkward timing;
- broken phrasing;
- abrupt handoff tone;
- continuing old speech after interruption.
## Add production metrics
Beyond rubric scores, teams should track:
```text
first audio latency
turn latency
conversation silence time
barge-in stop latency
tool P95 latency
call completion rate
abandonment rate
escalation accuracy
```
Silence time is especially important.
A tool may take four seconds, but if the agent keeps the interaction moving, perceived latency can remain acceptable.
## Build difficult cases
Do not test only happy paths.
Include:
- corrections;
- interruptions;
- irrelevant answers;
- tool timeouts;
- empty tool results;
- failed identity verification;
- unauthorized requests;
- mid-call goal changes;
- handoffs;
- long-context recall.
## Conclusion
ADK live evaluation matters because voice agents can finally move from:
```text
manual demo
```
to:
```text
scenario
+ synthetic audio user
+ rubrics
+ tool metrics
+ regression
+ CI gate
```
Teams deploying voice agents into support, healthcare scheduling, sales, finance, or internal operations should stop treating “sounds natural” as a release criterion.
The stronger standard is:
> Every prompt, model, and tool change should automatically prove that critical multi-turn behavior has not regressed.
For more Gemini Live, Google ADK, voice-agent, and production evaluation guides, visit **Zyentor Picks**: https://www.zyentorpicks.com/.