Files

T

History

Stephen Allen 5091f0a65a feat(eval): Make live and audio evals reachable via public entrypoints

Merge https://github.com/google/adk-python/pull/6458

Live/audio agent eval was only exercisable through private internal service imports; the public surface (CLI, dev-server, AgentEvaluator) always ran non-live text inference, so users had no supported path to evaluate Live API agents with a simulated audio user.

This threads `use_live` through all three public entrypoints, fixes the live-send path so native-audio models accept simulated user audio, and lets the dev-server select an audio (`llm_audio`) user simulator over HTTP. Live transcriptions are consolidated to text, with the text response preferred as the gradable output for turns carrying both audio and a transcript.

Adds a runnable sample (`live_non_blocking_tool_agent` evalset + `test_config` with `use_live: true` and a Gemini TTS audio simulator) plus unit tests covering `use_live` propagation, request validation, resampling, and the realtime-audio send path.

COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/6458 from allen-stephen:feat/live-eval-parity 3fc33a2616d1515c822387deb8d70c27d5bc6244
PiperOrigin-RevId: 954725627

2026-07-27 11:08:51 -07:00

__init__.py

feat(live): Run non-blocking tools in a background task

2026-07-09 14:50:20 -07:00

agent.py

feat(live): Run non-blocking tools in a background task

2026-07-09 14:50:20 -07:00

live_non_blocking_tool_agent.evalset.json

feat(eval): Make live and audio evals reachable via public entrypoints

2026-07-27 11:08:51 -07:00

README.md

feat(eval): Make live and audio evals reachable via public entrypoints

2026-07-27 11:08:51 -07:00

test_config.json

feat(eval): Make live and audio evals reachable via public entrypoints

2026-07-27 11:08:51 -07:00

README.md

Live Non-Blocking Tool Agent Sample

Overview

This sample provides a minimal agent to demonstrate non-blocking tool execution in ADK Live mode (adk web / run_live).

When a tool declaration is configured with response_scheduling set to WHEN_IDLE, SILENT, or INTERRUPT, it indicates to the model that response handling can occur asynchronously.

Sample Inputs

Please start a slow background task for data processing, and then let's keep talking.

Triggers slow_background_task which sleeps for 10 seconds. While it runs, continue speaking to the agent.

Reproduction Instructions

Run the sample via adk web:

uv run adk web contributing/samples/live/live_non_blocking_tool_agent

Open the ADK web interface and start a Live Session with the agent.
Trigger the tool by saying: "Please start a slow background task and keep talking with me."
Continue speaking to the agent while the background task runs in console ([Tool] Starting slow background task...).

Expected Behavior

The model should continue conversing and generating audio/transcription responses immediately while the tool executes in the background. The tool result is delivered later per the response_scheduling mode.

Evaluating this agent

test_config.json and live_non_blocking_tool_agent.evalset.json evaluate the agent in live mode with an llm_audio user simulator (each user turn is synthesized to audio and streamed to the live agent).

Install the eval extra: uv pip install -e ".[eval]".
Add a .env in this directory with Vertex AI credentials (see live_bidi_streaming_single_agent/.env). The project needs access to both the Live API and Gemini TTS models.

Run the eval:

uv run adk eval \
  contributing/samples/live/live_non_blocking_tool_agent \
  contributing/samples/live/live_non_blocking_tool_agent/live_non_blocking_tool_agent.evalset.json \
  --config_file_path contributing/samples/live/live_non_blocking_tool_agent/test_config.json