Files
Haran Rajkumar c8f5ae91da feat(cli): add adk eval --num_runs N with per-eval-case aggregation
**Problem:**
`adk eval` did not support repeated runs, so users had to script external loops
to reduce nondeterminism.

**Solution:**
- Add `--num_runs` to `adk eval` (default `1`, min `1`). The value is carried on
  `InferenceConfig.num_runs`, so the eval service repeats each eval case through
  its existing parallelism pool instead of the CLI multiplying the requests.
- Aggregate the per-run results for each eval case (`eval_set_id` + `eval_id`)
  with a mean-over-invocations strategy that matches `AgentEvaluator` (the pytest
  entrypoint), so multi-run summaries are consistent across entrypoints. The
  strategy is an input via `EvaluateConfig.aggregation_strategy`
  (`AggregationStrategy`); only `MEAN_OVER_INVOCATIONS` is supported today.
- `--print_detailed_results` continues to show per-run details.

Tests cover the aggregator (mean-over-invocations, grouping, threshold, and
runs that failed before producing metric results), the service-level `num_runs`
repetition, and the CLI validation/summary behavior.

Co-authored-by: ftnext <takuyafjp+develop@gmail.com>

Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 959841339
2026-08-20 17:58:38 -07:00
..
2026-01-20 14:50:09 -08:00