Files
nikkie babb11c83c feat(eval): support custom metrics in AgentEvaluator
Merge https://github.com/google/adk-python/pull/4344

**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**

### Link to Issue or Description of Change

**1. Link to an existing issue (if applicable):**

- Closes: #4343

**Problem:**
`AgentEvaluator.evaluate` did not register custom metrics from `EvalConfig`, so custom metrics worked in `adk eval` but not in pytest-based evals.

**Solution:**
Align `AgentEvaluator` with the CLI eval flow by registering custom metrics via a per-run metric registry and a shared default `MetricInfo` helper. The per-run registry is a fork of `DEFAULT_METRIC_EVALUATOR_REGISTRY` (new `MetricEvaluatorRegistry.fork()`), so the custom metrics declared by one eval config never leak into another run.

### Testing Plan

Add unit coverage for the registration behavior and a lightweight integration example that uses a custom metric.

**Unit Tests:**

- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.

```
% pytest tests/unittests/evaluation tests/unittests/cli

614 passed, 308 warnings in 12.74s
```

**Manual End-to-End (E2E) Tests:**

```
% pytest tests/integration/test_with_test_file.py::test_with_custom_metric

tests/integration/test_with_test_file.py .                               [100%]

1 passed, 12 warnings in 3.57s
```

### Checklist

- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.

### Additional context

This change keeps `AgentEvaluator` behavior consistent with `adk eval` while avoiding CLI-layer dependencies.

`fork()` returns an isolated copy seeded with the source registry's contents, rather than a bare `MetricEvaluatorRegistry()`. The seeding is what keeps this backwards compatible: registering an `Evaluator` subclass on `DEFAULT_METRIC_EVALUATOR_REGISTRY` is the only way to plug one in, since an eval config can only name a scoring function. Callers who do that today (including ones replacing the evaluator behind a standard metric name) would otherwise silently fall back to the stock evaluator, with no error and a different score. Covered by `test_evaluate_eval_set_keeps_evaluators_from_the_default_registry`.

Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4344 from ftnext:agent-evaluator-support-custom-metric 3c844a9817d16954c683b164a88a86a82e8f9cf1
PiperOrigin-RevId: 966416454
2026-08-18 00:17:15 -07:00
..
2026-01-20 14:50:09 -08:00
2026-01-20 14:50:09 -08:00
2026-01-20 14:50:09 -08:00
2025-04-17 21:47:59 +00:00