babb11c83c
Merge https://github.com/google/adk-python/pull/4344 **Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.** ### Link to Issue or Description of Change **1. Link to an existing issue (if applicable):** - Closes: #4343 **Problem:** `AgentEvaluator.evaluate` did not register custom metrics from `EvalConfig`, so custom metrics worked in `adk eval` but not in pytest-based evals. **Solution:** Align `AgentEvaluator` with the CLI eval flow by registering custom metrics via a per-run metric registry and a shared default `MetricInfo` helper. The per-run registry is a fork of `DEFAULT_METRIC_EVALUATOR_REGISTRY` (new `MetricEvaluatorRegistry.fork()`), so the custom metrics declared by one eval config never leak into another run. ### Testing Plan Add unit coverage for the registration behavior and a lightweight integration example that uses a custom metric. **Unit Tests:** - [x] I have added or updated unit tests for my change. - [x] All unit tests pass locally. ``` % pytest tests/unittests/evaluation tests/unittests/cli 614 passed, 308 warnings in 12.74s ``` **Manual End-to-End (E2E) Tests:** ``` % pytest tests/integration/test_with_test_file.py::test_with_custom_metric tests/integration/test_with_test_file.py . [100%] 1 passed, 12 warnings in 3.57s ``` ### Checklist - [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document. - [x] I have performed a self-review of my own code. - [x] I have commented my code, particularly in hard-to-understand areas. - [x] I have added tests that prove my fix is effective or that my feature works. - [x] New and existing unit tests pass locally with my changes. - [x] I have manually tested my changes end-to-end. - [x] Any dependent changes have been merged and published in downstream modules. ### Additional context This change keeps `AgentEvaluator` behavior consistent with `adk eval` while avoiding CLI-layer dependencies. `fork()` returns an isolated copy seeded with the source registry's contents, rather than a bare `MetricEvaluatorRegistry()`. The seeding is what keeps this backwards compatible: registering an `Evaluator` subclass on `DEFAULT_METRIC_EVALUATOR_REGISTRY` is the only way to plug one in, since an eval config can only name a scoring function. Callers who do that today (including ones replacing the evaluator behind a standard metric name) would otherwise silently fall back to the stock evaluator, with no error and a different score. Covered by `test_evaluate_eval_set_keeps_evaluators_from_the_default_registry`. Co-authored-by: Yi Liu <yiliuly@google.com> COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4344 from ftnext:agent-evaluator-support-custom-metric 3c844a9817d16954c683b164a88a86a82e8f9cf1 PiperOrigin-RevId: 966416454