Merge https://github.com/google/adk-python/pull/4344
**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
- Closes: #4343
**Problem:**
`AgentEvaluator.evaluate` did not register custom metrics from `EvalConfig`, so custom metrics worked in `adk eval` but not in pytest-based evals.
**Solution:**
Align `AgentEvaluator` with the CLI eval flow by registering custom metrics via a per-run metric registry and a shared default `MetricInfo` helper. The per-run registry is a fork of `DEFAULT_METRIC_EVALUATOR_REGISTRY` (new `MetricEvaluatorRegistry.fork()`), so the custom metrics declared by one eval config never leak into another run.
### Testing Plan
Add unit coverage for the registration behavior and a lightweight integration example that uses a custom metric.
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
```
% pytest tests/unittests/evaluation tests/unittests/cli
614 passed, 308 warnings in 12.74s
```
**Manual End-to-End (E2E) Tests:**
```
% pytest tests/integration/test_with_test_file.py::test_with_custom_metric
tests/integration/test_with_test_file.py . [100%]
1 passed, 12 warnings in 3.57s
```
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
### Additional context
This change keeps `AgentEvaluator` behavior consistent with `adk eval` while avoiding CLI-layer dependencies.
`fork()` returns an isolated copy seeded with the source registry's contents, rather than a bare `MetricEvaluatorRegistry()`. The seeding is what keeps this backwards compatible: registering an `Evaluator` subclass on `DEFAULT_METRIC_EVALUATOR_REGISTRY` is the only way to plug one in, since an eval config can only name a scoring function. Callers who do that today (including ones replacing the evaluator behind a standard metric name) would otherwise silently fall back to the stock evaluator, with no error and a different score. Covered by `test_evaluate_eval_set_keeps_evaluators_from_the_default_registry`.
Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4344 from ftnext:agent-evaluator-support-custom-metric 3c844a9817d16954c683b164a88a86a82e8f9cf1
PiperOrigin-RevId: 966416454
Merge https://github.com/google/adk-python/pull/4414
**Please ensure you have read the [contribution guide](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) before creating a pull request.**
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
- Related: #4410
- Fixes: #2602
**Problem:**
`AgentEvaluator.evaluate()` did not support built-in eval set result persistence, making it harder to reuse the same workflow as CLI/Web paths that already use `EvalSetResultsManager`.
Also, introducing new parameters in the middle of method signatures would break positional-argument compatibility for existing users.
**Solution:**
This PR adds optional eval result persistence to `AgentEvaluator` while preserving backward compatibility:
- Add optional parameters to `AgentEvaluator.evaluate()` and `AgentEvaluator.evaluate_eval_set()`:
- `app_name: Optional[str] = None`
- `eval_set_results_manager: Optional[EvalSetResultsManager] = None`
- Persist results per eval set (a single save aggregating all `EvalCaseResult`s), aligning `AgentEvaluator` with existing CLI/Web/API (`LocalEvalService`) persistence behavior.
- Resolve `app_name` from explicit input first, then derive from `agent_module` (including `.agent` suffix handling).
- Save results before failure assertion so failed eval runs still leave artifacts for inspection.
- Keep existing positional argument behavior by appending new parameters at the end of public method signatures.
- Add/extend tests to verify:
- explicit and derived `app_name`
- save-on-failure behavior
- argument propagation from `evaluate()` to `evaluate_eval_set()`
- positional-argument backward compatibility
- Add an integration usage example for `app_name` omission with `LocalEvalSetResultsManager`.
- For multi-run evals, all runs and eval cases are aggregated into a single result file per eval set (each run contributes one `EvalCaseResult`).
### Testing Plan
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
```
% pytest tests/unittests/evaluation
======================== 357 passed, 169 warnings in 9.68s =========================
```
**Manual End-to-End (E2E) Tests:**
```
% pytest tests/integration/test_with_test_file.py::test_with_single_test_file_saves_eval_set_result
======================== 1 passed, 14 warnings in 5.24s ========================
```
Verify a result file is created under: `<tmp_path>/<derived_app_name>/.adk/eval_history/*.evalset_result.json` (e.g., 1 file containing 2 `EvalCaseResult`s when num_runs=2 on a single-case eval fixture).
This is helpful for debugging failed integration tests.
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
### Additional context
- This PR intentionally preserves public API positional compatibility by appending new optional parameters at the tail of method signatures.
- A generated local eval result JSON file may exist in the working tree from manual verification and is intentionally not part of the code change.
Co-authored-by: Yi Liu <yiliuly@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4414 from ftnext:agent-evaluator-save-evalset-result 873973e549c0a4b25b83e1ef81e1b4148ab4e379
PiperOrigin-RevId: 962597058
Adds OCIGenAILlm under integrations/oci/, for Google Gemini and other models
hosted on Oracle Cloud Infrastructure Generative AI. Optional install:
pip install google-adk[oci]. LLMRegistry auto-routing and the
google.adk.models import surface are preserved.
The OpenAI-compatible transport from the source PR (OCIGenAIOpenAILlm) is
not taken. It reimplemented the message, tool and response conversion plus
the streaming loop that OpenAILlm already provides; the right form is a
small subclass overriding the OpenAI client, which cannot live in
integrations/ while OpenAILlm is still experimental. It can land separately
once that settles.
The OCI client is now built once per instance rather than per request, so a
call no longer re-reads the OCI config from disk.
Merge https://github.com/google/adk-python/pull/5285Closes#5069
Co-authored-by: George Weale <gweale@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/5285 from fede-kamel:feat/oci-generative-ai 0230acc0a93b7e43014f2ef3a8b89de463a50bd8
PiperOrigin-RevId: 955453382
ManagedAgent and Gemini(use_interactions_api=True) both reach the Interactions
API and surface identically as tool_name=google-adk in Google's usage pipeline,
with no way to tell them apart. Thread an optional framework_label through
merge_tracking_headers / get_tracking_headers / get_client_labels /
_get_default_labels, and have ManagedAgent emit google-adk/<version>+managed_agent
on the per-request extra_headers it sends to interactions.create, so its traffic
is distinguishable via the tool_version dimension while tool_name stays
google-adk. The suffix is applied on the request-time header path because that is
what reaches the Interactions wire (the per-request extra_headers override the
genai client's construction-time headers; verified by live capture). An explicit
framework_label takes precedence over the Agent Engine (+remote_reasoning_engine)
suffix; all other callers of merge_tracking_headers keep the no-arg default and
are unchanged. Follow-up to the ManagedAgent tracking-headers change.
Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 947239702
Add a runnable sample wiring `ManagedAgent` to the Maps Grounding Lite MCP
server via `RemoteMcpServer`, with a `header_provider` callback that reads
`GOOGLE_MAPS_API_KEY` and sends it as the `X-Goog-Api-Key` header. Demonstrates
server-side remote MCP execution with runtime header minting.
Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 947137422
Add `RemoteMcpServer`, a server-side remote MCP tool for `ManagedAgent`. The
Managed Agents / Interactions API runs the MCP server itself, so ADK only
forwards the server URL and headers as an `MCPServerParam` and never opens an
MCP session. A `header_provider` callback (the same contract as the `LlmAgent`
`McpToolset.header_provider`) mints auth headers at request time, driven by the
runner, and is merged over any static headers so a fresh token can be generated
per turn.
Only remote (HTTP/streamable) MCP servers are supported; raw
`types.Tool.mcp_servers` remains rejected. Includes a live integration test
against Maps Grounding Lite, scoped to the Gemini Developer API backend; the
Vertex Interactions endpoint does not yet accept the `mcp_server` tool param
(consistent with google-genai documenting `types.Tool.mcp_servers` as
unsupported on Vertex AI).
Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 945306712
Add a runnable sample under contributing/samples/managed_agent/code_execution
showing how to use ManagedAgent with the server-side code execution tool. Since
ManagedAgent has no code_executor field, code execution is enabled by passing the
raw types.Tool(code_execution=types.ToolCodeExecution()) config in tools. The
sample exposes a root_agent in agent.py and ships a README plus a matching
single-turn live integration test that verifies a code-executed prime-sum
computation.
Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 943548907
Add ManagedAgent(BaseAgent), which drives the Managed Agents interactions.create
API directly. This first cut supports server-side tools only (ADK built-in tools
and raw types.Tool configs); client-executed tools (FunctionTool/callables) and
MCP are rejected with NotImplementedError. Multi-turn chaining reuses
previous_interaction_id and recovers the sandbox environment across turns.
Co-authored-by: Haran Rajkumar <haranrk@google.com>
PiperOrigin-RevId: 943406685
Merge https://github.com/google/adk-python/pull/5638
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):**
- Closes: #5637
This change adds `invocation_context.end_invocation = True` after the auth event yield in `_postprocess_handle_function_calls_async`, mirroring the existing termination signal in `_resolve_toolset_auth`. Tool-level auth now terminates symmetrically with toolset-level auth at the EUC, instead of continuing for one more LLM call.
### Testing Plan
**Unit Tests:**
- [x] I have added or updated unit tests for my change.
- [x] All unit tests pass locally.
Three existing tests in `test_functions_request_euc.py` had assertions tied to the trailing post-EUC LLM call:
- `test_function_request_euc`: adds `assert len(mock_model.requests) == 1` to anchor the new termination behavior.
- `test_function_get_auth_response`: `events[-3]` → `events[-2]` for the auth event lookup, since the auth event is now second-to-last.
- `test_function_get_auth_response_partial`: same `events[-3]` → `events[-2]` change, plus the two `len(mock_model.requests)` assertions drop by 1 (3 → 2 and 4 → 3).
```
$ pytest tests/unittests/flows/llm_flows/test_functions_request_euc.py
======================== 3 passed, 17 warnings in 1.31s ========================
$ pytest tests/unittests/
=============== 5695 passed, 2308 warnings in 122.89s (0:02:02) ================
```
**Manual End-to-End (E2E) Tests:**
A self-contained Runner-based reproduction is at https://github.com/doughayden/adk-issue-examples/tree/main/04-tool_level_auth_continuation. The agent definition (`agent.py`) wires up an `OpenAPIToolset` against a local OAuth2 test server. `main.py` constructs an `InMemoryRunner`, applies the workaround for #5327 (`get_auth_config = lambda: None`) at runtime to land on the tool-level auth path, and sends a tool-triggering prompt. The `--apply-fix` flag monkey-patches the proposed fix to demonstrate the resolution end-to-end.
Without the fix:
```
👤 User: What's the weather in San Francisco?
🌤️ Weather Assistant event stream:
[function_call] get_weather by WeatherAssistant
[auth_event] adk_request_credential by WeatherAssistant
[function_response] get_weather by WeatherAssistant
[post_euc_text] WeatherAssistant: "I'm sorry, I cannot retrieve the weather for San Francisco at the moment. It ..."
Event counts:
function_calls: 1
auth_events: 1
function_responses: 1
text_events: 1
post_euc_text_events: 1
✅ Bug reproduced: 1 text event(s) after the EUC (agent loop continued past adk_request_credential).
```
With the fix:
```
👤 User: What's the weather in San Francisco?
🌤️ Weather Assistant event stream:
[function_call] get_weather by WeatherAssistant
[auth_event] adk_request_credential by WeatherAssistant
[function_response] get_weather by WeatherAssistant
Event counts:
function_calls: 1
auth_events: 1
function_responses: 1
text_events: 0
post_euc_text_events: 0
✅ Fix verified: no LLM events after the EUC.
```
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [ ] Any dependent changes have been merged and published in downstream modules.
### Additional context
**Alternative considered:**
A reorder of the yields (yield `auth_event` last so `last_event.is_final_response()` returns True) would also fix the loop termination in a single iteration without needing the flag. I went with `end_invocation = True` to preserve the observable event order and to match the existing pattern in `_resolve_toolset_auth`. Happy to switch if maintainers prefer the reorder.
**Related:**
The same yield site at lines 1126-1130 also produces `tool_confirmation_event` for HITL with the same `long_running_tool_ids` shape and the same termination gap. This PR scopes to `auth_event` only. Happy to open a follow-up PR with the same fix for `tool_confirmation_event` if the team agrees with the approach here.
Co-authored-by: George Weale <gweale@google.com>
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/5638 from doughayden:fix/tool-level-auth-terminates-at-euc 0a04d30aae4a284c8234218ac4e82e1f723bc9c1
PiperOrigin-RevId: 932731604
`gemini-1.*` and `gemini-2.0*` models are respectively deprecated and scheduled for shutdown on June 1, 2026. `gemini-2.5*` models are their successors.
No regressions in unit tests:
```
========================================================================================== 5583 passed, 2237 warnings in 84.91s (0:01:24) ===========================================================================================
```
PiperOrigin-RevId: 907663315
Merge https://github.com/google/adk-python/pull/4705
### Link to Issue or Description of Change
- Closes: #4666
**Problem:**
Gemini-2.0-flash model is still used in integration tests, but disabled for new users
**Solution:**
Replace all occurrences of Gemini-2.0-flash by Gemini-2.5-flash, in integration tests and docstrings.
Because gemini-2.0 is still enabled for some users, this PR doesn't clean the model from unit tests.
### Testing Plan
Run already existing unittests and integration tests
### Checklist
- [x] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [x] I have performed a self-review of my own code.
- [x] I have commented my code, particularly in hard-to-understand areas.
- [x] I have added tests that prove my fix is effective or that my feature works.
- [x] New and existing unit tests pass locally with my changes.
- [x] I have manually tested my changes end-to-end.
- [x] Any dependent changes have been merged and published in downstream modules.
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/4705 from morganroux:fix/issues/4666 3c390a2ed3abb7b2fc7bd99cfb801e8a2b1f5343
PiperOrigin-RevId: 900973552
Merge https://github.com/google/adk-python/pull/3944
### Link to Issue or Description of Change
**1. Link to an existing issue (if applicable):** N/A
**2. Or, if no issue exists, describe the change:** fixing various typos in multiple files: see commit diffs for details
**Problem:**
Discovered typos while reading ADK repo
**Solution:**
Submitted this PR to fix them
### Testing Plan
N/A: changes only in comments, .md and docstrings.
**Unit Tests:**
- [N/A ] I have added or updated unit tests for my change.
- [X] All unit tests pass locally.
_Please include a summary of passed `pytest` results._
**Manual End-to-End (E2E) Tests:**
N/A
### Checklist
- [X] I have read the [CONTRIBUTING.md](https://github.com/google/adk-python/blob/main/CONTRIBUTING.md) document.
- [X] I have performed a self-review of my own code.
- [N/A] I have commented my code, particularly in hard-to-understand areas.
- [N/A] I have added tests that prove my fix is effective or that my feature works.
- [X] New and existing unit tests pass locally with my changes.
- [N/A] I have manually tested my changes end-to-end.
- [N/A] Any dependent changes have been merged and published in downstream modules.
### Additional context
N/A
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/3944 from didier-durand:fix-typos-a 02378a488d9a87ac9b6b7397fe9ad7c393faf16a
PiperOrigin-RevId: 868245940
Merge https://github.com/google/adk-python/pull/2857
Adds support for invoking Gemma models via the Gemini API endpoint. To support agentic function, callbacks are added which can extract and transform function calls and responses into user and model messages in the history.
This change is intended to allow developers to explore the use of Gemma models for agentic purposes without requiring local deployment of the models. This should ease the burden of experimentation and testing for developers.
A basic "hello world" style agent example is provided to demonstrate proper functioning of Gemma 3 models inside an Agent container, using the dice roll + prime check framework of similar examples for other models.
## Testing
### Testing Plan
- add and run integration and unit tests
- manual run of example `multi_tool_agent` from quickstart using new `Gemma` model
- manual run of `hello_world_gemma` agent
### Automated Test Results:
| Test Command | Results |
|----------------|---------|
| pytest ./tests/unittests | 4386 passed, 2849 warnings in 58.43s |
| pytest ./tests/unittests/models/test_google_llm.py | 100 passed, 4 warnings in 5.83s |
| pytest ./tests/integration/models/test_google_llm.py | 5 passed, 2 warnings in 3.73s |
### Manual Testing
Here is a log of `multi_tool_agent` run with locally-built wheel and using Gemma model.
```
❯ adk run multi_tool_agent
Log setup complete: /var/folders/bg/_133c0ds2kb7cn699cpmmh_h0061bp/T/agents_log/agent.20250904_152617.log
To access latest log: tail -F /var/folders/bg/_133c0ds2kb7cn699cpmmh_h0061bp/T/agents_log/agent.latest.log
/Users/<redacted>/venvs/adk-quickstart/lib/python3.11/site-packages/google/adk/cli/cli.py:143: UserWarning: [EXPERIMENTAL] InMemoryCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
credential_service = InMemoryCredentialService()
/Users/<redacted>/venvs/adk-quickstart/lib/python3.11/site-packages/google/adk/auth/credential_service/in_memory_credential_service.py:33: UserWarning: [EXPERIMENTAL] BaseCredentialService: This feature is experimental and may change or be removed in future versions without notice. It may introduce breaking changes at any time.
super().__init__()
Running agent weather_time_agent, type exit to exit.
[user]: what's the weather like today?
[weather_time_agent]: Which city are you asking about?
[user]: new york
[weather_time_agent]: OK. The weather in New York is sunny with a temperature of 25 degrees Celsius (77 degrees Fahrenheit).
```
And here is a snippet of a log generated with DEBUG level logging of the `hello_world_gemma` sample. It demonstrates how function calls are extracted and inserted based on Gemma model interactions:
```
...
2025-09-04 15:32:41,708 - DEBUG - google_llm.py:138 -
LLM Request:
-----------------------------------------------------------
System Instruction:
None
-----------------------------------------------------------
Contents:
{"parts":[{"text":"\n You roll dice and answer questions about the outcome of the dice rolls.\n You can roll dice of different sizes...\n"}],"role":"user"}
{"parts":[{"text":"Hi, introduce yourself."}],"role":"user"}
{"parts":[{"text":"Hello! I am data_processing_agent, a hello world agent that can roll many-sided dice and check if numbers are prime. I'm ready to assist you with those tasks. Let's begin!\n\n\n\n"}],"role":"model"}
{"parts":[{"text":"Roll a die with 100 sides and check if it is prime"}],"role":"user"}
{"parts":[{"text":"{\"args\":{\"sides\":100},\"name\":\"roll_die\"}"}],"role":"model"}
{"parts":[{"text":"Invoking tool `roll_die` produced: `{\"result\": 82}`."}],"role":"user"}
{"parts":[{"text":"{\"args\":{\"nums\":[82]},\"name\":\"check_prime\"}"}],"role":"model"}
{"parts":[{"text":"Invoking tool `check_prime` produced: `{\"result\": \"No prime numbers found.\"}`."}],"role":"user"}
{"parts":[{"text":"The die roll was 82, and it is not a prime number.\n\n\n\n"}],"role":"model"}
{"parts":[{"text":"Roll it again."}],"role":"user"}
-----------------------------------------------------------
Functions:
-----------------------------------------------------------
2025-09-04 15:32:41,708 - INFO - models.py:8165 - AFC is enabled with max remote calls: 10.
2025-09-04 15:32:42,693 - INFO - google_llm.py:180 - Response received from the model.
2025-09-04 15:32:42,693 - DEBUG - google_llm.py:181 -
LLM Response:
-----------------------------------------------------------
Text:
{"args":{"sides":100},"name":"roll_die"}
-----------------------------------------------------------
...
```
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/adk-python/pull/2857 from douglas-reid:add-gemma-via-api e6d015f6a9ccbcf20ef7a7af8e4bbe1e9a5936b6
PiperOrigin-RevId: 816451001
We updated the one of the public methods on AgentEvaluator to take in eval metric configurations using a more formal EvalConfig data model.
We also mark "criteria" field on the method as deprecated.
Updated some integration test cases.
PiperOrigin-RevId: 814314134
We add a new metric for evaluating safety of Agent's response to ADK Eval. We delegate the actual implementation to Vertex Gen AI Eval SDK, so using this metric will require GCP project.
As a part of this change, we created (refactored) a simple Facade for vertex gen ai eval sdk.
PiperOrigin-RevId: 778580406
Additionally, few other small changes.
* Updated a test fixture to support the latest eval data schema. Somehow I missed doing that previously.
* Updated the `evaluation_generator.py` to use `run_async`, instead of `run`.
* Also, raise an informed error when dependencies required eval are not installed.
* Also, changed the behavior of AgentEvaluator.evaluate method to run all the evals, instead of failing at the first eval metric failure.
PiperOrigin-RevId: 775919127
nikkie