Add a client-side response cache honoring SEP-2549 caching hints (#3023)
This commit is contained in:
Max
+70
-17
@@ -4,61 +4,114 @@ Every result a server returns for `tools/list`, `prompts/list`, `resources/list`
|
||||
|
||||
The server doesn't cache anything. The fields are a *declaration*: "this tool list is the same for everyone and won't change for a minute." A client (or a gateway in front of you) may then skip the round trip. Honoring the hints is the client's choice; emitting them is the server's job, and the SDK does it for you.
|
||||
|
||||
Out of the box every result says `ttlMs: 0, cacheScope: "private"` — immediately stale, never shared. That is always safe and always conformant. If your lists really are stable and identical for all callers, say so at construction:
|
||||
Out of the box every result says `ttlMs: 0, cacheScope: "private"`: immediately stale, never shared. That is always safe and always conformant. If your lists really are stable and identical for all callers, say so at construction:
|
||||
|
||||
```python title="server.py" hl_lines="5-8"
|
||||
--8<-- "docs_src/caching/tutorial001.py"
|
||||
```
|
||||
|
||||
* The map is keyed by **method name** — the six cacheable methods are the only legal keys. The parameter is typed `Mapping[CacheableMethod, CacheHint]`, so your editor autocompletes the keys and flags a typo before you run; anything that slips past the type checker raises at construction.
|
||||
* The map is keyed by **method name**, and the six cacheable methods are the only legal keys. The parameter is typed `Mapping[CacheableMethod, CacheHint]`, so your editor autocompletes the keys and flags a typo before you run; anything that slips past the type checker raises at construction.
|
||||
* A method you don't mention keeps the defaults. The map is a set of overrides, not a manifest.
|
||||
* `CacheHint(ttl_ms=5_000)` left `scope` unset, so it stays `"private"`: five seconds of freshness, per caller. Scope and TTL are independent decisions.
|
||||
* `"server/discover"` is a legal key too — the handshake result is cacheable like any list.
|
||||
* `"server/discover"` is a legal key too, since the handshake result is cacheable like any list.
|
||||
|
||||
!!! warning
|
||||
`cacheScope: "public"` means *anyone* may be served your cached response — a shared
|
||||
`cacheScope: "public"` means *anyone* may be served your cached response. A shared
|
||||
gateway will happily hand one user's result to another, even when the request was
|
||||
authenticated. Mark a result `"public"` only when it is identical for every caller, and
|
||||
never use `cacheScope` as access control: it is a label, not a lock.
|
||||
|
||||
## Per-handler override
|
||||
|
||||
On the low-level `Server`, handlers build their results by hand — and `ttl_ms` / `cache_scope` are just fields on the result models. A handler that sets them explicitly always wins over the constructor map, field by field:
|
||||
On the low-level `Server`, handlers build their results by hand, and `ttl_ms` / `cache_scope` are just fields on the result models. A handler that sets them explicitly always wins over the constructor map, field by field:
|
||||
|
||||
```python title="server.py" hl_lines="11 17"
|
||||
--8<-- "docs_src/caching/tutorial002.py"
|
||||
```
|
||||
|
||||
The handler said `ttl_ms=1_000` and nothing about scope. On the wire: `ttlMs: 1000` (the handler's, not the map's `60_000`) and `cacheScope: "public"` (the map's — the handler left it unset). Explicit beats configured, configured beats default — per field, so a handler can pin one field and leave the other to the server-wide policy.
|
||||
The handler said `ttl_ms=1_000` and nothing about scope. On the wire: `ttlMs: 1000` (the handler's, not the map's `60_000`) and `cacheScope: "public"` (the map's, because the handler left it unset). Explicit beats configured, and configured beats default. This holds per field, so a handler can pin one field and leave the other to the server-wide policy.
|
||||
|
||||
This is also the escape hatch for dynamics the constructor can't know: a handler that filters `resources/read` per user can return `cache_scope="private"` for one URI from an otherwise-public server.
|
||||
|
||||
One caveat on paginated lists: the protocol requires the **same `cacheScope` on every page** of one list. The constructor map satisfies that by construction — it's keyed by method, not by page. But a handler that overrides the scope itself owns that consistency: override it on *every* page, never only when a cursor is present, or page one and page two will disagree.
|
||||
One caveat on paginated lists: the protocol requires the **same `cacheScope` on every page** of one list. The constructor map satisfies that by construction, since it's keyed by method, not by page. But a handler that overrides the scope itself owns that consistency: override it on *every* page, never only when a cursor is present, or page one and page two will disagree.
|
||||
|
||||
## What the client sees
|
||||
|
||||
On the client, the hints arrive as plain fields on every cacheable result — `ttl_ms` and `cache_scope`, already parsed:
|
||||
On a 2026-07-28 session, `Client` honors the hints for you: it has a built-in response cache, on by default. A result that arrives carrying a `ttlMs` is stored, and an identical call within that TTL is served from the cache with no round trip. A result that carries *no* hint is not cached: hint-less results get `CacheConfig.default_ttl_ms`, which defaults to `0` (immediately stale), so a server that declares nothing sees exactly the call-for-call traffic it always did.
|
||||
|
||||
```python title="client.py" hl_lines="15"
|
||||
```python title="client.py" hl_lines="34 36 39"
|
||||
--8<-- "docs_src/caching/tutorial003.py"
|
||||
```
|
||||
|
||||
The SDK parses; it does not (yet) act. There is no built-in response cache: calling `list_tools()` twice makes two round trips, whatever the TTL said. The spec makes honoring optional — a client that ignores the hints entirely is fully conformant — so until the SDK grows a response cache, the supported path is to read the fields and do your own bookkeeping:
|
||||
Four calls, three fetches. The second call found a fresh entry and never reached the server; advancing the (injected) clock past the TTL made the third fetch again; the fourth said `cache_mode="refresh"`. That kwarg exists on the five caching verbs (`list_tools`, `list_prompts`, `list_resources`, `list_resource_templates`, `read_resource`):
|
||||
|
||||
* **Freshness** is `now < t_received + ttl_ms / 1000`: record the clock when the response arrives, and treat the result as reusable until the TTL runs out. `ttl_ms == 0` means *immediately stale* — don't reuse it at all.
|
||||
* **Scope is a sharing rule, not a suggestion.** A `"private"` result may be reused only within the same authorization context — same access token, same cache. Never put `"private"` results in a cache shared across users.
|
||||
* **Notifications beat TTL.** If the server sends `list_changed` while your copy is still fresh, the copy is stale now — re-fetch.
|
||||
* `"use"` (the default) serves a fresh entry if there is one, and stores the fetch if not.
|
||||
* `"refresh"` never serves: it fetches and stores the result, replacing whatever was cached.
|
||||
* `"bypass"` makes the round trip without touching the cache at all: no read, no write.
|
||||
|
||||
Against an **older server** (pre-2026 protocol), the fields are simply absent from the wire, and the models show their conservative defaults: `ttl_ms == 0`, `cache_scope == "private"` — stale and unshared, the right assumption for a server that declared nothing. If you need to distinguish "the server said 0" from "the server said nothing", check `"ttl_ms" in result.model_fields_set`: it's only set when the field actually arrived.
|
||||
One rule sits above `"use"`: **calls carrying `meta` always reach the server.** A request with `meta` set (a progress token, tracing fields) expects a wire request, so under `cache_mode="use"` it is treated as `"refresh"`: the cache read is skipped, and the fetched result still replaces the cached entry. `"bypass"` and an explicit `"refresh"` behave as they always do.
|
||||
|
||||
To turn caching off entirely, construct with `Client(server, cache=False)`: every call is a round trip again, and `cache_mode`, while still accepted, does nothing.
|
||||
|
||||
Scope is honored automatically too: `"private"` entries are keyed to the cache's *partition* (below), while `"public"` ones may opt into wider sharing. And **notifications beat TTL** for the exact entries they name: a `list_changed` notification evicts the matching cached listing, and `resources/updated` evicts the cached read stored under exactly its URI, however fresh they were.
|
||||
|
||||
One caveat on `resources/updated`: eviction is exact-URI only. The store contract has no enumerate or scan operation (same as the reference TypeScript implementation), so a notification carrying a *sub*-resource URI does not evict a cached read of its parent. If your server signals sub-resources this way, refetch the parent with `cache_mode="refresh"`.
|
||||
|
||||
### Configuring it: `CacheConfig`
|
||||
|
||||
```python
|
||||
from mcp.client import CacheConfig
|
||||
|
||||
client = Client("https://api.example.com/mcp", cache=CacheConfig(default_ttl_ms=5_000))
|
||||
```
|
||||
|
||||
* `store`: where entries live. The default is a fresh in-memory store per client; pass your own `ResponseCacheStore` implementation (Redis-backed, say) to share a cache across clients or processes. The contract types (`ResponseCacheStore`, `CacheKey`, `CacheEntry`, and the default `InMemoryResponseCacheStore`) are importable from `mcp.client`. A lookup may issue up to two sequential store `get`s (the private arm, then the public one), so size a remote store's latency expectations accordingly. A custom store **requires** an explicit `partition`.
|
||||
* `partition`: the authorization-context label that keeps one principal's `"private"` entries from being served to another within a shared store.
|
||||
* `target_id`: explicit server identity, for custom transports and in-process servers (below).
|
||||
* `default_ttl_ms`: TTL applied to results that carry no `ttlMs` hint. The default `0` leaves hint-less results uncached.
|
||||
* `share_public`: serve server-asserted-`"public"` entries across partitions (below). Off by default.
|
||||
* `clock`: the wall-clock source, in epoch seconds. Inject one, as the example above does, and expiry tests need no sleeping.
|
||||
|
||||
!!! warning "Partition = verified principal"
|
||||
Derive `partition` from a **verified credential**, such as a validated token's subject. Never derive it from request-supplied data, and never from the server URL (server identity is a separate key axis). The SDK is a library with no authentication of its own: the trust anchor is whoever constructs the `CacheConfig`, which is the deployment, not the tenant. A multi-tenant gateway mints one `CacheConfig` per authenticated principal.
|
||||
|
||||
The partition is also fixed for the `Client`'s lifetime. If the connection's authorization context changes mid-session (a re-authentication as a different principal, say), the cache does not follow; construct a new `Client` for the new principal.
|
||||
|
||||
Cache keys also carry the **server's identity**: the URL string you dialed, with any `user:pass@` userinfo stripped and otherwise byte-exact. No case folding, no query reordering, no trailing-slash cleanup. Under-normalizing only costs sharing, while over-normalizing could merge two tenants (`?tenant=a` vs `?tenant=b`), so superficially different URLs simply don't share entries. When there is no URL (an in-process server, or a `Transport` instance), the client gets a random per-instance identity instead; set `CacheConfig.target_id` to name the server (with a custom store this is required, and construction says so). The identity is sha256-hashed before it enters key material, so a URL carrying secrets in its query string never appears in store keys. Don't log the pre-hash form yourself, either.
|
||||
|
||||
!!! warning "`share_public` trusts the server, fleet-wide"
|
||||
By default even `"public"` entries stay within their partition. `share_public=True` serves entries the server marked `cacheScope: "public"` to **every** partition using the store, trusting the server's classification on behalf of all of them. A server that stamps `"public"` on per-tenant data (by bug or by malice) then leaks one tenant's response to the others. The flag is deliberately constructor-level only: the per-call `cache_mode` can narrow caching, but nothing per-call can widen sharing.
|
||||
|
||||
### What the cache never does
|
||||
|
||||
* **Session-tier calls bypass it.** `client.session.list_tools()` and friends always make the round trip; the cache lives on the `Client` verbs.
|
||||
* **`server/discover` stays out of it.** The discover result is delivered once, at connect, and never enters the response cache, even when it carries a `ttlMs`. If you persist one yourself to skip the reconnect probe ([`prior_discover`](../client/protocol-versions.md#reconnecting-with-prior_discover)), its freshness is your bookkeeping: `DiscoverResult` carries `ttl_ms` and `cache_scope`, already parsed, for exactly that purpose.
|
||||
* **Continuation pages are never cached.** Only cursor-less calls participate. A continuation page rejected for an expired cursor does *evict* the cached listing, because the listing changed under it.
|
||||
* **Multi-round-trip reads are never cached.** A `read_resource` seeded with `input_responses`/`request_state`, or one that resolves through input rounds, never enters the cache (a spec MUST).
|
||||
* **Notification eviction needs notifications.** Eviction is only as good as the transport's delivery, and the modern in-process path (`Client(server)` with the default `mode="auto"`) does not deliver standalone notifications today.
|
||||
* **Eviction is eventual, not instantaneous.** Wire-path notifications are dispatched from spawned tasks, so a call racing a notification's arrival may be served the pre-eviction entry once more; the window is bounded by dispatch latency, and the eviction still lands.
|
||||
* **No stale-if-error.** An expired entry is never served because the refetch failed; the error propagates.
|
||||
* **No early re-fetch.** A stored entry is served until its TTL expires and the next call after that pays the round trip; nothing refreshes in the background.
|
||||
* **No coalescing.** Two concurrent identical calls are two fetches.
|
||||
* **No TTL beyond 24 hours.** A larger `ttlMs`, whether server-sent or configured, is clamped down on store (`mcp.client.caching.MAX_TTL_MS`), bounding how long any entry, however generously hinted, can be served.
|
||||
* On a **shared store**, clients race each other. Each client drops its own write when an eviction overtook the fetch in flight, but a *co-tenant* client can still write back an entry that an eviction it never saw had removed; and that race bookkeeping is itself bounded: past 4096 tracked keys the oldest key's guard is dropped first. Both windows are accepted, and closed by the TTL cap above.
|
||||
* **No serving across protocol eras.** Entries are scoped to the negotiated protocol version: on a shared persistent store, a session never serves an entry written under a different negotiated version (the same listing genuinely differs by era, since the SDK strips the 2026 fields for older sessions). Eviction likewise touches only the current era's entries; another era's entries simply age out by TTL.
|
||||
|
||||
### Reading the hints yourself
|
||||
|
||||
The hints are also plain fields on every cacheable result (`result.ttl_ms` and `result.cache_scope`, already parsed), in case you want to layer your own bookkeeping on top of (or instead of) the built-in cache.
|
||||
|
||||
Against an **older server** (pre-2026 protocol), the fields are simply absent from the wire, and the models show their conservative defaults: `ttl_ms == 0` and `cache_scope == "private"`, stale and unshared, the right assumption for a server that declared nothing. The cache treats a legacy session the same way: hints are never consulted there (whatever keys appear on the wire), only `default_ttl_ms` applies, and its default of `0` caches nothing, so a pre-2026 connection behaves exactly as it did before the cache existed. If you need to distinguish "the server said 0" from "the server said nothing", check `"ttl_ms" in result.model_fields_set`: it's only set when the field actually arrived.
|
||||
|
||||
## Older clients
|
||||
|
||||
Clients on pre-2026 protocol versions never see either field — the SDK strips them at serialization for those connections. Configure your hints once; there is nothing version-specific to write.
|
||||
Clients on pre-2026 protocol versions never see either field; the SDK strips them at serialization for those connections. Configure your hints once; there is nothing version-specific to write.
|
||||
|
||||
## Recap
|
||||
|
||||
* Six methods carry `ttlMs`/`cacheScope`; the SDK defaults them to `0`/`"private"` — stale and unshared, always safe.
|
||||
* Six methods carry `ttlMs`/`cacheScope`; the SDK defaults them to `0`/`"private"`, stale and unshared, always safe.
|
||||
* `cache_hints={method: CacheHint(...)}` at construction (both `MCPServer` and `Server`) sets server-wide values per method.
|
||||
* A handler that sets the fields on its result overrides the map, per field.
|
||||
* `"public"` is a promise that the result is identical for every caller. It is not access control.
|
||||
* Clients read the hints as `result.ttl_ms` / `result.cache_scope` and own the caching decision themselves — the SDK has no built-in response cache yet.
|
||||
* `Client` honors the hints automatically: its response cache is on by default, serves fresh entries instead of refetching, and caches nothing for servers (or sessions) that provide no hints.
|
||||
* Per call, `cache_mode="refresh"` refetches and `"bypass"` skips the cache; `cache=False` at construction turns it off entirely.
|
||||
|
||||
@@ -427,6 +427,10 @@ On `ClientSession`, `call_tool` / `get_prompt` / `read_resource` still return th
|
||||
|
||||
For protocol 2026-07-28 over Streamable HTTP, a tool's input-schema property may carry an `x-mcp-header` annotation. When a tool the client has listed is called, each annotated argument is mirrored into an `Mcp-Param-<name>` request header (string verbatim, integer as decimal, boolean as `true`/`false`, base64-sentinel-wrapped when not header-safe; `null`/absent arguments are omitted). The argument is also left in the request body. `list_tools` caches a tool's annotations, so list a tool before calling it to enable mirroring; a tool the client never listed emits no `Mcp-Param-*` headers. Other transports ignore the annotation.
|
||||
|
||||
### `Client` verbs may serve cached responses ([SEP-2549](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2549))
|
||||
|
||||
On protocol 2026-07-28, servers attach caching hints (`ttlMs`, `cacheScope`) to the cacheable results, and `Client` now honors them: `list_tools`, `list_prompts`, `list_resources`, `list_resource_templates`, and `read_resource` may serve a cached response instead of making a round trip, for as long as the server's `ttlMs` says the result is fresh. With the default configuration, servers that send no hints, including every pre-2026 server, see identical call-for-call behavior, because hint-less results are not cached (a `CacheConfig.default_ttl_ms` above zero caches them too). Pass `Client(..., cache=False)` to disable the cache and restore v1 behavior exactly; per-call control (`cache_mode`) and configuration (`CacheConfig`) are described in [Caching hints](advanced/caching.md).
|
||||
|
||||
### Server extensions API ([SEP-2133](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2133))
|
||||
|
||||
`MCPServer` now accepts opt-in extensions that bundle MCP behaviour behind a
|
||||
|
||||
@@ -1,15 +1,40 @@
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from mcp_types import ListToolsResult, PaginatedRequestParams, Tool
|
||||
|
||||
from mcp import Client
|
||||
from mcp.server import CacheHint, MCPServer
|
||||
|
||||
mcp = MCPServer("Weather", cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")})
|
||||
from mcp.client import CacheConfig
|
||||
from mcp.server import CacheHint, Server, ServerRequestContext
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
def forecast(city: str) -> str:
|
||||
return f"Sunny in {city}"
|
||||
@dataclass
|
||||
class DemoState:
|
||||
fetches: int = 0
|
||||
now: float = 1_000_000.0
|
||||
|
||||
|
||||
state = DemoState()
|
||||
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams | None) -> ListToolsResult:
|
||||
state.fetches += 1
|
||||
return ListToolsResult(tools=[Tool(name="forecast", input_schema={"type": "object"})])
|
||||
|
||||
|
||||
server = Server(
|
||||
"Weather",
|
||||
on_list_tools=list_tools,
|
||||
cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")},
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
async with Client(mcp) as client:
|
||||
tools = await client.list_tools()
|
||||
print(f"{len(tools.tools)} tools, fresh for {tools.ttl_ms / 1000:.0f}s, scope={tools.cache_scope}")
|
||||
start = state.fetches
|
||||
async with Client(server, cache=CacheConfig(clock=lambda: state.now)) as client:
|
||||
await client.list_tools() # fetch 1
|
||||
await client.list_tools() # fresh for 60s: served from the cache
|
||||
state.now += 60.0
|
||||
await client.list_tools() # the TTL ran out: fetch 2
|
||||
await client.list_tools(cache_mode="refresh") # skip the cache read: fetch 3
|
||||
print(f"4 calls, {state.fetches - start} fetches")
|
||||
|
||||
@@ -13,7 +13,7 @@ from __future__ import annotations
|
||||
from collections.abc import Mapping
|
||||
from functools import cache
|
||||
from types import MappingProxyType, UnionType
|
||||
from typing import Any, Final, TypeVar
|
||||
from typing import Any, Final, Literal, TypeVar, get_args
|
||||
|
||||
from pydantic import BaseModel, TypeAdapter
|
||||
|
||||
@@ -23,9 +23,11 @@ import mcp_types.v2026_07_28 as v2026
|
||||
from mcp_types.version import KNOWN_PROTOCOL_VERSIONS
|
||||
|
||||
__all__ = [
|
||||
"CACHEABLE_METHODS",
|
||||
"CLIENT_NOTIFICATIONS",
|
||||
"CLIENT_REQUESTS",
|
||||
"CLIENT_RESULTS",
|
||||
"CacheableMethod",
|
||||
"MONOLITH_NOTIFICATIONS",
|
||||
"MONOLITH_REQUESTS",
|
||||
"MONOLITH_RESULTS",
|
||||
@@ -404,6 +406,24 @@ MONOLITH_RESULTS: Final[Mapping[str, type[types.Result] | UnionType]] = MappingP
|
||||
"""Monolith result model (or two-arm union) per request method."""
|
||||
|
||||
|
||||
CacheableMethod = Literal[
|
||||
"prompts/list",
|
||||
"resources/list",
|
||||
"resources/read",
|
||||
"resources/templates/list",
|
||||
"server/discover",
|
||||
"tools/list",
|
||||
]
|
||||
"""Methods whose results carry `ttlMs`/`cacheScope`; hand-written Literal, welded to `CACHEABLE_METHODS` by tests."""
|
||||
|
||||
CACHEABLE_METHODS: Final[frozenset[str]] = frozenset(
|
||||
method
|
||||
for method, row in MONOLITH_RESULTS.items()
|
||||
if any(issubclass(arm, types.CacheableResult) for arm in (get_args(row) if isinstance(row, UnionType) else (row,)))
|
||||
)
|
||||
"""Runtime mirror of `CacheableMethod`, derived from `MONOLITH_RESULTS`."""
|
||||
|
||||
|
||||
# --- Parse functions ---
|
||||
|
||||
# Envelope stubs merged into bodies for surface validation (surface classes are full frames).
|
||||
|
||||
@@ -2,8 +2,28 @@
|
||||
|
||||
from mcp.client._input_required import InputRequiredRoundsExceededError
|
||||
from mcp.client._transport import Transport
|
||||
from mcp.client.caching import (
|
||||
CacheConfig,
|
||||
CacheEntry,
|
||||
CacheKey,
|
||||
CacheMode,
|
||||
InMemoryResponseCacheStore,
|
||||
ResponseCacheStore,
|
||||
)
|
||||
from mcp.client.client import Client
|
||||
from mcp.client.context import ClientRequestContext
|
||||
from mcp.client.session import ClientSession
|
||||
|
||||
__all__ = ["Client", "ClientRequestContext", "ClientSession", "InputRequiredRoundsExceededError", "Transport"]
|
||||
__all__ = [
|
||||
"CacheConfig",
|
||||
"CacheEntry",
|
||||
"CacheKey",
|
||||
"CacheMode",
|
||||
"Client",
|
||||
"ClientRequestContext",
|
||||
"ClientSession",
|
||||
"InMemoryResponseCacheStore",
|
||||
"InputRequiredRoundsExceededError",
|
||||
"ResponseCacheStore",
|
||||
"Transport",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,387 @@
|
||||
"""Client-side response caching primitives (SEP-2549, protocol revision 2026-07-28)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Final, Literal, Protocol
|
||||
|
||||
import anyio
|
||||
import anyio.lowlevel
|
||||
from mcp_types import (
|
||||
CacheableResult,
|
||||
PromptListChangedNotification,
|
||||
ResourceListChangedNotification,
|
||||
ResourceUpdatedNotification,
|
||||
ServerNotification,
|
||||
ToolListChangedNotification,
|
||||
)
|
||||
from mcp_types.version import MODERN_PROTOCOL_VERSIONS
|
||||
|
||||
__all__ = [
|
||||
"MAX_TTL_MS",
|
||||
"CacheConfig",
|
||||
"CacheEntry",
|
||||
"CacheKey",
|
||||
"CacheMode",
|
||||
"InMemoryResponseCacheStore",
|
||||
"ResponseCacheStore",
|
||||
]
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
CacheMode = Literal["use", "refresh", "bypass"]
|
||||
"""Per-call cache behavior: `"use"` serves and stores, `"refresh"` stores
|
||||
without serving, `"bypass"` skips the cache entirely."""
|
||||
|
||||
MAX_TTL_MS: Final[int] = 24 * 60 * 60 * 1000
|
||||
"""Cap on any entry's time-to-live (24 hours, in milliseconds); larger `ttlMs` values are clamped down."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CacheKey:
|
||||
"""Identity of one cached response; compare as the field tuple, never a flattened string (collision hazard)."""
|
||||
|
||||
method: str
|
||||
|
||||
params_key: str = ""
|
||||
"""Result-affecting params discriminator: the uri for `resources/read`, `""` for the list methods."""
|
||||
|
||||
partition: str = ""
|
||||
"""Coordinator-computed arm identifier; opaque to stores."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CacheEntry:
|
||||
"""One cached response with its freshness and sharing metadata."""
|
||||
|
||||
value: Any
|
||||
"""The cached result; the SDK deep-copies on write and on serve, so a store may hold it as-is."""
|
||||
|
||||
scope: Literal["public", "private"]
|
||||
"""Server-asserted `cacheScope`: only `"public"` entries may be shared across authorization contexts."""
|
||||
|
||||
expires_at: float | None
|
||||
"""Epoch seconds after which the entry is stale; `None` is never fresh."""
|
||||
|
||||
|
||||
class ResponseCacheStore(Protocol):
|
||||
"""Storage contract for the client response cache.
|
||||
|
||||
Each `Client` calls its store from a single event loop; per-operation
|
||||
atomicity is the implementation's responsibility. Operations may raise -
|
||||
the SDK degrades to a miss rather than failing the call. A serializing
|
||||
store must round-trip `value` back to the result model object (a
|
||||
wrong-shape entry is a miss, never an error). A lookup may issue two
|
||||
sequential `get` calls (private arm, then public).
|
||||
"""
|
||||
|
||||
async def get(self, key: CacheKey) -> CacheEntry | None: ...
|
||||
|
||||
async def set(self, key: CacheKey, entry: CacheEntry) -> None: ...
|
||||
|
||||
async def delete(self, key: CacheKey) -> None: ...
|
||||
|
||||
async def clear(self) -> None: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CacheConfig:
|
||||
"""Configuration for a `Client`'s response cache.
|
||||
|
||||
Raises:
|
||||
ValueError: On a custom `store` without `partition`, an empty `target_id`, or a negative `default_ttl_ms`.
|
||||
"""
|
||||
|
||||
store: ResponseCacheStore | None = None
|
||||
"""Backing store; `None` means a per-client `InMemoryResponseCacheStore`.
|
||||
A custom store requires an explicit `partition`."""
|
||||
|
||||
partition: str = ""
|
||||
"""Authorization-context identifier isolating `"private"`-scoped entries
|
||||
within a shared store. Derive it from a verified credential - never from
|
||||
request-supplied data or the server URL. Fixed for the `Client`'s
|
||||
lifetime: construct a new `Client` when the principal changes."""
|
||||
|
||||
target_id: str | None = None
|
||||
"""Server-identity override for custom transports and proxies where the
|
||||
SDK cannot derive one from a URL; must be non-empty when provided."""
|
||||
|
||||
default_ttl_ms: int = 0
|
||||
"""TTL in milliseconds for results carrying no `ttlMs` hint; the default `0` leaves them uncached."""
|
||||
|
||||
clock: Callable[[], float] = time.time
|
||||
"""Wall-clock source returning epoch seconds; injectable for expiry tests."""
|
||||
|
||||
share_public: bool = False
|
||||
"""Serve server-marked `"public"` entries across every partition in the store.
|
||||
|
||||
WARNING: this trusts the server's `"public"` classification for every
|
||||
principal sharing the store - a mislabeled response leaks across tenants.
|
||||
Constructor-level only: the per-call `cache_mode` can never widen sharing."""
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.store is not None and not self.partition:
|
||||
raise ValueError("a custom store requires an explicit partition")
|
||||
if self.target_id == "":
|
||||
raise ValueError("target_id must be a non-empty string or omitted")
|
||||
if self.default_ttl_ms < 0:
|
||||
raise ValueError(f"default_ttl_ms must be >= 0, got {self.default_ttl_ms}")
|
||||
|
||||
|
||||
class InMemoryResponseCacheStore:
|
||||
"""Default in-process `ResponseCacheStore`.
|
||||
|
||||
Method bodies are synchronous, so concurrent tasks never observe a torn
|
||||
write. `max_entries` caps the whole store, evicting least-recently-used
|
||||
at the cap (`0` disables it); `get` and `set` both refresh recency, so a
|
||||
hot entry survives churn from other keys.
|
||||
|
||||
Raises:
|
||||
ValueError: If `max_entries` is negative.
|
||||
"""
|
||||
|
||||
def __init__(self, *, max_entries: int = 1024) -> None:
|
||||
if max_entries < 0:
|
||||
raise ValueError(f"max_entries must be >= 0, got {max_entries}")
|
||||
self._max_entries = max_entries
|
||||
self._entries: dict[CacheKey, CacheEntry] = {}
|
||||
|
||||
async def get(self, key: CacheKey) -> CacheEntry | None:
|
||||
entry = self._entries.get(key)
|
||||
if entry is not None:
|
||||
# Pop-and-reinsert moves the key to the back: the dict's insertion order is the LRU ledger.
|
||||
self._entries[key] = self._entries.pop(key)
|
||||
return entry
|
||||
|
||||
async def set(self, key: CacheKey, entry: CacheEntry) -> None:
|
||||
self._entries.pop(key, None)
|
||||
self._entries[key] = entry
|
||||
if self._max_entries and len(self._entries) > self._max_entries:
|
||||
del self._entries[next(iter(self._entries))]
|
||||
|
||||
async def delete(self, key: CacheKey) -> None:
|
||||
self._entries.pop(key, None)
|
||||
|
||||
async def clear(self) -> None:
|
||||
self._entries.clear()
|
||||
|
||||
|
||||
_GENERATION_MAP_CAP: Final[int] = 4096
|
||||
"""Cap on the generation map; at the cap the oldest key's eviction-race guard is dropped (FIFO)."""
|
||||
|
||||
_STORE_CLEANUP_TIMEOUT: Final[float] = 5
|
||||
"""Bound for must-complete store cleanup deletes (mirrors the dispatcher's final-write bound);
|
||||
a wedged store delete must not hold client teardown uncancellably."""
|
||||
|
||||
|
||||
class ClientResponseCache:
|
||||
"""Coordinates the `Client` caching verbs with a `ResponseCacheStore`: keys, era gate, TTL/scope, eviction."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
store: ResponseCacheStore,
|
||||
partition: str,
|
||||
arm_id: str,
|
||||
default_ttl_ms: int,
|
||||
clock: Callable[[], float],
|
||||
share_public: bool,
|
||||
negotiated_version: Callable[[], str | None],
|
||||
generation_map_cap: int = _GENERATION_MAP_CAP,
|
||||
store_cleanup_timeout: float = _STORE_CLEANUP_TIMEOUT,
|
||||
) -> None:
|
||||
self._store = store
|
||||
self._partition = partition
|
||||
self._arm_id = arm_id
|
||||
self._share_public = share_public
|
||||
self._default_ttl_ms = default_ttl_ms
|
||||
self._clock = clock
|
||||
self._negotiated_version = negotiated_version
|
||||
# A key is eviction-race-guarded iff registered here.
|
||||
self._generations: dict[tuple[str, str], int] = {}
|
||||
self._generation_map_cap = generation_map_cap
|
||||
self._store_cleanup_timeout = store_cleanup_timeout
|
||||
self._warned_store_ops: set[str] = set()
|
||||
|
||||
def _arm(self, scope: Literal["public", "private"]) -> str:
|
||||
# JSON arrays so crafted arm_id/partition values cannot collide across field boundaries.
|
||||
# The negotiated version era-scopes every arm: a session never serves an entry written
|
||||
# under a different protocol era (its content differs - sieve-stripped fields, header
|
||||
# filtering). Every caller runs post-connect; were that ever untrue, the supplier's
|
||||
# None still partitions harmlessly.
|
||||
fields: list[str | None] = [scope, self._negotiated_version(), self._arm_id]
|
||||
if scope == "private" or not self._share_public:
|
||||
fields.append(self._partition)
|
||||
return json.dumps(fields)
|
||||
|
||||
async def read(self, method: str, params_key: str) -> CacheableResult | None:
|
||||
"""Serve a fresh entry for the key, or `None`; the served result is a deep copy."""
|
||||
# A hit completes without any other yielding await, so checkpoint here: a poll
|
||||
# loop over a fresh entry must not starve spawned tasks (eviction dispatch).
|
||||
await anyio.lowlevel.checkpoint()
|
||||
# A wrong-shape entry raises as late as the copy, so the boundary wraps the whole read path.
|
||||
try:
|
||||
entry = await self._get_fresh(CacheKey(method, params_key, self._arm("private")))
|
||||
if entry is None:
|
||||
# After a scope flip, a stale private entry must not shadow a fresh public one.
|
||||
entry = await self._get_fresh(CacheKey(method, params_key, self._arm("public")))
|
||||
if entry is not None and entry.scope != "public":
|
||||
# Never serve an entry the server scoped "private" out of the shared arm.
|
||||
entry = None
|
||||
copied: CacheableResult | None = None if entry is None else entry.value.model_copy(deep=True)
|
||||
except Exception: # boundary around user store code: any read-path failure is a miss, never a failed call
|
||||
self._warn_store_failure("get")
|
||||
return None
|
||||
self._warned_store_ops.discard("get")
|
||||
return copied
|
||||
|
||||
async def _get_fresh(self, key: CacheKey) -> CacheEntry | None:
|
||||
entry = await self._store.get(key)
|
||||
if entry is None or entry.expires_at is None or entry.expires_at <= self._clock():
|
||||
return None
|
||||
return entry
|
||||
|
||||
def capture(self, method: str, params_key: str) -> int:
|
||||
"""Register the key for eviction-race detection before the fetch; `write` takes the returned generation."""
|
||||
gen_key = (method, params_key)
|
||||
if gen_key not in self._generations:
|
||||
if len(self._generations) >= self._generation_map_cap:
|
||||
# FIFO overflow: the dropped key's race guard degrades to the accepted co-tenant class.
|
||||
del self._generations[next(iter(self._generations))]
|
||||
self._generations[gen_key] = 0
|
||||
return self._generations[gen_key]
|
||||
|
||||
async def write(
|
||||
self,
|
||||
method: str,
|
||||
params_key: str,
|
||||
result: CacheableResult,
|
||||
gen_at_capture: int,
|
||||
mode: Literal["use", "refresh"],
|
||||
) -> None:
|
||||
"""Store a fetched result under the arm its resolved scope selects."""
|
||||
gen_key = (method, params_key)
|
||||
if self._generation_moved(gen_key, gen_at_capture):
|
||||
return # the key was evicted while the fetch was in flight
|
||||
ttl_ms, scope = self._resolve(result)
|
||||
private_key = CacheKey(method, params_key, self._arm("private"))
|
||||
public_key = CacheKey(method, params_key, self._arm("public"))
|
||||
if ttl_ms <= 0:
|
||||
if mode == "refresh":
|
||||
# The refetch superseded the warm entry, which a cancellation must not leave serving.
|
||||
await self._cleanup_delete(private_key, public_key)
|
||||
return
|
||||
own, opposite = (public_key, private_key) if scope == "public" else (private_key, public_key)
|
||||
# Opposite arm first: a failed delete aborts before the set - never two arms answering for one key.
|
||||
if not await self._delete(opposite):
|
||||
# The own arm's entry is superseded too: best-effort delete, degrading to a full miss.
|
||||
await self._cleanup_delete(own)
|
||||
return
|
||||
entry = CacheEntry(value=result.model_copy(deep=True), scope=scope, expires_at=self._clock() + ttl_ms / 1000)
|
||||
try:
|
||||
if not await self._set(own, entry):
|
||||
# The fetch superseded any pre-existing own-arm entry, and the failed set
|
||||
# left it in place: purge it (mirrors the opposite-arm-failure path).
|
||||
await self._cleanup_delete(own)
|
||||
finally:
|
||||
# An eviction can land while the set commits - even when the await
|
||||
# is cancelled - so re-check on every exit; the delete must complete
|
||||
# so the pending cancellation cannot resurrect the evicted entry.
|
||||
if self._generation_moved(gen_key, gen_at_capture):
|
||||
await self._cleanup_delete(own)
|
||||
|
||||
async def evict_method(self, method: str) -> None:
|
||||
"""Evict the method's cursor-less entry."""
|
||||
await self.evict_key(method, "")
|
||||
|
||||
async def evict_key(self, method: str, params_key: str) -> None:
|
||||
"""Evict one key from both arms.
|
||||
|
||||
Only the current era's arms are touched; other-era entries in a persistent store age out by TTL.
|
||||
"""
|
||||
gen_key = (method, params_key)
|
||||
# Bump first so an in-flight fetch cannot write the evicted entry back.
|
||||
# Unregistered keys skip the bump (uris must not grow the map) but not
|
||||
# the deletes - a persistent store may hold uncaptured entries.
|
||||
if gen_key in self._generations:
|
||||
self._generations[gen_key] += 1
|
||||
# Must complete: a cancellation between the deletes would leave one arm serving the evicted entry.
|
||||
await self._cleanup_delete(
|
||||
CacheKey(method, params_key, self._arm("private")),
|
||||
CacheKey(method, params_key, self._arm("public")),
|
||||
)
|
||||
|
||||
async def evict_for_notification(self, notification: ServerNotification) -> None:
|
||||
"""Map a server notification to the entries it makes stale.
|
||||
|
||||
Eviction is eventual (spawned-task dispatch): the generation bump closes
|
||||
the write-back race; a racing read may briefly serve the old entry.
|
||||
"""
|
||||
match notification:
|
||||
case ToolListChangedNotification():
|
||||
await self.evict_method("tools/list")
|
||||
case PromptListChangedNotification():
|
||||
await self.evict_method("prompts/list")
|
||||
case ResourceListChangedNotification():
|
||||
# Templates enumerate the same changed resource space.
|
||||
await self.evict_method("resources/list")
|
||||
await self.evict_method("resources/templates/list")
|
||||
case ResourceUpdatedNotification():
|
||||
await self.evict_key("resources/read", notification.params.uri)
|
||||
case _:
|
||||
pass
|
||||
|
||||
def _resolve(self, result: CacheableResult) -> tuple[int, Literal["public", "private"]]:
|
||||
# A legacy peer can also put `ttlMs`/`cacheScope` keys on the wire, so
|
||||
# wire presence is not a peer-era signal - hints count only when modern.
|
||||
modern = self._negotiated_version() in MODERN_PROTOCOL_VERSIONS
|
||||
if modern and "ttl_ms" in result.model_fields_set:
|
||||
# An explicit `ttlMs: 0` stays 0, and negatives are unconstructible
|
||||
# upstream (model ge=0, parse-seam floor) - only the cap applies.
|
||||
ttl_ms = result.ttl_ms
|
||||
else:
|
||||
ttl_ms = self._default_ttl_ms
|
||||
scope: Literal["public", "private"] = "public" if modern and result.cache_scope == "public" else "private"
|
||||
return min(ttl_ms, MAX_TTL_MS), scope
|
||||
|
||||
def _generation_moved(self, gen_key: tuple[str, str], gen_at_capture: int) -> bool:
|
||||
# A FIFO-dropped key fails open (the accepted co-tenant race) rather than discarding the fetch.
|
||||
return self._generations.get(gen_key, gen_at_capture) != gen_at_capture
|
||||
|
||||
async def _set(self, key: CacheKey, entry: CacheEntry) -> bool:
|
||||
try:
|
||||
await self._store.set(key, entry)
|
||||
except Exception: # boundary around user store code: nothing cached, the fetch already succeeded
|
||||
self._warn_store_failure("set")
|
||||
return False
|
||||
self._warned_store_ops.discard("set")
|
||||
return True
|
||||
|
||||
async def _cleanup_delete(self, *keys: CacheKey) -> None:
|
||||
# Must-complete cleanup: shielded so a pending cancellation cannot skip the deletes,
|
||||
# bounded so a wedged store delete cannot hold client teardown uncancellably.
|
||||
with anyio.move_on_after(self._store_cleanup_timeout, shield=True) as scope:
|
||||
for key in keys:
|
||||
await self._delete(key)
|
||||
if scope.cancelled_caught:
|
||||
logger.warning("Response cache store delete timed out; the entry will age out by TTL")
|
||||
|
||||
async def _delete(self, key: CacheKey) -> bool:
|
||||
try:
|
||||
await self._store.delete(key)
|
||||
except Exception: # boundary around user store code: callers decide whether a failed delete aborts
|
||||
self._warn_store_failure("delete")
|
||||
return False
|
||||
self._warned_store_ops.discard("delete")
|
||||
return True
|
||||
|
||||
def _warn_store_failure(self, kind: Literal["get", "set", "delete"]) -> None:
|
||||
# One warning per failure burst, per op kind; re-armed only when that
|
||||
# same kind succeeds, so a healthy delete cannot re-arm a broken set.
|
||||
if kind not in self._warned_store_ops:
|
||||
self._warned_store_ops.add(kind)
|
||||
logger.warning("Response cache store operation failed; continuing without the cache", exc_info=True)
|
||||
+201
-9
@@ -2,14 +2,20 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import uuid
|
||||
from collections.abc import Awaitable, Callable, Mapping
|
||||
from contextlib import AsyncExitStack
|
||||
from dataclasses import KW_ONLY, dataclass, field
|
||||
from typing import Any, Literal, TypeVar
|
||||
from typing import Any, Literal, TypeVar, cast
|
||||
|
||||
import anyio
|
||||
import anyio.lowlevel
|
||||
import mcp_types as types
|
||||
from mcp_types import (
|
||||
INVALID_PARAMS,
|
||||
CacheableResult,
|
||||
CallToolResult,
|
||||
CompleteResult,
|
||||
EmptyResult,
|
||||
@@ -39,6 +45,7 @@ from mcp.client._input_required import DEFAULT_INPUT_REQUIRED_MAX_ROUNDS, run_in
|
||||
from mcp.client._memory import InMemoryTransport
|
||||
from mcp.client._probe import negotiate_auto
|
||||
from mcp.client._transport import Transport
|
||||
from mcp.client.caching import CacheConfig, CacheMode, ClientResponseCache, InMemoryResponseCacheStore
|
||||
from mcp.client.session import (
|
||||
ClientRequestContext,
|
||||
ClientSession,
|
||||
@@ -54,8 +61,11 @@ from mcp.server.mcpserver import MCPServer
|
||||
from mcp.server.runner import modern_on_request
|
||||
from mcp.shared.direct_dispatcher import create_direct_dispatcher_pair
|
||||
from mcp.shared.dispatcher import Dispatcher, ProgressFnT
|
||||
from mcp.shared.exceptions import MCPDeprecationWarning
|
||||
from mcp.shared.exceptions import MCPDeprecationWarning, MCPError
|
||||
from mcp.shared.jsonrpc_dispatcher import JSONRPCDispatcher
|
||||
from mcp.shared.session import RequestResponder
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
ConnectMode = Literal["legacy", "auto"] | str
|
||||
"""``mode=`` value: ``"legacy"`` (initialize handshake), ``"auto"`` (discover, fall back to
|
||||
@@ -64,6 +74,7 @@ forward-compat; ``Client.__post_init__`` rejects anything outside that set at co
|
||||
|
||||
_T = TypeVar("_T")
|
||||
_ResultT = TypeVar("_ResultT")
|
||||
_CacheableT = TypeVar("_CacheableT", bound=CacheableResult)
|
||||
|
||||
_Connector = Callable[[AsyncExitStack, ConnectMode, bool], Awaitable["Dispatcher[Any]"]]
|
||||
"""Resolved at ``__post_init__`` from the shape of ``server`` alone: enter whatever resources
|
||||
@@ -115,6 +126,46 @@ def _connected(value: _T | None) -> _T:
|
||||
return value
|
||||
|
||||
|
||||
def _strip_userinfo(url: str) -> str:
|
||||
"""Drop any userinfo from the URL's authority component; byte-exact otherwise.
|
||||
|
||||
Credentials must not enter cache-key material; any further normalization could merge distinct servers.
|
||||
"""
|
||||
# Pure text, no urlsplit: it strips embedded tab/CR/LF before parsing, which would misalign slices.
|
||||
sep = url.find("//")
|
||||
if sep == -1:
|
||||
return url
|
||||
start = sep + 2
|
||||
end = len(url)
|
||||
for delimiter in "/?#":
|
||||
if (found := url.find(delimiter, start)) != -1:
|
||||
end = min(end, found)
|
||||
authority = url[start:end]
|
||||
if "@" not in authority:
|
||||
return url
|
||||
return url[:start] + authority.rpartition("@")[2] + url[end:]
|
||||
|
||||
|
||||
def _evicting_message_handler(cache: ClientResponseCache, user_handler: MessageHandlerFnT | None) -> MessageHandlerFnT:
|
||||
"""Wrap the session message handler with cache eviction on server notifications."""
|
||||
|
||||
async def handler(
|
||||
message: RequestResponder[types.ServerRequest, types.ClientResult] | types.ServerNotification | Exception,
|
||||
) -> None:
|
||||
if isinstance(message, types.ServerNotification):
|
||||
try:
|
||||
await cache.evict_for_notification(message)
|
||||
except Exception: # boundary: eviction reaches user store code; a cache fault must not block delivery
|
||||
logger.exception("Response cache eviction failed; the notification is still delivered")
|
||||
if user_handler is not None:
|
||||
await user_handler(message)
|
||||
else:
|
||||
# Mirrors ClientSession's default handler (session._default_message_handler).
|
||||
await anyio.lowlevel.checkpoint()
|
||||
|
||||
return handler
|
||||
|
||||
|
||||
def _synthesize_discover(protocol_version: str) -> types.DiscoverResult:
|
||||
return types.DiscoverResult(
|
||||
supported_versions=[protocol_version],
|
||||
@@ -221,10 +272,20 @@ class Client:
|
||||
"""SEP-2133 extension support to advertise under `ClientCapabilities.extensions`
|
||||
(identifier -> settings), e.g. `{"io.modelcontextprotocol/ui": {"mimeTypes": [...]}}`."""
|
||||
|
||||
cache: CacheConfig | Literal[False] | None = None
|
||||
"""Client-side response caching for the SEP-2549 cacheable methods (2026-07-28).
|
||||
|
||||
`None` (the default) honors server `ttlMs`/`cacheScope` hints with a per-client
|
||||
in-memory store; pass a `CacheConfig` to customize, or `False` to disable. The
|
||||
cacheable verbs take a per-call `cache_mode` (see `CacheMode`); calls carrying
|
||||
`meta` always reach the server. A `CacheConfig` with a custom `store` requires
|
||||
`target_id` when the server is not a URL (no identity can be derived)."""
|
||||
|
||||
_entered: bool = field(init=False, default=False)
|
||||
_session: ClientSession | None = field(init=False, default=None)
|
||||
_exit_stack: AsyncExitStack | None = field(init=False, default=None)
|
||||
_connect: _Connector = field(init=False, repr=False, compare=False)
|
||||
_response_cache: ClientResponseCache | None = field(init=False, default=None, repr=False, compare=False)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.mode not in ("legacy", "auto") and self.mode not in MODERN_PROTOCOL_VERSIONS:
|
||||
@@ -247,16 +308,44 @@ class Client:
|
||||
else:
|
||||
self._connect = _connect_transport(srv)
|
||||
|
||||
if self.cache is not False:
|
||||
config = self.cache if self.cache is not None else CacheConfig()
|
||||
# Only the hash below leaves this scope - the raw identity may carry credentials; never log or store it.
|
||||
target_id = config.target_id
|
||||
if target_id is None and isinstance(self.server, str):
|
||||
target_id = _strip_userinfo(self.server)
|
||||
if target_id is None:
|
||||
if config.store is not None:
|
||||
raise ValueError(
|
||||
"a custom cache store requires CacheConfig.target_id when the server is not a URL: "
|
||||
"in-process servers and Transport instances get a random per-client identity, so "
|
||||
"their entries in a shared store could never be served to another client"
|
||||
)
|
||||
target_id = uuid.uuid4().hex
|
||||
self._response_cache = ClientResponseCache(
|
||||
store=config.store if config.store is not None else InMemoryResponseCacheStore(),
|
||||
partition=config.partition,
|
||||
arm_id=hashlib.sha256(target_id.encode()).hexdigest(),
|
||||
default_ttl_ms=config.default_ttl_ms,
|
||||
clock=config.clock,
|
||||
share_public=config.share_public,
|
||||
# Lazy: the negotiated version is unknown until __aenter__'s handshake.
|
||||
negotiated_version=lambda: self._session.protocol_version if self._session is not None else None,
|
||||
)
|
||||
|
||||
async def _build_session(self, exit_stack: AsyncExitStack) -> ClientSession:
|
||||
"""Enter the resolved connector and return an un-entered ClientSession."""
|
||||
dispatcher = await self._connect(exit_stack, self.mode, self.raise_exceptions)
|
||||
message_handler = self.message_handler
|
||||
if self._response_cache is not None:
|
||||
message_handler = _evicting_message_handler(self._response_cache, self.message_handler)
|
||||
return ClientSession(
|
||||
dispatcher=dispatcher,
|
||||
read_timeout_seconds=self.read_timeout_seconds,
|
||||
sampling_callback=self.sampling_callback,
|
||||
list_roots_callback=self.list_roots_callback,
|
||||
logging_callback=self.logging_callback,
|
||||
message_handler=self.message_handler,
|
||||
message_handler=message_handler,
|
||||
client_info=self.client_info,
|
||||
elicitation_callback=self.elicitation_callback,
|
||||
extensions=self.extensions,
|
||||
@@ -361,23 +450,76 @@ class Client:
|
||||
"""Set the logging level on the server."""
|
||||
return await self.session.set_logging_level(level=level, meta=meta) # pyright: ignore[reportDeprecated]
|
||||
|
||||
async def _cached_fetch(
|
||||
self,
|
||||
method: str,
|
||||
*,
|
||||
cursor: str | None,
|
||||
meta: RequestParamsMeta | None,
|
||||
cache_mode: CacheMode,
|
||||
send: Callable[[], Awaitable[_CacheableT]],
|
||||
absorb: Callable[[_CacheableT], _CacheableT] | None = None,
|
||||
) -> _CacheableT:
|
||||
"""Serve one of the four list verbs through the response cache.
|
||||
|
||||
`absorb` (tools/list only) re-applies session-side derived state to a served cache hit.
|
||||
"""
|
||||
cache = self._response_cache
|
||||
if cache is None or cache_mode == "bypass":
|
||||
return await send()
|
||||
# A closed (or never-entered) client must raise, never serve cached entries.
|
||||
_ = self.session
|
||||
if meta is not None and cache_mode == "use":
|
||||
# meta (a progress token, tracing fields) expects a wire request; fetch and replace the entry.
|
||||
cache_mode = "refresh"
|
||||
if cursor is not None:
|
||||
# Continuation pages skip the cache, but an expired cursor means the listing changed (spec SHOULD evict).
|
||||
try:
|
||||
return await send()
|
||||
except MCPError as e:
|
||||
if e.code == INVALID_PARAMS:
|
||||
await cache.evict_method(method)
|
||||
raise
|
||||
if cache_mode == "use" and (hit := await cache.read(method, "")) is not None:
|
||||
# The hit is a private deep copy, so absorption may mutate it freely.
|
||||
served = cast(_CacheableT, hit)
|
||||
return served if absorb is None else absorb(served)
|
||||
gen = cache.capture(method, "")
|
||||
result = await send()
|
||||
await cache.write(method, "", result, gen, cache_mode)
|
||||
return result
|
||||
|
||||
async def list_resources(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
meta: RequestParamsMeta | None = None,
|
||||
cache_mode: CacheMode = "use",
|
||||
) -> ListResourcesResult:
|
||||
"""List available resources from the server."""
|
||||
return await self.session.list_resources(params=PaginatedRequestParams(cursor=cursor, _meta=meta))
|
||||
return await self._cached_fetch(
|
||||
"resources/list",
|
||||
cursor=cursor,
|
||||
meta=meta,
|
||||
cache_mode=cache_mode,
|
||||
send=lambda: self.session.list_resources(params=PaginatedRequestParams(cursor=cursor, _meta=meta)),
|
||||
)
|
||||
|
||||
async def list_resource_templates(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
meta: RequestParamsMeta | None = None,
|
||||
cache_mode: CacheMode = "use",
|
||||
) -> ListResourceTemplatesResult:
|
||||
"""List available resource templates from the server."""
|
||||
return await self.session.list_resource_templates(params=PaginatedRequestParams(cursor=cursor, _meta=meta))
|
||||
return await self._cached_fetch(
|
||||
"resources/templates/list",
|
||||
cursor=cursor,
|
||||
meta=meta,
|
||||
cache_mode=cache_mode,
|
||||
send=lambda: self.session.list_resource_templates(params=PaginatedRequestParams(cursor=cursor, _meta=meta)),
|
||||
)
|
||||
|
||||
async def read_resource(
|
||||
self,
|
||||
@@ -386,6 +528,7 @@ class Client:
|
||||
input_responses: InputResponses | None = None,
|
||||
request_state: str | None = None,
|
||||
meta: RequestParamsMeta | None = None,
|
||||
cache_mode: CacheMode = "use",
|
||||
) -> ReadResourceResult:
|
||||
"""Read a resource from the server.
|
||||
|
||||
@@ -400,6 +543,8 @@ class Client:
|
||||
resuming from a persisted `InputRequiredResult`).
|
||||
request_state: Opaque state to seed the first call with.
|
||||
meta: Additional metadata for the request.
|
||||
cache_mode: Cache behavior for this call (see `CacheMode`); seeded
|
||||
calls (`input_responses` or `request_state` set) ignore it.
|
||||
|
||||
Returns:
|
||||
The resource content.
|
||||
@@ -414,7 +559,29 @@ class Client:
|
||||
uri, input_responses=r, request_state=s, meta=meta, allow_input_required=True
|
||||
)
|
||||
|
||||
return await self._drive_input_required(await retry(input_responses, request_state), retry)
|
||||
# Seeded calls resume a specific exchange and must never be cached (spec MUST).
|
||||
seeded = input_responses is not None or request_state is not None
|
||||
cache = None if seeded else self._response_cache
|
||||
if cache is None or cache_mode == "bypass":
|
||||
return await self._drive_input_required(await retry(input_responses, request_state), retry)
|
||||
# A closed (or never-entered) client must raise, never serve cached entries.
|
||||
_ = self.session
|
||||
if meta is not None and cache_mode == "use":
|
||||
# Calls carrying meta always reach the server (mirrors `_cached_fetch`).
|
||||
cache_mode = "refresh"
|
||||
if cache_mode == "use" and (hit := await cache.read("resources/read", uri)) is not None:
|
||||
# Only terminal first-round results are stored, so a hit legitimately skips the driver.
|
||||
return cast(ReadResourceResult, hit)
|
||||
gen = cache.capture("resources/read", uri)
|
||||
first = await retry(None, None)
|
||||
if not isinstance(first, InputRequiredResult):
|
||||
await cache.write("resources/read", uri, first, gen, cache_mode)
|
||||
elif cache_mode == "refresh":
|
||||
# The refresh superseded whatever was cached, but an input_required resolution
|
||||
# cannot be stored: purge the warm entry so it cannot be served again.
|
||||
await cache.evict_key("resources/read", uri)
|
||||
# Driver rounds carry inputResponses, so a terminal result reached through them is never cached (spec MUST).
|
||||
return await self._drive_input_required(first, retry)
|
||||
|
||||
async def subscribe_resource(self, uri: str, *, meta: RequestParamsMeta | None = None) -> EmptyResult:
|
||||
"""Subscribe to resource updates."""
|
||||
@@ -481,9 +648,16 @@ class Client:
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
meta: RequestParamsMeta | None = None,
|
||||
cache_mode: CacheMode = "use",
|
||||
) -> ListPromptsResult:
|
||||
"""List available prompts from the server."""
|
||||
return await self.session.list_prompts(params=PaginatedRequestParams(cursor=cursor, _meta=meta))
|
||||
return await self._cached_fetch(
|
||||
"prompts/list",
|
||||
cursor=cursor,
|
||||
meta=meta,
|
||||
cache_mode=cache_mode,
|
||||
send=lambda: self.session.list_prompts(params=PaginatedRequestParams(cursor=cursor, _meta=meta)),
|
||||
)
|
||||
|
||||
async def get_prompt(
|
||||
self,
|
||||
@@ -565,9 +739,27 @@ class Client:
|
||||
"""
|
||||
return await self.session.complete(ref=ref, argument=argument, context_arguments=context_arguments)
|
||||
|
||||
async def list_tools(self, *, cursor: str | None = None, meta: RequestParamsMeta | None = None) -> ListToolsResult:
|
||||
async def list_tools(
|
||||
self,
|
||||
*,
|
||||
cursor: str | None = None,
|
||||
meta: RequestParamsMeta | None = None,
|
||||
cache_mode: CacheMode = "use",
|
||||
) -> ListToolsResult:
|
||||
"""List available tools from the server."""
|
||||
return await self.session.list_tools(params=PaginatedRequestParams(cursor=cursor, _meta=meta))
|
||||
return await self._cached_fetch(
|
||||
"tools/list",
|
||||
cursor=cursor,
|
||||
meta=meta,
|
||||
cache_mode=cache_mode,
|
||||
send=lambda: self.session.list_tools(params=PaginatedRequestParams(cursor=cursor, _meta=meta)),
|
||||
# A cache hit skips session.list_tools, so the session re-absorbs the served
|
||||
# listing to rebuild its derived per-tool state. Hits are cursorless, but a
|
||||
# cached page 1 can carry next_cursor - never prune on a partial listing.
|
||||
absorb=lambda hit: self.session._absorb_tool_listing( # pyright: ignore[reportPrivateUsage]
|
||||
hit, complete=hit.next_cursor is None
|
||||
),
|
||||
)
|
||||
|
||||
@deprecated("The roots capability is deprecated as of 2026-07-28 (SEP-2577).", category=MCPDeprecationWarning)
|
||||
async def send_roots_list_changed(self) -> None:
|
||||
|
||||
@@ -55,6 +55,13 @@ DISCOVER_TIMEOUT_SECONDS = 10.0
|
||||
logger = logging.getLogger("client")
|
||||
|
||||
|
||||
def _clamp_inbound_ttl(raw: dict[str, Any]) -> None:
|
||||
"""Floor a negative inbound `ttlMs` to 0 before `ge=0` validation fails the call (2026-07-28 caching SHOULD)."""
|
||||
ttl = raw.get("ttlMs")
|
||||
if isinstance(ttl, int | float) and not isinstance(ttl, bool) and ttl < 0:
|
||||
raw["ttlMs"] = 0
|
||||
|
||||
|
||||
def _preconnect_stamp(data: dict[str, Any], opts: CallOptions) -> None:
|
||||
# initialize/discover forbid cancellation; other pre-handshake requests (lowlevel
|
||||
# ClientSession callers may skip the handshake entirely) keep the courtesy cancel.
|
||||
@@ -331,6 +338,7 @@ class ClientSession:
|
||||
if metadata.on_resumption_token_update is not None:
|
||||
opts["on_resumption_token"] = metadata.on_resumption_token_update
|
||||
raw = await self._dispatcher.send_raw_request(method, data.get("params"), opts)
|
||||
_clamp_inbound_ttl(raw)
|
||||
# Literal fallback covers pre-handshake and stateless; matches runner.py.
|
||||
version = self._negotiated_version or "2025-11-25"
|
||||
try:
|
||||
@@ -458,7 +466,10 @@ class ClientSession:
|
||||
"cancel_on_abandon": False,
|
||||
"headers": {MCP_PROTOCOL_VERSION_HEADER: version, MCP_METHOD_HEADER: data["method"]},
|
||||
}
|
||||
return await self._dispatcher.send_raw_request(data["method"], data.get("params"), opts)
|
||||
raw = await self._dispatcher.send_raw_request(data["method"], data.get("params"), opts)
|
||||
# Un-floored, a negative ttl fails the mode='auto' probe's validation and silently downgrades the handshake.
|
||||
_clamp_inbound_ttl(raw)
|
||||
return raw
|
||||
|
||||
async def discover(self) -> types.DiscoverResult:
|
||||
"""Probe `server/discover` and adopt the result.
|
||||
@@ -895,7 +906,15 @@ class ClientSession:
|
||||
types.ListToolsRequest(params=params),
|
||||
types.ListToolsResult,
|
||||
)
|
||||
complete = (params is None or params.cursor is None) and result.next_cursor is None
|
||||
return self._absorb_tool_listing(result, complete=complete)
|
||||
|
||||
def _absorb_tool_listing(self, result: types.ListToolsResult, *, complete: bool) -> types.ListToolsResult:
|
||||
"""Filter the listing per the 2026 x-mcp-header MUST and rebuild derived per-tool state, in place.
|
||||
|
||||
Idempotent: cached values are already post-filter, so the response cache can re-absorb a served listing.
|
||||
`complete` (an uncursored single-page listing) prunes per-tool state down to the listing's tools.
|
||||
"""
|
||||
if self._negotiated_version in MODERN_PROTOCOL_VERSIONS:
|
||||
# 2026-07-28: clients MUST drop tools whose x-mcp-header annotations are invalid.
|
||||
kept: list[types.Tool] = []
|
||||
@@ -911,11 +930,17 @@ class ClientSession:
|
||||
kept.append(tool)
|
||||
result.tools = kept
|
||||
|
||||
# Cache tool output schemas for future validation
|
||||
# Note: don't clear the cache, as we may be using a cursor
|
||||
# Cache tool output schemas for future validation; cursor pages only ever add.
|
||||
for tool in result.tools:
|
||||
self._tool_output_schemas[tool.name] = tool.output_schema
|
||||
|
||||
if complete:
|
||||
# The listing is the full tool universe, so state for unlisted tools is stale
|
||||
# (the server dropped them, or a shared-cache writer's filter did).
|
||||
names = {tool.name for tool in result.tools}
|
||||
self._x_mcp_header_maps = {k: v for k, v in self._x_mcp_header_maps.items() if k in names}
|
||||
self._tool_output_schemas = {k: v for k, v in self._tool_output_schemas.items() if k in names}
|
||||
|
||||
return result
|
||||
|
||||
@deprecated("The roots capability is deprecated as of 2026-07-28 (SEP-2577).", category=MCPDeprecationWarning)
|
||||
|
||||
@@ -11,27 +11,13 @@ from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Final, Literal, TypeVar, get_args
|
||||
from typing import Any, Literal, TypeVar
|
||||
|
||||
import mcp_types as types
|
||||
from mcp_types.methods import CACHEABLE_METHODS, CacheableMethod
|
||||
|
||||
__all__ = ["CACHEABLE_METHODS", "CacheHint", "CacheableMethod", "apply_cache_hint", "validate_cache_hints"]
|
||||
|
||||
CacheableMethod = Literal[
|
||||
"prompts/list",
|
||||
"resources/list",
|
||||
"resources/read",
|
||||
"resources/templates/list",
|
||||
"server/discover",
|
||||
"tools/list",
|
||||
]
|
||||
"""The methods whose results carry `ttlMs`/`cacheScope`. Closed set: the spec
|
||||
defines caching hints on exactly these six (tests pin it to which result models
|
||||
mix in `CacheableResult`)."""
|
||||
|
||||
CACHEABLE_METHODS: Final[frozenset[str]] = frozenset(get_args(CacheableMethod))
|
||||
"""Runtime mirror of `CacheableMethod`, for callers the type checker can't see."""
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CacheHint:
|
||||
@@ -87,7 +73,8 @@ def validate_cache_hints(cache_hints: Mapping[Any, Any] | None) -> dict[str, Cac
|
||||
"""
|
||||
if cache_hints is None:
|
||||
return {}
|
||||
unknown = sorted(method for method in cache_hints if method not in CACHEABLE_METHODS)
|
||||
# repr-format keys so a non-string key raises this ValueError, not a TypeError from sorted/join.
|
||||
unknown = sorted(repr(method) for method in cache_hints if method not in CACHEABLE_METHODS)
|
||||
if unknown:
|
||||
raise ValueError(f"cache_hints keys must be cacheable methods (see CacheableMethod); got: {', '.join(unknown)}")
|
||||
validated: dict[str, CacheHint] = {}
|
||||
|
||||
@@ -198,12 +198,15 @@ class ServerRunner(Generic[LifespanT]):
|
||||
if isinstance(result, ErrorData):
|
||||
# Raise inside the chain so middleware observes the failure.
|
||||
raise MCPError.from_error_data(result)
|
||||
# Fill cache hints on the typed result, before the serialize sieve
|
||||
# Fill cache hints on the handler result, before the serialize sieve
|
||||
# decides whether the negotiated version carries the fields at all.
|
||||
# `input_required` interim results are not `CacheableResult` models,
|
||||
# so the MRTR carve-out (no hints on them) holds by shape.
|
||||
if isinstance(result, CacheableResult) and (hint := self.server.cache_hints.get(method)) is not None:
|
||||
result = apply_cache_hint(result, hint)
|
||||
# MRTR carve-out: `input_required` interim results, typed or mapping, never get hints.
|
||||
if (hint := self.server.cache_hints.get(method)) is not None:
|
||||
if isinstance(result, CacheableResult):
|
||||
result = apply_cache_hint(result, hint)
|
||||
elif isinstance(result, Mapping) and result.get("resultType") != "input_required":
|
||||
# Hint keys first so wire keys the handler set win, matching `apply_cache_hint` precedence.
|
||||
result = {"ttlMs": hint.ttl_ms, "cacheScope": hint.scope, **result}
|
||||
# Dump and serialize inside the chain so the OpenTelemetry span (the
|
||||
# outermost middleware) records a failing handler return shape too.
|
||||
return self._serialize(method, version, result)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -506,6 +506,100 @@ async def test_modern_list_tools_drops_tools_with_invalid_x_mcp_header_but_legac
|
||||
assert [t.name for t in result.tools] == ["ok", "dropme"]
|
||||
|
||||
|
||||
_RETIRED_TOOL = Tool(
|
||||
name="retired",
|
||||
input_schema={"type": "object", "properties": {"region": {"type": "string", "x-mcp-header": "Region"}}},
|
||||
output_schema={"type": "object"},
|
||||
)
|
||||
_SURVIVOR_TOOL = Tool(name="survivor", input_schema={"type": "object"})
|
||||
|
||||
|
||||
def _scripted_listing_server(listings: list[ListToolsResult]) -> Server:
|
||||
"""Serves the given listings in order, one per tools/list request."""
|
||||
|
||||
async def on_list_tools(ctx: ServerRequestContext, params: types.PaginatedRequestParams | None) -> ListToolsResult:
|
||||
return listings.pop(0)
|
||||
|
||||
return Server("test", on_list_tools=on_list_tools)
|
||||
|
||||
|
||||
async def test_a_complete_listing_prunes_per_tool_state_for_tools_it_no_longer_contains() -> None:
|
||||
"""SDK-defined: a complete (uncursored, cursorless) listing is the full tool universe, so the
|
||||
header map and output schema derived from an earlier listing of a now-absent tool are dropped."""
|
||||
server = _scripted_listing_server(
|
||||
[
|
||||
ListToolsResult(tools=[_RETIRED_TOOL, _SURVIVOR_TOOL]),
|
||||
ListToolsResult(tools=[_SURVIVOR_TOOL]),
|
||||
]
|
||||
)
|
||||
|
||||
with anyio.fail_after(5):
|
||||
async with Client(server) as client:
|
||||
await client.session.list_tools()
|
||||
assert set(client.session._x_mcp_header_maps) == {"retired", "survivor"}
|
||||
assert set(client.session._tool_output_schemas) == {"retired", "survivor"}
|
||||
|
||||
await client.session.list_tools()
|
||||
assert set(client.session._x_mcp_header_maps) == {"survivor"}
|
||||
assert set(client.session._tool_output_schemas) == {"survivor"}
|
||||
|
||||
|
||||
async def test_a_complete_listing_prunes_output_schemas_on_a_legacy_session_too() -> None:
|
||||
"""SDK-defined: the prune is era-independent -- legacy sessions cache output schemas the same
|
||||
way (their header-map dict just stays empty, since the x-mcp-header filter is 2026-only)."""
|
||||
server = _scripted_listing_server(
|
||||
[
|
||||
ListToolsResult(tools=[_RETIRED_TOOL, _SURVIVOR_TOOL]),
|
||||
ListToolsResult(tools=[_SURVIVOR_TOOL]),
|
||||
]
|
||||
)
|
||||
|
||||
with anyio.fail_after(5):
|
||||
async with Client(server, mode="legacy") as client:
|
||||
await client.session.list_tools()
|
||||
assert set(client.session._tool_output_schemas) == {"retired", "survivor"}
|
||||
assert client.session._x_mcp_header_maps == {}
|
||||
|
||||
await client.session.list_tools()
|
||||
assert set(client.session._tool_output_schemas) == {"survivor"}
|
||||
|
||||
|
||||
async def test_a_listing_with_a_next_cursor_prunes_no_per_tool_state() -> None:
|
||||
"""SDK-defined: a first page carrying next_cursor is not the full universe -- state for tools
|
||||
expected on later pages must survive it."""
|
||||
server = _scripted_listing_server(
|
||||
[
|
||||
ListToolsResult(tools=[_RETIRED_TOOL, _SURVIVOR_TOOL]),
|
||||
ListToolsResult(tools=[_SURVIVOR_TOOL], next_cursor="2"),
|
||||
]
|
||||
)
|
||||
|
||||
with anyio.fail_after(5):
|
||||
async with Client(server) as client:
|
||||
await client.session.list_tools()
|
||||
await client.session.list_tools()
|
||||
assert set(client.session._x_mcp_header_maps) == {"retired", "survivor"}
|
||||
assert set(client.session._tool_output_schemas) == {"retired", "survivor"}
|
||||
|
||||
|
||||
async def test_a_cursor_page_fetch_prunes_no_per_tool_state() -> None:
|
||||
"""SDK-defined: a continuation page is partial even when it ends the pagination (no
|
||||
next_cursor) -- only an uncursored single-page listing prunes."""
|
||||
server = _scripted_listing_server(
|
||||
[
|
||||
ListToolsResult(tools=[_RETIRED_TOOL, _SURVIVOR_TOOL]),
|
||||
ListToolsResult(tools=[_SURVIVOR_TOOL]),
|
||||
]
|
||||
)
|
||||
|
||||
with anyio.fail_after(5):
|
||||
async with Client(server) as client:
|
||||
await client.session.list_tools()
|
||||
await client.session.list_tools(params=types.PaginatedRequestParams(cursor="2"))
|
||||
assert set(client.session._x_mcp_header_maps) == {"retired", "survivor"}
|
||||
assert set(client.session._tool_output_schemas) == {"retired", "survivor"}
|
||||
|
||||
|
||||
def test_client_rejects_handshake_era_mode_at_construction() -> None:
|
||||
"""A handshake-era protocol-version string passed as `mode=` is rejected by
|
||||
`__post_init__` with a hint to use `mode='legacy'` — the version-pin path is
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1661,6 +1661,33 @@ async def test_discover_reraises_unsupported_version_with_malformed_error_data()
|
||||
assert [m for m, _ in dispatcher.calls] == ["server/discover"]
|
||||
|
||||
|
||||
# --- inbound ttlMs clamp ---
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_positive_inbound_ttl_reaches_the_result_unchanged() -> None:
|
||||
listing: dict[str, Any] = {"resultType": "complete", "tools": [], "ttlMs": 60_000, "cacheScope": "private"}
|
||||
dispatcher = _ScriptedDispatcher(_discover_result_dict(), listing)
|
||||
with anyio.fail_after(5):
|
||||
async with ClientSession(dispatcher=dispatcher) as session:
|
||||
await session.discover()
|
||||
result = await session.list_tools()
|
||||
assert result.ttl_ms == 60_000
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
@pytest.mark.parametrize("wire_ttl", [True, False])
|
||||
async def test_a_boolean_inbound_ttl_is_not_clamped_only_coerced_by_validation(wire_ttl: bool) -> None:
|
||||
"""SDK-defined: `bool` is an `int` subclass; the clamp skips it and pydantic's lax mode coerces it instead."""
|
||||
listing: dict[str, Any] = {"resultType": "complete", "tools": [], "ttlMs": wire_ttl, "cacheScope": "private"}
|
||||
dispatcher = _ScriptedDispatcher(_discover_result_dict(), listing)
|
||||
with anyio.fail_after(5):
|
||||
async with ClientSession(dispatcher=dispatcher) as session:
|
||||
await session.discover()
|
||||
result = await session.list_tools()
|
||||
assert result.ttl_ms == int(wire_ttl)
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_session_call_tool_returns_input_required_result_when_opted_in() -> None:
|
||||
"""`ClientSession.call_tool(..., allow_input_required=True)` surfaces the
|
||||
|
||||
@@ -1,13 +1,19 @@
|
||||
"""`docs/advanced/caching.md`: every claim the page makes, proved against the real SDK."""
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import Any, cast
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
from inline_snapshot import snapshot
|
||||
from mcp_types import INTERNAL_ERROR, ListToolsResult, PaginatedRequestParams, Tool
|
||||
|
||||
from docs_src.caching import tutorial001, tutorial002, tutorial003
|
||||
from mcp import Client
|
||||
from mcp.server import CacheHint, MCPServer
|
||||
from mcp import Client, MCPError
|
||||
from mcp.client import CacheConfig
|
||||
from mcp.client.caching import InMemoryResponseCacheStore
|
||||
from mcp.server import CacheHint, MCPServer, Server, ServerRequestContext
|
||||
from mcp.server.caching import CacheableMethod
|
||||
|
||||
# See test_index.py for why this is a per-module mark and not a conftest hook.
|
||||
pytestmark = [pytest.mark.anyio, pytest.mark.filterwarnings("error::mcp.MCPDeprecationWarning")]
|
||||
@@ -42,7 +48,7 @@ async def test_a_non_cacheable_method_is_rejected_at_construction() -> None:
|
||||
with pytest.raises(ValueError) as exc:
|
||||
MCPServer("Weather", cache_hints=cast(Any, {"tools/call": CacheHint(ttl_ms=1_000)}))
|
||||
assert str(exc.value) == snapshot(
|
||||
"cache_hints keys must be cacheable methods (see CacheableMethod); got: tools/call"
|
||||
"cache_hints keys must be cacheable methods (see CacheableMethod); got: 'tools/call'"
|
||||
)
|
||||
|
||||
|
||||
@@ -55,16 +61,149 @@ async def test_the_handler_value_wins_over_the_map_per_field() -> None:
|
||||
assert tools.cache_scope == "public"
|
||||
|
||||
|
||||
async def test_the_client_program_on_the_page_reads_the_hints(capsys: pytest.CaptureFixture[str]) -> None:
|
||||
"""tutorial003: `main()` is the literal client program on the page - the hints
|
||||
arrive as parsed fields on the result."""
|
||||
async def test_the_client_program_on_the_page_makes_three_fetches_for_four_calls(
|
||||
capsys: pytest.CaptureFixture[str],
|
||||
) -> None:
|
||||
"""tutorial003: a cache hit, an expiry, and `cache_mode="refresh"` make four calls cost three fetches."""
|
||||
await tutorial003.main()
|
||||
assert capsys.readouterr().out == "1 tools, fresh for 60s, scope=public\n"
|
||||
assert capsys.readouterr().out == "4 calls, 3 fetches\n"
|
||||
|
||||
|
||||
def _counting_tools_server(*, ttl_ms: int | None = 60_000) -> tuple[Server[Any], list[str | None]]:
|
||||
"""Each tools/list fetch returns a distinct tool name, so a cache hit is
|
||||
payload-distinguishable from a refetch; `ttl_ms=None` sends no hints."""
|
||||
fetches: list[str | None] = []
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams | None) -> ListToolsResult:
|
||||
fetches.append(params.cursor if params is not None else None)
|
||||
return ListToolsResult(tools=[Tool(name=f"t{len(fetches) - 1}", input_schema={"type": "object"})])
|
||||
|
||||
hints: Mapping[CacheableMethod, CacheHint] | None = None
|
||||
if ttl_ms is not None:
|
||||
hints = {"tools/list": CacheHint(ttl_ms=ttl_ms)}
|
||||
return Server("counting", on_list_tools=list_tools, cache_hints=hints), fetches
|
||||
|
||||
|
||||
async def test_caching_is_on_by_default_the_second_call_makes_no_fetch() -> None:
|
||||
server, fetches = _counting_tools_server()
|
||||
async with Client(server) as client:
|
||||
first = await client.list_tools()
|
||||
second = await client.list_tools()
|
||||
assert fetches == [None]
|
||||
assert second == first
|
||||
|
||||
|
||||
async def test_a_hintless_result_is_not_cached_by_default() -> None:
|
||||
"""`default_ttl_ms` defaults to 0, so a hintless server sees its usual call-for-call traffic."""
|
||||
server, fetches = _counting_tools_server(ttl_ms=None)
|
||||
async with Client(server) as client:
|
||||
await client.list_tools()
|
||||
await client.list_tools()
|
||||
assert fetches == [None, None]
|
||||
|
||||
|
||||
async def test_cache_false_makes_every_call_a_round_trip() -> None:
|
||||
server, fetches = _counting_tools_server()
|
||||
async with Client(server, cache=False) as client:
|
||||
await client.list_tools()
|
||||
await client.list_tools()
|
||||
assert fetches == [None, None]
|
||||
|
||||
|
||||
async def test_refresh_refetches_and_replaces_the_cached_entry() -> None:
|
||||
server, fetches = _counting_tools_server()
|
||||
async with Client(server) as client:
|
||||
await client.list_tools()
|
||||
refreshed = await client.list_tools(cache_mode="refresh")
|
||||
served = await client.list_tools()
|
||||
assert fetches == [None, None]
|
||||
assert [tool.name for tool in refreshed.tools] == ["t1"]
|
||||
assert served == refreshed
|
||||
|
||||
|
||||
async def test_bypass_fetches_without_reading_or_writing_the_cache() -> None:
|
||||
server, fetches = _counting_tools_server()
|
||||
async with Client(server) as client:
|
||||
first = await client.list_tools()
|
||||
bypassed = await client.list_tools(cache_mode="bypass")
|
||||
served = await client.list_tools()
|
||||
assert fetches == [None, None]
|
||||
assert [tool.name for tool in bypassed.tools] == ["t1"]
|
||||
assert served == first
|
||||
|
||||
|
||||
async def test_an_expired_entry_is_not_revived_when_the_refetch_fails() -> None:
|
||||
"""SDK ruling: no stale-if-error - the refetch failure propagates."""
|
||||
now = 1_000_000.0
|
||||
fetches: list[None] = []
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams | None) -> ListToolsResult:
|
||||
fetches.append(None)
|
||||
if len(fetches) > 1:
|
||||
raise MCPError(code=INTERNAL_ERROR, message="backend down")
|
||||
return ListToolsResult(tools=[Tool(name="t0", input_schema={"type": "object"})])
|
||||
|
||||
server = Server("flaky", on_list_tools=list_tools, cache_hints={"tools/list": CacheHint(ttl_ms=60_000)})
|
||||
async with Client(server, cache=CacheConfig(clock=lambda: now)) as client:
|
||||
await client.list_tools()
|
||||
now += 60.0 # past the 60s TTL
|
||||
with pytest.raises(MCPError) as exc:
|
||||
await client.list_tools()
|
||||
assert exc.value.code == INTERNAL_ERROR
|
||||
assert len(fetches) == 2
|
||||
|
||||
|
||||
async def test_two_concurrent_identical_calls_are_two_fetches() -> None:
|
||||
"""SDK ruling: no coalescing. The handler barrier releases only once both
|
||||
calls are inside it, so the test passes only if the fetches were concurrent."""
|
||||
both_fetching = anyio.Event()
|
||||
fetches: list[None] = []
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams | None) -> ListToolsResult:
|
||||
fetches.append(None)
|
||||
if len(fetches) == 2:
|
||||
both_fetching.set()
|
||||
with anyio.fail_after(5):
|
||||
await both_fetching.wait()
|
||||
return ListToolsResult(tools=[Tool(name="t", input_schema={"type": "object"})])
|
||||
|
||||
server = Server("concurrent", on_list_tools=list_tools, cache_hints={"tools/list": CacheHint(ttl_ms=60_000)})
|
||||
async with Client(server) as client:
|
||||
async with anyio.create_task_group() as tg:
|
||||
tg.start_soon(client.list_tools)
|
||||
tg.start_soon(client.list_tools)
|
||||
assert len(fetches) == 2
|
||||
|
||||
|
||||
async def test_a_session_tier_call_always_makes_the_round_trip() -> None:
|
||||
"""The cache lives on the `Client` verbs; `client.session` sits below it."""
|
||||
server, fetches = _counting_tools_server()
|
||||
async with Client(server) as client:
|
||||
await client.list_tools()
|
||||
await client.session.list_tools()
|
||||
assert fetches == [None, None]
|
||||
|
||||
|
||||
async def test_a_custom_store_requires_a_partition() -> None:
|
||||
with pytest.raises(ValueError) as exc:
|
||||
CacheConfig(store=InMemoryResponseCacheStore())
|
||||
assert str(exc.value) == snapshot("a custom store requires an explicit partition")
|
||||
|
||||
|
||||
async def test_a_custom_store_with_an_in_process_server_requires_target_id() -> None:
|
||||
server, _ = _counting_tools_server()
|
||||
with pytest.raises(ValueError) as exc:
|
||||
Client(server, cache=CacheConfig(store=InMemoryResponseCacheStore(), partition="user-1"))
|
||||
assert str(exc.value) == snapshot(
|
||||
"a custom cache store requires CacheConfig.target_id when the server is not a URL: in-process servers "
|
||||
"and Transport instances get a random per-client identity, so their entries in a shared store could "
|
||||
"never be served to another client"
|
||||
)
|
||||
|
||||
|
||||
async def test_the_wire_presence_check_the_page_recommends_works() -> None:
|
||||
"""The page's claim: `"ttl_ms" in result.model_fields_set` distinguishes a
|
||||
server that sent the field from one that said nothing (model defaults)."""
|
||||
async with Client(tutorial003.mcp) as client:
|
||||
async with Client(tutorial001.mcp) as client:
|
||||
tools = await client.list_tools()
|
||||
assert "ttl_ms" in tools.model_fields_set
|
||||
|
||||
@@ -511,7 +511,8 @@ async def test_modern_client_stops_mirroring_after_a_re_list_drops_the_tool() ->
|
||||
bad_schema = {"type": "object", "properties": {"a": {"type": "string", "x-mcp-header": "bad name"}}}
|
||||
valid = Tool(name="run", input_schema=schema)
|
||||
invalid = Tool(name="run", input_schema=bad_schema)
|
||||
listings = iter([valid, invalid])
|
||||
# Three pages: the call after the drop re-lists once because the prune also cleared `run`'s schema entry.
|
||||
listings = iter([valid, invalid, invalid])
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext, params: PaginatedRequestParams | None) -> ListToolsResult:
|
||||
return ListToolsResult(tools=[next(listings)], ttl_ms=0, cache_scope="public")
|
||||
|
||||
@@ -1,40 +1,27 @@
|
||||
"""`mcp.server.caching`: `CacheHint` validation, per-field fills, and the
|
||||
`cache_hints` constructor map reaching the wire on both server tiers."""
|
||||
|
||||
from types import UnionType
|
||||
from typing import Any, cast, get_args
|
||||
from typing import Any, cast
|
||||
|
||||
import pytest
|
||||
from inline_snapshot import snapshot
|
||||
from mcp_types import (
|
||||
CacheableResult,
|
||||
InputRequiredResult,
|
||||
ListResourcesResult,
|
||||
ListToolsResult,
|
||||
PaginatedRequestParams,
|
||||
ReadResourceRequestParams,
|
||||
Resource,
|
||||
Tool,
|
||||
methods,
|
||||
)
|
||||
|
||||
from mcp import Client
|
||||
from mcp.server import CacheHint, MCPServer, Server, ServerRequestContext
|
||||
from mcp.server.caching import CACHEABLE_METHODS, apply_cache_hint
|
||||
from mcp.server.caching import apply_cache_hint
|
||||
|
||||
pytestmark = pytest.mark.anyio
|
||||
|
||||
|
||||
def test_cacheable_methods_match_the_result_models() -> None:
|
||||
"""Spec-mandated set (SEP-2549): `CACHEABLE_METHODS` mirrors exactly the
|
||||
methods whose monolith result models mix in `CacheableResult` - if the
|
||||
schema gains or loses a cacheable result, this weld breaks."""
|
||||
derived: set[str] = set()
|
||||
for method, model in methods.MONOLITH_RESULTS.items():
|
||||
arms = get_args(model) if isinstance(model, UnionType) else (model,)
|
||||
if any(isinstance(arm, type) and issubclass(arm, CacheableResult) for arm in arms):
|
||||
derived.add(method)
|
||||
assert CACHEABLE_METHODS == derived
|
||||
|
||||
|
||||
def test_cache_hint_defaults_match_the_conservative_model_defaults() -> None:
|
||||
"""SDK-defined: an unconfigured hint fills the same values the result models
|
||||
already default to - immediately stale, not shared - so stamping it is
|
||||
@@ -83,7 +70,7 @@ def test_a_non_cacheable_method_in_cache_hints_is_rejected_at_server_constructio
|
||||
with pytest.raises(ValueError) as exc:
|
||||
Server("srv", cache_hints=cast(Any, {"tools/call": CacheHint()}))
|
||||
assert str(exc.value) == snapshot(
|
||||
"cache_hints keys must be cacheable methods (see CacheableMethod); got: tools/call"
|
||||
"cache_hints keys must be cacheable methods (see CacheableMethod); got: 'tools/call'"
|
||||
)
|
||||
|
||||
|
||||
@@ -96,6 +83,72 @@ def test_a_non_cache_hint_value_is_rejected_at_server_construction() -> None:
|
||||
assert str(exc.value) == snapshot("cache_hints['tools/list'] must be a CacheHint, got dict")
|
||||
|
||||
|
||||
def test_a_non_string_cache_hints_key_is_rejected_with_the_unknown_key_error() -> None:
|
||||
"""A non-string key takes the same unknown-key ValueError as a typo, not a TypeError from message formatting."""
|
||||
with pytest.raises(ValueError) as exc:
|
||||
Server("srv", cache_hints=cast(Any, {42: CacheHint()}))
|
||||
assert str(exc.value) == snapshot("cache_hints keys must be cacheable methods (see CacheableMethod); got: 42")
|
||||
|
||||
|
||||
async def test_a_dict_returning_handler_takes_the_configured_hint() -> None:
|
||||
"""The stamp covers raw-dict results too - 2026-07-28 requires both fields on the wire."""
|
||||
hint = CacheHint(ttl_ms=60_000, scope="public")
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams) -> dict[str, Any]:
|
||||
return {"tools": [], "resultType": "complete"}
|
||||
|
||||
server = Server("srv", cache_hints={"tools/list": hint})
|
||||
server.add_request_handler("tools/list", PaginatedRequestParams, list_tools)
|
||||
async with Client(server) as client:
|
||||
result = await client.list_tools()
|
||||
assert result.ttl_ms == hint.ttl_ms
|
||||
assert result.cache_scope == hint.scope
|
||||
|
||||
|
||||
async def test_a_dict_provided_ttl_wins_and_the_hint_fills_only_the_missing_scope() -> None:
|
||||
"""Dict path mirrors the model path's `model_fields_set` precedence: present wire keys win."""
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams) -> dict[str, Any]:
|
||||
return {"tools": [], "resultType": "complete", "ttlMs": 25}
|
||||
|
||||
server = Server("srv", cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")})
|
||||
server.add_request_handler("tools/list", PaginatedRequestParams, list_tools)
|
||||
async with Client(server) as client:
|
||||
result = await client.list_tools()
|
||||
assert result.ttl_ms == 25
|
||||
assert result.cache_scope == "public"
|
||||
|
||||
|
||||
async def test_a_dict_returning_handler_leaks_no_hint_fields_to_a_2025_session() -> None:
|
||||
"""The stamp runs version-independently; the 2025 serialize sieve strips the fields."""
|
||||
|
||||
async def list_tools(ctx: ServerRequestContext[Any], params: PaginatedRequestParams) -> dict[str, Any]:
|
||||
return {"tools": []}
|
||||
|
||||
server = Server("srv", cache_hints={"tools/list": CacheHint(ttl_ms=60_000, scope="public")})
|
||||
server.add_request_handler("tools/list", PaginatedRequestParams, list_tools)
|
||||
async with Client(server, mode="legacy") as client:
|
||||
result = await client.list_tools()
|
||||
assert "ttl_ms" not in result.model_fields_set
|
||||
assert "cache_scope" not in result.model_fields_set
|
||||
|
||||
|
||||
async def test_an_input_required_shaped_dict_is_never_stamped() -> None:
|
||||
"""Spec carve-out: interim `input_required` results carry no cache hints, even on a hinted method."""
|
||||
|
||||
async def read_resource(ctx: ServerRequestContext[Any], params: ReadResourceRequestParams) -> dict[str, Any]:
|
||||
return {"resultType": "input_required", "requestState": "s1"}
|
||||
|
||||
server = Server("srv", cache_hints={"resources/read": CacheHint(ttl_ms=60_000, scope="public")})
|
||||
server.add_request_handler("resources/read", ReadResourceRequestParams, read_resource)
|
||||
async with Client(server) as client:
|
||||
result = await client.session.read_resource("res://x", allow_input_required=True)
|
||||
assert isinstance(result, InputRequiredResult)
|
||||
assert result.model_dump(by_alias=True, exclude_none=True) == snapshot(
|
||||
{"resultType": "input_required", "requestState": "s1"}
|
||||
)
|
||||
|
||||
|
||||
async def test_server_cache_hints_reach_the_wire_for_a_bare_handler_result() -> None:
|
||||
"""SDK-defined: a lowlevel handler that never thinks about caching emits the
|
||||
server-wide hint configured at construction."""
|
||||
|
||||
@@ -548,6 +548,11 @@ def test_built_in_maps_are_immutable():
|
||||
_assign_item(built_in)
|
||||
|
||||
|
||||
def test_cacheable_methods_mirror_the_cacheable_method_literal():
|
||||
"""SEP-2549 weld: the hand-written Literal and the set derived from `MONOLITH_RESULTS` must agree."""
|
||||
assert methods.CACHEABLE_METHODS == frozenset(get_args(methods.CacheableMethod))
|
||||
|
||||
|
||||
def test_minimal_request_bodies_parse_through_every_request_row():
|
||||
for (method, version), surface_type in methods.CLIENT_REQUESTS.items():
|
||||
parsed = methods.parse_client_request(method, version, REQUEST_PARAMS_FIXTURES[surface_type])
|
||||
|
||||
Reference in New Issue
Block a user