Max Isbey
ab24c34d89
- pre_parse_json leaves a string alone when json.loads refuses it with something other than JSONDecodeError (over-long integer, deep nesting), so validation rejects it as a bad argument instead of it surfacing as a crash with a traceback per request. - convert_result skips output-schema validation for a returned CallToolResult(is_error=True); an error result has no structured content to check, and the author's message now reaches the client as written. - read_resource checks that Resource.read() returned str or bytes, so a mistyped custom resource is logged as a crash and answered with -32603 rather than "Invalid request parameters" with no log record. - Docs and examples that still said "raise any exception and the model reads it" now say ToolError; deprecated.md lists the deprecated FuncMetadata helper; docstrings spell out the MCPError carve-out and the nested-crash __cause__. - Two tests tightened: the prompt argument-validation test proves the body never ran, and the invalid-types check asserts on validate_arguments.
110 lines
4.1 KiB
Markdown
110 lines
4.1 KiB
Markdown
# Testing
|
|
|
|
The Python SDK ships a `Client` class with an **in-memory transport**: pass it your server object and it connects to it directly.
|
|
|
|
No subprocess. No port. No transport at all. It's the same idea as FastAPI's `TestClient`.
|
|
|
|
## Basic usage
|
|
|
|
Let's assume you have a simple server with a single tool:
|
|
|
|
```python title="server.py"
|
|
--8<-- "docs_src/testing/tutorial001.py"
|
|
```
|
|
|
|
To run the test below you'll need two extra (development) dependencies:
|
|
|
|
=== "uv"
|
|
|
|
```bash
|
|
uv add --dev pytest inline-snapshot
|
|
```
|
|
|
|
=== "pip"
|
|
|
|
```bash
|
|
pip install pytest inline-snapshot
|
|
```
|
|
|
|
!!! info
|
|
These docs assume you already know [`pytest`](https://docs.pytest.org/en/stable/).
|
|
|
|
[`inline-snapshot`](https://15r10nk.github.io/inline-snapshot/latest/) is what the test below
|
|
uses to assert on the whole result object in one line. It records the output of a test as the
|
|
`snapshot(...)` literal you see. If you'd rather not use it, drop the import and assert on the
|
|
fields you care about (`result.content[0].text == "3"`) like in any other test.
|
|
|
|
Now the test:
|
|
|
|
```python title="test_server.py"
|
|
import pytest
|
|
from inline_snapshot import snapshot
|
|
from mcp import Client
|
|
from mcp.types import CallToolResult, TextContent
|
|
|
|
from server import mcp
|
|
|
|
|
|
@pytest.fixture
|
|
def anyio_backend(): # (1)!
|
|
return "asyncio"
|
|
|
|
|
|
@pytest.fixture
|
|
async def client(): # (2)!
|
|
async with Client(mcp, raise_exceptions=True) as c:
|
|
yield c
|
|
|
|
|
|
@pytest.mark.anyio
|
|
async def test_call_add_tool(client: Client):
|
|
result = await client.call_tool("add", {"a": 1, "b": 2})
|
|
# Drop the server identity stamp in `_meta`; it is not what this test is about.
|
|
result.meta = None
|
|
assert result == snapshot(
|
|
CallToolResult(
|
|
content=[TextContent(type="text", text="3")],
|
|
structured_content={"result": 3},
|
|
)
|
|
)
|
|
```
|
|
|
|
1. If you are using `trio`, return `"trio"` instead. See the [anyio documentation](https://anyio.readthedocs.io/en/stable/testing.html#specifying-the-backends-to-run-on) for the details.
|
|
2. The fixture yields a connected client. Every test that takes `client` gets a fresh in-memory connection to the same server.
|
|
|
|
There you go! You can now extend your tests to cover more scenarios.
|
|
|
|
## Why `raise_exceptions=True`?
|
|
|
|
Two different things can go wrong, and this flag only touches one of them.
|
|
|
|
An exception inside one of **your tools** is not a protocol failure. It becomes a normal result with
|
|
`is_error=True` (and if it was a `ToolError`, the model reads your message). `raise_exceptions` doesn't
|
|
change that: with or without it, `call_tool` returns the same `is_error=True` result. There's a whole page on it:
|
|
**[Handling errors](../servers/handling-errors.md)**.
|
|
|
|
A failure **outside** a tool body is different. On the connection `Client(mcp)` gives you, the
|
|
server sanitises it into a generic `"Internal server error"` before the client sees it. You should
|
|
never leak the details of an unexpected crash to a remote caller. In a test that is exactly what
|
|
you *don't* want, and it is what `raise_exceptions=True` changes: your test sees the real message
|
|
instead of the sanitised one.
|
|
|
|
Leave it on in tests. It has no meaning in production code.
|
|
|
|
## In-process by default
|
|
|
|
!!! note
|
|
`Client(mcp)` connects in-process and is **era-neutral** by default: it probes the server and
|
|
picks the appropriate protocol path. Pin `mode="legacy"` if your test exercises legacy-specific
|
|
semantics (sampling or elicitation push, `message_handler`), and drop `raise_exceptions=True`
|
|
there: a legacy connection never sanitises in the first place, and the flag re-raises the
|
|
failure inside the server task instead of in your test.
|
|
|
|
That one line is also why these docs can promise you that their examples work: every
|
|
example file is exercised by the SDK's own test suite, almost all of them through exactly this
|
|
client. You're using the same tool the SDK uses on itself.
|
|
|
|
You have a working, tested server. Putting it inside a real application (Claude Desktop, an
|
|
IDE) is **[Connect to a real host](real-host.md)**; every other way to serve it is
|
|
**[Running your server](../run/index.md)**.
|