Fix stdio client shutdown bugs and rebuild the stdio test suite (#2773)
This commit is contained in:
Max
@@ -105,6 +105,46 @@ The `headers`, `timeout`, `sse_read_timeout`, and `auth` parameters have been re
|
||||
|
||||
Note: `sse_client` retains its `headers`, `timeout`, `sse_read_timeout`, and `auth` parameters — only the streamable HTTP transport changed.
|
||||
|
||||
### `terminate_windows_process` removed
|
||||
|
||||
The deprecated `mcp.os.win32.utilities.terminate_windows_process` function has been
|
||||
removed. Process termination is handled internally by the `stdio_client` context
|
||||
manager; there is no replacement API. The Windows tree-termination helper
|
||||
`terminate_windows_process_tree` no longer accepts a `timeout_seconds` argument —
|
||||
the value was never used (Job Object termination is immediate).
|
||||
|
||||
### `stdio_client` no longer kills children of a gracefully-exited server on POSIX
|
||||
|
||||
When a server exits on its own after `stdio_client` closes its stdin, background
|
||||
child processes the server leaves behind are no longer killed on POSIX — their
|
||||
lifetime is the server's business. The old behavior was a side effect of a shutdown
|
||||
wait gated on the stdio pipes closing rather than on process exit: a child holding
|
||||
an inherited pipe made a well-behaved server look hung, so its whole process tree
|
||||
was killed. (That gating is an asyncio behavior specific to Python 3.11+ — on
|
||||
Python 3.10 and the trio backend the old wait already resolved on process exit, so
|
||||
the spurious kill never fired there.) A server that does not exit within the grace
|
||||
period is still terminated
|
||||
along with its entire process group. On Windows, children stay in the server's Job
|
||||
Object and are still killed at shutdown — now deterministically when the job handle
|
||||
is closed, rather than whenever the handle happened to be garbage-collected.
|
||||
|
||||
If you relied on `stdio_client` killing everything the server spawned, make the
|
||||
server terminate its own children on shutdown (its stdin reaching EOF is the
|
||||
shutdown signal), or clean up the process tree from the host application after
|
||||
`stdio_client` exits.
|
||||
|
||||
Two related shutdown refinements: `stdio_client` now closes its end of the pipes
|
||||
deterministically at shutdown, so a surviving child that keeps writing to an
|
||||
inherited stdout receives `EPIPE`/`SIGPIPE` once the client is gone (previously the
|
||||
pipe lingered until garbage collection); and a failed write to a server that is
|
||||
still running now surfaces as a closed connection (`CONNECTION_CLOSED`) on the read
|
||||
side instead of leaving requests waiting indefinitely.
|
||||
|
||||
`terminate_posix_process_tree` now requires the process to lead its own process
|
||||
group (spawned with `start_new_session=True`); the `getpgid()` lookup and the
|
||||
per-process terminate/kill fallback are gone. The win32 utilities logger is now
|
||||
named `mcp.os.win32.utilities` (was `client.stdio.win32`).
|
||||
|
||||
### Removed type aliases and classes
|
||||
|
||||
The following deprecated type aliases and classes have been removed from `mcp.types`:
|
||||
|
||||
+224
-140
@@ -1,21 +1,33 @@
|
||||
"""stdio client transport.
|
||||
|
||||
Runs an MCP server as a subprocess and exchanges newline-delimited JSON-RPC
|
||||
messages with it over stdin/stdout. Two pipe tasks bridge the server's pipes
|
||||
to the session's in-memory streams; shutdown follows the MCP spec sequence
|
||||
(close stdin, wait, then kill the process tree) inside a cancellation shield
|
||||
with every wait bounded, so a cancelled caller can neither leak a live server
|
||||
process nor hang on one.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from contextlib import asynccontextmanager
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager, suppress
|
||||
from pathlib import Path
|
||||
from typing import Literal, TextIO
|
||||
|
||||
import anyio
|
||||
import anyio.lowlevel
|
||||
from anyio.abc import Process
|
||||
from anyio.streams.memory import MemoryObjectReceiveStream, MemoryObjectSendStream
|
||||
from anyio.abc import AsyncResource, Process
|
||||
from anyio.streams.text import TextReceiveStream
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from mcp import types
|
||||
from mcp.client._transport import TransportStreams
|
||||
from mcp.os.posix.utilities import terminate_posix_process_tree
|
||||
from mcp.os.win32.utilities import (
|
||||
FallbackProcess,
|
||||
ServerProcess,
|
||||
close_process_job,
|
||||
create_windows_process,
|
||||
get_windows_executable_command,
|
||||
terminate_windows_process_tree,
|
||||
@@ -44,14 +56,24 @@ DEFAULT_INHERITED_ENV_VARS = (
|
||||
else ["HOME", "LOGNAME", "PATH", "SHELL", "TERM", "USER"]
|
||||
)
|
||||
|
||||
# Timeout for process termination before falling back to force kill
|
||||
# Grace period for the server to exit on its own after its stdin closes.
|
||||
PROCESS_TERMINATION_TIMEOUT = 2.0
|
||||
|
||||
# Extra time after SIGTERM before SIGKILL; POSIX only (Windows kills hard).
|
||||
FORCE_KILL_TIMEOUT = 2.0
|
||||
|
||||
# Time for the event loop to observe a kill; only an unkillable process runs this out.
|
||||
_KILL_REAP_TIMEOUT = 2.0
|
||||
|
||||
# Time for the writer to flush accepted messages before stdin closes.
|
||||
_WRITER_FLUSH_TIMEOUT = 0.5
|
||||
|
||||
# How often to poll returncode while waiting for the process to die.
|
||||
_EXIT_POLL_INTERVAL = 0.01
|
||||
|
||||
|
||||
def get_default_environment() -> dict[str, str]:
|
||||
"""Returns a default environment object including only environment variables deemed
|
||||
safe to inherit.
|
||||
"""
|
||||
"""Returns only the environment variables that are safe to inherit."""
|
||||
env: dict[str, str] = {}
|
||||
|
||||
for key in DEFAULT_INHERITED_ENV_VARS:
|
||||
@@ -76,150 +98,227 @@ class StdioServerParameters(BaseModel):
|
||||
"""Command line arguments to pass to the executable."""
|
||||
|
||||
env: dict[str, str] | None = None
|
||||
"""
|
||||
The environment to use when spawning the process.
|
||||
|
||||
If not specified, the result of get_default_environment() will be used.
|
||||
"""
|
||||
"""Extra environment variables, merged over get_default_environment()."""
|
||||
|
||||
cwd: str | Path | None = None
|
||||
"""The working directory to use when spawning the process."""
|
||||
|
||||
encoding: str = "utf-8"
|
||||
"""
|
||||
The text encoding used when sending/receiving messages to the server.
|
||||
|
||||
Defaults to utf-8.
|
||||
"""
|
||||
"""Text encoding for messages to and from the server."""
|
||||
|
||||
encoding_error_handler: Literal["strict", "ignore", "replace"] = "strict"
|
||||
"""
|
||||
The text encoding error handler.
|
||||
|
||||
See https://docs.python.org/3/library/codecs.html#codec-base-classes for
|
||||
explanations of possible values.
|
||||
"""
|
||||
"""Encoding error handler; see https://docs.python.org/3/library/codecs.html#error-handlers."""
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def stdio_client(server: StdioServerParameters, errlog: TextIO = sys.stderr):
|
||||
"""Client transport for stdio: this will connect to a server by spawning a
|
||||
process and communicating with it over stdin/stdout.
|
||||
async def stdio_client(
|
||||
server: StdioServerParameters, errlog: TextIO = sys.stderr
|
||||
) -> AsyncGenerator[TransportStreams, None]:
|
||||
"""Spawns an MCP server subprocess and connects to it over stdin/stdout.
|
||||
|
||||
Raises:
|
||||
OSError: If the server process cannot be spawned.
|
||||
ValueError: If the spawn parameters are invalid (embedded NUL bytes).
|
||||
"""
|
||||
read_stream: MemoryObjectReceiveStream[SessionMessage | Exception]
|
||||
read_stream_writer: MemoryObjectSendStream[SessionMessage | Exception]
|
||||
command = _get_executable_command(server.command)
|
||||
|
||||
write_stream: MemoryObjectSendStream[SessionMessage]
|
||||
write_stream_reader: MemoryObjectReceiveStream[SessionMessage]
|
||||
process = await _create_platform_compatible_process(
|
||||
command=command,
|
||||
args=server.args,
|
||||
env=get_default_environment() | (server.env or {}),
|
||||
errlog=errlog,
|
||||
cwd=server.cwd,
|
||||
)
|
||||
|
||||
read_stream_writer, read_stream = anyio.create_memory_object_stream(0)
|
||||
write_stream, write_stream_reader = anyio.create_memory_object_stream(0)
|
||||
# The spawn succeeded; no awaits until the task group is entered, or a
|
||||
# cancellation delivered in the gap would leak the live process.
|
||||
read_stream_writer, read_stream = anyio.create_memory_object_stream[SessionMessage | Exception](0)
|
||||
write_stream, write_stream_reader = anyio.create_memory_object_stream[SessionMessage](0)
|
||||
|
||||
try:
|
||||
command = _get_executable_command(server.command)
|
||||
shutting_down = False
|
||||
writer_done = anyio.Event()
|
||||
|
||||
# Open process with stderr piped for capture
|
||||
process = await _create_platform_compatible_process(
|
||||
command=command,
|
||||
args=server.args,
|
||||
env=({**get_default_environment(), **server.env} if server.env is not None else get_default_environment()),
|
||||
errlog=errlog,
|
||||
cwd=server.cwd,
|
||||
)
|
||||
except OSError:
|
||||
# Clean up streams if process creation fails
|
||||
await read_stream.aclose()
|
||||
await write_stream.aclose()
|
||||
await read_stream_writer.aclose()
|
||||
await write_stream_reader.aclose()
|
||||
raise
|
||||
|
||||
async def stdout_reader():
|
||||
async def stdout_reader() -> None:
|
||||
assert process.stdout, "Opened process is missing stdout"
|
||||
|
||||
stdout = TextReceiveStream(process.stdout, encoding=server.encoding, errors=server.encoding_error_handler)
|
||||
try:
|
||||
async with read_stream_writer:
|
||||
buffer = ""
|
||||
async for chunk in TextReceiveStream(
|
||||
process.stdout,
|
||||
encoding=server.encoding,
|
||||
errors=server.encoding_error_handler,
|
||||
):
|
||||
lines = (buffer + chunk).split("\n")
|
||||
buffer = lines.pop()
|
||||
try:
|
||||
# One line at a time; no read-ahead while a delivery is blocked.
|
||||
buffer = ""
|
||||
async for chunk in stdout:
|
||||
lines = (buffer + chunk).split("\n")
|
||||
buffer = lines.pop()
|
||||
for line in lines:
|
||||
try:
|
||||
await read_stream_writer.send(_parse_line(line))
|
||||
except (anyio.ClosedResourceError, anyio.BrokenResourceError):
|
||||
return # the session is gone; only the drain below remains
|
||||
finally:
|
||||
await _drain_stdout(process)
|
||||
except anyio.ClosedResourceError:
|
||||
pass # our own shutdown closed the stdout stream under the read
|
||||
except (anyio.BrokenResourceError, ConnectionError):
|
||||
# Teardown noise during shutdown, a real failure otherwise; either way
|
||||
# the session sees clean closure when the read stream closes.
|
||||
if not shutting_down:
|
||||
logger.exception("Reading from the MCP server's stdout failed mid-session")
|
||||
|
||||
for line in lines:
|
||||
try:
|
||||
message = types.jsonrpc_message_adapter.validate_json(line, by_name=False)
|
||||
except Exception as exc: # pragma: no cover
|
||||
logger.exception("Failed to parse JSONRPC message from server")
|
||||
await read_stream_writer.send(exc)
|
||||
continue
|
||||
|
||||
session_message = SessionMessage(message)
|
||||
await read_stream_writer.send(session_message)
|
||||
except anyio.ClosedResourceError: # pragma: lax no cover
|
||||
await anyio.lowlevel.checkpoint()
|
||||
|
||||
async def stdin_writer():
|
||||
async def stdin_writer() -> None:
|
||||
assert process.stdin, "Opened process is missing stdin"
|
||||
|
||||
try:
|
||||
async with write_stream_reader:
|
||||
async for session_message in write_stream_reader:
|
||||
json = session_message.message.model_dump_json(by_alias=True, exclude_unset=True)
|
||||
await process.stdin.send(
|
||||
(json + "\n").encode(
|
||||
encoding=server.encoding,
|
||||
errors=server.encoding_error_handler,
|
||||
)
|
||||
)
|
||||
except anyio.ClosedResourceError: # pragma: no cover
|
||||
await anyio.lowlevel.checkpoint()
|
||||
data = (json + "\n").encode(encoding=server.encoding, errors=server.encoding_error_handler)
|
||||
await process.stdin.send(data)
|
||||
except (anyio.ClosedResourceError, anyio.BrokenResourceError, OSError):
|
||||
# The server may still be alive: close the read stream so the session
|
||||
# sees the connection end instead of a request hanging forever.
|
||||
await read_stream_writer.aclose()
|
||||
finally:
|
||||
writer_done.set()
|
||||
|
||||
async with anyio.create_task_group() as tg, process:
|
||||
async def shutdown() -> None:
|
||||
"""Winds the transport down: stop traffic, flush, stop the server, release the streams."""
|
||||
# Unblock the reader into its drain: a server stuck writing stdout cannot
|
||||
# read its stdin, so draining is what lets the flush below complete.
|
||||
read_stream.close()
|
||||
# Bounded window for the writer to flush already-accepted messages.
|
||||
write_stream.close()
|
||||
with anyio.move_on_after(_WRITER_FLUSH_TIMEOUT) as flush_scope:
|
||||
await writer_done.wait()
|
||||
if flush_scope.cancelled_caught:
|
||||
await anyio.lowlevel.cancel_shielded_checkpoint() # resync coverage on 3.11 (gh-106749)
|
||||
await _stop_server_process(process)
|
||||
await _aclose_all(read_stream, write_stream, read_stream_writer, write_stream_reader)
|
||||
# One pass so unblocked tasks exit via their except paths before the cancel.
|
||||
await anyio.lowlevel.checkpoint()
|
||||
|
||||
async with anyio.create_task_group() as tg:
|
||||
tg.start_soon(stdout_reader)
|
||||
tg.start_soon(stdin_writer)
|
||||
try:
|
||||
yield read_stream, write_stream
|
||||
finally:
|
||||
# MCP spec: stdio shutdown sequence
|
||||
# 1. Close input stream to server
|
||||
# 2. Wait for server to exit, or send SIGTERM if it doesn't exit in time
|
||||
# 3. Send SIGKILL if still not exited
|
||||
if process.stdin: # pragma: no branch
|
||||
try:
|
||||
await process.stdin.aclose()
|
||||
except Exception: # pragma: no cover
|
||||
# stdin might already be closed, which is fine
|
||||
pass
|
||||
shutting_down = True
|
||||
# Shutdown must finish even under caller cancellation, or the server
|
||||
# process would leak; every wait inside is bounded. (Native
|
||||
# task.cancel() and the fallback's worker threads can still defeat it.)
|
||||
with anyio.CancelScope(shield=True):
|
||||
await shutdown()
|
||||
# Unstick pipe tasks a kill survivor's open pipe end could still block.
|
||||
tg.cancel_scope.cancel()
|
||||
# The cancel lands via throw(); one yield resyncs 3.11 coverage (gh-106749).
|
||||
await anyio.lowlevel.cancel_shielded_checkpoint()
|
||||
|
||||
try:
|
||||
# Give the process time to exit gracefully after stdin closes
|
||||
with anyio.fail_after(PROCESS_TERMINATION_TIMEOUT):
|
||||
await process.wait()
|
||||
except TimeoutError:
|
||||
# Process didn't exit from stdin closure, use platform-specific termination
|
||||
# which handles SIGTERM -> SIGKILL escalation
|
||||
await _terminate_process_tree(process)
|
||||
except ProcessLookupError: # pragma: no cover
|
||||
# Process already exited, which is fine
|
||||
pass
|
||||
await read_stream.aclose()
|
||||
await write_stream.aclose()
|
||||
await read_stream_writer.aclose()
|
||||
await write_stream_reader.aclose()
|
||||
|
||||
def _parse_line(line: str) -> SessionMessage | Exception:
|
||||
"""Parses one stdout line, returning parse errors as values for the session to surface."""
|
||||
try:
|
||||
message = types.jsonrpc_message_adapter.validate_json(line, by_name=False)
|
||||
except ValueError as exc:
|
||||
logger.exception("Failed to parse JSONRPC message from server")
|
||||
return exc
|
||||
return SessionMessage(message)
|
||||
|
||||
|
||||
async def _drain_stdout(process: ServerProcess) -> None:
|
||||
"""Consumes and discards the server's remaining stdout.
|
||||
|
||||
Keeps a server flushing buffered output from blocking on a full pipe and
|
||||
missing its chance to exit; shielded, raw bytes, ends when shutdown closes
|
||||
the pipe.
|
||||
"""
|
||||
assert process.stdout
|
||||
with anyio.CancelScope(shield=True):
|
||||
with suppress(
|
||||
anyio.EndOfStream,
|
||||
anyio.ClosedResourceError,
|
||||
anyio.BrokenResourceError,
|
||||
ConnectionError,
|
||||
OSError,
|
||||
):
|
||||
while True:
|
||||
await process.stdout.receive()
|
||||
|
||||
|
||||
async def _stop_server_process(process: ServerProcess) -> None:
|
||||
"""Closes stdin, waits out the grace period, then kills the whole tree.
|
||||
|
||||
The escalation order is spec text; timeouts and tree-wide scope are SDK policy:
|
||||
https://modelcontextprotocol.io/specification/2025-11-25/basic/lifecycle#shutdown
|
||||
"""
|
||||
assert process.stdin and process.stdout, "server process is spawned with pipes"
|
||||
|
||||
await _close_pipe(process.stdin)
|
||||
if not await _wait_for_process_exit(process, PROCESS_TERMINATION_TIMEOUT):
|
||||
await _terminate_process_tree(process)
|
||||
# Until the event loop observes the death, the transport cannot close.
|
||||
if not await _wait_for_process_exit(process, _KILL_REAP_TIMEOUT):
|
||||
logger.warning("MCP server process %d is still alive after the kill escalation; abandoning it", process.pid)
|
||||
|
||||
# Reaps surviving Windows job members now, not at GC; no-op on POSIX.
|
||||
close_process_job(process)
|
||||
# A kill survivor can hold the stdout pipe open; poison the reader anyway.
|
||||
await _close_pipe(process.stdout)
|
||||
_close_subprocess_transport(process)
|
||||
|
||||
|
||||
async def _close_pipe(stream: AsyncResource) -> None:
|
||||
"""Closes a pipe stream, tolerating one already closed, broken, or contended."""
|
||||
with suppress(OSError, anyio.BrokenResourceError, anyio.ClosedResourceError):
|
||||
await stream.aclose()
|
||||
|
||||
|
||||
async def _wait_for_process_exit(process: ServerProcess, timeout: float) -> bool:
|
||||
"""Returns whether the process died within the timeout, by polling returncode.
|
||||
|
||||
Not process.wait(): on asyncio 3.11+ it also waits for pipe EOF, and a
|
||||
child that inherited the pipes makes an exited server look hung.
|
||||
"""
|
||||
deadline = anyio.current_time() + timeout
|
||||
while process.returncode is None:
|
||||
if anyio.current_time() >= deadline:
|
||||
return False
|
||||
await anyio.sleep(_EXIT_POLL_INTERVAL)
|
||||
return True
|
||||
|
||||
|
||||
async def _terminate_process_tree(process: ServerProcess) -> None:
|
||||
"""Kills the process and all its descendants.
|
||||
|
||||
POSIX: SIGTERM to the process group, SIGKILL after FORCE_KILL_TIMEOUT.
|
||||
Windows: immediate Job Object termination (already a hard kill).
|
||||
"""
|
||||
if sys.platform == "win32": # pragma: no cover
|
||||
await terminate_windows_process_tree(process)
|
||||
else: # pragma: lax no cover
|
||||
# The Windows-only FallbackProcess never reaches the POSIX path.
|
||||
assert isinstance(process, Process)
|
||||
await terminate_posix_process_tree(process, FORCE_KILL_TIMEOUT)
|
||||
|
||||
|
||||
def _close_subprocess_transport(process: ServerProcess) -> None:
|
||||
"""Closes the asyncio subprocess transport, if there is one.
|
||||
|
||||
The transport otherwise stays open (and warns at GC) while a surviving
|
||||
descendant holds a pipe end; nothing public exposes it, hence the attribute
|
||||
walk. No-op on trio and the Windows fallback.
|
||||
"""
|
||||
transport = getattr(getattr(process, "_process", None), "_transport", None)
|
||||
# Duck-typed: uvloop's UVProcessTransport is not an asyncio.SubprocessTransport.
|
||||
close = getattr(transport, "close", None)
|
||||
if callable(close):
|
||||
# close() on <=3.12 can raise PermissionError re-killing a setuid child.
|
||||
with suppress(PermissionError):
|
||||
close()
|
||||
|
||||
|
||||
def _get_executable_command(command: str) -> str:
|
||||
"""Get the correct executable command normalized for the current platform.
|
||||
|
||||
Args:
|
||||
command: Base command (e.g., 'uvx', 'npx')
|
||||
|
||||
Returns:
|
||||
str: Platform-appropriate command
|
||||
"""
|
||||
"""Normalizes the command for the current platform."""
|
||||
if sys.platform == "win32": # pragma: no cover
|
||||
return get_windows_executable_command(command)
|
||||
else: # pragma: lax no cover
|
||||
@@ -232,16 +331,15 @@ async def _create_platform_compatible_process(
|
||||
env: dict[str, str] | None = None,
|
||||
errlog: TextIO = sys.stderr,
|
||||
cwd: Path | str | None = None,
|
||||
):
|
||||
"""Creates a subprocess in a platform-compatible way.
|
||||
) -> ServerProcess:
|
||||
"""Spawns the server in its own kill scope.
|
||||
|
||||
Unix: Creates process in a new session/process group for killpg support
|
||||
Windows: Creates process in a Job Object for reliable child termination
|
||||
A new session/process group on POSIX, a Job Object on Windows.
|
||||
"""
|
||||
if sys.platform == "win32": # pragma: no cover
|
||||
process = await create_windows_process(command, args, env, errlog, cwd)
|
||||
return await create_windows_process(command, args, env, errlog, cwd)
|
||||
else: # pragma: lax no cover
|
||||
process = await anyio.open_process(
|
||||
return await anyio.open_process(
|
||||
[command, *args],
|
||||
env=env,
|
||||
stderr=errlog,
|
||||
@@ -249,22 +347,8 @@ async def _create_platform_compatible_process(
|
||||
start_new_session=True,
|
||||
)
|
||||
|
||||
return process
|
||||
|
||||
|
||||
async def _terminate_process_tree(process: Process | FallbackProcess, timeout_seconds: float = 2.0) -> None:
|
||||
"""Terminate a process and all its children using platform-specific methods.
|
||||
|
||||
Unix: Uses os.killpg() for atomic process group termination
|
||||
Windows: Uses Job Objects via pywin32 for reliable child process cleanup
|
||||
|
||||
Args:
|
||||
process: The process to terminate
|
||||
timeout_seconds: Timeout in seconds before force killing (default: 2.0)
|
||||
"""
|
||||
if sys.platform == "win32": # pragma: no cover
|
||||
await terminate_windows_process_tree(process, timeout_seconds)
|
||||
else: # pragma: lax no cover
|
||||
# FallbackProcess should only be used for Windows compatibility
|
||||
assert isinstance(process, Process)
|
||||
await terminate_posix_process_tree(process, timeout_seconds)
|
||||
async def _aclose_all(*streams: AsyncResource) -> None:
|
||||
"""Closes every given stream."""
|
||||
for stream in streams:
|
||||
await stream.aclose()
|
||||
|
||||
@@ -3,55 +3,61 @@
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
from contextlib import suppress
|
||||
|
||||
import anyio
|
||||
from anyio.abc import Process
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# How often to probe for surviving group members between SIGTERM and SIGKILL.
|
||||
_GROUP_POLL_INTERVAL = 0.01
|
||||
|
||||
|
||||
async def terminate_posix_process_tree(process: Process, timeout_seconds: float = 2.0) -> None:
|
||||
"""Terminate a process and all its children on POSIX systems.
|
||||
"""Terminates a process and all its descendants on POSIX.
|
||||
|
||||
Uses os.killpg() for atomic process group termination.
|
||||
|
||||
Args:
|
||||
process: The process to terminate
|
||||
timeout_seconds: Timeout in seconds before force killing (default: 2.0)
|
||||
SIGTERMs the process group, waits up to timeout_seconds for it to
|
||||
disappear, then SIGKILLs whatever remains. killpg reaches every descendant
|
||||
atomically, even ones whose parent already exited; daemonizers that left
|
||||
the group escape by design. A group only disappears once every member is
|
||||
dead and reaped, so a client running as PID 1 should reap orphans (e.g.
|
||||
docker run --init) or the wait below runs its full timeout.
|
||||
"""
|
||||
pid = getattr(process, "pid", None) or getattr(getattr(process, "popen", None), "pid", None)
|
||||
if not pid:
|
||||
# No PID means there's no process to terminate - it either never started,
|
||||
# already exited, or we have an invalid process object
|
||||
return
|
||||
# The leader's pid is the pgid (start_new_session). Never use getpgid():
|
||||
# it fails once the leader is reaped, even with live members left.
|
||||
pgid = process.pid
|
||||
|
||||
try:
|
||||
pgid = os.getpgid(pid)
|
||||
os.killpg(pgid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
return # the whole group is already gone
|
||||
except PermissionError:
|
||||
# EPERM never proves the group is gone (macOS raises it for zombie or
|
||||
# foreign-euid members), so keep waiting and escalating.
|
||||
logger.warning(
|
||||
"No permission to signal some of process group %d; waiting for it to exit anyway", pgid, exc_info=True
|
||||
)
|
||||
|
||||
with anyio.move_on_after(timeout_seconds):
|
||||
while True:
|
||||
try:
|
||||
# Check if process group still exists (signal 0 = check only)
|
||||
os.killpg(pgid, 0)
|
||||
await anyio.sleep(0.1)
|
||||
except ProcessLookupError:
|
||||
return
|
||||
with anyio.move_on_after(timeout_seconds):
|
||||
while _group_alive(pgid):
|
||||
# Reading returncode reaps the leader on trio; a zombie leader would
|
||||
# otherwise keep the group alive for the full timeout.
|
||||
_ = process.returncode
|
||||
await anyio.sleep(_GROUP_POLL_INTERVAL)
|
||||
return
|
||||
|
||||
try:
|
||||
os.killpg(pgid, signal.SIGKILL)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
# ESRCH: died since the last probe. EPERM: we killed what we were allowed to.
|
||||
with suppress(ProcessLookupError, PermissionError):
|
||||
os.killpg(pgid, signal.SIGKILL)
|
||||
|
||||
except (ProcessLookupError, PermissionError, OSError) as e:
|
||||
logger.warning(f"Process group termination failed for PID {pid}: {e}, falling back to simple terminate")
|
||||
try:
|
||||
process.terminate()
|
||||
with anyio.fail_after(timeout_seconds):
|
||||
await process.wait()
|
||||
except Exception:
|
||||
logger.warning(f"Process termination failed for PID {pid}, attempting force kill")
|
||||
try:
|
||||
process.kill()
|
||||
except Exception:
|
||||
logger.exception(f"Failed to kill process {pid}")
|
||||
|
||||
def _group_alive(pgid: int) -> bool:
|
||||
"""Probes the group with signal 0; only ESRCH proves it is gone."""
|
||||
try:
|
||||
os.killpg(pgid, 0)
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
pass # unsignalable survivors or unreaped zombies; EPERM is ambiguous
|
||||
return True
|
||||
|
||||
+125
-189
@@ -4,16 +4,16 @@ import logging
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import weakref
|
||||
from contextlib import suppress
|
||||
from pathlib import Path
|
||||
from typing import BinaryIO, TextIO, cast
|
||||
from typing import BinaryIO, TextIO, TypeAlias, cast
|
||||
|
||||
import anyio
|
||||
from anyio import to_thread
|
||||
from anyio.abc import Process
|
||||
from anyio.streams.file import FileReadStream, FileWriteStream
|
||||
from typing_extensions import deprecated
|
||||
|
||||
logger = logging.getLogger("client.stdio.win32")
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Windows-specific imports for Job Objects
|
||||
if sys.platform == "win32":
|
||||
@@ -28,110 +28,86 @@ else:
|
||||
win32job = None
|
||||
pywintypes = None
|
||||
|
||||
JobHandle = int
|
||||
# How often FallbackProcess polls the underlying Popen for exit.
|
||||
_EXIT_POLL_INTERVAL = 0.01
|
||||
|
||||
# Job Object handle per spawned process, for tree termination at shutdown.
|
||||
# Values stay pywin32 PyHANDLEs: if no pop site ever runs, the dying weak entry
|
||||
# drops the last reference and the PyHANDLE destructor closes the handle, which
|
||||
# is what makes KILL_ON_JOB_CLOSE reap an abandoned tree.
|
||||
_process_jobs: "weakref.WeakKeyDictionary[Process | FallbackProcess, object]" = weakref.WeakKeyDictionary()
|
||||
|
||||
|
||||
def get_windows_executable_command(command: str) -> str:
|
||||
"""Get the correct executable command normalized for Windows.
|
||||
"""Resolves the command to a Windows executable path.
|
||||
|
||||
On Windows, commands might exist with specific extensions (.exe, .cmd, etc.)
|
||||
that need to be located for proper execution.
|
||||
|
||||
Args:
|
||||
command: Base command (e.g., 'uvx', 'npx')
|
||||
|
||||
Returns:
|
||||
str: Windows-appropriate command path
|
||||
Tries the bare name first, then the common script extensions (.cmd, .bat,
|
||||
.exe, .ps1).
|
||||
"""
|
||||
try:
|
||||
# First check if command exists in PATH as-is
|
||||
if command_path := shutil.which(command):
|
||||
return command_path
|
||||
|
||||
# Check for Windows-specific extensions
|
||||
for ext in [".cmd", ".bat", ".exe", ".ps1"]:
|
||||
ext_version = f"{command}{ext}"
|
||||
if ext_path := shutil.which(ext_version):
|
||||
return ext_path
|
||||
|
||||
# For regular commands or if we couldn't find special versions
|
||||
return command
|
||||
except OSError:
|
||||
# Handle file system errors during path resolution
|
||||
# (permissions, broken symlinks, etc.)
|
||||
return command
|
||||
return command # path probing failed (permissions, broken symlinks)
|
||||
|
||||
|
||||
class FallbackProcess:
|
||||
"""A fallback process wrapper for Windows to handle async I/O
|
||||
when using subprocess.Popen, which provides sync-only FileIO objects.
|
||||
"""Async wrapper around subprocess.Popen for SelectorEventLoop.
|
||||
|
||||
This wraps stdin and stdout into async-compatible
|
||||
streams (FileReadStream, FileWriteStream),
|
||||
so that MCP clients expecting async streams can work properly.
|
||||
Windows event loops without async subprocess support get this Popen-backed
|
||||
fallback, with anyio file streams wrapping the pipes.
|
||||
"""
|
||||
|
||||
def __init__(self, popen_obj: subprocess.Popen[bytes]):
|
||||
def __init__(self, popen_obj: subprocess.Popen[bytes]) -> None:
|
||||
self.popen: subprocess.Popen[bytes] = popen_obj
|
||||
self.stdin_raw = popen_obj.stdin # type: ignore[assignment]
|
||||
self.stdout_raw = popen_obj.stdout # type: ignore[assignment]
|
||||
self.stderr = popen_obj.stderr # type: ignore[assignment]
|
||||
stdin = popen_obj.stdin
|
||||
stdout = popen_obj.stdout
|
||||
|
||||
self.stdin = FileWriteStream(cast(BinaryIO, self.stdin_raw)) if self.stdin_raw else None
|
||||
self.stdout = FileReadStream(cast(BinaryIO, self.stdout_raw)) if self.stdout_raw else None
|
||||
self.stdin = FileWriteStream(cast(BinaryIO, stdin)) if stdin else None
|
||||
self.stdout = FileReadStream(cast(BinaryIO, stdout)) if stdout else None
|
||||
|
||||
async def __aenter__(self):
|
||||
"""Support async context manager entry."""
|
||||
return self
|
||||
async def wait(self) -> int:
|
||||
"""Waits for exit by polling the Popen.
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: BaseException | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: object | None,
|
||||
) -> None:
|
||||
"""Terminate and wait on process exit inside a thread."""
|
||||
A thread blocked in Popen.wait() cannot be cancelled by anyio, which
|
||||
would defeat every timeout placed around this call.
|
||||
"""
|
||||
while (returncode := self.popen.poll()) is None:
|
||||
await anyio.sleep(_EXIT_POLL_INTERVAL)
|
||||
return returncode
|
||||
|
||||
def terminate(self) -> None:
|
||||
"""Terminates the subprocess."""
|
||||
self.popen.terminate()
|
||||
await to_thread.run_sync(self.popen.wait)
|
||||
|
||||
# Close the file handles to prevent ResourceWarning
|
||||
if self.stdin:
|
||||
await self.stdin.aclose()
|
||||
if self.stdout:
|
||||
await self.stdout.aclose()
|
||||
if self.stdin_raw:
|
||||
self.stdin_raw.close()
|
||||
if self.stdout_raw:
|
||||
self.stdout_raw.close()
|
||||
if self.stderr:
|
||||
self.stderr.close()
|
||||
|
||||
async def wait(self):
|
||||
"""Async wait for process completion."""
|
||||
return await to_thread.run_sync(self.popen.wait)
|
||||
|
||||
def terminate(self):
|
||||
"""Terminate the subprocess immediately."""
|
||||
return self.popen.terminate()
|
||||
|
||||
def kill(self) -> None:
|
||||
"""Kill the subprocess immediately (alias for terminate)."""
|
||||
self.terminate()
|
||||
"""Kills the subprocess (on Windows the same hard kill as terminate)."""
|
||||
self.popen.kill()
|
||||
|
||||
@property
|
||||
def pid(self) -> int:
|
||||
"""Return the process ID."""
|
||||
"""Returns the process ID."""
|
||||
return self.popen.pid
|
||||
|
||||
@property
|
||||
def returncode(self) -> int | None:
|
||||
"""Return the exit code, or ``None`` if the process has not yet terminated."""
|
||||
return self.popen.returncode
|
||||
"""The exit code, or None while the process is still running.
|
||||
|
||||
Polls the Popen so death is observable without anyone calling wait().
|
||||
"""
|
||||
return self.popen.poll()
|
||||
|
||||
|
||||
# ------------------------
|
||||
# Updated function
|
||||
# ------------------------
|
||||
# The process handle stdio_client drives: anyio's Process, or the Popen-backed
|
||||
# fallback used on Windows event loops without async subprocess support.
|
||||
ServerProcess: TypeAlias = Process | FallbackProcess
|
||||
|
||||
|
||||
async def create_windows_process(
|
||||
@@ -141,53 +117,35 @@ async def create_windows_process(
|
||||
errlog: TextIO | None = sys.stderr,
|
||||
cwd: Path | str | None = None,
|
||||
) -> Process | FallbackProcess:
|
||||
"""Creates a subprocess in a Windows-compatible way with Job Object support.
|
||||
"""Creates a subprocess with Job Object support for tree termination.
|
||||
|
||||
Attempts to use anyio's open_process for async subprocess creation.
|
||||
In some cases this will throw NotImplementedError on Windows, e.g.,
|
||||
when using the SelectorEventLoop, which does not support async subprocesses.
|
||||
In that case, we fall back to using subprocess.Popen.
|
||||
|
||||
The process is automatically added to a Job Object to ensure all child
|
||||
processes are terminated when the parent is terminated.
|
||||
|
||||
Args:
|
||||
command (str): The executable to run
|
||||
args (list[str]): List of command line arguments
|
||||
env (dict[str, str] | None): Environment variables
|
||||
errlog (TextIO | None): Where to send stderr output (defaults to sys.stderr)
|
||||
cwd (Path | str | None): Working directory for the subprocess
|
||||
Spawns via anyio's open_process; event loops without async subprocess
|
||||
support (notably the SelectorEventLoop) raise NotImplementedError, in which
|
||||
case the spawn falls back to a Popen-backed FallbackProcess. Either way the
|
||||
process is then assigned to a Job Object so its children can be terminated
|
||||
with it; children spawned before the assignment completes are not captured
|
||||
(see the inline note below).
|
||||
|
||||
Returns:
|
||||
Process | FallbackProcess: Async-compatible subprocess with stdin and stdout streams
|
||||
Process | FallbackProcess: The spawned process with async stdin/stdout streams.
|
||||
"""
|
||||
job = _create_job_object()
|
||||
process = None
|
||||
|
||||
try:
|
||||
# First try using anyio with Windows-specific flags to hide console window
|
||||
process = await anyio.open_process(
|
||||
[command, *args],
|
||||
env=env,
|
||||
# Ensure we don't create console windows for each process
|
||||
creationflags=subprocess.CREATE_NO_WINDOW # type: ignore
|
||||
if hasattr(subprocess, "CREATE_NO_WINDOW")
|
||||
else 0,
|
||||
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
|
||||
stderr=errlog,
|
||||
cwd=cwd,
|
||||
)
|
||||
except NotImplementedError:
|
||||
# If Windows doesn't support async subprocess creation, use fallback
|
||||
# Windows event loops without async subprocess support (SelectorEventLoop)
|
||||
process = await _create_windows_fallback_process(command, args, env, errlog, cwd)
|
||||
except Exception:
|
||||
# Try again without creation flags
|
||||
process = await anyio.open_process(
|
||||
[command, *args],
|
||||
env=env,
|
||||
stderr=errlog,
|
||||
cwd=cwd,
|
||||
)
|
||||
|
||||
# Children spawned before the assignment completes land outside the job
|
||||
# (membership is inherited at CreateProcess, never acquired retroactively);
|
||||
# if that ever bites, the fix is a CREATE_SUSPENDED spawn -> assign -> resume.
|
||||
job = _create_job_object()
|
||||
_maybe_assign_process_to_job(process, job)
|
||||
return process
|
||||
|
||||
@@ -199,41 +157,26 @@ async def _create_windows_fallback_process(
|
||||
errlog: TextIO | None = sys.stderr,
|
||||
cwd: Path | str | None = None,
|
||||
) -> FallbackProcess:
|
||||
"""Create a subprocess using subprocess.Popen as a fallback when anyio fails.
|
||||
|
||||
This function wraps the sync subprocess.Popen in an async-compatible interface.
|
||||
"""
|
||||
try:
|
||||
# Try launching with creationflags to avoid opening a new console window
|
||||
popen_obj = subprocess.Popen(
|
||||
[command, *args],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=errlog,
|
||||
env=env,
|
||||
cwd=cwd,
|
||||
bufsize=0, # Unbuffered output
|
||||
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
|
||||
)
|
||||
except Exception:
|
||||
# If creationflags failed, fallback without them
|
||||
popen_obj = subprocess.Popen(
|
||||
[command, *args],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=errlog,
|
||||
env=env,
|
||||
cwd=cwd,
|
||||
bufsize=0,
|
||||
)
|
||||
"""Spawns via subprocess.Popen and wraps it in FallbackProcess."""
|
||||
popen_obj = subprocess.Popen(
|
||||
[command, *args],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=errlog,
|
||||
env=env,
|
||||
cwd=cwd,
|
||||
bufsize=0, # Unbuffered output
|
||||
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
|
||||
)
|
||||
return FallbackProcess(popen_obj)
|
||||
|
||||
|
||||
def _create_job_object() -> int | None:
|
||||
"""Create a Windows Job Object configured to terminate all processes when closed."""
|
||||
if sys.platform != "win32" or not win32job:
|
||||
def _create_job_object() -> object | None:
|
||||
"""Creates a Windows Job Object configured to terminate all its processes when closed."""
|
||||
if sys.platform != "win32" or not win32api or not win32job:
|
||||
return None
|
||||
|
||||
job = None
|
||||
try:
|
||||
job = win32job.CreateJobObject(None, "")
|
||||
extended_info = win32job.QueryInformationJobObject(job, win32job.JobObjectExtendedLimitInformation)
|
||||
@@ -241,17 +184,20 @@ def _create_job_object() -> int | None:
|
||||
extended_info["BasicLimitInformation"]["LimitFlags"] |= win32job.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
|
||||
win32job.SetInformationJobObject(job, win32job.JobObjectExtendedLimitInformation, extended_info)
|
||||
return job
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to create Job Object for process tree management: {e}")
|
||||
except pywintypes.error:
|
||||
logger.warning("Failed to create Job Object for process tree management", exc_info=True)
|
||||
# If creation succeeded but configuration failed, close the handle now.
|
||||
if job is not None:
|
||||
_close_job_handle(job)
|
||||
return None
|
||||
|
||||
|
||||
def _maybe_assign_process_to_job(process: Process | FallbackProcess, job: JobHandle | None) -> None:
|
||||
"""Try to assign a process to a job object.
|
||||
def _maybe_assign_process_to_job(process: Process | FallbackProcess, job: object | None) -> None:
|
||||
"""Assigns the process to the job and records it for tree termination.
|
||||
|
||||
If assignment fails for any reason, the job handle is closed.
|
||||
On any failure the job handle is closed instead.
|
||||
"""
|
||||
if not job:
|
||||
if job is None:
|
||||
return
|
||||
|
||||
if sys.platform != "win32" or not win32api or not win32con or not win32job:
|
||||
@@ -262,72 +208,62 @@ def _maybe_assign_process_to_job(process: Process | FallbackProcess, job: JobHan
|
||||
win32con.PROCESS_SET_QUOTA | win32con.PROCESS_TERMINATE, False, process.pid
|
||||
)
|
||||
if not process_handle:
|
||||
raise Exception("Failed to open process handle")
|
||||
raise pywintypes.error(0, "OpenProcess", "Failed to open process handle")
|
||||
|
||||
try:
|
||||
win32job.AssignProcessToJobObject(job, process_handle)
|
||||
process._job_object = job
|
||||
finally:
|
||||
win32api.CloseHandle(process_handle)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to assign process {process.pid} to Job Object: {e}")
|
||||
if win32api:
|
||||
win32api.CloseHandle(job)
|
||||
# Record only after the CloseHandle above succeeded: had it failed, the
|
||||
# except below would close the job and KILL_ON_JOB_CLOSE takes the server.
|
||||
_process_jobs[process] = job
|
||||
except pywintypes.error:
|
||||
logger.warning("Failed to assign process %d to Job Object", process.pid, exc_info=True)
|
||||
_close_job_handle(job)
|
||||
|
||||
|
||||
async def terminate_windows_process_tree(process: Process | FallbackProcess, timeout_seconds: float = 2.0) -> None:
|
||||
"""Terminate a process and all its children on Windows.
|
||||
def close_process_job(process: Process | FallbackProcess) -> None:
|
||||
"""Closes the process's Job Object handle, if it still has one.
|
||||
|
||||
If the process has an associated job object, it will be terminated.
|
||||
Otherwise, falls back to basic process termination.
|
||||
|
||||
Args:
|
||||
process: The process to terminate
|
||||
timeout_seconds: Timeout in seconds before force killing (default: 2.0)
|
||||
KILL_ON_JOB_CLOSE makes the close also kill any members still alive,
|
||||
deterministically rather than at GC time; a deliberate divergence from
|
||||
POSIX, where a graceful server's children are left alive.
|
||||
"""
|
||||
if sys.platform != "win32":
|
||||
return
|
||||
|
||||
job = getattr(process, "_job_object", None)
|
||||
if job and win32job:
|
||||
try:
|
||||
win32job.TerminateJobObject(job, 1)
|
||||
except Exception:
|
||||
# Job might already be terminated
|
||||
pass
|
||||
finally:
|
||||
if win32api:
|
||||
try:
|
||||
win32api.CloseHandle(job)
|
||||
except Exception:
|
||||
pass
|
||||
job = _process_jobs.pop(process, None)
|
||||
if job is not None:
|
||||
_close_job_handle(job)
|
||||
|
||||
# Always try to terminate the process itself as well
|
||||
|
||||
async def terminate_windows_process_tree(process: Process | FallbackProcess) -> None:
|
||||
"""Terminates the process's job, or just the process if it has no job.
|
||||
|
||||
Job termination is an immediate hard kill of every member. Windows has no
|
||||
tree-wide SIGTERM; the stdin-close grace period is the server's chance to
|
||||
exit cleanly.
|
||||
"""
|
||||
if sys.platform != "win32":
|
||||
return
|
||||
|
||||
job = _process_jobs.pop(process, None)
|
||||
if job is not None and win32job:
|
||||
try:
|
||||
with suppress(pywintypes.error): # the job might already be terminated
|
||||
win32job.TerminateJobObject(job, 1)
|
||||
finally:
|
||||
_close_job_handle(job)
|
||||
|
||||
# The process may have no job (creation or assignment failed); kill it directly too.
|
||||
try:
|
||||
process.terminate()
|
||||
except Exception:
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
@deprecated(
|
||||
"terminate_windows_process is deprecated and will be removed in a future version. "
|
||||
"Process termination is now handled internally by the stdio_client context manager."
|
||||
)
|
||||
async def terminate_windows_process(process: Process | FallbackProcess):
|
||||
"""Terminate a Windows process.
|
||||
|
||||
Note: On Windows, terminating a process with process.terminate() doesn't
|
||||
always guarantee immediate process termination.
|
||||
If the process does not exit within 2 seconds, process.kill() is called
|
||||
to send a SIGKILL-equivalent signal.
|
||||
|
||||
Args:
|
||||
process: The process to terminate
|
||||
"""
|
||||
try:
|
||||
process.terminate()
|
||||
with anyio.fail_after(2.0):
|
||||
await process.wait()
|
||||
except TimeoutError:
|
||||
# Force kill if it doesn't terminate
|
||||
process.kill()
|
||||
def _close_job_handle(job: object) -> None:
|
||||
"""Closes a Job Object handle, tolerating one that is already closed."""
|
||||
if win32api and pywintypes:
|
||||
with suppress(pywintypes.error):
|
||||
win32api.CloseHandle(job)
|
||||
|
||||
+1234
-394
File diff suppressed because it is too large
Load Diff
@@ -1,18 +1,13 @@
|
||||
"""The stdio transport: one subprocess end-to-end test and one in-process framing test.
|
||||
|
||||
Everything else in the suite runs in a single process; the subprocess test exists to prove the same
|
||||
client↔server round trip works over the stdio transport's real boundary (a child process whose
|
||||
stdin/stdout carry one newline-delimited JSON-RPC message per line). The server lives in
|
||||
`_stdio_server.py` and is launched via `python -m` so subprocess coverage measurement applies.
|
||||
The subprocess test proves the client-server round trip over the transport's real process
|
||||
boundary; its server lives in `_stdio_server.py` and is launched via `python -m` so subprocess
|
||||
coverage measurement applies. The framing test drives `stdio_server` over injected in-process
|
||||
streams instead.
|
||||
|
||||
The framing test drives `stdio_server` in-process by passing it injected text streams instead of the
|
||||
real stdin/stdout, so the raw lines the transport writes can be asserted directly without a process
|
||||
boundary.
|
||||
|
||||
stdio is deliberately not a leg of the `connect`-fixture matrix: spawning a subprocess per test
|
||||
would be slow, and the matrix already proves transport-agnosticism over three in-process
|
||||
transports. Process-lifecycle edge cases (escalation to terminate/kill, parse errors) are covered by
|
||||
`tests/client/test_stdio.py` and stay deferred here.
|
||||
stdio is deliberately not a leg of the `connect`-fixture matrix: a subprocess per test would be
|
||||
slow, and the matrix already proves transport-agnosticism in-process. Process-lifecycle edge
|
||||
cases (terminate/kill escalation, parse errors) stay in `tests/client/test_stdio.py`.
|
||||
"""
|
||||
|
||||
import io
|
||||
@@ -26,6 +21,7 @@ import anyio
|
||||
import pytest
|
||||
from inline_snapshot import snapshot
|
||||
|
||||
from mcp.client import stdio
|
||||
from mcp.client.client import Client
|
||||
from mcp.client.stdio import StdioServerParameters, stdio_client
|
||||
from mcp.server.stdio import stdio_server
|
||||
@@ -51,10 +47,21 @@ _REPO_ROOT = Path(__file__).parents[3]
|
||||
@requirement("transport:stdio")
|
||||
@requirement("transport:stdio:clean-shutdown")
|
||||
@requirement("transport:stdio:stderr-passthrough")
|
||||
async def test_tool_call_and_notification_round_trip_over_a_stdio_subprocess() -> None:
|
||||
"""A Client connected over stdio initializes, calls a tool with arguments, receives the
|
||||
server's log notification before the call returns, and the server exits when the transport
|
||||
closes its stdin."""
|
||||
async def test_tool_call_and_notification_round_trip_over_a_stdio_subprocess(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""A stdio-subprocess Client round-trips a tool call, a notification, and a clean exit.
|
||||
|
||||
The Client initializes, calls a tool with arguments, and receives the server's log
|
||||
notification before the call returns; the server exits when the transport closes its
|
||||
stdin.
|
||||
"""
|
||||
# After stdin closes, the child must unwind, write the clean-exit line, and let coverage's
|
||||
# atexit hook persist its subprocess data file before escalation. The production 2s default
|
||||
# was too tight on slow Windows runners: the child was killed mid-atexit (test stayed green)
|
||||
# and the silently missing data file tripped the 100% coverage gate. Not under test.
|
||||
monkeypatch.setattr(stdio, "PROCESS_TERMINATION_TIMEOUT", 10.0)
|
||||
|
||||
received: list[LoggingMessageNotificationParams] = []
|
||||
|
||||
async def collect(params: LoggingMessageNotificationParams) -> None:
|
||||
@@ -66,15 +73,18 @@ async def test_tool_call_and_notification_round_trip_over_a_stdio_subprocess() -
|
||||
command=sys.executable,
|
||||
args=["-m", _stdio_server.__name__],
|
||||
cwd=str(_REPO_ROOT),
|
||||
# stdio_client deliberately filters the inherited environment to a safe minimum,
|
||||
# which drops the variables coverage.py's subprocess support uses; pass them through
|
||||
# so the server module is measured. Empty when not running under coverage.
|
||||
env={key: value for key, value in os.environ.items() if key.startswith("COVERAGE_")},
|
||||
# stdio_client filters the inherited environment, dropping the variables
|
||||
# coverage.py's subprocess support uses; pass them through so the server module is
|
||||
# measured. PYTHONWARNINGS: the child recompiles anyio (pytest's pyc tag differs),
|
||||
# and on 3.14 anyio's return-in-finally SyntaxWarning would land on the snapshot stderr.
|
||||
env={key: value for key, value in os.environ.items() if key.startswith("COVERAGE_")}
|
||||
| {"PYTHONWARNINGS": "ignore::SyntaxWarning"},
|
||||
),
|
||||
errlog=errlog,
|
||||
)
|
||||
|
||||
with anyio.fail_after(10):
|
||||
# Must exceed session time plus the patched PROCESS_TERMINATION_TIMEOUT (10s).
|
||||
with anyio.fail_after(20):
|
||||
async with Client(transport, logging_callback=collect) as client:
|
||||
assert client.initialize_result.server_info.name == "stdio-echo"
|
||||
result = await client.call_tool("echo", {"text": "across\nprocesses"})
|
||||
@@ -83,28 +93,23 @@ async def test_tool_call_and_notification_round_trip_over_a_stdio_subprocess() -
|
||||
captured_stderr = errlog.read()
|
||||
|
||||
assert result == snapshot(CallToolResult(content=[TextContent(text="across\nprocesses")]))
|
||||
# stdio carries one ordered server→client stream, so the same notification-before-response
|
||||
# stdio carries one ordered server-to-client stream, so the same notification-before-response
|
||||
# guarantee holds here as for the in-memory transport.
|
||||
assert received == snapshot(
|
||||
[LoggingMessageNotificationParams(level="info", logger="echo", data="echoing across\nprocesses")]
|
||||
)
|
||||
# The server writes this line only after its run loop returns, which happens when stdin closes:
|
||||
# seeing it proves the process exited on its own rather than via the transport's terminate
|
||||
# escalation, without a timing-based assertion. The capture itself proves stderr passthrough:
|
||||
# the transport routes the child's stderr to the caller's `errlog` without consuming it.
|
||||
# The server writes this line only after its run loop returns on stdin close: seeing it proves
|
||||
# a self-exit, not the terminate escalation. The capture itself proves stderr passthrough.
|
||||
assert captured_stderr == snapshot("stdio-echo: clean exit\n")
|
||||
|
||||
|
||||
@requirement("transport:stdio:stream-purity")
|
||||
@requirement("transport:stdio:no-embedded-newlines")
|
||||
async def test_stdio_server_writes_one_jsonrpc_message_per_line() -> None:
|
||||
"""Everything `stdio_server` writes is a valid JSON-RPC message on its own line, and nothing else.
|
||||
"""Every `stdio_server` write is one valid JSON-RPC message on its own line.
|
||||
|
||||
The transport's stdin/stdout parameters are public, so the test injects in-process text streams
|
||||
instead of the real process handles and drives the read/write streams directly: a JSON-RPC line on
|
||||
stdin is parsed and delivered, and every message sent on the write stream appears as exactly one
|
||||
newline-terminated line whose payload newlines are JSON-escaped. This proves the transport's own
|
||||
framing; it does not guard `sys.stdout` against handler code that prints to it directly (see the
|
||||
Each line is newline-terminated with payload newlines JSON-escaped. This proves the
|
||||
transport's own framing; it does not guard `sys.stdout` against handler code (see the
|
||||
divergence on `transport:stdio:stream-purity`).
|
||||
"""
|
||||
captured = io.StringIO()
|
||||
|
||||
@@ -1,240 +0,0 @@
|
||||
"""Regression test for issue #1027: Ensure cleanup procedures run properly during shutdown
|
||||
|
||||
Issue #1027 reported that cleanup code after "yield" in lifespan was unreachable when
|
||||
processes were terminated. This has been fixed by implementing the MCP spec-compliant
|
||||
stdio shutdown sequence that closes stdin first, allowing graceful exit.
|
||||
|
||||
These tests verify the fix continues to work correctly across all platforms.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import tempfile
|
||||
import textwrap
|
||||
from pathlib import Path
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import _create_platform_compatible_process, stdio_client
|
||||
from tests.shared.test_win32_utils import escape_path_for_python
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_lifespan_cleanup_executed():
|
||||
"""Regression test ensuring MCP server cleanup code runs during shutdown.
|
||||
|
||||
This test verifies that the fix for issue #1027 works correctly by:
|
||||
1. Starting an MCP server that writes a marker file on startup
|
||||
2. Shutting down the server normally via stdio_client
|
||||
3. Verifying the cleanup code (after yield) executed and wrote its marker file
|
||||
|
||||
The fix implements proper stdin closure before termination, giving servers
|
||||
time to run their cleanup handlers.
|
||||
"""
|
||||
|
||||
# Create marker files to track server lifecycle
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
|
||||
startup_marker = f.name
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
|
||||
cleanup_marker = f.name
|
||||
|
||||
# Remove the files so we can detect when they're created
|
||||
Path(startup_marker).unlink()
|
||||
Path(cleanup_marker).unlink()
|
||||
|
||||
# Create a minimal MCP server using MCPServer that tracks lifecycle
|
||||
server_code = textwrap.dedent(f"""
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from contextlib import asynccontextmanager
|
||||
from mcp.server.mcpserver import MCPServer
|
||||
|
||||
STARTUP_MARKER = {escape_path_for_python(startup_marker)}
|
||||
CLEANUP_MARKER = {escape_path_for_python(cleanup_marker)}
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(server):
|
||||
# Write startup marker
|
||||
Path(STARTUP_MARKER).write_text("started")
|
||||
try:
|
||||
yield {{"started": True}}
|
||||
finally:
|
||||
# This cleanup code now runs properly during shutdown
|
||||
Path(CLEANUP_MARKER).write_text("cleaned up")
|
||||
|
||||
mcp = MCPServer("test-server", lifespan=lifespan)
|
||||
|
||||
@mcp.tool()
|
||||
def echo(text: str) -> str:
|
||||
return text
|
||||
|
||||
if __name__ == "__main__":
|
||||
mcp.run()
|
||||
""")
|
||||
|
||||
# Write the server script to a temporary file
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".py") as f:
|
||||
server_script = f.name
|
||||
f.write(server_code)
|
||||
|
||||
try:
|
||||
# Launch the MCP server
|
||||
params = StdioServerParameters(command=sys.executable, args=[server_script])
|
||||
|
||||
async with stdio_client(params) as (read, write):
|
||||
async with ClientSession(read, write) as session:
|
||||
# Initialize the session
|
||||
result = await session.initialize()
|
||||
assert result.protocol_version in ["2024-11-05", "2025-06-18", "2025-11-25"]
|
||||
|
||||
# Verify startup marker was created
|
||||
assert Path(startup_marker).exists(), "Server startup marker not created"
|
||||
assert Path(startup_marker).read_text() == "started"
|
||||
|
||||
# Make a test request to ensure server is working
|
||||
response = await session.call_tool("echo", {"text": "hello"})
|
||||
assert response.content[0].type == "text"
|
||||
assert getattr(response.content[0], "text") == "hello"
|
||||
|
||||
# Session will be closed when exiting the context manager
|
||||
|
||||
# Give server a moment to complete cleanup
|
||||
with anyio.move_on_after(5.0):
|
||||
while not Path(cleanup_marker).exists(): # pragma: lax no cover
|
||||
await anyio.sleep(0.1)
|
||||
|
||||
# Verify cleanup marker was created - this works now that stdio_client
|
||||
# properly closes stdin before termination, allowing graceful shutdown
|
||||
assert Path(cleanup_marker).exists(), "Server cleanup marker not created - regression in issue #1027 fix"
|
||||
assert Path(cleanup_marker).read_text() == "cleaned up"
|
||||
|
||||
finally:
|
||||
# Clean up files
|
||||
for path in [server_script, startup_marker, cleanup_marker]:
|
||||
try: # pragma: lax no cover
|
||||
Path(path).unlink()
|
||||
except FileNotFoundError: # pragma: lax no cover
|
||||
pass
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
@pytest.mark.filterwarnings("ignore::ResourceWarning" if sys.platform == "win32" else "default")
|
||||
async def test_stdin_close_triggers_cleanup():
|
||||
"""Regression test verifying the stdin-based graceful shutdown mechanism.
|
||||
|
||||
This test ensures the core fix for issue #1027 continues to work by:
|
||||
1. Manually managing a server process
|
||||
2. Closing stdin to trigger graceful shutdown
|
||||
3. Verifying cleanup handlers run before the process exits
|
||||
|
||||
This mimics the behavior now implemented in stdio_client's shutdown sequence.
|
||||
|
||||
Note on Windows ResourceWarning:
|
||||
On Windows, we may see ResourceWarning about unclosed file descriptors.
|
||||
This is expected behavior because:
|
||||
- We're manually managing the process lifecycle
|
||||
- Windows file handle cleanup works differently than Unix
|
||||
- The warning doesn't indicate a real issue - cleanup still works
|
||||
We filter this warning on Windows only to avoid test noise.
|
||||
"""
|
||||
|
||||
# Create marker files to track server lifecycle
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
|
||||
startup_marker = f.name
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".txt") as f:
|
||||
cleanup_marker = f.name
|
||||
|
||||
# Remove the files so we can detect when they're created
|
||||
Path(startup_marker).unlink()
|
||||
Path(cleanup_marker).unlink()
|
||||
|
||||
# Create an MCP server that handles stdin closure gracefully
|
||||
server_code = textwrap.dedent(f"""
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from contextlib import asynccontextmanager
|
||||
from mcp.server.mcpserver import MCPServer
|
||||
|
||||
STARTUP_MARKER = {escape_path_for_python(startup_marker)}
|
||||
CLEANUP_MARKER = {escape_path_for_python(cleanup_marker)}
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(server):
|
||||
# Write startup marker
|
||||
Path(STARTUP_MARKER).write_text("started")
|
||||
try:
|
||||
yield {{"started": True}}
|
||||
finally:
|
||||
# This cleanup code runs when stdin closes, enabling graceful shutdown
|
||||
Path(CLEANUP_MARKER).write_text("cleaned up")
|
||||
|
||||
mcp = MCPServer("test-server", lifespan=lifespan)
|
||||
|
||||
@mcp.tool()
|
||||
def echo(text: str) -> str:
|
||||
return text
|
||||
|
||||
if __name__ == "__main__":
|
||||
# The server should exit gracefully when stdin closes
|
||||
try:
|
||||
mcp.run()
|
||||
except Exception:
|
||||
# Server might get EOF or other errors when stdin closes
|
||||
pass
|
||||
""")
|
||||
|
||||
# Write the server script to a temporary file
|
||||
with tempfile.NamedTemporaryFile(mode="w", delete=False, suffix=".py") as f:
|
||||
server_script = f.name
|
||||
f.write(server_code)
|
||||
|
||||
try:
|
||||
# This test manually manages the process to verify stdin-based shutdown
|
||||
# Start the server process
|
||||
process = await _create_platform_compatible_process(
|
||||
command=sys.executable, args=[server_script], env=None, errlog=sys.stderr, cwd=None
|
||||
)
|
||||
|
||||
# Wait for server to start
|
||||
with anyio.move_on_after(10.0):
|
||||
while not Path(startup_marker).exists():
|
||||
await anyio.sleep(0.1)
|
||||
|
||||
# Check if process is still running
|
||||
if hasattr(process, "returncode") and process.returncode is not None: # pragma: lax no cover
|
||||
pytest.fail(f"Server process exited with code {process.returncode}")
|
||||
|
||||
assert Path(startup_marker).exists(), "Server startup marker not created"
|
||||
|
||||
# Close stdin to signal shutdown
|
||||
if process.stdin: # pragma: no branch
|
||||
await process.stdin.aclose()
|
||||
|
||||
# Wait for process to exit gracefully
|
||||
try:
|
||||
with anyio.fail_after(5.0): # Increased from 2.0 to 5.0
|
||||
await process.wait()
|
||||
except TimeoutError: # pragma: lax no cover
|
||||
# If it doesn't exit after stdin close, terminate it
|
||||
process.terminate()
|
||||
await process.wait()
|
||||
|
||||
# Check if cleanup ran
|
||||
with anyio.move_on_after(5.0):
|
||||
while not Path(cleanup_marker).exists(): # pragma: lax no cover
|
||||
await anyio.sleep(0.1)
|
||||
|
||||
# Verify the cleanup ran - stdin closure enables graceful shutdown
|
||||
assert Path(cleanup_marker).exists(), "Server cleanup marker not created - stdin-based shutdown failed"
|
||||
assert Path(cleanup_marker).read_text() == "cleaned up"
|
||||
|
||||
finally:
|
||||
# Clean up files
|
||||
for path in [server_script, startup_marker, cleanup_marker]:
|
||||
try: # pragma: lax no cover
|
||||
Path(path).unlink()
|
||||
except FileNotFoundError: # pragma: lax no cover
|
||||
pass
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Test for issue #552: stdio_client hangs on Windows."""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from textwrap import dedent
|
||||
|
||||
@@ -8,41 +9,36 @@ import pytest
|
||||
|
||||
from mcp import ClientSession, StdioServerParameters
|
||||
from mcp.client.stdio import stdio_client
|
||||
from mcp.types import LATEST_PROTOCOL_VERSION, InitializeResult
|
||||
|
||||
|
||||
@pytest.mark.skipif(sys.platform != "win32", reason="Windows-specific test") # pragma: no cover
|
||||
@pytest.mark.anyio
|
||||
async def test_windows_stdio_client_with_session():
|
||||
"""Test the exact scenario from issue #552: Using ClientSession with stdio_client.
|
||||
async def test_initialize_succeeds_and_shutdown_returns_after_the_server_exits_mid_session():
|
||||
"""Initialize completes and shutdown returns when the server exits mid-session.
|
||||
|
||||
This reproduces the original bug report where stdio_client hangs on Windows 11
|
||||
when used with ClientSession.
|
||||
This is the proactor pipe scenario that hung on Windows 11 (issue #552). The positive
|
||||
assertion matters: a session that errors quickly would also "not hang".
|
||||
"""
|
||||
# Create a minimal MCP server that responds to initialization
|
||||
server_script = dedent("""
|
||||
# A minimal server: answer initialize correctly, then exit.
|
||||
server_script = dedent(f"""
|
||||
import json
|
||||
import sys
|
||||
|
||||
# Read initialization request
|
||||
line = sys.stdin.readline()
|
||||
request = json.loads(line)
|
||||
|
||||
# Send initialization response
|
||||
response = {
|
||||
response = {{
|
||||
"jsonrpc": "2.0",
|
||||
"id": 1,
|
||||
"result": {
|
||||
"protocolVersion": "1.0",
|
||||
"capabilities": {},
|
||||
"serverInfo": {"name": "test-server", "version": "1.0"}
|
||||
}
|
||||
}
|
||||
"id": request["id"],
|
||||
"result": {{
|
||||
"protocolVersion": {json.dumps(LATEST_PROTOCOL_VERSION)},
|
||||
"capabilities": {{}},
|
||||
"serverInfo": {{"name": "test-server", "version": "1.0"}}
|
||||
}}
|
||||
}}
|
||||
print(json.dumps(response))
|
||||
sys.stdout.flush()
|
||||
|
||||
# Exit after a short delay
|
||||
import time
|
||||
time.sleep(0.1)
|
||||
sys.exit(0)
|
||||
""").strip()
|
||||
|
||||
params = StdioServerParameters(
|
||||
@@ -50,14 +46,11 @@ async def test_windows_stdio_client_with_session():
|
||||
args=["-c", server_script],
|
||||
)
|
||||
|
||||
# This is the exact pattern from the bug report
|
||||
with anyio.fail_after(10):
|
||||
try:
|
||||
async with stdio_client(params) as (read, write):
|
||||
async with ClientSession(read, write) as session:
|
||||
await session.initialize()
|
||||
# Should exit ClientSession without hanging
|
||||
# Should exit stdio_client without hanging
|
||||
except Exception:
|
||||
# Connection errors are expected when process exits
|
||||
pass
|
||||
async with stdio_client(params) as (read, write):
|
||||
async with ClientSession(read, write) as session:
|
||||
result = await session.initialize()
|
||||
assert isinstance(result, InitializeResult)
|
||||
assert result.server_info.name == "test-server"
|
||||
# Exiting ClientSession and stdio_client must not hang even though the
|
||||
# server process is already gone.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Test the elicitation feature using stdio transport."""
|
||||
"""Test the elicitation feature over the in-memory client transport."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
@@ -58,9 +58,9 @@ async def call_tool_and_assert(
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_stdio_elicitation():
|
||||
"""Test the elicitation feature using stdio transport."""
|
||||
mcp = MCPServer(name="StdioElicitationServer")
|
||||
async def test_elicitation_accept_returns_the_users_answer_to_the_tool():
|
||||
"""An accepted elicitation delivers the user's content back to the requesting tool."""
|
||||
mcp = MCPServer(name="ElicitationServer")
|
||||
create_ask_user_tool(mcp)
|
||||
|
||||
# Create a custom handler for elicitation requests
|
||||
@@ -76,9 +76,9 @@ async def test_stdio_elicitation():
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_stdio_elicitation_decline():
|
||||
"""Test elicitation with user declining."""
|
||||
mcp = MCPServer(name="StdioElicitationDeclineServer")
|
||||
async def test_elicitation_decline_reaches_the_tool_without_content():
|
||||
"""A declined elicitation reports the decline to the tool, with no content attached."""
|
||||
mcp = MCPServer(name="ElicitationDeclineServer")
|
||||
create_ask_user_tool(mcp)
|
||||
|
||||
async def elicitation_callback(context: RequestContext[ClientSession], params: ElicitRequestParams):
|
||||
|
||||
+108
-31
@@ -1,17 +1,26 @@
|
||||
import io
|
||||
import sys
|
||||
import threading
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
from io import TextIOWrapper
|
||||
|
||||
import anyio
|
||||
import pytest
|
||||
|
||||
from mcp.server.mcpserver import MCPServer
|
||||
from mcp.server.stdio import stdio_server
|
||||
from mcp.shared.message import SessionMessage
|
||||
from mcp.types import JSONRPCMessage, JSONRPCRequest, JSONRPCResponse, jsonrpc_message_adapter
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_stdio_server():
|
||||
async def test_stdio_server_round_trips_messages_over_injected_streams() -> None:
|
||||
"""stdio_server frames JSON-RPC messages as one line each in both directions.
|
||||
|
||||
Parses one message per stdin line and writes each outgoing message as exactly one
|
||||
line, driven over injected in-process streams.
|
||||
"""
|
||||
stdin = io.StringIO()
|
||||
stdout = io.StringIO()
|
||||
|
||||
@@ -24,52 +33,45 @@ async def test_stdio_server():
|
||||
stdin.write(message.model_dump_json(by_alias=True, exclude_none=True) + "\n")
|
||||
stdin.seek(0)
|
||||
|
||||
async with stdio_server(stdin=anyio.AsyncFile(stdin), stdout=anyio.AsyncFile(stdout)) as (
|
||||
read_stream,
|
||||
write_stream,
|
||||
):
|
||||
received_messages: list[JSONRPCMessage] = []
|
||||
async with read_stream:
|
||||
async for message in read_stream:
|
||||
if isinstance(message, Exception): # pragma: no cover
|
||||
raise message
|
||||
received_messages.append(message.message)
|
||||
if len(received_messages) == 2:
|
||||
break
|
||||
with anyio.fail_after(5):
|
||||
async with stdio_server(stdin=anyio.AsyncFile(stdin), stdout=anyio.AsyncFile(stdout)) as (
|
||||
read_stream,
|
||||
write_stream,
|
||||
):
|
||||
async with read_stream:
|
||||
received_messages: list[JSONRPCMessage] = []
|
||||
for _ in range(2):
|
||||
received = await read_stream.receive()
|
||||
assert not isinstance(received, Exception)
|
||||
received_messages.append(received.message)
|
||||
|
||||
# Verify received messages
|
||||
assert len(received_messages) == 2
|
||||
assert received_messages[0] == JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
assert received_messages[1] == JSONRPCResponse(jsonrpc="2.0", id=2, result={})
|
||||
assert received_messages[0] == JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
assert received_messages[1] == JSONRPCResponse(jsonrpc="2.0", id=2, result={})
|
||||
|
||||
# Test sending responses from the server
|
||||
responses = [
|
||||
JSONRPCRequest(jsonrpc="2.0", id=3, method="ping"),
|
||||
JSONRPCResponse(jsonrpc="2.0", id=4, result={}),
|
||||
]
|
||||
responses = [
|
||||
JSONRPCRequest(jsonrpc="2.0", id=3, method="ping"),
|
||||
JSONRPCResponse(jsonrpc="2.0", id=4, result={}),
|
||||
]
|
||||
|
||||
async with write_stream:
|
||||
for response in responses:
|
||||
session_message = SessionMessage(response)
|
||||
await write_stream.send(session_message)
|
||||
await write_stream.send(SessionMessage(response))
|
||||
await write_stream.aclose()
|
||||
|
||||
stdout.seek(0)
|
||||
output_lines = stdout.readlines()
|
||||
assert len(output_lines) == 2
|
||||
|
||||
received_responses = [jsonrpc_message_adapter.validate_json(line.strip()) for line in output_lines]
|
||||
assert len(received_responses) == 2
|
||||
assert received_responses[0] == JSONRPCRequest(jsonrpc="2.0", id=3, method="ping")
|
||||
assert received_responses[1] == JSONRPCResponse(jsonrpc="2.0", id=4, result={})
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_stdio_server_invalid_utf8(monkeypatch: pytest.MonkeyPatch):
|
||||
"""Non-UTF-8 bytes on stdin must not crash the server.
|
||||
async def test_stdio_server_invalid_utf8(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Non-UTF-8 stdin bytes surface as an in-stream exception without killing the stream.
|
||||
|
||||
Invalid bytes are replaced with U+FFFD, which then fails JSON parsing and
|
||||
is delivered as an in-stream exception. Subsequent valid messages must
|
||||
still be processed.
|
||||
Invalid bytes are replaced with U+FFFD, fail JSON parsing, and arrive as an in-stream
|
||||
exception; subsequent valid messages are still processed.
|
||||
"""
|
||||
# \xff\xfe are invalid UTF-8 start bytes.
|
||||
valid = JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
@@ -92,3 +94,78 @@ async def test_stdio_server_invalid_utf8(monkeypatch: pytest.MonkeyPatch):
|
||||
second = await read_stream.receive()
|
||||
assert isinstance(second, SessionMessage)
|
||||
assert second.message == valid
|
||||
|
||||
|
||||
class _KeepOpenBytesIO(io.BytesIO):
|
||||
"""A BytesIO that survives its TextIOWrapper being closed.
|
||||
|
||||
Lets the test read what was written after `run()` has torn the wrapper down.
|
||||
"""
|
||||
|
||||
def close(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
def _run_stdio_bounded(server: MCPServer) -> None:
|
||||
"""Run the blocking `server.run("stdio")` in a daemon thread joined with a 5s bound.
|
||||
|
||||
`run()` creates its own event loop, so a sync test cannot arm `anyio.fail_after`;
|
||||
the join timeout turns a run loop that never returns on stdin EOF into a red test
|
||||
instead of a silent CI hang. An exception escaping `run()` still fails the test:
|
||||
pytest's unhandled-thread warning is escalated by `filterwarnings = ["error"]`.
|
||||
"""
|
||||
|
||||
def target() -> None:
|
||||
server.run("stdio")
|
||||
|
||||
thread = threading.Thread(target=target, daemon=True)
|
||||
thread.start()
|
||||
thread.join(5)
|
||||
assert not thread.is_alive(), 'run("stdio") did not return after stdin EOF'
|
||||
|
||||
|
||||
def test_mcpserver_run_stdio_serves_until_stdin_closes(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""`MCPServer.run("stdio")` serves over process stdio and returns at stdin EOF.
|
||||
|
||||
Answers a request over the process's stdio and returns when stdin reaches EOF,
|
||||
rather than serving forever.
|
||||
"""
|
||||
ping = JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
stdin_bytes = io.BytesIO(ping.model_dump_json(by_alias=True, exclude_none=True).encode() + b"\n")
|
||||
captured = _KeepOpenBytesIO()
|
||||
monkeypatch.setattr(sys, "stdin", TextIOWrapper(stdin_bytes, encoding="utf-8"))
|
||||
monkeypatch.setattr(sys, "stdout", TextIOWrapper(captured, encoding="utf-8"))
|
||||
|
||||
_run_stdio_bounded(MCPServer(name="RunStdioServer"))
|
||||
|
||||
response = jsonrpc_message_adapter.validate_json(captured.getvalue().decode().strip())
|
||||
assert response == JSONRPCResponse(jsonrpc="2.0", id=1, result={})
|
||||
|
||||
|
||||
def test_mcpserver_run_stdio_runs_lifespan_cleanup_after_stdin_closes(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Code after `yield` in a lifespan runs when stdin EOF ends `run("stdio")`.
|
||||
|
||||
Regression lock for the issue #1027 shutdown chain: the run loop must end on
|
||||
stdin EOF and unwind the lifespan rather than be killed before returning.
|
||||
"""
|
||||
events: list[str] = []
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(server: MCPServer) -> AsyncIterator[None]:
|
||||
events.append("setup")
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
events.append("cleanup")
|
||||
|
||||
ping = JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
stdin_bytes = io.BytesIO(ping.model_dump_json(by_alias=True, exclude_none=True).encode() + b"\n")
|
||||
captured = _KeepOpenBytesIO()
|
||||
monkeypatch.setattr(sys, "stdin", TextIOWrapper(stdin_bytes, encoding="utf-8"))
|
||||
monkeypatch.setattr(sys, "stdout", TextIOWrapper(captured, encoding="utf-8"))
|
||||
|
||||
_run_stdio_bounded(MCPServer(name="LifespanStdioServer", lifespan=lifespan))
|
||||
|
||||
assert events == ["setup", "cleanup"]
|
||||
response = jsonrpc_message_adapter.validate_json(captured.getvalue().decode().strip())
|
||||
assert response == JSONRPCResponse(jsonrpc="2.0", id=1, result={})
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
"""Windows-specific test utilities."""
|
||||
|
||||
|
||||
def escape_path_for_python(path: str) -> str:
|
||||
"""Escape a file path for use in Python code strings.
|
||||
|
||||
Converts backslashes to forward slashes which work on all platforms
|
||||
and don't need escaping in Python strings.
|
||||
"""
|
||||
return repr(path.replace("\\", "/"))
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Kernel-synchronized liveness probes for the real-subprocess stdio lifecycle suite.
|
||||
|
||||
A spawned (grand)child connects back to a test-owned TCP listener and sends
|
||||
`b'alive'`; the kernel then provides every signal a test needs, with no sleeps or
|
||||
polling. The kernel closes all of a process's file descriptors on exit, so EOF
|
||||
(clean close / FIN) or `BrokenResourceError` (abrupt close / RST, typical of
|
||||
SIGKILL and Windows job termination) proves death; only a running process can
|
||||
answer an echo, so a reply proves liveness without racing a kill.
|
||||
|
||||
Extracted from the real-process section of tests/client/test_stdio.py; the two
|
||||
copies on this branch are deliberate -- consolidating them is follow-up work.
|
||||
"""
|
||||
|
||||
import anyio
|
||||
import anyio.abc
|
||||
import pytest
|
||||
|
||||
|
||||
def connect_back_script(port: int, *, echo: bool = False) -> str:
|
||||
"""Return a `python -c` script body that connects to 127.0.0.1:`port` and sends `b'alive'`.
|
||||
|
||||
After the banner the script blocks forever -- or, with `echo=True`, echoes every
|
||||
received chunk back so `assert_peer_echoes` can prove the process still runs.
|
||||
"""
|
||||
# lax no cover: echo mode is used only by POSIX-gated tests; Windows runners enforce 100% per job.
|
||||
if echo: # pragma: lax no cover
|
||||
tail = "while True:\n data = s.recv(65536)\n if not data:\n break\n s.sendall(data)\n"
|
||||
else:
|
||||
tail = "time.sleep(3600)\n"
|
||||
return f"import socket, time\ns = socket.create_connection(('127.0.0.1', {port}))\ns.sendall(b'alive')\n" + tail
|
||||
|
||||
|
||||
async def open_liveness_listener() -> tuple[anyio.abc.SocketListener, int]:
|
||||
"""Open a TCP listener on localhost and return it along with its port."""
|
||||
multi = await anyio.create_tcp_listener(local_host="127.0.0.1")
|
||||
sock = multi.listeners[0]
|
||||
assert isinstance(sock, anyio.abc.SocketListener)
|
||||
addr = sock.extra(anyio.abc.SocketAttribute.local_address)
|
||||
# IPv4 local_address is (host: str, port: int)
|
||||
assert isinstance(addr, tuple) and len(addr) >= 2 and isinstance(addr[1], int)
|
||||
return sock, addr[1]
|
||||
|
||||
|
||||
async def accept_alive(sock: anyio.abc.SocketListener) -> anyio.abc.SocketStream:
|
||||
"""Accept one connection and assert the peer sent `b'alive'`.
|
||||
|
||||
Reads until the full 5-byte banner arrives (TCP may legally split even a tiny
|
||||
send). Callers bound this with `anyio.fail_after` to catch a subprocess that
|
||||
never started.
|
||||
"""
|
||||
stream = await sock.accept()
|
||||
msg = b""
|
||||
while len(msg) < 5:
|
||||
msg += await stream.receive(5 - len(msg))
|
||||
assert msg == b"alive", f"expected b'alive', got {msg!r}"
|
||||
return stream
|
||||
|
||||
|
||||
async def assert_stream_closed(stream: anyio.abc.SocketStream) -> None:
|
||||
"""Assert the peer holding the other end of `stream` has terminated."""
|
||||
with anyio.fail_after(5.0), pytest.raises((anyio.EndOfStream, anyio.BrokenResourceError)):
|
||||
await stream.receive(1)
|
||||
|
||||
|
||||
async def assert_peer_echoes(stream: anyio.abc.SocketStream) -> None: # pragma: lax no cover
|
||||
"""Assert the peer holding the other end of `stream` is still running.
|
||||
|
||||
Round-trips one echo through the stream (the peer must use `echo=True`); a dead
|
||||
process can never answer, so this cannot pass spuriously.
|
||||
|
||||
lax no cover: only POSIX-gated survival tests call this; Windows runners
|
||||
enforce 100% coverage per job.
|
||||
"""
|
||||
with anyio.fail_after(5.0):
|
||||
await stream.send(b"ping")
|
||||
# Read until the full echo has arrived: TCP may legally split even a tiny send.
|
||||
echoed = b""
|
||||
while len(echoed) < 4:
|
||||
echoed += await stream.receive(4 - len(echoed))
|
||||
assert echoed == b"ping", f"expected b'ping', got {echoed!r}"
|
||||
@@ -0,0 +1,77 @@
|
||||
"""Fixtures for the stdio lifecycle suite.
|
||||
|
||||
Provides recording seams around `stdio_client`'s spawn and tree-termination
|
||||
internals (the real implementations still run), plus a teardown that keeps a
|
||||
crashed test from orphaning its sleep-forever subprocesses.
|
||||
"""
|
||||
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
from collections.abc import Generator
|
||||
from contextlib import suppress
|
||||
from pathlib import Path
|
||||
from typing import TextIO
|
||||
|
||||
import anyio.abc
|
||||
import pytest
|
||||
|
||||
from mcp.client import stdio
|
||||
from mcp.client.stdio import _create_platform_compatible_process, _terminate_process_tree
|
||||
from mcp.os.win32.utilities import FallbackProcess
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def spawned_processes(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> Generator[list[anyio.abc.Process | FallbackProcess]]:
|
||||
"""Record every process `stdio_client` spawns; the real spawn still runs.
|
||||
|
||||
Teardown SIGKILLs each spawn-time process group on POSIX: the safety net for a
|
||||
test that dies mid-body and the reaper for deliberate survivors. On Windows
|
||||
there is no group to signal (the Job Object covers strays).
|
||||
"""
|
||||
spawned: list[anyio.abc.Process | FallbackProcess] = []
|
||||
|
||||
async def recording_spawn(
|
||||
command: str,
|
||||
args: list[str],
|
||||
env: dict[str, str] | None = None,
|
||||
errlog: TextIO = sys.stderr,
|
||||
cwd: Path | str | None = None,
|
||||
) -> anyio.abc.Process | FallbackProcess:
|
||||
process = await _create_platform_compatible_process(command, args, env, errlog, cwd)
|
||||
spawned.append(process)
|
||||
return process
|
||||
|
||||
monkeypatch.setattr(stdio, "_create_platform_compatible_process", recording_spawn)
|
||||
yield spawned
|
||||
_kill_spawn_groups(spawned)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def terminate_calls(monkeypatch: pytest.MonkeyPatch) -> list[anyio.abc.Process | FallbackProcess]:
|
||||
"""Record every invocation of `stdio_client`'s tree-termination seam; the real termination still runs.
|
||||
|
||||
An empty list after the context exits proves the graceful path: a FIN looks the
|
||||
same whether the peer exited on stdin closure or was killed.
|
||||
"""
|
||||
terminated: list[anyio.abc.Process | FallbackProcess] = []
|
||||
|
||||
async def recording_terminate(process: anyio.abc.Process | FallbackProcess) -> None:
|
||||
terminated.append(process)
|
||||
await _terminate_process_tree(process)
|
||||
|
||||
monkeypatch.setattr(stdio, "_terminate_process_tree", recording_terminate)
|
||||
return terminated
|
||||
|
||||
|
||||
# lax no cover: registered on every platform but a no-op on Windows, whose runners enforce 100% per job.
|
||||
def _kill_spawn_groups(spawned: list[anyio.abc.Process | FallbackProcess]) -> None: # pragma: lax no cover
|
||||
"""SIGKILL each spawn-time process group; see `spawned_processes`."""
|
||||
if sys.platform == "win32":
|
||||
return
|
||||
for process in spawned:
|
||||
# macOS killpg raises EPERM for a group holding only unreaped zombies.
|
||||
with suppress(ProcessLookupError, PermissionError):
|
||||
os.killpg(process.pid, signal.SIGKILL)
|
||||
@@ -0,0 +1,276 @@
|
||||
"""Real-subprocess stdio lifecycle tests that hold on both POSIX and Windows.
|
||||
|
||||
The `stdio_client` tests each launch a real server through the public API and pin
|
||||
one lifecycle behaviour, with kernel-level liveness sockets as the only
|
||||
synchronization; the `FallbackProcess` tests wrap a raw `subprocess.Popen`
|
||||
directly. Platform-divergent shutdown policy lives in test_posix.py /
|
||||
test_windows.py; the full protocol round trip is pinned by
|
||||
tests/interaction/transports/test_stdio.py and in-process shutdown logic by
|
||||
tests/client/test_stdio.py.
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from contextlib import AsyncExitStack
|
||||
from pathlib import Path
|
||||
|
||||
import anyio
|
||||
import anyio.abc
|
||||
import pytest
|
||||
|
||||
from mcp.client import stdio
|
||||
from mcp.client.stdio import StdioServerParameters, stdio_client
|
||||
from mcp.os.win32.utilities import FallbackProcess
|
||||
from tests.transports.stdio._liveness import (
|
||||
accept_alive,
|
||||
assert_stream_closed,
|
||||
connect_back_script,
|
||||
open_liveness_listener,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_server_that_exits_on_stdin_close_is_reaped_and_never_terminated(
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""The happy path: closing stdin alone shuts a well-behaved server down.
|
||||
|
||||
The server exits with code 0 and the escalation seam is never invoked.
|
||||
"""
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
# The server exits on its own at stdin EOF -- the well-behaved response
|
||||
# to shutdown's first step.
|
||||
server = (
|
||||
f"import socket, sys\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"sys.stdin.read()\n"
|
||||
)
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
# The bound covers one interpreter cold start on a loaded runner; a healthy
|
||||
# run takes well under a second.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(params):
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
|
||||
await assert_stream_closed(stream)
|
||||
|
||||
assert spawned_processes[0].returncode == 0
|
||||
assert terminate_calls == []
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_cancelling_the_client_mid_session_terminates_the_whole_server_tree(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""Cancellation still runs the full shutdown against a real process tree.
|
||||
|
||||
Cancellation here stands in for a client timeout or app shutdown: a server that
|
||||
ignores stdin closure is escalated against, and its child dies with it.
|
||||
"""
|
||||
monkeypatch.setattr(stdio, "PROCESS_TERMINATION_TIMEOUT", 0.2)
|
||||
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
child = connect_back_script(port)
|
||||
# The parent never reads stdin and blocks forever, so only the escalation
|
||||
# can end it -- which cancellation must not skip.
|
||||
parent = f"import subprocess, sys\nsubprocess.Popen([sys.executable, '-c', {child!r}])\n" + connect_back_script(
|
||||
port
|
||||
)
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", parent])
|
||||
|
||||
entered = anyio.Event()
|
||||
# Cancel a scope owned by the client's task, not the test's task group: a
|
||||
# host self-cancel is delivered by throwing through this test function's
|
||||
# suspended frames, and Python 3.11's tracer loses coverage events after
|
||||
# such a throw() traversal (python/cpython#106749).
|
||||
cancel_scope = anyio.CancelScope()
|
||||
|
||||
async def run_client_until_cancelled() -> None:
|
||||
with cancel_scope:
|
||||
async with stdio_client(params):
|
||||
entered.set()
|
||||
await anyio.sleep_forever()
|
||||
|
||||
streams: list[anyio.abc.SocketStream] = []
|
||||
# The bound covers two interpreter cold starts on a loaded runner plus the
|
||||
# shortened escalation wait; a healthy run takes around a second.
|
||||
with anyio.fail_after(10.0):
|
||||
async with anyio.create_task_group() as tg:
|
||||
tg.start_soon(run_client_until_cancelled)
|
||||
await entered.wait()
|
||||
for _ in range(2):
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
streams.append(stream)
|
||||
cancel_scope.cancel()
|
||||
|
||||
for stream in streams:
|
||||
await assert_stream_closed(stream)
|
||||
|
||||
assert terminate_calls == spawned_processes
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_a_server_that_exits_mid_session_keeps_its_own_exit_code(
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""A server that dies on its own mid-session is reaped with the exit code it chose.
|
||||
|
||||
The client surfaces the child's true status rather than synthesizing one, and
|
||||
the escalation seam confirms nothing was terminated along the way.
|
||||
"""
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
server = (
|
||||
f"import socket, sys\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"sys.exit(7)\n"
|
||||
)
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
# The bound covers one interpreter cold start on a loaded runner; a healthy
|
||||
# run takes well under a second.
|
||||
with anyio.fail_after(10.0):
|
||||
# no branch: coverage mis-traces the exit arcs of a nested `async with` on 3.11+.
|
||||
async with stdio_client(params): # pragma: no branch
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
# The server is already gone before shutdown begins.
|
||||
await assert_stream_closed(stream)
|
||||
|
||||
assert spawned_processes[0].returncode == 7
|
||||
assert terminate_calls == []
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_server_stderr_output_reaches_the_errlog_file(
|
||||
tmp_path: Path,
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""What the server writes to stderr lands in the file passed as `errlog`.
|
||||
|
||||
The spawn hands over errlog's file descriptor as the child's stderr, so it must
|
||||
be a real file -- an in-memory StringIO has no fileno.
|
||||
"""
|
||||
marker = "stdio-lifecycle stderr marker 4242"
|
||||
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
server = (
|
||||
f"import socket, sys\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"sys.stderr.write({marker!r} + '\\n')\n"
|
||||
f"sys.stderr.flush()\n"
|
||||
f"sys.stdin.read()\n"
|
||||
)
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
with (tmp_path / "errlog.txt").open("w+", encoding="utf-8") as errlog:
|
||||
# The bound covers one interpreter cold start on a loaded runner; a
|
||||
# healthy run takes well under a second.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(params, errlog=errlog):
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
|
||||
# The server exited on stdin EOF, so every stderr write it made has
|
||||
# reached the file descriptor.
|
||||
errlog.seek(0)
|
||||
content = errlog.read()
|
||||
|
||||
assert marker in content
|
||||
assert spawned_processes[0].returncode == 0
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
not hasattr(os, "waitid"), reason="needs os.waitid(WNOWAIT); absent on Windows and macOS before 3.13"
|
||||
)
|
||||
# lax no cover: Windows runners enforce 100% per job but lack os.waitid and skip this
|
||||
# test; test_windows.py's SelectorEventLoop lifecycle test exercises the property there.
|
||||
def test_fallback_process_reports_death_through_returncode_without_a_wait_call() -> None: # pragma: lax no cover
|
||||
"""`FallbackProcess.returncode` observes process death on its own.
|
||||
|
||||
Pre-fix it returned Popen's cached value, which stays None until someone calls wait()/poll().
|
||||
|
||||
`os.waitid(WEXITED | WNOWAIT)` waits for the child to become reapable without
|
||||
reaping it or priming Popen's cache (which would mask the regression); the
|
||||
pre-fix cached read would still see None here. stdout EOF is NOT such a signal:
|
||||
the kernel closes the pipes before the exit status is published, so an
|
||||
EOF-then-assert version flakes.
|
||||
"""
|
||||
popen = subprocess.Popen(
|
||||
[sys.executable, "-c", "pass"],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
assert popen.stdin is not None and popen.stdout is not None
|
||||
try:
|
||||
process = FallbackProcess(popen)
|
||||
|
||||
os.waitid(os.P_PID, popen.pid, os.WEXITED | os.WNOWAIT)
|
||||
assert process.returncode == 0
|
||||
finally:
|
||||
popen.stdin.close()
|
||||
popen.stdout.close()
|
||||
# The WNOWAIT above left the child unreaped; reap it so no zombie (and no
|
||||
# Popen ResourceWarning) outlives the test.
|
||||
popen.wait()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_fallback_process_wait_is_cancellable_while_the_child_lives() -> None:
|
||||
"""`FallbackProcess.wait()` honours cancellation while the child is still running.
|
||||
|
||||
Pre-fix it parked `Popen.wait()` in a worker thread anyio will not abandon,
|
||||
which blocks every cancellation aimed at it. Runs everywhere: the wrapper holds
|
||||
a plain Popen.
|
||||
"""
|
||||
popen = subprocess.Popen(
|
||||
[sys.executable, "-c", "import sys; sys.stdin.read()"],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
assert popen.stdin is not None and popen.stdout is not None
|
||||
# Pre-fix, no timeout below can fire while the worker thread is parked in
|
||||
# Popen.wait(); killing the child turns that regression's hang into a clean failure.
|
||||
watchdog = threading.Timer(8.0, popen.kill)
|
||||
watchdog.start()
|
||||
try:
|
||||
process = FallbackProcess(popen)
|
||||
|
||||
# move_on_after's short deadline is the time-based feature under test --
|
||||
# cancellability -- not a wait for an async condition.
|
||||
with anyio.fail_after(5):
|
||||
with anyio.move_on_after(0.1) as scope:
|
||||
await process.wait()
|
||||
|
||||
assert scope.cancelled_caught
|
||||
# Only the wait was cancelled; the child itself is untouched.
|
||||
assert popen.poll() is None
|
||||
finally:
|
||||
watchdog.cancel()
|
||||
popen.kill()
|
||||
popen.wait()
|
||||
popen.stdin.close()
|
||||
popen.stdout.close()
|
||||
@@ -0,0 +1,116 @@
|
||||
"""POSIX-only stdio lifecycle tests: a gracefully-exited server's children survive the client shutdown.
|
||||
|
||||
SDK-defined policy, not spec-mandated (docs/migration.md, "`stdio_client` no
|
||||
longer kills children of a gracefully-exited server on POSIX"). Windows has the
|
||||
opposite documented outcome; see tests/transports/stdio/test_windows.py.
|
||||
"""
|
||||
|
||||
import errno
|
||||
import sys
|
||||
from contextlib import suppress
|
||||
|
||||
import anyio
|
||||
import anyio.abc
|
||||
import pytest
|
||||
|
||||
from mcp.client.stdio import StdioServerParameters, stdio_client
|
||||
from mcp.os.win32.utilities import FallbackProcess
|
||||
from tests.transports.stdio._liveness import (
|
||||
accept_alive,
|
||||
assert_peer_echoes,
|
||||
connect_back_script,
|
||||
open_liveness_listener,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.skipif(sys.platform == "win32", reason="POSIX process-group semantics")
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
# lax no cover: the per-job 100% coverage gate also runs on Windows, where this file is skipped.
|
||||
async def test_a_gracefully_exiting_servers_child_survives_the_client_shutdown( # pragma: lax no cover
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""A server that exits on stdin closure keeps its background child running after `stdio_client` returns.
|
||||
|
||||
The client never escalates against the gracefully-exited server. SDK-defined
|
||||
policy per docs/migration.md; regression for the pre-fix client that
|
||||
tree-killed the child. The Windows twin in test_windows.py pins the opposite outcome.
|
||||
"""
|
||||
sock, port = await open_liveness_listener()
|
||||
async with sock:
|
||||
child = connect_back_script(port, echo=True)
|
||||
# The server hands its inherited pipes to a child, then exits as soon as
|
||||
# its stdin closes: the well-behaved graceful path.
|
||||
server = f"import subprocess, sys\nsubprocess.Popen([sys.executable, '-c', {child!r}])\nsys.stdin.read()\n"
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
# Two interpreter cold starts on a loaded runner; healthy runs take ~0.3s.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(params):
|
||||
child_stream = await accept_alive(sock)
|
||||
async with child_stream:
|
||||
# Only a live process answers an echo: the child survived shutdown.
|
||||
await assert_peer_echoes(child_stream)
|
||||
|
||||
# A FIN-shaped probe cannot tell graceful exit from a kill; the seam can:
|
||||
# no escalation was invoked, and the leader exited 0 on stdin closure.
|
||||
assert terminate_calls == []
|
||||
leader = spawned_processes[0]
|
||||
assert leader.returncode == 0
|
||||
# The child is deliberately left running; the spawned_processes teardown
|
||||
# SIGKILLs the spawn-time process group to reap it.
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
@pytest.mark.usefixtures("spawned_processes") # failure-path safety net for the parked child
|
||||
# lax no cover: same Windows-runner coverage-gate reason as above.
|
||||
async def test_a_surviving_childs_write_to_the_inherited_stdout_fails_with_epipe() -> None: # pragma: lax no cover
|
||||
"""A surviving child writing to the stdout pipe it inherited from the server gets EPIPE once the client is gone.
|
||||
|
||||
The pipe's only read end was the client's, and shutdown closed it
|
||||
deterministically rather than at GC time. Pins the docs/migration.md claim
|
||||
"a surviving child that keeps writing to an inherited stdout receives
|
||||
EPIPE/SIGPIPE once the client is gone" (SDK-defined).
|
||||
|
||||
Steps: the server hands its stdio pipes to a child and exits on stdin closure;
|
||||
the child parks on its socket until `stdio_client` has fully exited (so the
|
||||
write cannot race transport teardown), then writes one byte to its inherited
|
||||
fd 1 and reports the errno (0 on success) back over the socket.
|
||||
"""
|
||||
sock, port = await open_liveness_listener()
|
||||
async with sock:
|
||||
# Pin SIGPIPE to SIG_IGN explicitly (CPython already starts that way) so
|
||||
# the write fails with EPIPE instead of relying on interpreter startup details.
|
||||
child = (
|
||||
f"import os, signal, socket\n"
|
||||
f"signal.signal(signal.SIGPIPE, signal.SIG_IGN)\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"s.recv(4)\n"
|
||||
f"try:\n"
|
||||
f" os.write(1, b'x')\n"
|
||||
f" result = b'0'\n"
|
||||
f"except OSError as e:\n"
|
||||
f" result = str(e.errno).encode()\n"
|
||||
f"s.sendall(result)\n"
|
||||
)
|
||||
server = f"import subprocess, sys\nsubprocess.Popen([sys.executable, '-c', {child!r}])\nsys.stdin.read()\n"
|
||||
params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
# Two interpreter cold starts on a loaded runner; healthy runs take ~0.3s.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(params):
|
||||
child_stream = await accept_alive(sock)
|
||||
async with child_stream:
|
||||
# The context has fully exited: the transport, and with it the
|
||||
# pipe's only read end, is closed. Release the child's write.
|
||||
await child_stream.send(b"go")
|
||||
# The child sends its errno report and exits, so read to EOF: the
|
||||
# complete reply is everything before the kernel's FIN.
|
||||
reply = b""
|
||||
with suppress(anyio.EndOfStream):
|
||||
while True:
|
||||
reply += await child_stream.receive(16)
|
||||
|
||||
assert int(reply) == errno.EPIPE, f"child reported errno {reply!r}, expected EPIPE"
|
||||
@@ -0,0 +1,235 @@
|
||||
"""Windows-only stdio lifecycle behaviors, against real subprocesses.
|
||||
|
||||
Each test pins a contract that exists only on Windows: Job-Object reaping of a
|
||||
gracefully-exited server's children (the deliberate divergence from the POSIX
|
||||
policy in test_posix.py), the SelectorEventLoop fallback wrapper, and the CRLF
|
||||
line endings a native text-mode server emits. Synchronization is kernel-level
|
||||
only (liveness sockets); see `_liveness`.
|
||||
|
||||
Per-test no-cover pragmas (as in tests/issues/test_552_windows_hang.py): bodies run
|
||||
only on windows-latest CI legs, the per-job 100% gate would count them uncovered on
|
||||
non-Windows runners, and strict-no-cover is skipped on Windows where they execute.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
from contextlib import AsyncExitStack
|
||||
from pathlib import Path
|
||||
|
||||
import anyio
|
||||
import anyio.abc
|
||||
import pytest
|
||||
|
||||
from mcp.client.stdio import StdioServerParameters, stdio_client
|
||||
from mcp.os.win32.utilities import FallbackProcess
|
||||
from mcp.shared.message import SessionMessage
|
||||
from mcp.types import JSONRPCRequest, JSONRPCResponse
|
||||
from tests.transports.stdio._liveness import (
|
||||
accept_alive,
|
||||
assert_stream_closed,
|
||||
connect_back_script,
|
||||
open_liveness_listener,
|
||||
)
|
||||
|
||||
pytestmark = [
|
||||
pytest.mark.anyio,
|
||||
pytest.mark.skipif(sys.platform != "win32", reason="Windows Job Object / event-loop semantics"),
|
||||
]
|
||||
|
||||
|
||||
async def test_a_gracefully_exited_servers_child_is_reaped_when_the_job_handle_closes( # pragma: no cover
|
||||
tmp_path: Path,
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""A gracefully-exited server's child is killed deterministically when shutdown closes the job handle.
|
||||
|
||||
The server exits cleanly on stdin closure, leaving a child behind; shutdown's
|
||||
close of the server's Job Object handle (`close_process_job` +
|
||||
`JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`) kills that child deterministically, not at
|
||||
GC time. Documented divergence from POSIX (docs/migration.md; the POSIX twin is
|
||||
test_posix.py::test_a_gracefully_exiting_servers_child_survives_the_client_shutdown).
|
||||
|
||||
`terminate_calls == []` is the load-bearing distinction: the child died through
|
||||
the graceful path's job-handle close, not the escalation's `TerminateJobObject`;
|
||||
the two kills are indistinguishable on the socket.
|
||||
|
||||
Both processes connect back and their stderr is captured via `errlog`, so a
|
||||
timeout failure can report which process never showed and the child's fate
|
||||
(xdist swallows subprocess stderr on CI).
|
||||
"""
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
# The startup marker (and any child traceback, via stderr=sys.stderr below)
|
||||
# lands in errlog, splitting "never started" from "started but never connected".
|
||||
child = "import sys\nprint('child-started', file=sys.stderr, flush=True)\n" + connect_back_script(port)
|
||||
# The server spawns a child, connects back itself, then exits as soon as
|
||||
# its stdin closes: the graceful path, so the escalation never runs.
|
||||
# The child inherits Job membership: the SDK assigns the server to the Job
|
||||
# synchronously after spawn, long before the cold-starting interpreter can
|
||||
# Popen the child (membership is inherited at CreateProcess, never
|
||||
# acquired retroactively).
|
||||
#
|
||||
# The child's stdin must be DEVNULL: CPython startup queries fd 0, and
|
||||
# Windows serializes that query behind the server's pending blocking
|
||||
# `sys.stdin.read()` on the inherited pipe, so the child would freeze at
|
||||
# interpreter startup until the next inbound byte or EOF.
|
||||
#
|
||||
# After stdin EOF ends the server, it reports the child's `poll()` status:
|
||||
# `None` means alive at server exit; an exit/NTSTATUS code names the killer.
|
||||
server = (
|
||||
f"import socket, subprocess, sys\n"
|
||||
f"try:\n"
|
||||
f" p = subprocess.Popen([sys.executable, '-c', {child!r}], "
|
||||
f"stdin=subprocess.DEVNULL, stderr=sys.stderr)\n"
|
||||
f"except BaseException as exc:\n"
|
||||
f" print(exc, file=sys.stderr, flush=True)\n"
|
||||
f" raise\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"sys.stdin.read()\n"
|
||||
f"print('child-rc:%s' % p.poll(), file=sys.stderr, flush=True)\n"
|
||||
)
|
||||
server_params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
with (tmp_path / "errlog.txt").open("w+", encoding="utf-8") as errlog:
|
||||
|
||||
def server_stderr() -> str:
|
||||
errlog.seek(0)
|
||||
return errlog.read()
|
||||
|
||||
streams: list[anyio.abc.SocketStream] = []
|
||||
spawn_started = anyio.current_time()
|
||||
entered_at: float | None = None
|
||||
try:
|
||||
# Two interpreter cold starts on a loaded runner; healthy runs
|
||||
# take well under a second.
|
||||
with anyio.fail_after(15.0):
|
||||
async with stdio_client(server_params, errlog=errlog):
|
||||
entered_at = anyio.current_time()
|
||||
# The server and child race to connect; accept both,
|
||||
# order-agnostic (accept_alive verifies each banner).
|
||||
for _ in range(2):
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
streams.append(stream)
|
||||
except TimeoutError:
|
||||
# `stdio_client.__aexit__` has already completed its shielded shutdown,
|
||||
# so the stderr read carries the server's final `child-rc` line, not a
|
||||
# mid-flight snapshot.
|
||||
missing_leg = "the server never ran its connect line" if not streams else "the child never connected"
|
||||
spawn_split = (
|
||||
"the context never entered"
|
||||
if entered_at is None
|
||||
else f"the context entered {entered_at - spawn_started:.1f}s after spawn began"
|
||||
)
|
||||
pytest.fail(
|
||||
f"{len(streams)}/2 liveness connections arrived ({missing_leg}); "
|
||||
f"{spawn_split}; server stderr: {server_stderr()!r}"
|
||||
)
|
||||
|
||||
# Context exit closed the job handle: KILL_ON_JOB_CLOSE killed the
|
||||
# child and the server exited gracefully, so both sockets close.
|
||||
# The `spawned_processes` strong reference is load-bearing: `_process_jobs`
|
||||
# is weak-keyed, so without it a GC between context exit and this assert
|
||||
# could close the job handle itself and mask a regression in the
|
||||
# deterministic close.
|
||||
try:
|
||||
for stream in streams:
|
||||
await assert_stream_closed(stream)
|
||||
except TimeoutError:
|
||||
pytest.fail(f"a socket stayed open after shutdown; server stderr: {server_stderr()!r}")
|
||||
|
||||
leader = spawned_processes[0]
|
||||
# The graceful path: the server exited on stdin closure with code 0,
|
||||
# and the tree-termination escalation was never invoked.
|
||||
assert leader.returncode == 0, server_stderr()
|
||||
assert terminate_calls == [], server_stderr()
|
||||
|
||||
|
||||
# Overrides the suite-wide anyio_backend fixture for this test only: a selector
|
||||
# event loop cannot run asyncio subprocesses, forcing stdio_client onto FallbackProcess.
|
||||
@pytest.mark.parametrize("anyio_backend", [("asyncio", {"loop_factory": asyncio.SelectorEventLoop})])
|
||||
async def test_a_selector_event_loop_session_uses_the_fallback_process_and_exits_cleanly( # pragma: no cover
|
||||
spawned_processes: list[anyio.abc.Process | FallbackProcess],
|
||||
terminate_calls: list[anyio.abc.Process | FallbackProcess],
|
||||
) -> None:
|
||||
"""Under a `SelectorEventLoop`, `stdio_client` falls back to `FallbackProcess` and still exits cleanly.
|
||||
|
||||
A selector event loop has no asyncio subprocess support, so `stdio_client`
|
||||
falls back to the Popen-based `FallbackProcess` wrapper; a well-behaved server
|
||||
still completes the full clean lifecycle: spawn, liveness, exit on stdin
|
||||
closure, reaped, never escalated against.
|
||||
|
||||
The `isinstance` check is the engagement proof: if a future anyio gains selector
|
||||
subprocess support, the spawn would silently return a normal Process. A hang here
|
||||
most likely means the known fallback hazard documented in `stdio_client`'s
|
||||
shutdown comment (reader thread parked in a synchronous `ReadFile`), which is
|
||||
why this test pins only the clean-exit path, never a kill path.
|
||||
"""
|
||||
async with AsyncExitStack() as stack:
|
||||
sock, port = await open_liveness_listener()
|
||||
stack.push_async_callback(sock.aclose)
|
||||
|
||||
# Connect back for liveness, then exit as soon as stdin closes: the
|
||||
# well-behaved server, so shutdown's first step suffices.
|
||||
server = (
|
||||
f"import socket, sys\n"
|
||||
f"s = socket.create_connection(('127.0.0.1', {port}))\n"
|
||||
f"s.sendall(b'alive')\n"
|
||||
f"sys.stdin.read()\n"
|
||||
)
|
||||
server_params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
# One interpreter cold start on a loaded runner; healthy runs take ~0.3s.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(server_params):
|
||||
stream = await accept_alive(sock)
|
||||
stack.push_async_callback(stream.aclose)
|
||||
# The engagement proof, asserted while the session is live.
|
||||
assert isinstance(spawned_processes[0], FallbackProcess)
|
||||
|
||||
# The server exited on stdin closure: socket closed, exit code 0, and the
|
||||
# escalation never fired.
|
||||
await assert_stream_closed(stream)
|
||||
assert spawned_processes[0].returncode == 0
|
||||
assert terminate_calls == []
|
||||
|
||||
|
||||
async def test_a_native_server_emitting_crlf_line_endings_round_trips_messages() -> None: # pragma: no cover
|
||||
"""The client round-trips messages from a text-mode Windows server that frames its output with \\r\\n.
|
||||
|
||||
`TextIOWrapper`'s `newline=None` translates "\\n" to `os.linesep`, so such a
|
||||
server emits \\r\\n; the client still parses each line because the reader
|
||||
splits on "\\n" only and the JSON parser tolerates the trailing "\\r" as
|
||||
whitespace. The SDK's own server writes through such a wrapper, so this
|
||||
tolerance is load-bearing for Windows interop.
|
||||
|
||||
tests/issues/test_552_windows_hang.py exercises the same wire form implicitly
|
||||
through `initialize()`; this test is the explicit owner of the framing claim.
|
||||
"""
|
||||
# Read one request, answer it via print() (which emits \r\n on Windows), then
|
||||
# exit when stdin closes. json.loads/dumps keep the script free of SDK imports.
|
||||
server = (
|
||||
"import json, sys\n"
|
||||
"line = sys.stdin.readline()\n"
|
||||
"request = json.loads(line)\n"
|
||||
"print(json.dumps({'jsonrpc': '2.0', 'id': request['id'], 'result': {}}))\n"
|
||||
"sys.stdout.flush()\n"
|
||||
"sys.stdin.read()\n"
|
||||
)
|
||||
server_params = StdioServerParameters(command=sys.executable, args=["-c", server])
|
||||
|
||||
ping = JSONRPCRequest(jsonrpc="2.0", id=1, method="ping")
|
||||
|
||||
# One interpreter cold start on a loaded runner; healthy runs take ~0.3s.
|
||||
with anyio.fail_after(10.0):
|
||||
async with stdio_client(server_params) as (read_stream, write_stream):
|
||||
await write_stream.send(SessionMessage(ping))
|
||||
received = await read_stream.receive()
|
||||
# A reader that choked on the trailing \r would deliver a ValueError
|
||||
# here instead of a parsed message.
|
||||
assert isinstance(received, SessionMessage)
|
||||
assert received.message == JSONRPCResponse(jsonrpc="2.0", id=1, result={})
|
||||
Reference in New Issue
Block a user