fix(proxy): scope the signed-thinking lock to blocks that actually changed (#3124)
## Description Anthropic signs the thinking **block**, not the request — the signature covers that block's own content. #2254 responded to real 400s by freezing the **entire body** whenever any thinking block appeared anywhere in history. That protects bytes no signature covers, including top-level `tools` and `system`, which are not even inside `messages`. Measured on 227,777 lines of real proxy logs from a user reporting ~1% savings: - **618 of 1,802 requests (34.3%)** had every computed compression discarded. **100%** were `client=claude-code`; Codex/GPT traffic was untouched. - One session logged turn 1 saving 428 tokens, then **229 consecutive turns saving exactly 0**. - **491.9s — 34.2% of all optimization time** — was spent computing compressions that were then thrown away. One request paid 21.2s to compute a real 8.0% reduction that never shipped. - It orphaned the turn-1 cache prefix on **12 of 35** sessions, corroborated by Headroom's own `CACHE-MISS-ATTRIBUTION` events (21/21 are `reason=prefix_change`, **none** TTL expiry), with an exact token match: `expected_cached=27,541` equalling turn 1's write. ## Changes Made - Replace the presence test with a **positional, order-sensitive fingerprint** of every `thinking` / `redacted_thinking` block, compared against the client's original. Byte-equal blocks → forward the edits. Any difference (edited text, edited signature, dropped, reordered, moved) or any failure to prove equality → today's verbatim passthrough. Keys are sorted so a dict rebuilt in a different order is not mistaken for an edit. - `outbound_body_is_client_bytes` mirrors the relaxation exactly, or the CCR buffering probe and the forwarder would disagree and re-create #2952 in reverse. - The #2990/#3015 accounting reset now **recomputes** the lock immediately before use instead of reusing the probe taken before the CCR branch. The predicate tests block *content* now, and `enforce_cache_control_ttl_order` rewrites `body["messages"]` in between, so the early answer can go stale. (Latent before this PR; load-bearing after.) - **Perf:** parse the client body once per decision, plus a substring prescreen. A 9.3 MB body (the real production maximum) could otherwise be parsed four times per request on a stage that already carries a 30s timeout whose expiry quarantines compression process-wide. ## Rollout safety **On by default at the maintainer's explicit direction.** `HEADROOM_THINKING_PRESERVING_MUTATIONS=0` restores the previous blanket lock with no deploy. The risk is recorded in the module rather than smoothed over: #2254's stated cause — a plain canonical re-encode — cannot alter parsed values and therefore cannot by itself invalidate a signature, and that report's own log shows a transform (`tool_search_deferral`) firing on the failing turn. So the stated cause does not hold up, **but the failure was real and its true trigger was never isolated.** This relaxation is strictly narrower than what broke: it forwards edits only when every block is provably identical, which is the property the blanket rule was a crude proxy for. ## Testing ```text uv run pytest tests/test_proxy_byte_faithful_forwarding.py tests/test_ccr_buffered_stream_signed_thinking.py \ tests/test_proxy/test_anthropic_ccr_deferred_injection.py 92 passed uv run mypy headroom/proxy/body_forwarding.py headroom/proxy/handlers/anthropic.py # Success uv run ruff check . && ruff format --check . # clean ``` Existing tests that encoded the blanket lock were **re-pointed at the correct trigger, not deleted** — each now tampers with a thinking block so it still guards what it was written for. `test_signed_thinking_discarded_mutation_uses_wire_truth_for_all_accounting` (#3015) now runs under the kill switch, which proves both that the accounting neutralisation still works and that the env-var rollback is a complete restoration. ## Real behavior proof - **Setup:** macOS arm64, Python 3.12, this branch, byte-capturing transport. - **After-fix evidence** — end-to-end through `/v1/messages` with a signed thinking block in history and a compactable tool schema (`test_untouched_thinking_lets_tool_compaction_reach_the_wire`): the annotation keys the compaction strips (`$schema`, `title`) are **absent from the captured upstream bytes**, and `wire["messages"][1]["content"][0]` is **byte-identical to the client's signed block**. Under the kill switch the same request forwards the client's bytes unchanged with accounting zeroed. - **Parse-count measured, not assumed:** 7.2 MB thinking-bearing body → 2 parses became 1. 2 MB body with no thinking blocks (~2 of 3 requests) → 1 parse became **0**, i.e. faster than before this feature existed. - **Projected effect on the reporting user's traffic**, derived from their unlocked requests: Claude Code headline **2.27% → roughly 5–6%**. Their unlocked requests already achieve 5.62% overall and 7.2–7.4% in the 20K–150K band, which matches our fleet beacon (~8%); the 2.27% is a blend where 60% of tokens sat in requests that shipped nothing. - **NOT tested: live paid Anthropic traffic with a real signed thinking block.** This is the one thing that matters most and I could not do it here. The signature-verification behaviour is Anthropic's, and no local test can prove it accepts a re-serialized body carrying an untouched block. **Please validate on live traffic before relying on the default.** Watch for 400 `invalid_request_error` mentioning `thinking`, and `CACHE-MISS-ATTRIBUTION reason=prefix_change` rates. ## Known risk not eliminated Enabling this changes the wire bytes for in-flight sessions, so expect a **one-time prefix change** on the first affected turn of each live conversation. Supporting evidence that this is bounded: canonical serialization is already the norm for the ~66% of traffic without thinking blocks, and that traffic sustains a 94.3% cache hit rate. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Tejas Chopra
@@ -81,14 +81,157 @@ def has_signed_thinking_blocks(body: dict[str, Any]) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _original_body_has_signed_thinking_blocks(original_body_bytes: bytes | None) -> bool:
|
||||
#: A body carrying a ``thinking`` or ``redacted_thinking`` block contains this
|
||||
#: substring, because it appears in the block's own ``type`` value. Scanning for
|
||||
#: it is orders of magnitude cheaper than parsing, and request bodies here reach
|
||||
#: 9.3 MB in real agent traffic -- so this prescreen keeps the common case (no
|
||||
#: thinking anywhere, ~2 of every 3 requests) from paying a full JSON parse it
|
||||
#: cannot learn anything from.
|
||||
#:
|
||||
#: A false positive costs one parse we would have done anyway. A false negative
|
||||
#: is not strictly impossible -- a ``type`` value whose ASCII letters are written
|
||||
#: as JSON unicode escapes still parses to ``thinking`` while containing no
|
||||
#: literal match, and that is legal JSON no standard encoder emits
|
||||
#: (``json.dumps`` does not escape ASCII even under
|
||||
#: ``ensure_ascii=True``, and neither does any client we forward for) -- and its
|
||||
#: consequences are asymmetric in our favour: when the mutated body still holds
|
||||
#: the block, ``has_signed_thinking_blocks(body)`` sees it on the parsed dict and
|
||||
#: we lock anyway. The single reachable gap needs an escaped ``type`` in the
|
||||
#: original AND a transform that removed the block from the body, which is the
|
||||
#: rare #3015 shape crossed with an encoder nobody uses. Documented rather than
|
||||
#: defended, because closing it means re-parsing every large body to catch a case
|
||||
#: no observed client can produce.
|
||||
_THINKING_SUBSTRING = b"thinking"
|
||||
|
||||
|
||||
def _parse_original_body(original_body_bytes: bytes | None) -> dict[str, Any] | None:
|
||||
"""Parse the client's body once, or ``None`` when it cannot be used.
|
||||
|
||||
Every caller here needs the same parsed document, and a 9.3 MB body parsed
|
||||
twice per decision (and twice again in the handler's earlier probe) is real
|
||||
latency on a stage that already has a 30s timeout whose expiry quarantines
|
||||
compression process-wide. Parse in one place and pass the result down.
|
||||
|
||||
``None`` means "cannot prove anything from the original", which every caller
|
||||
must treat as the conservative answer.
|
||||
"""
|
||||
if original_body_bytes is None:
|
||||
return False
|
||||
return None
|
||||
if _THINKING_SUBSTRING not in original_body_bytes:
|
||||
return None
|
||||
try:
|
||||
original = json.loads(original_body_bytes)
|
||||
except (json.JSONDecodeError, UnicodeDecodeError, ValueError, MemoryError, RecursionError):
|
||||
return None
|
||||
return original if isinstance(original, dict) else None
|
||||
|
||||
|
||||
def _original_body_has_signed_thinking_blocks(original_body_bytes: bytes | None) -> bool:
|
||||
original = _parse_original_body(original_body_bytes)
|
||||
return original is not None and has_signed_thinking_blocks(original)
|
||||
|
||||
|
||||
#: Allow canonical re-serialization on a thinking-bearing request when every
|
||||
#: thinking block is provably unchanged. **Default ON**; set this to ``0`` (or
|
||||
#: ``false``/``no``/``off``) to restore the previous blanket lock.
|
||||
#:
|
||||
#: Risk note, recorded deliberately. The lock this relaxes was added by #2254
|
||||
#: after real upstream 400s ("`thinking` blocks ... cannot be modified"). That
|
||||
#: report attributed the failure to a plain canonical re-encode, which cannot
|
||||
#: alter parsed values and therefore cannot by itself invalidate a signature --
|
||||
#: and the report's own log shows a transform (`tool_search_deferral`) firing on
|
||||
#: the failing turn. So the stated cause does not hold up, but the failure was
|
||||
#: real and its true trigger was never isolated. This relaxation is narrower
|
||||
#: than what broke: it forwards edits ONLY when every thinking block is
|
||||
#: byte-identical to the client's, which is exactly the property #2254's blanket
|
||||
#: rule was a crude proxy for.
|
||||
#:
|
||||
#: It is on by default at the maintainer's direction, to recover the savings the
|
||||
#: lock was discarding. If Anthropic starts rejecting thinking-bearing turns,
|
||||
#: set the env var to ``0`` -- that is a single-variable, no-deploy rollback to
|
||||
#: the previous behaviour, and the 400s stop immediately.
|
||||
THINKING_PRESERVING_MUTATIONS_ENV = "HEADROOM_THINKING_PRESERVING_MUTATIONS"
|
||||
|
||||
|
||||
def thinking_preserving_mutations_enabled() -> bool:
|
||||
"""Whether to forward edits that provably left every thinking block intact.
|
||||
|
||||
Defaults to enabled. Only an explicit falsey value restores the blanket lock,
|
||||
so an unset or unparseable variable keeps the documented default rather than
|
||||
silently reverting behaviour.
|
||||
"""
|
||||
raw = os.environ.get(THINKING_PRESERVING_MUTATIONS_ENV)
|
||||
if raw is None:
|
||||
return True
|
||||
return raw.strip().lower() not in ("0", "false", "no", "off")
|
||||
|
||||
|
||||
def thinking_block_fingerprint(body: Any) -> list[tuple[int, int, str]]:
|
||||
"""Positional, order-sensitive fingerprint of every thinking block.
|
||||
|
||||
Each entry is ``(message_index, block_index, canonical_json_of_block)``, so
|
||||
the comparison catches a block whose text or ``signature`` changed, one that
|
||||
was added, removed, reordered, or moved between messages. Keys are sorted so
|
||||
a dict rebuilt in a different order is not mistaken for an edit -- the wire
|
||||
contract is over parsed values, not key order.
|
||||
"""
|
||||
out: list[tuple[int, int, str]] = []
|
||||
messages = body.get("messages") if isinstance(body, dict) else None
|
||||
if not isinstance(messages, list):
|
||||
return out
|
||||
for message_index, message in enumerate(messages):
|
||||
if not isinstance(message, dict):
|
||||
continue
|
||||
content = message.get("content")
|
||||
if not isinstance(content, list):
|
||||
continue
|
||||
for block_index, block in enumerate(content):
|
||||
if isinstance(block, dict) and block.get("type") in {
|
||||
"thinking",
|
||||
"redacted_thinking",
|
||||
}:
|
||||
out.append(
|
||||
(
|
||||
message_index,
|
||||
block_index,
|
||||
json.dumps(block, sort_keys=True, ensure_ascii=False),
|
||||
)
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def thinking_blocks_survived_mutation(
|
||||
body: dict[str, Any],
|
||||
original_body_bytes: bytes | None,
|
||||
original: dict[str, Any] | None = None,
|
||||
) -> bool:
|
||||
"""True when every thinking block is byte-equal to the one the client sent.
|
||||
|
||||
This is the whole point of the relaxation. Anthropic signs the thinking
|
||||
block, not the request: the signature covers that block's own content, so
|
||||
edits to a ``tool_result`` twenty turns back, or to the top-level ``tools``
|
||||
and ``system`` fields (which are not even inside ``messages`` and therefore
|
||||
cannot be covered by any per-block signature), leave every seal intact.
|
||||
Treating the presence of a sealed block as a reason to freeze the entire
|
||||
body is a category error -- it protects bytes the signature says nothing
|
||||
about, and on Claude Code traffic that is nearly the whole request.
|
||||
|
||||
Conservative by construction: any parse failure, or any detectable
|
||||
difference at all, returns False and the caller keeps today's passthrough.
|
||||
|
||||
Pass ``original`` when the caller has already parsed the client body, so a
|
||||
multi-megabyte document is not re-parsed once per predicate.
|
||||
"""
|
||||
if original is None:
|
||||
original = _parse_original_body(original_body_bytes)
|
||||
if original is None:
|
||||
return False
|
||||
try:
|
||||
return thinking_block_fingerprint(body) == thinking_block_fingerprint(original)
|
||||
except (TypeError, ValueError, RecursionError):
|
||||
# An unserializable block (or pathological nesting) means we cannot prove
|
||||
# the blocks are untouched, so we must not claim they are.
|
||||
return False
|
||||
return isinstance(original, dict) and has_signed_thinking_blocks(original)
|
||||
|
||||
|
||||
class BodyMutationTracker:
|
||||
@@ -133,10 +276,31 @@ def select_outbound_body(
|
||||
upstream instead of silently claiming the edit landed.
|
||||
"""
|
||||
mode = forwarder_mode if forwarder_mode is not None else get_python_forwarder_mode()
|
||||
# Parse the client body at most once for the whole decision (bodies here
|
||||
# reach 9.3 MB in real traffic; this used to cost two full parses).
|
||||
original_parsed = _parse_original_body(original_body_bytes)
|
||||
if original_body_bytes is not None and (
|
||||
has_signed_thinking_blocks(body)
|
||||
or _original_body_has_signed_thinking_blocks(original_body_bytes)
|
||||
or (original_parsed is not None and has_signed_thinking_blocks(original_parsed))
|
||||
):
|
||||
# The lock is about the SEAL, not the box. When every thinking block is
|
||||
# provably identical to the one the client sent, no signature can have
|
||||
# been invalidated, so the remaining edits (compressed tool_result text,
|
||||
# compacted tool schemas, tool-search deferral) are safe to forward.
|
||||
# Without this, one thinking block anywhere in history froze the entire
|
||||
# request for the rest of the session -- on real Claude Code traffic that
|
||||
# discarded every computed compression on 34% of requests.
|
||||
if thinking_preserving_mutations_enabled() and thinking_blocks_survived_mutation(
|
||||
body, original_body_bytes, original=original_parsed
|
||||
):
|
||||
if mode == "legacy_json_kwarg":
|
||||
content = json.dumps(body, separators=(", ", ": "), ensure_ascii=True).encode(
|
||||
"utf-8"
|
||||
)
|
||||
return OutboundBody(content=content, source="legacy")
|
||||
if body_mutated:
|
||||
return OutboundBody(content=serialize_body_canonical(body), source="canonical")
|
||||
return OutboundBody(content=original_body_bytes, source="passthrough")
|
||||
return OutboundBody(
|
||||
content=original_body_bytes,
|
||||
source="passthrough",
|
||||
@@ -190,10 +354,28 @@ def outbound_body_is_client_bytes(
|
||||
send the client's bytes verbatim. Asking before acting is cheaper than
|
||||
discovering it from a reply in the wrong wire format.
|
||||
|
||||
Mirrors the first branch of :func:`select_outbound_body`; the forwarder mode
|
||||
is deliberately not consulted because that branch overrides it too.
|
||||
Mirrors the first branch of :func:`select_outbound_body`, INCLUDING the
|
||||
thinking-preserving relaxation. These two must agree exactly: the caller
|
||||
flips ``stream`` to False to buy a buffered reply, and that flip only lands
|
||||
if the mutated body actually reaches the wire. If this said "locked" while
|
||||
``select_outbound_body`` shipped canonical bytes, we would ask upstream for a
|
||||
stream and then parse the reply as buffered JSON -- the 200-with-empty-body
|
||||
failure from #2952, in reverse.
|
||||
|
||||
The forwarder mode is deliberately not consulted: the lock branch overrides
|
||||
it, and the relaxation only ever returns bytes derived from ``body``, so
|
||||
either way the caller's edits are on the wire.
|
||||
"""
|
||||
return original_body_bytes is not None and (
|
||||
if original_body_bytes is None:
|
||||
return False
|
||||
original_parsed = _parse_original_body(original_body_bytes)
|
||||
if not (
|
||||
has_signed_thinking_blocks(body)
|
||||
or _original_body_has_signed_thinking_blocks(original_body_bytes)
|
||||
)
|
||||
or (original_parsed is not None and has_signed_thinking_blocks(original_parsed))
|
||||
):
|
||||
return False
|
||||
if thinking_preserving_mutations_enabled() and thinking_blocks_survived_mutation(
|
||||
body, original_body_bytes, original=original_parsed
|
||||
):
|
||||
return False
|
||||
return True
|
||||
|
||||
@@ -3567,7 +3567,23 @@ class AnthropicHandlerMixin:
|
||||
# edits that will not be sent (#2990). This covers PERF, /stats,
|
||||
# durable savings, response headers, pipeline events, and the
|
||||
# prefix tracker rather than fixing only one reporting surface.
|
||||
if outbound_locked_to_client_bytes and body_mutation_tracker.mutated:
|
||||
#
|
||||
# RECOMPUTE rather than reusing the probe from before the CCR
|
||||
# branch. That probe answers "will my stream flip survive?" and
|
||||
# has to be asked early; this asks "what are we actually about to
|
||||
# bill for?" and must be asked last. The two diverge now that the
|
||||
# predicate tests thinking-block CONTENT and not merely presence:
|
||||
# `enforce_cache_control_ttl_order` immediately above rewrites
|
||||
# `body["messages"]`, so a marker moved onto or off a message
|
||||
# holding a thinking block changes the answer after the early
|
||||
# probe was taken. Evaluating the same predicate on the same
|
||||
# final `body` that `select_outbound_body` will see is what keeps
|
||||
# the accounting and the wire in agreement.
|
||||
final_locked_to_client_bytes = outbound_body_is_client_bytes(
|
||||
body=body,
|
||||
original_body_bytes=original_body_bytes,
|
||||
)
|
||||
if final_locked_to_client_bytes and body_mutation_tracker.mutated:
|
||||
discarded_reasons = body_mutation_tracker.reasons
|
||||
try:
|
||||
wire_body = json.loads(original_body_bytes or b"")
|
||||
|
||||
@@ -1857,7 +1857,11 @@ class ContentRouter(Transform):
|
||||
).strip().lower() in ("1", "true", "yes", "on")
|
||||
self._text_crusher: Any = None
|
||||
# Cross-turn dedup: config field OR env HEADROOM_DEDUPE (robust to how the
|
||||
# config was built). Effective only in lossless mode (guarded in apply()).
|
||||
# config was built). Runs in BOTH modes — the call site in ``apply()`` has
|
||||
# no lossless guard, and ``_cross_turn_dedup_messages`` documents working
|
||||
# against lossless folds and CCR-recoverable forms alike. (This comment
|
||||
# previously claimed "lossless mode only", which reads as "inert in your
|
||||
# config" to anyone auditing why dedup never fired.)
|
||||
self._cross_turn_dedup_enabled: bool = (
|
||||
self.config.enable_cross_turn_dedup
|
||||
or os.environ.get("HEADROOM_DEDUPE", "").strip().lower() in ("1", "true", "yes", "on")
|
||||
|
||||
@@ -108,13 +108,30 @@ def _headers() -> dict[str, str]:
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("with_thinking", "expect_plain_streaming"),
|
||||
[(True, True), (False, False)],
|
||||
("with_thinking", "relaxation_enabled", "expect_plain_streaming"),
|
||||
[
|
||||
# Locked (kill switch engaged): the flip cannot reach upstream, so
|
||||
# buffering would ask for a stream and then parse it as JSON, stranding
|
||||
# the client with an unreadable 200. This is #2952 exactly.
|
||||
(True, False, True),
|
||||
# Relaxed (default): no transform touched the thinking block, so the
|
||||
# flip DOES land and buffered retrieval becomes the coherent choice --
|
||||
# the outcome #2952 wanted before the blanket lock made it unreachable.
|
||||
(True, True, False),
|
||||
# No thinking block: unaffected in either regime.
|
||||
(False, True, False),
|
||||
(False, False, False),
|
||||
],
|
||||
)
|
||||
def test_signed_thinking_history_skips_the_buffered_ccr_path(
|
||||
with_thinking: bool, expect_plain_streaming: bool, ccr_marker: str
|
||||
with_thinking: bool,
|
||||
relaxation_enabled: bool,
|
||||
expect_plain_streaming: bool,
|
||||
ccr_marker: str,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""The buffered path is only chosen when the stream:false flip can land."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1" if relaxation_enabled else "0")
|
||||
calls: dict[str, object] = {}
|
||||
|
||||
async def fake_stream_response(url, headers, body, *args, **kwargs): # noqa: ANN001
|
||||
|
||||
@@ -38,6 +38,7 @@ from headroom.proxy.body_forwarding import (
|
||||
prepare_outbound_body_bytes,
|
||||
select_outbound_body,
|
||||
serialize_body_canonical,
|
||||
thinking_blocks_survived_mutation,
|
||||
)
|
||||
from headroom.proxy.helpers import (
|
||||
_reset_session_beta_tracker_for_test,
|
||||
@@ -196,6 +197,11 @@ def test_signed_thinking_history_with_original_bytes_uses_passthrough(
|
||||
],
|
||||
}
|
||||
original = json.dumps(body, indent=2).encode("utf-8")
|
||||
# The lock triggers on a MODIFIED thinking block, not merely a present one:
|
||||
# the signature covers the block's own content, so a body whose blocks are
|
||||
# untouched has no seal to break. Tamper with the block so this test still
|
||||
# exercises the guard it was written for.
|
||||
body["messages"][1]["content"][0]["thinking"] = "rewritten by a transform"
|
||||
|
||||
outbound = select_outbound_body(
|
||||
body=body,
|
||||
@@ -245,6 +251,11 @@ def test_signed_thinking_history_overrides_legacy_encoder() -> None:
|
||||
]
|
||||
}
|
||||
original = json.dumps(body, indent=2).encode("utf-8")
|
||||
# Modified block => the lock engages and still outranks the legacy encoder.
|
||||
# (An UNmodified block no longer overrides legacy mode: with every seal
|
||||
# provably intact there is nothing for the override to protect, and honoring
|
||||
# the operator's explicit rollback request is the more useful behaviour.)
|
||||
body["messages"][0]["content"][0]["signature"] = "tampered"
|
||||
|
||||
outbound = select_outbound_body(
|
||||
body=body,
|
||||
@@ -268,6 +279,12 @@ def test_signed_thinking_passthrough_reports_the_mutations_it_discarded() -> Non
|
||||
],
|
||||
}
|
||||
original = json.dumps({**body, "stream": True}).encode("utf-8")
|
||||
# A thinking block the transforms did NOT touch no longer forces
|
||||
# passthrough, so the CCR stream flip in this very scenario now reaches
|
||||
# upstream — which is the outcome #2952 wanted in the first place. The
|
||||
# discard-reporting path still has to work when a block really was edited,
|
||||
# so tamper with it here and keep guarding that.
|
||||
body["messages"][0]["content"][0]["signature"] = "rewritten"
|
||||
|
||||
outbound = select_outbound_body(
|
||||
body=body,
|
||||
@@ -591,7 +608,15 @@ def _make_no_optimize_app() -> tuple[TestClient, _CapturingTransport]:
|
||||
return _make_anthropic_app(optimize=False)
|
||||
|
||||
|
||||
def test_signed_thinking_discarded_mutation_uses_wire_truth_for_all_accounting() -> None:
|
||||
def test_signed_thinking_discarded_mutation_uses_wire_truth_for_all_accounting(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
# Exercised with the relaxation DISABLED, which is both the documented
|
||||
# rollback and the pre-existing behaviour. Keeping #3015's wire-truth
|
||||
# assertions under the kill switch proves two things at once: the accounting
|
||||
# neutralisation still works whenever the lock does engage, and the env-var
|
||||
# rollback really is a complete restoration rather than a partial one.
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "0")
|
||||
config = ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
@@ -673,6 +698,89 @@ def test_signed_thinking_discarded_mutation_uses_wire_truth_for_all_accounting()
|
||||
assert tracker._last_forwarded_messages[: len(inbound["messages"])] == inbound["messages"]
|
||||
|
||||
|
||||
def test_untouched_thinking_lets_tool_compaction_reach_the_wire(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""End-to-end counterpart: the same request, with the relaxation active.
|
||||
|
||||
This is the whole point of the change. Tool schemas are a top-level field --
|
||||
not inside ``messages`` at all, so no per-block thinking signature can
|
||||
possibly cover them -- yet the blanket lock discarded their compaction on
|
||||
every turn of a thinking-bearing session. Here the compaction must reach
|
||||
upstream AND be credited, while the thinking block goes out untouched.
|
||||
"""
|
||||
monkeypatch.delenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", raising=False)
|
||||
config = ProxyConfig(
|
||||
optimize=False,
|
||||
cache_enabled=False,
|
||||
rate_limit_enabled=False,
|
||||
cost_tracking_enabled=False,
|
||||
log_requests=False,
|
||||
ccr_inject_tool=False,
|
||||
ccr_handle_responses=False,
|
||||
ccr_context_tracking=False,
|
||||
image_optimize=False,
|
||||
)
|
||||
app = create_app(config)
|
||||
proxy = app.state.proxy
|
||||
transport = _CapturingTransport()
|
||||
proxy.http_client = httpx.AsyncClient(transport=transport)
|
||||
proxy._record_request_outcome = AsyncMock(wraps=proxy._record_request_outcome)
|
||||
|
||||
tracker = _FakePrefixTracker(frozen_count=0)
|
||||
proxy.session_tracker_store.compute_session_id = lambda request, model, messages: "signed"
|
||||
proxy.session_tracker_store.get_or_create = lambda session_id, provider: tracker
|
||||
|
||||
signed_block = {"type": "thinking", "thinking": "private", "signature": "sig123"}
|
||||
inbound = {
|
||||
"model": "claude-opus-5",
|
||||
"max_tokens": 64,
|
||||
"messages": [
|
||||
{"role": "user", "content": "Solve this."},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [dict(signed_block), {"type": "text", "text": "Working."}],
|
||||
},
|
||||
{"role": "user", "content": "Continue."},
|
||||
],
|
||||
"tools": [
|
||||
{
|
||||
"name": "lookup",
|
||||
"description": " Look up a value. ",
|
||||
"input_schema": {
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"title": "LookupArgs",
|
||||
"type": "object",
|
||||
"properties": {"q": {"type": "string", "title": "Q"}},
|
||||
},
|
||||
}
|
||||
],
|
||||
}
|
||||
inbound_bytes = json.dumps(inbound, indent=2).encode()
|
||||
|
||||
response = TestClient(app).post(
|
||||
"/v1/messages",
|
||||
headers={
|
||||
"x-api-key": "test-key",
|
||||
"anthropic-version": "2023-06-01",
|
||||
"content-type": "application/json",
|
||||
},
|
||||
content=inbound_bytes,
|
||||
)
|
||||
|
||||
assert response.status_code == 200
|
||||
# The edit shipped: the annotation keys the compaction strips are gone.
|
||||
assert transport.captured_body != inbound_bytes
|
||||
wire = json.loads(transport.captured_body)
|
||||
assert "$schema" not in wire["tools"][0]["input_schema"]
|
||||
assert "title" not in wire["tools"][0]["input_schema"]
|
||||
# ...and the seal went out byte-identical, which is what makes it safe.
|
||||
assert wire["messages"][1]["content"][0] == signed_block
|
||||
|
||||
outcome = proxy._record_request_outcome.await_args.args[0]
|
||||
assert "wire_mutations_discarded" not in outcome.tags
|
||||
|
||||
|
||||
def _openai_responses_body_bytes(*, stream: bool) -> bytes:
|
||||
payload = {
|
||||
"model": "gpt-5.5",
|
||||
@@ -1608,3 +1716,180 @@ def test_ws_http_fallback_uses_canonical_serializer() -> None:
|
||||
assert b"\\u" not in out
|
||||
# Round-trip equality via JSON parse.
|
||||
assert json.loads(out.decode("utf-8")) == body
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thinking-preserving mutations (relaxing the blanket signed-thinking byte-lock)
|
||||
#
|
||||
# Anthropic signs the thinking BLOCK, not the request. The signature covers that
|
||||
# block's own content, so edits elsewhere -- a compressed ``tool_result`` twenty
|
||||
# turns back, a compacted ``tools`` array that is not even inside ``messages``
|
||||
# -- cannot invalidate it. The original lock (#2254) froze the entire body
|
||||
# whenever any thinking block was present, which on Claude Code traffic meant
|
||||
# every computed compression was discarded from turn 2 of a session onward.
|
||||
#
|
||||
# The relaxation ships dark behind ``HEADROOM_THINKING_PRESERVING_MUTATIONS``
|
||||
# and only engages when every thinking block is provably byte-equal to the one
|
||||
# the client sent. These tests pin both directions: what must now ship, and what
|
||||
# must still lock.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_TB_SIGNED_BLOCK = {
|
||||
"type": "thinking",
|
||||
"thinking": "Let me think about this… café",
|
||||
"signature": "EuYBCkQYBCKMAQ==",
|
||||
}
|
||||
|
||||
|
||||
def _tb_body(tool_text: str = "LOG LINE\n" * 500) -> dict:
|
||||
"""Claude-Code-shaped turn: signed thinking in history + a fat tool_result."""
|
||||
return {
|
||||
"model": "claude-sonnet-5",
|
||||
"tools": [{"name": "Bash", "description": "x" * 400, "input_schema": {"type": "object"}}],
|
||||
"messages": [
|
||||
{"role": "user", "content": [{"type": "text", "text": "hi"}]},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": [dict(_TB_SIGNED_BLOCK), {"type": "text", "text": "ok"}],
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "tool_result",
|
||||
"tool_use_id": "t1",
|
||||
"content": [{"type": "text", "text": tool_text}],
|
||||
}
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _tb_select(mutated: dict, original_bytes: bytes):
|
||||
return select_outbound_body(
|
||||
body=mutated,
|
||||
original_body_bytes=original_bytes,
|
||||
body_mutated=True,
|
||||
forwarder_mode="byte_faithful",
|
||||
mutation_reasons=["content_router", "anthropic:tool_schema_compaction"],
|
||||
)
|
||||
|
||||
|
||||
def test_thinking_preserving_mutation_ships_compression(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""Untouched thinking block => the rest of the body may be re-serialized."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1")
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
mutated = json.loads(original)
|
||||
mutated["messages"][2]["content"][0]["content"][0]["text"] = "[compressed]"
|
||||
|
||||
outbound = _tb_select(mutated, original)
|
||||
|
||||
assert outbound.source == "canonical"
|
||||
assert outbound.dropped_mutations is False
|
||||
assert b"[compressed]" in outbound.content
|
||||
# The seal itself must survive verbatim, or we have merely moved the bug.
|
||||
assert _TB_SIGNED_BLOCK["signature"].encode() in outbound.content
|
||||
assert json.loads(outbound.content)["messages"][1]["content"][0] == _TB_SIGNED_BLOCK
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("label", "tamper"),
|
||||
[
|
||||
("edited_text", lambda b: b["messages"][1]["content"][0].__setitem__("thinking", "x")),
|
||||
(
|
||||
"edited_signature",
|
||||
lambda b: b["messages"][1]["content"][0].__setitem__("signature", "x"),
|
||||
),
|
||||
("dropped_block", lambda b: b["messages"][1]["content"].__delitem__(0)),
|
||||
("reordered_blocks", lambda b: b["messages"][1]["content"].reverse()),
|
||||
],
|
||||
)
|
||||
def test_touching_a_thinking_block_still_locks(
|
||||
monkeypatch: pytest.MonkeyPatch, label: str, tamper
|
||||
) -> None:
|
||||
"""Any detectable change to a thinking block keeps today's verbatim passthrough."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1")
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
mutated = json.loads(original)
|
||||
tamper(mutated)
|
||||
|
||||
outbound = _tb_select(mutated, original)
|
||||
|
||||
assert outbound.source == "passthrough", label
|
||||
assert outbound.dropped_mutations is True
|
||||
assert outbound.content == original
|
||||
|
||||
|
||||
def test_thinking_relaxation_is_on_by_default(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""Unset flag => relaxation active (maintainer chose on-by-default)."""
|
||||
monkeypatch.delenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", raising=False)
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
mutated = json.loads(original)
|
||||
mutated["messages"][2]["content"][0]["content"][0]["text"] = "[compressed]"
|
||||
|
||||
outbound = _tb_select(mutated, original)
|
||||
|
||||
assert outbound.source == "canonical"
|
||||
assert outbound.dropped_mutations is False
|
||||
|
||||
|
||||
@pytest.mark.parametrize("off_value", ["0", "false", "no", "off", "OFF"])
|
||||
def test_kill_switch_restores_the_blanket_lock(
|
||||
monkeypatch: pytest.MonkeyPatch, off_value: str
|
||||
) -> None:
|
||||
"""The documented rollback must work without a deploy, on every falsey spelling."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", off_value)
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
mutated = json.loads(original)
|
||||
mutated["messages"][2]["content"][0]["content"][0]["text"] = "[compressed]"
|
||||
|
||||
outbound = _tb_select(mutated, original)
|
||||
|
||||
assert outbound.source == "passthrough", off_value
|
||||
assert outbound.dropped_mutations is True
|
||||
assert outbound.content == original
|
||||
|
||||
|
||||
def test_thinking_block_key_reorder_is_not_an_edit(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The contract is over parsed values, so dict key order must not matter."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1")
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
mutated = json.loads(original)
|
||||
block = mutated["messages"][1]["content"][0]
|
||||
mutated["messages"][1]["content"][0] = {
|
||||
"signature": block["signature"],
|
||||
"thinking": block["thinking"],
|
||||
"type": block["type"],
|
||||
}
|
||||
|
||||
assert thinking_blocks_survived_mutation(mutated, original) is True
|
||||
|
||||
|
||||
def test_is_client_bytes_agrees_with_selection(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
"""The CCR buffering probe must never disagree with the forwarder (#2952)."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1")
|
||||
original = json.dumps(_tb_body()).encode()
|
||||
|
||||
preserved = json.loads(original)
|
||||
preserved["messages"][2]["content"][0]["content"][0]["text"] = "[compressed]"
|
||||
tampered = json.loads(original)
|
||||
tampered["messages"][1]["content"][0]["thinking"] = "tampered"
|
||||
|
||||
for mutated in (preserved, tampered):
|
||||
probe_says_locked = outbound_body_is_client_bytes(
|
||||
body=mutated, original_body_bytes=original
|
||||
)
|
||||
forwarder_locked = _tb_select(mutated, original).source == "passthrough"
|
||||
assert probe_says_locked is forwarder_locked
|
||||
|
||||
|
||||
def test_unparseable_original_cannot_prove_preservation(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""No proof => no relaxation. Fail closed."""
|
||||
monkeypatch.setenv("HEADROOM_THINKING_PRESERVING_MUTATIONS", "1")
|
||||
assert thinking_blocks_survived_mutation(_tb_body(), b"{not json") is False
|
||||
assert thinking_blocks_survived_mutation(_tb_body(), None) is False
|
||||
|
||||
Reference in New Issue
Block a user