fix: skip cross-turn dedup pointers on OpenAI chat streaming (#3191)

## Description Cross-turn dedup (`HEADROOM_DEDUPE` / `enable_cross_turn_dedup`, plus the cold-prefix recompaction router) folds a repeated tool-output span into a one-line in-context pointer, `[↑NL same as msg M: 'anchor']`. That pointer is only recoverable where the model can resolve the reference. On the OpenAI chat-completions STREAMING path (what `headroom wrap copilot` serves) it cannot, for two independent reasons: 1. The proxy itself logs `CCR: skipping retrieval-tool injection for OpenAI chat streaming; this path cannot intercept tool calls`, so no `headroom_retrieve` tool exists on this path and nothing can mechanically resolve a fold. 2. The pointer names its source as `msg M`, Headroom's internal message index. OpenAI-compatible chat clients never show the model numbered messages, so the reference is unresolvable even though the original bytes are technically still earlier in the same request. Observed with Kimi k2.7-code / k3 via `wrap copilot`: the model treats the pointer as deleted output, reports "the renderer is deduplicating/compressing", and retry-loops near-identical reads (one session burned ~200 turns; a folded conflicted-files listing hid 4 of 5 conflicted files and the agent committed unresolved `<<<<<<<` markers). The router already keeps unrecoverable LOSSY output verbatim (`lossy_unrecoverable_skipped`). Dedup folds are lossless in theory but unrecoverable in practice on this path; this PR gives them the same recoverability gate. Closes #3190 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - `headroom/transforms/content_router.py`: `ContentRouter.apply()` accepts a per-request `cross_turn_dedup_recoverable` kwarg (default `True`, so every existing caller is byte-identical). When `False`, the cross-turn dedup pass is skipped and repeated spans stay verbatim, mirroring the recoverability posture of the lossy `lossy_unrecoverable_skipped` guard. Config comment on `enable_cross_turn_dedup` documents the gate. - `headroom/proxy/handlers/openai.py`: `handle_openai_chat` computes the gate from the same predicate that already gates CCR retrieval-tool injection, `_should_inject_openai_chat_ccr_tool(ccr_inject_tool, stream)`, and threads it into both `openai_pipeline.apply(...)` call sites (token-mode and non-token-mode branches). Streaming chat requests skip the fold; buffered (non-streaming) chat, which can inject and redeem the retrieval tool, keeps folding. - `headroom/transforms/cold_prefix.py`: `cold_recompact_messages` no longer hardcodes pointer emission; new keyword-only `cross_turn_dedup_recoverable: bool = True` is forwarded to the router gate. The only caller (Anthropic cache-mode cold turn) keeps the default and is unchanged. - `tests/test_cross_turn_dedup.py`: router-gate regression tests (unrecoverable path keeps verbatim bytes for both the OpenAI `role:tool` string shape and the Anthropic `tool_result` block shape; default/explicit-`True` still folds). - `tests/test_cold_prefix.py` (new): recompaction folds by default (Anthropic path unchanged) and keeps verbatim bytes with `cross_turn_dedup_recoverable=False`. - `tests/test_openai_chat_dedup_recoverability.py` (new): end-to-end through the real `/v1/chat/completions` handler with `HEADROOM_DEDUPE=1`, capturing the exact upstream request body: `stream=True` keeps both copies byte-verbatim with no `[↑` pointer; `stream=False` still folds; `stream=False` under `--lossless` (which forces `ccr_inject_tool=False`) also keeps verbatim bytes, locking the intended coupling of "no retrieval tool" to "no bare pointer". ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text # BEFORE (branch base, fix reverted): the streaming regression test fails, # the upstream body carries the unresolvable pointer and drops the bytes. $ git stash push headroom/ && uv run pytest -q \ tests/test_openai_chat_dedup_recoverability.py::test_streaming_chat_keeps_verbatim_bytes_no_dedup_pointer E assert '[↑' not in "fix the ove...t merge.py']" E '[↑' is contained here: E [↑14L same as msg 2: '$ cat merge.py'] FAILED tests/test_openai_chat_dedup_recoverability.py::test_streaming_chat_keeps_verbatim_bytes_no_dedup_pointer (same run: test_cold_recompact_unrecoverable_path_keeps_verbatim_bytes also fails pre-fix; both recoverable-path legs pass before and after) # AFTER (full diff applied): $ uv run pytest tests/test_cross_turn_dedup.py tests/test_cold_prefix.py \ tests/test_openai_chat_dedup_recoverability.py \ tests/test_proxy/test_openai_chat_ccr_injection.py tests/test_no_ccr_lossy.py \ tests/test_openai_chat_turn_hooks.py tests/test_openai_chat_tool_desc_compaction.py \ tests/test_responses_cross_turn_dedup.py -q 45 passed, 2 warnings in 8.75s $ uv run pytest tests/test_proxy/ tests/test_openai_codex_routing.py \ tests/test_openai_chat_turn_hooks.py tests/test_openai_chat_tool_desc_compaction.py \ tests/test_openai_beta_session_sticky.py tests/test_openai_max_completion_tokens.py \ tests/test_no_ccr_lossy.py tests/test_netcost_gate.py tests/test_agent_savings.py \ tests/test_cross_turn_dedup.py tests/test_cold_prefix.py \ tests/test_openai_chat_dedup_recoverability.py -q 424 passed, 2 warnings in 73.52s $ uv run ruff format --check <touched files> && uv run ruff check <touched files> All checks passed! $ uv run mypy headroom/transforms/cold_prefix.py headroom/transforms/content_router.py headroom/proxy/handlers/openai.py Success: no issues found in 3 source files $ cargo fmt --all -- --check # FMT_OK $ cargo clippy --all-targets # 2 pre-existing warnings in untouched lib-test code, no errors $ cargo test # all targets green; see Additional Notes for the one environmental exception ``` ## Real Behavior Proof - Environment: macOS (Darwin), Python 3.13, repo tip `upstream/main` 5e0ce242 (v0.36.2). No secrets, no external network: the proof drives the real proxy handler in-process via FastAPI `TestClient` with the upstream send stubbed, capturing the exact request body the provider would receive. - Exact command / steps (copy-pasteable, self-contained): next lines ```sh # 1. The bug, on the branch base (pointer emitted on the streaming path): git stash push headroom/ # or check out upstream/main uv run pytest -q tests/test_openai_chat_dedup_recoverability.py # streaming leg FAILS git stash pop # 2. The fix: uv run pytest -q tests/test_openai_chat_dedup_recoverability.py # both legs pass ``` The test posts a chat-completions request whose history contains two identical multi-line tool outputs (the shape that folds), with `HEADROOM_DEDUPE=1`, and asserts on the captured upstream body: - `stream=True` (the `wrap copilot` shape): both copies forwarded byte-verbatim, no `[↑NL same as msg M]` pointer anywhere. - `stream=False` (buffered, retrieval tool injectable): the repeated span still folds to a pointer; the earliest copy stays verbatim as the in-context original. - Observed result: BEFORE, the streaming leg fails with the pointer present in the upstream body (same `transforms=router:cross_turn_dedup:N` evidence seen in proxy.log when the bug bit). AFTER, streaming keeps verbatim bytes and buffered keeps folding; the full touched-module suite (423 tests) is green. - Not tested: a live `wrap copilot` session against the real Copilot API (needs a subscription token; the in-process test captures the identical upstream body the handler produces). The Responses API path (`_dedup_responses_output_items`, Codex) is intentionally untouched: Responses streaming has a separate buffered-CCR path that can intercept tool calls. `/v1/compress` derived pipelines keep the default (recoverable) behavior. Separately worth verifying in a follow-up: whether `headroom_retrieve` resolves `msg M` dedup pointers on the paths that keep folding, or only CCR `hash=` content markers (the Anthropic-path fold is retained per the issue's scope, where it has not been observed to cause retry loops). ## Runtime Rollout Safety - Rollout-managed feature(s): none - Minimum rollout channel: N/A - Stable/default behavior changed: only the OpenAI chat-completions request path, and only when cross-turn dedup is active (opt-in `HEADROOM_DEDUPE=1`, or cold-prefix recompaction): streaming chat now keeps repeated tool-output bytes verbatim instead of emitting `[↑NL same as msg M]` pointers, and (because `--lossless` forces `ccr_inject_tool=False`) buffered chat in lossless mode does the same. Buffered chat with CCR on, Anthropic, Responses, and `/v1/compress` are byte-identical to before (default `cross_turn_dedup_recoverable=True`; the Responses fold is covered by the untouched, still-green `tests/test_responses_cross_turn_dedup.py`). - Kill switch / disable path: dedup remains opt-in via `HEADROOM_DEDUPE`; the gate itself can be overridden per request by passing `cross_turn_dedup_recoverable=True`. - Unsafe override required: no - Qualification impact: none - Rollback path: revert the single commit; no state, schema, or config migration involved. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation (docstrings + config comments) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Mirrors the existing recoverability precedent: the lossy path already refuses to emit unrecoverable output (`lossy_unrecoverable_skipped`, issue #1307); this extends the same posture to cross-turn dedup folds. - The gate reuses `_should_inject_openai_chat_ccr_tool`, the predicate that already decides whether the chat path can redeem an injected retrieval tool, so the two can never drift apart. - Prefer-false-negatives posture: a skipped fold only ever means bytes stay verbatim; no content is dropped, reordered, or lossy-transformed by this change. - Secondary operational bug noticed while diagnosing (NOT fixed here, separate issue candidate): all concurrent proxy processes write the same `~/.headroom/logs/proxy.log` with independent rotating handlers, so rotation stomps history across `wrap` instances on different ports. - Local environment note: `cargo test` on this machine hangs inside `crates/headroom-core/tests/kompress_parity.rs` (both tests stall in `ort` ONNX-runtime environment init, reproducible on the untouched branch base; this PR changes no Rust). With those two tests skipped, the full Rust suite is green (all targets `ok`, 0 failed). `cargo clippy --all-targets` and `cargo fmt --all -- --check` pass as-is.
2026-08-22 00:51:05 +02:00
parent 202c1895e1
commit 9c30b62962
6 changed files with 359 additions and 3 deletions
@@ -3511,6 +3511,19 @@ class OpenAIHandlerMixin:

        _compression_failed = False
        original_messages = messages  # Preserve for 400-retry fallback
+        # Cross-turn dedup rewrites repeated tool-output spans to bare
+        # `[↑NL same as msg M]` in-context pointers. Those are recoverable only
+        # where the model can resolve the reference; on the streaming chat path
+        # the CCR retrieval tool cannot be injected (this path cannot intercept
+        # tool calls) and OpenAI-compatible clients never show the model
+        # numbered messages, so a folded pointer reads as deleted content and
+        # models retry-loop on the "missing" output. Gate the fold on the same
+        # recoverability predicate that gates CCR tool injection: the buffered
+        # (non-streaming) chat path keeps dedup, the streaming path skips it.
+        _dedup_pointers_recoverable = _should_inject_openai_chat_ccr_tool(
+            ccr_inject_tool=self.config.ccr_inject_tool,
+            stream=stream,
+        )
        _decision = CompressionDecision.decide(
            headers=request.headers,
            config=self.config,
@@ -3565,6 +3578,7 @@ class OpenAIHandlerMixin:
                            ),
                            biases=_hook_biases,
                            compression_policy=compression_policy,
+                            cross_turn_dedup_recoverable=_dedup_pointers_recoverable,
                            # Thread the savings-profile knobs (e.g.
                            # HEADROOM_SAVINGS_PROFILE=agent-90) onto the live
                            # chat-completions path, matching handlers/
@@ -3605,6 +3619,7 @@ class OpenAIHandlerMixin:
                            frozen_message_count=apply_frozen_count,
                            biases=_hook_biases,
                            compression_policy=compression_policy,
+                            cross_turn_dedup_recoverable=_dedup_pointers_recoverable,
                            # Same savings-profile threading as the token-mode
                            # branch above — the non-token chat path must honor
                            # the configured profile too (#1534).
@@ -179,7 +179,11 @@ def has_plaintext_reasoning(messages: list[dict[str, Any]]) -> bool:


 def cold_recompact_messages(
-    messages: list[dict[str, Any]], *, tokenizer: Any, context: str = ""
+    messages: list[dict[str, Any]],
+    *,
+    tokenizer: Any,
+    context: str = "",
+    cross_turn_dedup_recoverable: bool = True,
 ) -> tuple[list[dict[str, Any]], list[str]]:
    """Lossless whole-prefix recompaction for a confirmed-cold turn.

@@ -191,6 +195,15 @@ def cold_recompact_messages(
    preserve nothing. Lossless + prefix-monotonic ⇒ deterministic per content ⇒
    the recompacted prefix re-caches and stays byte-stable on later warm turns.

+    ``cross_turn_dedup_recoverable`` is forwarded to the router's dedup gate:
+    pass False on paths where a bare ``[↑NL same as msg M]`` pointer cannot be
+    resolved — no CCR retrieval tool can be injected and the client never shows
+    the model numbered messages (OpenAI chat-completions streaming, e.g.
+    ``wrap copilot``). The fold is then skipped and the bytes stay verbatim;
+    the lossless folds still run. The Anthropic cache-mode caller keeps the
+    default True (the in-context reference resolves there and the retrieval
+    tool is injectable).
+
    Returns (new_messages, transforms_applied). Fail-open: returns the input
    unchanged on any error (never breaks the request).
    """
@@ -200,8 +213,17 @@ def cold_recompact_messages(
            ContentRouterConfig,
        )

+        # enable_cross_turn_dedup stays on: whether the fold may EMIT pointers
+        # is decided per-path by the router's recoverability gate below, not
+        # hardcoded here.
        router = ContentRouter(ContentRouterConfig(lossless=True, enable_cross_turn_dedup=True))
-        res = router.apply(list(messages), tokenizer, frozen_message_count=0, context=context)
+        res = router.apply(
+            list(messages),
+            tokenizer,
+            frozen_message_count=0,
+            context=context,
+            cross_turn_dedup_recoverable=cross_turn_dedup_recoverable,
+        )
        return res.messages, list(res.transforms_applied)
    except Exception as e:  # never break the request
        log.warning("cold-prefix recompaction failed (%s); leaving prefix unchanged", e)
@@ -1528,6 +1528,11 @@ class ContentRouterConfig:
    # Runs in both modes: lossless references verbatim/folded content; CCR mode
    # references the earlier block's kompressed-but-CCR-recoverable form
    # (deterministic content-hash → stable → still cache-safe, no added loss).
+    # Per request the fold is skipped when the caller reports the serving path
+    # cannot resolve the in-context `[↑NL same as msg M]` pointer
+    # (`apply(cross_turn_dedup_recoverable=False)`, e.g. OpenAI chat-completions
+    # streaming, where no CCR retrieval tool can be injected and clients never
+    # show the model numbered messages).
    enable_cross_turn_dedup: bool = False
    # Lossless-then-lossy. In lossy mode (not `lossless`), after a byte/data
    # lossless fold (search/log/text) run the aggressive lossy compressor
@@ -4757,6 +4762,20 @@ class ContentRouter(Transform):
        # pass a policy — ``_record_to_toin`` treats that as "no gate"
        # to preserve pre-F2.2 behaviour for non-proxy callers.
        self._runtime_compression_policy = kwargs.get("compression_policy")
+        # Cross-turn dedup recoverability gate. The fold rewrites a repeated
+        # span to a bare in-context pointer (``[↑NL same as msg M]``) that names
+        # Headroom's internal message index. That reference is only resolvable
+        # where the model can locate the original: on the OpenAI
+        # chat-completions streaming path (e.g. ``wrap copilot``) no CCR
+        # retrieval tool can be injected (the path cannot intercept tool calls)
+        # and the client never shows the model numbered messages, so the
+        # pointer reads as deleted content and the model retry-loops on the
+        # "missing" output. Same recoverability posture as the lossy
+        # ``lossy_unrecoverable_skipped`` guard: when the caller reports the
+        # path cannot resolve in-context pointers, skip the fold and keep the
+        # bytes verbatim. Default True: every path that does not opt out keeps
+        # today's behavior.
+        dedup_pointers_recoverable = bool(kwargs.get("cross_turn_dedup_recoverable", True))

        tokens_before = sum(tokenizer.count_text(str(m.get("content", ""))) for m in messages)
        context = kwargs.get("context", "")
@@ -5538,7 +5557,7 @@ class ContentRouter(Transform):
        # later duplicate would carry the same (recoverable) form anyway; dedup
        # just points to the earlier copy instead of repeating it. Frozen +
        # cache_control blocks are reference targets only (never rewritten).
-        if self._cross_turn_dedup_enabled:
+        if self._cross_turn_dedup_enabled and dedup_pointers_recoverable:
            transformed_messages = self._cross_turn_dedup_messages(
                transformed_messages, frozen_message_count, transforms_applied, route_counts
            )
@@ -0,0 +1,69 @@
+"""Cold-prefix recompaction: the cross-turn dedup fold must be path-aware.
+
+``cold_recompact_messages`` builds its own lossless ContentRouter with
+``enable_cross_turn_dedup=True``. The fold rewrites repeated tool-output spans
+to bare ``[↑NL same as msg M]`` in-context pointers — unresolvable on paths
+where no CCR retrieval tool can be injected and the client never shows the
+model numbered messages (OpenAI chat-completions streaming, wrap copilot).
+The recompaction therefore takes ``cross_turn_dedup_recoverable`` and forwards
+it to the router gate: unrecoverable paths keep the bytes verbatim, the
+Anthropic cache-mode caller (default True) keeps folding.
+"""
+
+from headroom.transforms.cold_prefix import cold_recompact_messages
+
+
+def _mk_tok():
+    from headroom.providers import OpenAIProvider
+    from headroom.tokenizer import Tokenizer
+
+    return Tokenizer(OpenAIProvider().get_token_counter("gpt-4o"), "gpt-4o")
+
+
+def _toolmsg(text, tid):
+    return {
+        "role": "user",
+        "content": [{"type": "tool_result", "tool_use_id": tid, "content": text}],
+    }
+
+
+def _conversation():
+    span = "\n".join(f"    result_{i} = compute_overdraft(business_id={i})" for i in range(12))
+    return [
+        {"role": "user", "content": "fix the overdraft bug"},
+        {"role": "assistant", "content": "cat merge.py"},
+        _toolmsg(f"$ cat merge.py\n{span}\n# end", "t1"),
+        {"role": "assistant", "content": "sed -n range"},
+        _toolmsg(f"$ sed -n 1,20p merge.py\n{span}\n# more", "t2"),
+    ]
+
+
+def test_cold_recompact_folds_by_default():
+    # Anthropic cache-mode path (the only caller today): unchanged — the
+    # repeated span still folds to an in-context pointer.
+    msgs = _conversation()
+    out, transforms = cold_recompact_messages(msgs, tokenizer=_mk_tok())
+    later = out[-1]["content"][0]["content"]
+    assert "[↑" in later
+    assert any("cross_turn_dedup" in t for t in transforms)
+
+
+def test_cold_recompact_unrecoverable_path_keeps_verbatim_bytes():
+    # Unresolvable-pointer path (OpenAI chat streaming shape): the fold is
+    # skipped, the repeated span stays byte-verbatim, and no pointer is
+    # emitted — while the recompaction itself still runs (message count and
+    # order unchanged).
+    msgs = _conversation()
+    out, transforms = cold_recompact_messages(
+        msgs, tokenizer=_mk_tok(), cross_turn_dedup_recoverable=False
+    )
+    later = out[-1]["content"][0]["content"]
+    assert "[↑" not in later
+    assert (
+        later
+        == "$ sed -n 1,20p merge.py\n"
+        + "\n".join(f"    result_{i} = compute_overdraft(business_id={i})" for i in range(12))
+        + "\n# more"
+    )
+    assert not any("cross_turn_dedup" in t for t in transforms)
+    assert len(out) == len(msgs)
@@ -399,3 +399,76 @@ def test_dedup_folds_role_function_output():
    ]
    out = _dedup_only(msgs)
    assert "[↑" in out[2]["content"]
+
+
+# --------------------------------------------------------------------------
+# Recoverability gate (unresolvable-pointer paths). The fold rewrites a
+# repeated span to a bare `[↑NL same as msg M]` pointer naming Headroom's
+# internal message index. On the OpenAI chat-completions streaming path
+# (wrap copilot) no CCR retrieval tool can be injected and the client never
+# shows the model numbered messages, so the pointer is unresolvable: the
+# model reads it as deleted content and retry-loops. `apply()` therefore
+# accepts `cross_turn_dedup_recoverable=False` — the same recoverability
+# posture as the lossy `lossy_unrecoverable_skipped` guard — and keeps the
+# repeated bytes verbatim. Default True preserves every other path.
+# --------------------------------------------------------------------------
+def _apply_with_recoverable(messages, recoverable):
+    import copy
+
+    from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
+
+    r = ContentRouter(ContentRouterConfig(lossless=True, enable_cross_turn_dedup=True))
+    return r.apply(
+        copy.deepcopy(messages), _mk_tok(), cross_turn_dedup_recoverable=recoverable
+    ).messages
+
+
+def test_apply_unrecoverable_path_keeps_verbatim_bytes():
+    # The OpenAI chat streaming shape (role:tool strings): with dedup ENABLED
+    # but the path flagged unrecoverable, the re-read must NOT fold — the
+    # request keeps the verbatim bytes, no bare pointer.
+    span = _readspan()
+    msgs = [
+        {"role": "tool", "tool_call_id": "c1", "content": f"$ cat f.py\n{span}"},
+        {"role": "assistant", "content": "again"},
+        {"role": "tool", "tool_call_id": "c2", "content": f"$ cat f.py\n{span}"},
+    ]
+    out = _apply_with_recoverable(msgs, recoverable=False)
+    assert out[2]["content"] == f"$ cat f.py\n{span}"  # verbatim, no pointer
+    assert "[↑" not in out[2]["content"]
+
+
+def test_apply_unrecoverable_gate_also_covers_tool_result_blocks():
+    # Anthropic tool_result block shape, same gate: nothing folds when the
+    # caller reports the pointer is unresolvable on this path.
+    span = _readspan()
+    msgs = [_toolmsg(f"a\n{span}", "t1"), _toolmsg(f"b\n{span}", "t2")]
+    out = _apply_with_recoverable(msgs, recoverable=False)
+    joined = "".join(b["content"] for m in out for b in m["content"] if isinstance(b, dict))
+    assert "[↑" not in joined
+    assert out[-1]["content"][0]["content"] == f"b\n{span}"  # verbatim bytes kept
+
+
+def test_apply_recoverable_default_and_true_still_fold():
+    # The recoverable paths (Anthropic, buffered/non-streaming chat — anywhere
+    # the reference resolves) keep folding: default kwarg-absent behavior is
+    # unchanged, and an explicit True folds too.
+    span = _readspan()
+    msgs = [
+        {"role": "tool", "tool_call_id": "c1", "content": f"$ cat f.py\n{span}"},
+        {"role": "assistant", "content": "again"},
+        {"role": "tool", "tool_call_id": "c2", "content": f"$ cat f.py\n{span}"},
+    ]
+    import copy
+
+    from headroom.transforms.content_router import ContentRouter, ContentRouterConfig
+
+    default_out = (
+        ContentRouter(ContentRouterConfig(lossless=True, enable_cross_turn_dedup=True))
+        .apply(copy.deepcopy(msgs), _mk_tok())
+        .messages
+    )
+    assert "[↑" in default_out[2]["content"]  # no kwarg -> still folds
+
+    true_out = _apply_with_recoverable(msgs, recoverable=True)
+    assert "[↑" in true_out[2]["content"]  # explicit recoverable -> folds
@@ -0,0 +1,158 @@
+"""OpenAI chat-completions: cross-turn dedup pointers are recoverability-gated.
+
+The fold rewrites a repeated tool-output span to a bare ``[↑NL same as msg M]``
+pointer naming Headroom's internal message index. On the STREAMING chat path
+(``wrap copilot``) the CCR retrieval tool cannot be injected — the path cannot
+intercept tool calls — and OpenAI-compatible clients never show the model
+numbered messages, so the pointer is unresolvable: models read it as deleted
+content and retry-loop. The chat handler therefore threads
+``cross_turn_dedup_recoverable=_should_inject_openai_chat_ccr_tool(...)`` into
+the router: streaming requests keep the repeated bytes verbatim, while the
+buffered (non-streaming) path — where the retrieval tool IS injectable — keeps
+folding.
+
+These tests drive the real ``/v1/chat/completions`` handler through a TestClient
+with dedup force-enabled (``HEADROOM_DEDUPE=1``) and capture the exact upstream
+request body, the same evidence the proxy logs showed when the bug bit.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+fastapi = pytest.importorskip("fastapi")
+httpx = pytest.importorskip("httpx")
+
+from fastapi.responses import StreamingResponse  # noqa: E402
+from fastapi.testclient import TestClient  # noqa: E402
+
+from headroom.proxy.server import ProxyConfig, create_app  # noqa: E402
+
+_SPAN = "\n".join(f"    result_{i} = compute_overdraft(business_id={i})" for i in range(12))
+
+
+def _messages() -> list[dict]:
+    """Two identical multi-line tool outputs — the re-read dedup folds."""
+    return [
+        {"role": "user", "content": "fix the overdraft bug"},
+        {"role": "assistant", "content": "cat merge.py"},
+        {"role": "tool", "tool_call_id": "call_1", "content": f"$ cat merge.py\n{_SPAN}\n# end"},
+        {"role": "assistant", "content": "sed -n range"},
+        {"role": "tool", "tool_call_id": "call_2", "content": f"$ cat merge.py\n{_SPAN}\n# end"},
+    ]
+
+
+def _config() -> ProxyConfig:
+    return ProxyConfig(optimize=True, cache_enabled=False, rate_limit_enabled=False)
+
+
+def _post(client: TestClient, *, stream: bool):
+    return client.post(
+        "/v1/chat/completions",
+        json={"model": "gpt-4o", "messages": _messages(), "stream": stream},
+        headers={"Authorization": "******"},
+    )
+
+
+def _sent_text(body: dict) -> str:
+    """Concatenate the upstream message contents (parsed, so newlines are real)."""
+    return "\n".join(str(m.get("content", "")) for m in body["messages"])
+
+
+def test_streaming_chat_keeps_verbatim_bytes_no_dedup_pointer(monkeypatch):
+    """The bug: a streaming chat request with a repeated span got a bare
+    ``[↑NL same as msg M]`` pointer the model cannot resolve. Now the upstream
+    body must carry the repeated bytes verbatim."""
+    monkeypatch.setenv("HEADROOM_DEDUPE", "1")  # before create_app: router reads env at init
+    captured: list[dict] = []
+
+    async def fake_stream(url, headers, body, *args, **kwargs):
+        captured.append(body)
+        return StreamingResponse(iter([b"data: {}\n\n"]), media_type="text/event-stream")
+
+    app = create_app(_config())
+    with TestClient(app) as client:
+        client.app.state.proxy._stream_response = fake_stream
+        resp = _post(client, stream=True)
+
+    assert resp.status_code == 200, resp.text
+    assert captured, "streaming upstream send was not captured"
+    sent = _sent_text(captured[0])
+    assert "[↑" not in sent  # no unresolvable pointer on the streaming path
+    assert sent.count(_SPAN) == 2  # both copies forwarded byte-verbatim
+
+
+def test_lossless_buffered_chat_also_skips_the_fold(monkeypatch):
+    """Coupling lock: --lossless forces ccr_inject_tool=False (server.py), so
+    the recoverability predicate is False for buffered chat too and the fold
+    is skipped there as well (no retrieval tool exists to redeem anything in
+    no-CCR mode). Bytes stay verbatim; the conservative direction is intended."""
+    monkeypatch.setenv("HEADROOM_DEDUPE", "1")
+    captured: list[dict] = []
+
+    async def fake_retry(method, url, headers, body, *args, **kwargs):
+        captured.append(body)
+        payload = {
+            "id": "chatcmpl-1",
+            "object": "chat.completion",
+            "model": "gpt-4o",
+            "choices": [
+                {
+                    "index": 0,
+                    "message": {"role": "assistant", "content": "done"},
+                    "finish_reason": "stop",
+                }
+            ],
+            "usage": {"prompt_tokens": 100, "completion_tokens": 5, "total_tokens": 105},
+        }
+        return httpx.Response(200, json=payload, headers={"content-type": "application/json"})
+
+    config = ProxyConfig(
+        optimize=True, lossless=True, cache_enabled=False, rate_limit_enabled=False
+    )
+    app = create_app(config)
+    with TestClient(app) as client:
+        client.app.state.proxy._retry_request = fake_retry
+        resp = _post(client, stream=False)
+
+    assert resp.status_code == 200, resp.text
+    assert captured, "buffered upstream send was not captured"
+    sent = _sent_text(captured[0])
+    assert "[↑" not in sent  # no retrieval tool in lossless mode -> no bare pointer
+    assert sent.count(_SPAN) == 2  # both copies forwarded byte-verbatim
+
+
+def test_buffered_chat_still_folds_repeated_tool_output(monkeypatch):
+    """The recoverable counterpart: non-streaming chat can inject the CCR
+    retrieval tool, so the in-context pointer stays resolvable and the
+    repeated span still folds (today's behavior, unchanged)."""
+    monkeypatch.setenv("HEADROOM_DEDUPE", "1")
+    captured: list[dict] = []
+
+    async def fake_retry(method, url, headers, body, *args, **kwargs):
+        captured.append(body)
+        payload = {
+            "id": "chatcmpl-1",
+            "object": "chat.completion",
+            "model": "gpt-4o",
+            "choices": [
+                {
+                    "index": 0,
+                    "message": {"role": "assistant", "content": "done"},
+                    "finish_reason": "stop",
+                }
+            ],
+            "usage": {"prompt_tokens": 100, "completion_tokens": 5, "total_tokens": 105},
+        }
+        return httpx.Response(200, json=payload, headers={"content-type": "application/json"})
+
+    app = create_app(_config())
+    with TestClient(app) as client:
+        client.app.state.proxy._retry_request = fake_retry
+        resp = _post(client, stream=False)
+
+    assert resp.status_code == 200, resp.text
+    assert captured, "buffered upstream send was not captured"
+    sent = _sent_text(captured[0])
+    assert "[↑" in sent  # fold still fires where the pointer resolves
+    assert sent.count(_SPAN) == 1  # earliest copy stays as the in-context original