fix(cli): stop the macOS malloc re-exec replacing an embedder's process (#3064)

## Description

**`main` cannot currently run its own test suite on macOS.** `pytest
tests/` dies at roughly 2% with exit code 2 — no traceback, no summary,
no failing test named. The pytest process is simply gone.

Two independent defects, both landed today, both invisible to CI.

### 1. The macOS malloc re-exec replaces the calling process

`headroom proxy` re-execs itself once on Darwin to apply two libmalloc
knobs that libmalloc only reads before `main()` (#2820, PR #2879):

```python
os.execv(sys.executable, [sys.executable, "-m", "headroom.cli", *sys.argv[1:]])
```

That reconstruction is only faithful when the process really *is* the
Headroom CLI. Ten-plus test files invoke the `proxy` command in-process
through Click's `CliRunner`. There, `os.execv` replaces **pytest** with
a Headroom process holding pytest's argv. Run with `-s`, the mechanism
is visible:

```
tests/test_agent_savings.py Usage: python -m headroom.cli [OPTIONS] COMMAND [ARGS]...
Error: No such command 'tests/test_agent_savings.py::test_proxy_cli_reads_agent_90_profile_env'.
```

Everything after the first such test — roughly 98% of the suite — never
runs. The same hazard applies to any application embedding the CLI.

**The documented kill switch does not help.** `tests/conftest.py:41`
scrubs every `HEADROOM_*` variable for hermeticity, so
`HEADROOM_MALLOC_TUNING` is deleted before the guard reads it. Only the
private `_HEADROOM_MALLOC_TUNED` survives, because it starts with an
underscore.

**CI could not have caught this.** The tuning is Darwin-only, and while
the repo *does* have macOS jobs (`macos-native-wrapper`, `wrap-native
(macos-latest)`), neither runs the Python test suite — the `test` shards
are `ubuntu-latest` only. So `sys.platform != "darwin"` returns first
everywhere pytest actually runs. #2879 merged with 37 green checks.

### 2. A semantic merge conflict between two green PRs

#3051 added `bind_scope(tags, request.scope)` at `gemini.py:325` and
updated the three Gemini fakes it knew about. #3035 branched earlier and
added a fourth `_FakeRequest` without `.scope`. Each was green against
its own base; together they fail:

```
AttributeError: '_FakeRequest' object has no attribute 'scope'
```

Git merged both cleanly. Only running the suite on merged `main`
surfaces it.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Added `_process_is_headroom_cli_entrypoint()`: the re-exec now
verifies its own precondition — `argv[0]` must be the `headroom` console
script or `headroom/cli/__main__.py`.
- The embedded path returns **before** stamping
`_HEADROOM_MALLOC_TUNED`, so a genuine CLI child inheriting the
environment can still apply the tuning.
- Gave the Gemini `_FakeRequest` the `.scope` every real Starlette
`Request` carries.
- `test_reexec_skips_when_operator_already_set_vars` now sets a
realistic `argv[0]`, matching its sibling exec test.
- New `tests/test_cli_proxy_malloc_reexec_guard.py` asserting the
guard's logic on **every** platform, since no CI runner is macOS.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes (`uv run mypy headroom`) — not run
- [x] New tests added for new functionality

### Test Output

Before, on `main`:

```text
$ .venv/bin/python -m pytest tests/ -q
collected 11622 items / 8 skipped
... tests/test_agent_savings.py ............................
$ echo $?
2
```

No summary line — the run does not end, it is replaced.

After, on this branch:

```text
$ .venv/bin/python -m pytest tests/ -q
3 failed, 11055 passed, 581 skipped, 6034 warnings in 303.69s (0:05:03)
```

All three remaining failures reproduce at `f9807fd6`, before today's
merges, and are unrelated:

| test | cause |
|---|---|
|
`test_graceful_shutdown::test_run_server_installs_cancelled_error_filter`
| full-suite ordering; passes in isolation (11 passed) |
|
`test_learn/test_integration::TestCodexIntegration::test_full_pipeline`
| pre-existing |
| `test_release_workflows::test_no_native_tls_in_wheel_build_tree` |
requires `cargo`, absent on this host |

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13, arm64, real
checkout of `main` at `ef7e07e0`.
- Exact command / steps: bisected the crash to a single test, then to a
single commit — `be5b26d8` (parent) exits 0, `6d87825f` (#2879) exits 2.
Confirmed causation by temporarily replacing the `os.execv` line with
`return`, which makes the test pass. Recovered the mechanism by running
the crashing test with `-s`, which prints the Headroom CLI rejecting
pytest's own argv.
- Observed result: on `main` the suite cannot reach a summary; on this
branch it completes with 11,055 passing. The two-file reproduction
(`test_agent_savings.py` + `test_anthropic_beta_session_sticky.py`) goes
from exit 2 to 62 passed.
- Not tested: a real `headroom proxy` launch on macOS confirming
libmalloc still receives the knobs after re-exec. The guard is covered
by unit tests asserting `execv` is still called with `["-m",
"headroom.cli", "proxy", "--port", "8787"]` for a console-script
`argv[0]`, but I have not watched `vmmap` on a live proxy. **A macOS
maintainer should confirm #2820's RSS fix still works end to end before
this ships.**

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: no for a real CLI launch; the re-exec
no longer fires when the CLI is invoked in-process, which was never
intended to work.
- Kill switch / disable path: `HEADROOM_MALLOC_TUNING=0` still disables
the tuning outright.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit — but that restores a `main` whose
test suite cannot run on macOS.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

**This is my fault and worth recording.** I merged both #2879 and #3035
earlier today on the rule "approved + green CI". Both were genuinely
approved and genuinely green. Neither was rebased onto current `main`
first, and CI has no macOS runner, so green meant less than it appeared
to.

Two process gaps this exposes, neither of which this PR fixes:

1. **The Python test suite never runs on macOS.** The repo has macOS
jobs (`macos-native-wrapper`, `wrap-native (macos-latest)`), but the
`test` shards are `ubuntu-latest` only, so Darwin-only code paths — the
allocator tuning is one, `wrap` has others — are unreachable by pytest
in CI. Even a reduced macOS shard would have caught this.
2. **Nothing requires a PR to be current with `main` before merging.**
Both defects here are cross-PR interactions that no per-PR check can
see. Enabling "require branches to be up to date before merging" on
`main` would have forced a rebase and surfaced the Gemini fake.

I would suggest an issue for each rather than folding them in here.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>

headroom/cli/proxy.py

@@ -5,6 +5,7 @@ import os
 import sys
 import warnings
 from importlib import import_module
+from pathlib import Path
 from typing import Any, Literal, cast
 
 import click
@@ -125,6 +126,24 @@ _MALLOC_TUNING = {
 }
 
 
+def _process_is_headroom_cli_entrypoint() -> bool:
+    """Is this process the Headroom CLI itself, rather than an embedder?
+
+    ``_reexec_with_malloc_tuning`` rebuilds the command line as
+    ``python -m headroom.cli <argv[1:]>``. That is only a faithful
+    reconstruction when the process really was started as the Headroom CLI. If
+    something else invoked the ``proxy`` command in-process — pytest's
+    ``CliRunner``, an embedding application, ``runpy`` — then ``argv[1:]``
+    belongs to *that* program, and ``os.execv`` would replace it with a Headroom
+    process parsing arguments that were never meant for us.
+    """
+    argv0 = Path(sys.argv[0] or "")
+    if argv0.name in {"headroom", "headroom.exe"}:
+        return True
+    # `python -m headroom.cli` sets argv[0] to .../headroom/cli/__main__.py.
+    return argv0.parts[-3:] == ("headroom", "cli", "__main__.py")
+
+
 def _reexec_with_malloc_tuning() -> None:
     if sys.platform != "darwin":
         return
@@ -132,6 +151,8 @@ def _reexec_with_malloc_tuning() -> None:
         return
     if os.environ.get("_HEADROOM_MALLOC_TUNED") == "1":
         return
+    if not _process_is_headroom_cli_entrypoint():
+        return
     missing = {k: v for k, v in _MALLOC_TUNING.items() if k not in os.environ}
     # Set the loop guard before the re-exec so the replacement process (which
     # inherits this environment) skips this path instead of re-execing forever.

tests/test_cli_proxy_malloc_reexec_guard.py

@@ -0,0 +1,103 @@
+"""The macOS malloc re-exec must never replace an embedder's process.
+
+``headroom proxy`` re-execs itself once on Darwin to apply two libmalloc knobs
+that libmalloc only reads before ``main()`` (#2820). The re-exec rebuilds the
+command as ``python -m headroom.cli <argv[1:]>``, which is only a faithful
+reconstruction when this process really is the Headroom CLI.
+
+When the ``proxy`` command is invoked *in-process* — pytest's ``CliRunner``, an
+embedding application — ``os.execv`` replaces that process instead. The whole
+pytest run is destroyed mid-suite with no traceback, and the replacement
+Headroom process is handed pytest's own argv.
+
+CI cannot catch this: the tuning is Darwin-only and no CI runner is macOS, so
+these tests assert the guard's *logic* on every platform rather than relying on
+the re-exec being reachable.
+"""
+
+from __future__ import annotations
+
+import sys
+
+import pytest
+
+from headroom.cli import proxy as proxy_cli
+
+
+@pytest.mark.parametrize(
+    "argv0",
+    [
+        "/usr/local/bin/headroom",
+        "/opt/homebrew/bin/headroom",
+    ],
+)
+def test_console_script_is_recognised_as_the_entrypoint(
+    argv0: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    monkeypatch.setattr(sys, "argv", [argv0, "proxy"])
+    assert proxy_cli._process_is_headroom_cli_entrypoint() is True
+
+
+def test_module_invocation_is_recognised_as_the_entrypoint(
+    monkeypatch: pytest.MonkeyPatch,
+) -> None:
+    monkeypatch.setattr(
+        sys, "argv", ["/venv/lib/python3.12/site-packages/headroom/cli/__main__.py", "proxy"]
+    )
+    assert proxy_cli._process_is_headroom_cli_entrypoint() is True
+
+
+@pytest.mark.parametrize(
+    "argv0",
+    [
+        "/venv/bin/pytest",
+        # `python -m pytest` — same basename as a module run, different package.
+        "/venv/lib/python3.12/site-packages/pytest/__main__.py",
+        "/usr/bin/uvicorn",
+        "",
+    ],
+)
+def test_embedders_are_not_mistaken_for_the_entrypoint(
+    argv0: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    monkeypatch.setattr(sys, "argv", [argv0, "proxy"])
+    assert proxy_cli._process_is_headroom_cli_entrypoint() is False
+
+
+def test_reexec_does_not_exec_when_embedded(monkeypatch: pytest.MonkeyPatch) -> None:
+    """The end-to-end guard: no execv when another program owns the process."""
+    monkeypatch.setattr(sys, "platform", "darwin")
+    monkeypatch.setattr(sys, "argv", ["/venv/bin/pytest", "tests/"])
+    monkeypatch.delenv("_HEADROOM_MALLOC_TUNED", raising=False)
+    for key in proxy_cli._MALLOC_TUNING:
+        monkeypatch.delenv(key, raising=False)
+
+    calls: list[object] = []
+    monkeypatch.setattr(proxy_cli.os, "execv", lambda *a, **k: calls.append(a))
+
+    proxy_cli._reexec_with_malloc_tuning()
+
+    assert calls == []
+    # The loop guard must not be set either: this process never applied the
+    # tuning, so a genuine CLI child inheriting the env must still be free to.
+    assert "_HEADROOM_MALLOC_TUNED" not in proxy_cli.os.environ
+
+
+def test_reexec_still_execs_for_a_real_cli_launch(monkeypatch: pytest.MonkeyPatch) -> None:
+    """The fix must not disable the feature it is guarding."""
+    monkeypatch.setattr(sys, "platform", "darwin")
+    monkeypatch.setattr(sys, "argv", ["/usr/local/bin/headroom", "proxy", "--port", "8787"])
+    monkeypatch.delenv("_HEADROOM_MALLOC_TUNED", raising=False)
+    for key in proxy_cli._MALLOC_TUNING:
+        monkeypatch.delenv(key, raising=False)
+
+    calls: list[tuple] = []
+    monkeypatch.setattr(proxy_cli.os, "execv", lambda *a, **k: calls.append(a))
+
+    proxy_cli._reexec_with_malloc_tuning()
+
+    assert len(calls) == 1
+    _executable, argv = calls[0]
+    assert argv[1:] == ["-m", "headroom.cli", "proxy", "--port", "8787"]
+    for key, value in proxy_cli._MALLOC_TUNING.items():
+        assert proxy_cli.os.environ[key] == value

tests/test_gemini_ccr_continuation_usage.py

@@ -25,6 +25,10 @@ class _FakeRequest:
         self.headers: dict[str, str] = {}
         self.query_params: dict[str, str] = {}
         self.url = SimpleNamespace(path="/v1beta/models/gemini-pro:generateContent", query="")
+        # Every real Starlette Request carries a scope, and the Gemini handler
+        # binds the savings-attribution ledger to it (#3051). Without this the
+        # double is a shape that cannot occur in production.
+        self.scope: dict = {"type": "http", "method": "POST"}
 
 
 class _CcrToolCallResponse:

tests/test_malloc_tuning.py

@@ -65,6 +65,10 @@ def test_reexec_guard_prevents_loop(monkeypatch):
 
 def test_reexec_skips_when_operator_already_set_vars(monkeypatch):
     monkeypatch.setattr(proxy_cli.sys, "platform", "darwin")
+    # A real CLI launch, like the sibling exec test below: the tuning path is
+    # only reachable when this process is the Headroom CLI entrypoint, and
+    # under pytest argv[0] is pytest's own.
+    monkeypatch.setattr(proxy_cli.sys, "argv", ["headroom", "proxy"])
     monkeypatch.setenv("MallocAggressiveMadvise", "1")
     monkeypatch.setenv("MallocLargeCache", "0")
     rec: dict = {}