fix(proxy): keep prefixed core tools resident (#3046)

## Description

Headroom's Tool Search deferral lowercased core tool names but did not
account for client namespace prefixes. Oh My Pi sends built-ins such as
`_read`, `_edit`, `_write`, and `_bash`, so those core tools were
incorrectly marked `defer_loading=True`.

This change centralizes resident-name normalization for both the
Anthropic and OpenAI paths. It lowercases names and removes only leading
underscores, preserving internal separators such as `mcp__server__read`
so unrelated tools do not become resident.

Closes #3031

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added a shared resident-tool name normalizer in
`headroom/proxy/helpers.py`.
- Applied the same normalization to Anthropic and OpenAI Tool Search
deferral.
- Added a regression test for Oh My Pi's exact 12-tool surface at the
deferral threshold.
- Added OpenAI coverage for prefixed resident tools and negative
namespace cases.

## Testing

<!-- Check what you actually ran, then paste the real command output
below. -->

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run --no-sync pytest --noconftest -q tests/test_openai_tool_search_deferral.py tests/test_issue_746_tool_search.py -k 'not normalize_tool_search_mode and not configure_'
72 passed, 23 deselected in 0.25s

$ uv run --no-sync ruff check .
All checks passed!

$ uv run --no-sync ruff format --check .
1499 files already formatted

$ UV_CACHE_DIR=/tmp/headroom-uv-cache uv run --no-sync mypy headroom
Success: no issues found in 520 source files
```

## Real Behavior Proof

- Environment: Linux x86_64 sandbox; Python 3.12.13; uv 0.11.33; no
provider credentials.
- Exact command / steps: Exercised the exact 12-tool Oh My Pi fixture
through the Anthropic deferral helper and prefixed resident plus
negative names through the OpenAI helper.
- Observed result: Anthropic kept `_edit`, `_task`, `_read`, `_bash`,
`_glob`, `_grep`, `_write`, `computer`, and `web_search` resident while
deferring `_hub`, `_todo`, and `_eval`. OpenAI kept prefixed core tools
resident while `mcp__server__read` and `terminal_helper` remained
deferred.
- Not tested: Live Oh My Pi traffic against Anthropic, provider E2E
tests, and the full native-backed pytest suite.

## Runtime Rollout Safety

- Rollout-managed feature(s): Existing server-side Tool Search deferral
for Anthropic and OpenAI.
- Minimum rollout channel: N/A; targeted bug fix to existing behavior.
- Stable/default behavior changed: Yes. Leading-underscore names that
normalize to known resident names now remain resident.
- Kill switch / disable path: Set `HEADROOM_TOOL_SEARCH=0`.
- Unsafe override required: No.
- Qualification impact: Prefixed core tools remain immediately
available; non-core and MCP namespace behavior is unchanged.
- Rollback path: Revert this commit or disable Tool Search with
`HEADROOM_TOOL_SEARCH=0`.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Screenshots (if applicable)

N/A

## Additional Notes
This commit is contained in:
Joseph Benno
2026-08-15 17:10:44 -04:00
committed by GitHub
parent 322425c43b
commit 2f4d001c9f
3 changed files with 68 additions and 11 deletions
+18 -11
View File
@@ -2811,6 +2811,13 @@ _TOOL_SEARCH_DEFAULT_NAME = "tool_search_tool_regex"
_TOOL_SEARCH_MIN_TOOLS = 12
def _tool_search_resident_key(name: Any) -> str:
"""Normalize a client tool name for resident-tool membership checks."""
# Oh My Pi prefixes every built-in with ``_``. Strip only leading namespace
# markers so internal separators such as ``mcp__server__read`` stay intact.
return str(name or "").lower().lstrip("_")
def anthropic_first_party_tool_search_supported(api_base_url: str | None) -> bool:
"""Return whether Anthropic server-side tool search is valid for this upstream."""
from headroom.providers.claude.runtime import is_custom_anthropic_base_url
@@ -2872,17 +2879,17 @@ def inject_tool_search_deferral(
last_resident_real: dict[str, Any] | None = None
resident_has_cache_control = False
# Clients disagree on casing for the same tool: Claude Code sends ``Bash`` /
# ``ToolSearch`` where opencode sends ``bash``. Compare case-insensitively so
# the exemption applies to both — an exact match silently deferred *every*
# tool for PascalCase clients, including their own tool-search tool.
core_lower = {name.lower() for name in core_tools}
# Clients disagree on casing and leading namespace markers for the same tool:
# Claude Code sends ``Bash``, opencode sends ``bash``, and Oh My Pi sends
# ``_bash``. Normalize both the configured names and each candidate so the
# exemption applies consistently across clients.
core_keys = {_tool_search_resident_key(name) for name in core_tools}
for tool in tools:
if (
not isinstance(tool, dict)
or tool.get("type")
or str(tool.get("name") or "").lower() in core_lower
or _tool_search_resident_key(tool.get("name")) in core_keys
):
# Non-dict, server/typed tools (web_search, computer, …), and core
# tools stay resident and unchanged.
@@ -3259,10 +3266,10 @@ def inject_tool_search_deferral_openai(
out: list[Any] = [{"type": _OPENAI_TOOL_SEARCH_TYPE}]
deferred = 0
# Case-insensitive for the same reason as the Anthropic path above: the
# resident-name sets are lowercase, clients are not required to be.
resident_lower = {name.lower() for name in core_tools} | {
name.lower() for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
# Normalize for the same reason as the Anthropic path above: clients may use
# different casing or a leading namespace marker for the same resident tool.
resident_keys = {_tool_search_resident_key(name) for name in core_tools} | {
_tool_search_resident_key(name) for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
}
for tool in tools:
if not isinstance(tool, dict):
@@ -3273,7 +3280,7 @@ def inject_tool_search_deferral_openai(
# trained to search namespaces / MCP servers). Everything else — core
# coding tools and other hosted tools — stays resident.
deferrable = (
ttype == "function" and str(tool.get("name") or "").lower() not in resident_lower
ttype == "function" and _tool_search_resident_key(tool.get("name")) not in resident_keys
) or ttype == "mcp"
if deferrable and not tool.get("defer_loading"):
new_tool = dict(tool)
+36
View File
@@ -350,6 +350,42 @@ def test_resident_real_tool_survives_pascal_case_surface() -> None:
assert any(not t.get("type") and not t.get("defer_loading") for t in out)
def _omp_tools() -> list[dict]:
"""Oh My Pi's 12-tool surface: underscore-prefixed built-ins plus typed tools."""
named = [
"_hub",
"_edit",
"_task",
"_todo",
"_eval",
"_read",
"_bash",
"_glob",
"_grep",
"_write",
]
return [
*[{"name": name, "description": name, "input_schema": {}} for name in named],
{"type": "computer_20250124", "name": "computer"},
{"type": "web_search_20250305", "name": "web_search"},
]
def test_core_tools_match_leading_underscore_namespace() -> None:
tools = _omp_tools()
assert len(tools) == _TOOL_SEARCH_MIN_TOOLS
out = inject_tool_search_deferral(tools)
by_name = {tool.get("name"): tool for tool in out if isinstance(tool, dict)}
for name in ("_edit", "_task", "_read", "_bash", "_glob", "_grep", "_write"):
assert by_name[name].get("defer_loading") is None, name
for name in ("_hub", "_todo", "_eval"):
assert by_name[name].get("defer_loading") is True, name
for name in ("computer", "web_search"):
assert by_name[name].get("defer_loading") is None, name
# ---------------------------------------------------------------------------
# Tool-search history repair (#2805)
#
+14
View File
@@ -123,6 +123,20 @@ def test_terminal_helper_remains_deferrable():
assert helper.get("defer_loading") is True
def test_prefixed_core_and_terminal_names_stay_resident():
resident = ["_bash", "_read", "_write", "_edit", "_glob", "_grep", "_terminal"]
noncore = ["_hub", "_todo", "_eval", "mcp__server__read", "terminal_helper"]
tools = [_fn(name) for name in resident + noncore]
out = inject_tool_search_deferral_openai(tools, "gpt-5.6-terra")
by_name = {tool["name"]: tool for tool in out if tool.get("type") == "function"}
for name in resident:
assert by_name[name].get("defer_loading") is None, name
for name in noncore:
assert by_name[name].get("defer_loading") is True, name
def test_defers_mcp_server():
tools = [_fn(n) for n in _CORE] + [{"type": "mcp", "server_label": "sentry"}]
tools += [_fn(f"x{i}") for i in range(8)]