fix(proxy): keep prefixed core tools resident (#3046)
## Description Headroom's Tool Search deferral lowercased core tool names but did not account for client namespace prefixes. Oh My Pi sends built-ins such as `_read`, `_edit`, `_write`, and `_bash`, so those core tools were incorrectly marked `defer_loading=True`. This change centralizes resident-name normalization for both the Anthropic and OpenAI paths. It lowercases names and removes only leading underscores, preserving internal separators such as `mcp__server__read` so unrelated tools do not become resident. Closes #3031 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added a shared resident-tool name normalizer in `headroom/proxy/helpers.py`. - Applied the same normalization to Anthropic and OpenAI Tool Search deferral. - Added a regression test for Oh My Pi's exact 12-tool surface at the deferral threshold. - Added OpenAI coverage for prefixed resident tools and negative namespace cases. ## Testing <!-- Check what you actually ran, then paste the real command output below. --> - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ uv run --no-sync pytest --noconftest -q tests/test_openai_tool_search_deferral.py tests/test_issue_746_tool_search.py -k 'not normalize_tool_search_mode and not configure_' 72 passed, 23 deselected in 0.25s $ uv run --no-sync ruff check . All checks passed! $ uv run --no-sync ruff format --check . 1499 files already formatted $ UV_CACHE_DIR=/tmp/headroom-uv-cache uv run --no-sync mypy headroom Success: no issues found in 520 source files ``` ## Real Behavior Proof - Environment: Linux x86_64 sandbox; Python 3.12.13; uv 0.11.33; no provider credentials. - Exact command / steps: Exercised the exact 12-tool Oh My Pi fixture through the Anthropic deferral helper and prefixed resident plus negative names through the OpenAI helper. - Observed result: Anthropic kept `_edit`, `_task`, `_read`, `_bash`, `_glob`, `_grep`, `_write`, `computer`, and `web_search` resident while deferring `_hub`, `_todo`, and `_eval`. OpenAI kept prefixed core tools resident while `mcp__server__read` and `terminal_helper` remained deferred. - Not tested: Live Oh My Pi traffic against Anthropic, provider E2E tests, and the full native-backed pytest suite. ## Runtime Rollout Safety - Rollout-managed feature(s): Existing server-side Tool Search deferral for Anthropic and OpenAI. - Minimum rollout channel: N/A; targeted bug fix to existing behavior. - Stable/default behavior changed: Yes. Leading-underscore names that normalize to known resident names now remain resident. - Kill switch / disable path: Set `HEADROOM_TOOL_SEARCH=0`. - Unsafe override required: No. - Qualification impact: Prefixed core tools remain immediately available; non-core and MCP namespace behavior is unchanged. - Rollback path: Revert this commit or disable Tool Search with `HEADROOM_TOOL_SEARCH=0`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes
This commit is contained in:
+18
-11
@@ -2811,6 +2811,13 @@ _TOOL_SEARCH_DEFAULT_NAME = "tool_search_tool_regex"
|
||||
_TOOL_SEARCH_MIN_TOOLS = 12
|
||||
|
||||
|
||||
def _tool_search_resident_key(name: Any) -> str:
|
||||
"""Normalize a client tool name for resident-tool membership checks."""
|
||||
# Oh My Pi prefixes every built-in with ``_``. Strip only leading namespace
|
||||
# markers so internal separators such as ``mcp__server__read`` stay intact.
|
||||
return str(name or "").lower().lstrip("_")
|
||||
|
||||
|
||||
def anthropic_first_party_tool_search_supported(api_base_url: str | None) -> bool:
|
||||
"""Return whether Anthropic server-side tool search is valid for this upstream."""
|
||||
from headroom.providers.claude.runtime import is_custom_anthropic_base_url
|
||||
@@ -2872,17 +2879,17 @@ def inject_tool_search_deferral(
|
||||
last_resident_real: dict[str, Any] | None = None
|
||||
resident_has_cache_control = False
|
||||
|
||||
# Clients disagree on casing for the same tool: Claude Code sends ``Bash`` /
|
||||
# ``ToolSearch`` where opencode sends ``bash``. Compare case-insensitively so
|
||||
# the exemption applies to both — an exact match silently deferred *every*
|
||||
# tool for PascalCase clients, including their own tool-search tool.
|
||||
core_lower = {name.lower() for name in core_tools}
|
||||
# Clients disagree on casing and leading namespace markers for the same tool:
|
||||
# Claude Code sends ``Bash``, opencode sends ``bash``, and Oh My Pi sends
|
||||
# ``_bash``. Normalize both the configured names and each candidate so the
|
||||
# exemption applies consistently across clients.
|
||||
core_keys = {_tool_search_resident_key(name) for name in core_tools}
|
||||
|
||||
for tool in tools:
|
||||
if (
|
||||
not isinstance(tool, dict)
|
||||
or tool.get("type")
|
||||
or str(tool.get("name") or "").lower() in core_lower
|
||||
or _tool_search_resident_key(tool.get("name")) in core_keys
|
||||
):
|
||||
# Non-dict, server/typed tools (web_search, computer, …), and core
|
||||
# tools stay resident and unchanged.
|
||||
@@ -3259,10 +3266,10 @@ def inject_tool_search_deferral_openai(
|
||||
|
||||
out: list[Any] = [{"type": _OPENAI_TOOL_SEARCH_TYPE}]
|
||||
deferred = 0
|
||||
# Case-insensitive for the same reason as the Anthropic path above: the
|
||||
# resident-name sets are lowercase, clients are not required to be.
|
||||
resident_lower = {name.lower() for name in core_tools} | {
|
||||
name.lower() for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
|
||||
# Normalize for the same reason as the Anthropic path above: clients may use
|
||||
# different casing or a leading namespace marker for the same resident tool.
|
||||
resident_keys = {_tool_search_resident_key(name) for name in core_tools} | {
|
||||
_tool_search_resident_key(name) for name in _OPENAI_TOOL_SEARCH_RESIDENT_NAMES
|
||||
}
|
||||
for tool in tools:
|
||||
if not isinstance(tool, dict):
|
||||
@@ -3273,7 +3280,7 @@ def inject_tool_search_deferral_openai(
|
||||
# trained to search namespaces / MCP servers). Everything else — core
|
||||
# coding tools and other hosted tools — stays resident.
|
||||
deferrable = (
|
||||
ttype == "function" and str(tool.get("name") or "").lower() not in resident_lower
|
||||
ttype == "function" and _tool_search_resident_key(tool.get("name")) not in resident_keys
|
||||
) or ttype == "mcp"
|
||||
if deferrable and not tool.get("defer_loading"):
|
||||
new_tool = dict(tool)
|
||||
|
||||
@@ -350,6 +350,42 @@ def test_resident_real_tool_survives_pascal_case_surface() -> None:
|
||||
assert any(not t.get("type") and not t.get("defer_loading") for t in out)
|
||||
|
||||
|
||||
def _omp_tools() -> list[dict]:
|
||||
"""Oh My Pi's 12-tool surface: underscore-prefixed built-ins plus typed tools."""
|
||||
named = [
|
||||
"_hub",
|
||||
"_edit",
|
||||
"_task",
|
||||
"_todo",
|
||||
"_eval",
|
||||
"_read",
|
||||
"_bash",
|
||||
"_glob",
|
||||
"_grep",
|
||||
"_write",
|
||||
]
|
||||
return [
|
||||
*[{"name": name, "description": name, "input_schema": {}} for name in named],
|
||||
{"type": "computer_20250124", "name": "computer"},
|
||||
{"type": "web_search_20250305", "name": "web_search"},
|
||||
]
|
||||
|
||||
|
||||
def test_core_tools_match_leading_underscore_namespace() -> None:
|
||||
tools = _omp_tools()
|
||||
assert len(tools) == _TOOL_SEARCH_MIN_TOOLS
|
||||
|
||||
out = inject_tool_search_deferral(tools)
|
||||
|
||||
by_name = {tool.get("name"): tool for tool in out if isinstance(tool, dict)}
|
||||
for name in ("_edit", "_task", "_read", "_bash", "_glob", "_grep", "_write"):
|
||||
assert by_name[name].get("defer_loading") is None, name
|
||||
for name in ("_hub", "_todo", "_eval"):
|
||||
assert by_name[name].get("defer_loading") is True, name
|
||||
for name in ("computer", "web_search"):
|
||||
assert by_name[name].get("defer_loading") is None, name
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool-search history repair (#2805)
|
||||
#
|
||||
|
||||
@@ -123,6 +123,20 @@ def test_terminal_helper_remains_deferrable():
|
||||
assert helper.get("defer_loading") is True
|
||||
|
||||
|
||||
def test_prefixed_core_and_terminal_names_stay_resident():
|
||||
resident = ["_bash", "_read", "_write", "_edit", "_glob", "_grep", "_terminal"]
|
||||
noncore = ["_hub", "_todo", "_eval", "mcp__server__read", "terminal_helper"]
|
||||
tools = [_fn(name) for name in resident + noncore]
|
||||
|
||||
out = inject_tool_search_deferral_openai(tools, "gpt-5.6-terra")
|
||||
|
||||
by_name = {tool["name"]: tool for tool in out if tool.get("type") == "function"}
|
||||
for name in resident:
|
||||
assert by_name[name].get("defer_loading") is None, name
|
||||
for name in noncore:
|
||||
assert by_name[name].get("defer_loading") is True, name
|
||||
|
||||
|
||||
def test_defers_mcp_server():
|
||||
tools = [_fn(n) for n in _CORE] + [{"type": "mcp", "server_label": "sentry"}]
|
||||
tools += [_fn(f"x{i}") for i in range(8)]
|
||||
|
||||
Reference in New Issue
Block a user