5860ae08f0
* fix: let a healthy route finish before the routing hook gives up The first-message ladder was sized from the routing call alone, but the server prepares the candidate catalog before it calls the router — about three seconds on a first message. A healthy route therefore cost ~4.8s against a 7s relay budget that started earlier, so the runner abandoned verdicts that did arrive: the attempt was wasted, the prompt was replayed a second time, and the transcript showed it twice. Each hop now covers preparation plus the call, with the hook budget at the 15s ceiling and the harness kill still under Claude Code's own 30s UserPromptSubmit default. A wedged router costs 15s instead of the 45s it cost before this ladder existed. The magnitude test gains a floor as well as a ceiling, so a future tightening cannot re-open the gap. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(web): say claude and codex on spawn chips, without the native suffix A spawn chip's harness id is how the spawn runs, not something the chip needs to spell out; the native suffix reads as noise there. SDK-brain sub-agents (a bundle agent's codex / claude-sdk children) carry no suffix and render unchanged, as do the session's own session/turn chips. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * test: align the spawn-gate budget assertion with the widened ladder Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(routing): keep a pinned session's spawns in its own family at the source A pinned Smart Routing session was offered every agent by ``sys_agent_list``, so a codex session could stand up a claude-native child and only then have routing decline it. Refuse the spawn before it happens instead: - ``sys_agent_list`` drops built-ins outside the caller's family when the caller routes its spawns and is not auto-harness. - ``POST /v1/sessions`` refuses an out-of-family child of such a parent, naming the rule. Auto-harness parents still cross families (the router owns theirs), and a plain session sees and spawns exactly what it did before. The routing decline stays as the fail-safe for a pane that exists anyway. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(routing): decline a route-turn whose parent routes another family ``route_turn_hook`` routed a pane's first typed prompt in the pane's own family with no look at its parent, so a child pane on another family's CLI could be pinned to a model its parent's family serves and the pane cannot speak. The policy now declines (fail-open, nothing pinned, no chip) when the pane's parent is a pinned Smart Routing session of another family. The create gate refuses such a pane outright, so this only catches a row that predates it — hence non-terminal, and the parent's switch stays togglable. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(routing): a failed auto-harness route must not claim the route-once label The auto-harness path stamped the routing-decision label on its own "unavailable" card, and that label is the route-once gate — so a router that happened to be down when the session started made every later in-harness prompt decline as "already routed". Leave the label unclaimed on failure, the way the turn, native-pane and child-spawn paths already do; the declined card still says what happened. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(routing): stop routing a Smart Routing create's prompt twice A native Smart Routing create routes the landing screen's prompt and pins what it picked; the harness then submits that same prompt, and the first-prompt hook scored it again — a second judge call tens of seconds later, for the verdict the pane was already running on, and a needless block-and-replay of the turn. The create now fingerprints the prompt it routed (a hash: the label is metadata, and the user's prompt does not belong there). When the hook sees that prompt again it claims the create's decision instead of making a new one — one router call, one chip. A prompt the user edited before sending does not match and still routes on its own, as does the first prompt of a session whose create-time route failed. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * perf(routing): take catalog preparation off the turn path A first routed message spent ~3.2s preparing routing candidates before the routes:select POST went out, and nothing in the logs named where it went. Two runner-derived catalogs were being resolved while the user's prompt was held: the claude-native picker vocabulary, whose stale entry the turn path awaits for up to _ROUTING_CATALOG_WAIT_S (3.0s) while the fetch retries a booting runner, and the runner model catalog, a round trip per turn for every pane that has no picker vocabulary of its own. Warm both when the runner binds instead. _on_runner_connect now calls prefetch_session_routing_catalogs once the session-init handshake has created the terminal, so the catalogs land before the first prompt rather than under it. The runner catalog also gains a per-session cache behind _fetch_runner_catalog (single-flight, 5-minute backstop TTL) whose entries drop through the seam that already invalidates runner-derived snapshot overlays — a rebind or relaunch can change which models a pane accepts, so it must not keep routing off the previous runner's list. A cold cache still takes the inline fetch, so nothing depends on the prefetch having run. route_turn now logs its two phases separately (prep vs router) and the stale catalog refresh logs what it waited, so the timeout ladder can be revisited against measurements instead of a guess. The ladder constants are unchanged here. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(codex): check a routed slug is reachable before switching the pane The routing verdict comes from a server-side gateway map that can go stale, so the routed model is not necessarily one this pane's gateway serves. The hook switched onto it regardless: codex accepted the id, the next turn failed, and nothing anywhere said why — the failure mode the #4074 review flagged. The pane's live model/list is the only authority on what it can be moved onto, and the hook already reads it to translate the routed id into codex's spelling. Make that read the reachability check too: codex_model_slug becomes codex_reachable_model_slug and answers None when no row names the model, and _apply_thread_model returns a decline reason instead of a bare bool. An unreachable pick leaves the pane on its own model, writes no marker, blocks nothing, and records "routed model not in this pane's catalog" to the routing trace and stderr — the same fail-open shape the claude side uses when a routed model has no spelling its picker accepts. A model/list that cannot be read is now distinguished from an empty catalog and also declines: an unreadable catalog is not evidence of reachability, and declining costs a turn of routing where switching blind costs the turn itself. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(auth): one workspace identity, and a refresh that can fall back Two credential faults that made a healthy workspace look unreachable. **One identity.** A pane and the server could authenticate as different ~/.databrickscfg profiles for the same host. The server's router client uses the config's `kind: databricks` provider profile; the claude-native pane installed ucode's recorded token command, which selects the workspace however ucode was set up — usually by host. Two profiles on one host are two identities, so re-authing one left the other's token expired and the two halves disagreed about whether the workspace was up. The named profile is now the authority on both sides: the pane's apiKeyHelper is regenerated against it (only for the recognizable `databricks auth token` shape — an enterprise deployment's own token command has a selector we have no business guessing at), and a `routing:` block that names no profile falls back to the provider block's rather than to the ambient SDK chain. Host selection stays the fallback for when nothing names a profile. **A refresh that can fall back.** The generated helper forced a refresh on every call. The reason is real — `--force-refresh` renews a still-valid token and keeps a long gateway session off a mid-session 401 — but it fails outright once the refresh token has gone stale, which turned a perfectly usable cached access token into a hard auth failure (twice in one day). The forced attempt is now speculative: its output is captured, its stderr dropped, and an empty result falls back to plain `auth token`, which serves the cached token and renews it near expiry. The fallback keeps its stderr so a genuine auth failure is still visible. Both harnesses generated this command separately, so the shape now has one definition (databricks_bearer_token_command) and the claude and codex helpers delegate to it. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * test: align both hook-budget assertions with the widened ladder Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * test: keep the catalog-cache reset import-free; cover the spawn chip in e2e_ui The autouse cache-reset fixture imported omnigent.server.smart_routing in every teardown, which detonated inside the spec suite's import-blocker test and taxed lanes that never load the server. A sys.modules lookup clears the cache only where it exists. The new Playwright case pins the shortened spawn-chip harness label the UI judge flagged. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix: leave a visible declined chip when the turn hook's routing call fails The create and dispatch paths already card a failed route; the in-harness first-message hook failed open silently, so a router 401 looked like the session simply ignoring Smart Routing. The hook now persists the same unavailable card with the cause, without claiming the route-once label — the next prompt can still route. Benign allows (already routed, routing off, the family guard) are not failures and stay chipless. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * feat(cli): drop create-time Smart Routing; keep first-message routing The CLI can only route a prompt it never shows: `--smart-routing -p` picked a model (and, on `run`, a harness) before the TUI existed, so the user typed at a session whose pick they could neither see nor change. The web UI is the surface that can do that. So the CLI keeps the one routing shape a terminal can honour — arm the session, let the harness's own hook route the first message typed — and rejects the rest. `omni claude|codex --smart-routing` stay, bare only. `-p` alongside them is now a usage error pointing at the TUI or the web UI, and `run --smart-routing` (with it the CLI's auto-harness route) is rejected outright; its flag stays hidden purely to say where routing moved, and comes out in 0.11. That leaves nothing behind the create-time path: the routed create no longer sends a message or the `auto` sentinel, reads back no verdict, and the launch-side plumbing that applied one is gone. `create_smart_routing_session` becomes `arm_smart_routing_session` and `RoutingDecision` becomes `ArmedSession` (session id + fail-open notice), because neither decides anything any more. The preflight gate, the `--resume` rejection and every server-side create path are untouched. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(web): drop "-native" from every routing chip, not just spawn chips A session-scope chip read "codex-native", which leaks how the pane runs into a label that only needs to name the brain. The shortening was scoped to sub-agent decisions; it belongs on every chip, so harnessDisplayLabel no longer takes a scope and always trims the trailing suffix. SDK ids (codex / claude-sdk / auto) carry no suffix and render unchanged. The e2e session-chip assertion now also pins the negative: a bare "claude" substring-matches "claude-native", so only not_to_contain_text catches a regression. Same for the card unit test, which anchors on the full label. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(web): render an auto-harness create chip below its prompt A session created with Smart Routing as both the model AND the harness records the pick as a `session` chip at create time, and its first turn routes again and records a `turn` chip — so two chips sit above the session's first user message. `deferredRoutingChips` only paired a chip whose immediate next content block was that message, so the first of the two was left in place and rendered ABOVE the prompt, reading as a preamble instead of the verdict on it. It only looked right when the two verdicts matched and the create chip was dropped by the collapse. Look forward past the sibling chips waiting on the same message (and past superseded ones, which render nothing) and defer them all below the message, in transcript order. A sub-agent chip still stops the scan: it renders standalone where it occurred, and stepping over it would reorder the two. The cache's pending-pair guard learns the same rule so the pair stays stable frame by frame. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * perf(runner): skip the sys_agent_list routing lookup on plain sessions Family confinement made every sys_agent_list pay a serial GET /v1/sessions/{id} with a 30s budget before discovering the session was not routed at all. Plain sessions — the overwhelming majority — carried seconds of fan-out latency for a feature they never use, and a wedged server stalled the listing for the full 30s. Read the runner-local routing class first: a session with no routing armed, or an auto-harness one, answers without a server hop. Only a locally pinned routed session spends the lookup, now on a 5s budget that fails open to the unfiltered listing, and its answer is cached for the session (routing state is fixed at create). The create-path gate still refuses out-of-family creates, so a fail-open listing stays safe. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(auth): fall back to ucode's recorded token command Pinning the pane's apiKeyHelper to the config-named Databricks profile fixed one outage and opened its mirror image: when the named profile holds no usable credential — a config naming DEFAULT while the user authenticated under another profile on the same host — the helper now prints nothing and every turn 401s, where before the rewrite ucode's own recorded command served a working token. The named profile stays the preferred identity; the recorded command becomes the helper's last resort, after the forced refresh and the cached token have both come up empty. An injected DATABRICKS_BEARER still short-circuits everything. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * perf(routing): only warm catalogs for routed, live sessions A runner reconnect walks every session bound to that runner, and the catalog prefetch fired for all of them — archived rows included — with no Smart Routing gate. One host's tunnel flap with ~25 plain codex panes launched 50 fire-and-forget tasks whose provider listings run on worker threads, so the session re-init running alongside them timed out and the panes came back stranded, all to warm a cache only Smart Routing reads. Gate the prefetch on the canonical routing reader (routing_class_from_snapshot), skip archived sessions, cap concurrent warm-ups with a small semaphore, and have each task retrieve its own exception: a tunnel dropped mid-prefetch raised RuntimeError that nothing ever retrieved, which surfaced only as asyncio unretrieved-exception noise. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * feat(routing): route a pinned native create before its pane launches Picking Claude Code or Codex with Smart Routing as the model created the session with no prompt to route on, so routing fell through to the in-pane first-message hook: the prompt was blocked, routed, switched with `/model` and replayed. The user watched their own message disappear for seconds, and the composer's model pill stayed stale because the pin landed mid-turn instead of before the snapshot bound. The web create now sends `smart_routing_message` for a pinned claude-native / codex-native pane too, whenever routing owns the model. The server already routes the MODEL only on that path and pins `model_override` before the terminal launches; the client still delivers the real first message after navigation, exactly as the auto path does. Bundle agents are untouched — their harness isn't decided until the first message event, so there is nothing to route at create. With the model pinned and the routing-decision label stamped before the pane exists, the `UserPromptSubmit` turn-routing hook has no answer left but "already routed" — paid for with a held prompt and a round trip per prompt. The session's routing class now carries a `turn_routing` flag that drops to false once the row has a routing decision, and the native launch skips the loopback router; the absent advertisement is what leaves the hook out of the generated settings. A create whose routing failed stamps nothing and keeps its hook, so the first message is still its retry, and spawn routing plus the extended catalog are untouched. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * fix(web): keep a create-time routing chip below the prompt it decides A pinned Smart Routing create routes at create time, so the session-scope decision is persisted before the pane launches while the landing composer's prompt is only posted after navigation. The prompt is on screen the whole time, but as an optimistic `pendingUserMessages` entry merged in AFTER the bubble walk — never a `user_message` block — so `pairableMessageAfter` cannot see it and the chip renders above the message until the server persists it, then visibly moves below. Splice the pending prompt above a run of session-scope chips that opens the committed timeline, matching the position `buildBubbles` gives the chip once the message is persisted. The chip renders once, below the prompt, and stays put across the pending → committed swap. Chips anywhere else (paired with their message, or a standalone sub-agent spawn) keep their place, and a chip with no message — including a declined create route — still renders. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> * chore: trigger CI on the rebased tip The rebase onto main and the chip-ordering fix never ran the test lanes; only CodeQL and DCO reported. Co-authored-by: Isaac Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com> --------- Signed-off-by: Bryan Qiu <bryan.qiu@databricks.com>
1275 lines
44 KiB
Python
1275 lines
44 KiB
Python
"""Tests for the codex-native policy hook entrypoint (``evaluate-policy``)."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import io
|
|
import json
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
import httpx
|
|
import pytest
|
|
|
|
from omnigent import codex_native_hook, native_policy_hook
|
|
from omnigent.codex_native_bridge import (
|
|
CodexNativeBridgeState,
|
|
codex_home_for_bridge_dir,
|
|
prepare_bridge_dir,
|
|
write_bridge_state,
|
|
write_policy_hook_config,
|
|
)
|
|
from tests.native_hook_helpers import make_failing_client
|
|
|
|
|
|
class _DenyHttpxClient:
|
|
"""
|
|
Sync HTTP client stub that records the request and returns a DENY verdict.
|
|
|
|
Returns a real :class:`httpx.Response` so the hook exercises its real
|
|
JSON parsing + verdict-mapping path rather than a mock's attributes.
|
|
|
|
:param headers: Headers passed to :class:`httpx.Client`.
|
|
:param timeout: Timeout passed to :class:`httpx.Client`.
|
|
"""
|
|
|
|
captured: dict[str, object] = {}
|
|
|
|
def __init__(self, *, headers: dict[str, str], timeout: object) -> None:
|
|
"""
|
|
Capture constructor inputs for later assertions.
|
|
|
|
:param headers: HTTP headers the hook builds for AP.
|
|
:param timeout: HTTP timeout object.
|
|
:returns: None.
|
|
"""
|
|
_DenyHttpxClient.captured["headers"] = headers
|
|
|
|
def __enter__(self) -> _DenyHttpxClient:
|
|
"""
|
|
Enter the context manager.
|
|
|
|
:returns: This stub client.
|
|
"""
|
|
return self
|
|
|
|
def __exit__(self, *args: object) -> None:
|
|
"""
|
|
Exit the context manager.
|
|
|
|
:param args: Exception details (unused).
|
|
:returns: None.
|
|
"""
|
|
del args
|
|
|
|
def post(self, url: str, *, json: dict[str, object]) -> httpx.Response:
|
|
"""
|
|
Record the outgoing request and return a DENY EvaluationResponse.
|
|
|
|
:param url: Target Omnigent URL.
|
|
:param json: Request body (the EvaluationRequest).
|
|
:returns: A real 200 response carrying a DENY verdict.
|
|
"""
|
|
_DenyHttpxClient.captured["url"] = url
|
|
_DenyHttpxClient.captured["json"] = json
|
|
return httpx.Response(
|
|
200,
|
|
text='{"result":"POLICY_ACTION_DENY","reason":"rm blocked by admin policy"}',
|
|
request=httpx.Request("POST", url),
|
|
)
|
|
|
|
|
|
class _RaisesIfCalled:
|
|
"""
|
|
HTTP client stub that fails the test if the hook ever POSTs.
|
|
|
|
Used by fail-open tests where the hook must short-circuit (missing
|
|
bridge state or policy config) before reaching the network.
|
|
|
|
:param headers: Headers passed to :class:`httpx.Client` (unused).
|
|
:param timeout: Timeout passed to :class:`httpx.Client` (unused).
|
|
"""
|
|
|
|
def __init__(self, *, headers: dict[str, str], timeout: object) -> None:
|
|
"""
|
|
Accept the constructor shape; do nothing.
|
|
|
|
:param headers: HTTP headers (unused).
|
|
:param timeout: HTTP timeout (unused).
|
|
:returns: None.
|
|
"""
|
|
del headers, timeout
|
|
|
|
def __enter__(self) -> _RaisesIfCalled:
|
|
"""
|
|
Enter the context manager.
|
|
|
|
:returns: This stub client.
|
|
"""
|
|
return self
|
|
|
|
def __exit__(self, *args: object) -> None:
|
|
"""
|
|
Exit the context manager.
|
|
|
|
:param args: Exception details (unused).
|
|
:returns: None.
|
|
"""
|
|
del args
|
|
|
|
def post(self, url: str, *, json: dict[str, object]) -> httpx.Response:
|
|
"""
|
|
Fail loudly — the hook should never reach the network here.
|
|
|
|
:param url: Target Omnigent URL (unused).
|
|
:param json: Request body (unused).
|
|
:returns: Never returns.
|
|
:raises AssertionError: Always.
|
|
"""
|
|
del url, json
|
|
raise AssertionError(
|
|
"evaluate-policy POSTed to Omnigent when it should have short-circuited "
|
|
"(missing bridge state or policy_hook config)."
|
|
)
|
|
|
|
|
|
@pytest.fixture
|
|
def bridge_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
|
"""
|
|
Create an isolated codex-native bridge directory with state.
|
|
|
|
Redirects the bridge root under ``tmp_path`` so the test never
|
|
touches the real ``~/.omnigent`` tree, then writes a valid bridge
|
|
state whose ``session_id`` the hook reads to build the Omnigent URL.
|
|
|
|
:param tmp_path: pytest temp directory.
|
|
:param monkeypatch: pytest monkeypatch fixture.
|
|
:returns: Prepared bridge directory.
|
|
"""
|
|
monkeypatch.setattr("omnigent.codex_native_bridge._BRIDGE_ROOT", tmp_path / "codex-native")
|
|
bdir = prepare_bridge_dir("bridge_test")
|
|
write_bridge_state(
|
|
bdir,
|
|
CodexNativeBridgeState(
|
|
session_id="conv_active",
|
|
socket_path=str(bdir / "app-server.sock"),
|
|
thread_id="thread_abc",
|
|
codex_home=str(bdir / "codex-home"),
|
|
),
|
|
)
|
|
return bdir
|
|
|
|
|
|
def _run_hook(
|
|
bridge_dir: Path, payload: dict[str, object], monkeypatch: pytest.MonkeyPatch
|
|
) -> int:
|
|
"""
|
|
Feed *payload* on stdin and run the ``evaluate-policy`` subcommand.
|
|
|
|
:param bridge_dir: The session's bridge directory.
|
|
:param payload: The codex hook JSON payload, e.g.
|
|
``{"hook_event_name": "PreToolUse", "tool_name": "Bash", ...}``.
|
|
:param monkeypatch: pytest monkeypatch fixture.
|
|
:returns: The hook process exit code.
|
|
"""
|
|
monkeypatch.setattr(sys, "stdin", io.StringIO(json.dumps(payload)))
|
|
return codex_native_hook.main(["evaluate-policy", "--bridge-dir", str(bridge_dir)])
|
|
|
|
|
|
def test_pre_tool_use_converts_posts_and_returns_deny(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
A PreToolUse hook converts to proto, POSTs to AP, and maps DENY back.
|
|
|
|
This is the full codex enforcement path: read bridge state +
|
|
policy_hook config → convert payload → POST /policies/evaluate →
|
|
map the DENY verdict to ``permissionDecision: deny``. It fails if any
|
|
link breaks (wrong URL/session, missing conversion, missing auth, or
|
|
a mis-mapped verdict that would let the blocked command run).
|
|
"""
|
|
_DenyHttpxClient.captured = {}
|
|
write_policy_hook_config(
|
|
bridge_dir,
|
|
ap_server_url="http://127.0.0.1:8787",
|
|
ap_auth_headers={"Authorization": "Bearer test-token"},
|
|
)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{
|
|
"hook_event_name": "PreToolUse",
|
|
"tool_name": "Bash",
|
|
"tool_input": {"command": "rm -rf /"},
|
|
},
|
|
monkeypatch,
|
|
)
|
|
captured = capsys.readouterr()
|
|
|
|
assert exit_code == 0
|
|
# URL is built from the bridge state's session_id, not the payload.
|
|
assert _DenyHttpxClient.captured["url"] == (
|
|
"http://127.0.0.1:8787/v1/sessions/conv_active/policies/evaluate"
|
|
)
|
|
# The codex payload is converted to the proto EvaluationRequest shape.
|
|
sent = _DenyHttpxClient.captured["json"]
|
|
assert sent["event"]["type"] == "PHASE_TOOL_CALL"
|
|
assert sent["event"]["data"] == {"name": "Bash", "arguments": {"command": "rm -rf /"}}
|
|
# Auth headers from policy_hook.json reach AP.
|
|
assert _DenyHttpxClient.captured["headers"] == {"Authorization": "Bearer test-token"}
|
|
# The DENY verdict maps back to codex's PreToolUse deny output.
|
|
result = json.loads(captured.out)
|
|
assert result["hookSpecificOutput"]["permissionDecision"] == "deny"
|
|
assert result["hookSpecificOutput"]["permissionDecisionReason"] == "rm blocked by admin policy"
|
|
assert captured.err == ""
|
|
|
|
|
|
def test_user_prompt_submit_converts_posts_and_blocks(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
A UserPromptSubmit hook converts to PHASE_REQUEST and maps DENY to block.
|
|
|
|
This is the request-phase enforcement path for native Codex sessions
|
|
(the server-level ``_evaluate_input_policy`` skips native message
|
|
events). The prompt rides in ``event.data.text``; a DENY maps to the
|
|
top-level ``decision: "block"`` contract — NOT ``permissionDecision`` —
|
|
which drops the prompt before the model sees it. A break here means a
|
|
blocked prompt would still reach the model.
|
|
"""
|
|
_DenyHttpxClient.captured = {}
|
|
write_policy_hook_config(
|
|
bridge_dir,
|
|
ap_server_url="http://127.0.0.1:8787",
|
|
ap_auth_headers={"Authorization": "Bearer test-token"},
|
|
)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{
|
|
"hook_event_name": "UserPromptSubmit",
|
|
"prompt": "delete the prod database",
|
|
},
|
|
monkeypatch,
|
|
)
|
|
captured = capsys.readouterr()
|
|
|
|
assert exit_code == 0
|
|
sent = _DenyHttpxClient.captured["json"]
|
|
assert sent["event"]["type"] == "PHASE_REQUEST"
|
|
assert sent["event"]["data"] == {"text": "delete the prod database"}
|
|
# DENY → top-level decision/reason block (not permissionDecision).
|
|
result = json.loads(captured.out)
|
|
assert result == {"decision": "block", "reason": "rm blocked by admin policy"}
|
|
assert captured.err == ""
|
|
|
|
|
|
def test_pre_tool_use_stamps_model_from_config(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""
|
|
The hook stamps ``config.toml``'s model onto the request context.
|
|
|
|
This is the race-free model source for the codex cost gate: the hook
|
|
reads the user's live ``/model`` selection from ``config.toml`` at gate
|
|
time and puts it on ``event.context.model`` so the server evaluates
|
|
against it (preferred over the engine's resolved model). If this
|
|
regresses, a terminal ``/model`` downgrade never reaches the gate and
|
|
the session stays wrongly blocked.
|
|
"""
|
|
_DenyHttpxClient.captured = {}
|
|
home = codex_home_for_bridge_dir(bridge_dir)
|
|
home.mkdir(parents=True, exist_ok=True)
|
|
# The current /model selection, as codex persists it to config.toml.
|
|
(home / "config.toml").write_text('model = "gpt-5.4"\n')
|
|
write_policy_hook_config(
|
|
bridge_dir,
|
|
ap_server_url="http://127.0.0.1:8787",
|
|
ap_auth_headers={"Authorization": "Bearer test-token"},
|
|
)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {}},
|
|
monkeypatch,
|
|
)
|
|
|
|
assert exit_code == 0
|
|
sent = _DenyHttpxClient.captured["json"]
|
|
# The live model is carried in the request so the gate sees gpt-5.4.
|
|
assert sent["event"]["context"]["model"] == "gpt-5.4"
|
|
# The harness is stamped so the gate can tailor messages to codex.
|
|
assert sent["event"]["context"]["harness"] == "codex-native"
|
|
|
|
|
|
def test_pre_tool_use_stamps_harness_without_config_model(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""The harness is stamped even when config.toml has no model.
|
|
|
|
The harness drives the deny message's switch-instruction wording, which
|
|
must be correct regardless of whether the model is determinable — so it
|
|
is stamped unconditionally (unlike the model, which is only stamped when
|
|
config.toml provides one).
|
|
"""
|
|
_DenyHttpxClient.captured = {}
|
|
# No config.toml written → read_codex_config_model returns None.
|
|
write_policy_hook_config(
|
|
bridge_dir,
|
|
ap_server_url="http://127.0.0.1:8787",
|
|
ap_auth_headers={"Authorization": "Bearer test-token"},
|
|
)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {}},
|
|
monkeypatch,
|
|
)
|
|
|
|
assert exit_code == 0
|
|
sent = _DenyHttpxClient.captured["json"]
|
|
assert sent["event"]["context"]["harness"] == "codex-native"
|
|
# Model absent (no config) — stays unstamped, the gate falls back.
|
|
assert "model" not in sent["event"]["context"]
|
|
|
|
|
|
def test_missing_bridge_state_is_fail_open(
|
|
tmp_path: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
With no bridge state, the hook emits nothing and never POSTs.
|
|
|
|
A bridge dir that has not been initialized must not crash codex or
|
|
block tools — the hook returns 0 with no verdict. ``_RaisesIfCalled``
|
|
asserts the network was never reached.
|
|
"""
|
|
monkeypatch.setattr("omnigent.codex_native_bridge._BRIDGE_ROOT", tmp_path / "codex-native")
|
|
empty_dir = prepare_bridge_dir("bridge_no_state")
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _RaisesIfCalled)
|
|
|
|
exit_code = _run_hook(
|
|
empty_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {}},
|
|
monkeypatch,
|
|
)
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
# No verdict emitted → codex applies its own default (fail-open).
|
|
assert captured.out == ""
|
|
|
|
|
|
def test_missing_policy_config_is_fail_open(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
With bridge state but no policy_hook config, the hook never POSTs.
|
|
|
|
The session has state but no Omnigent coordinates were written (e.g. a
|
|
local run with no Omnigent server), so there is nothing to enforce against.
|
|
The hook returns 0 with no output and does not touch the network.
|
|
"""
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _RaisesIfCalled)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {}},
|
|
monkeypatch,
|
|
)
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert captured.out == ""
|
|
|
|
|
|
@pytest.mark.parametrize("mode", ["connect_error", "non_2xx", "empty_body", "malformed_json"])
|
|
def test_pre_tool_use_fails_closed_when_verdict_unavailable(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
mode: str,
|
|
) -> None:
|
|
"""
|
|
A governed PreToolUse call denies when no usable verdict is returned.
|
|
|
|
For native harnesses this hook is the sole TOOL_CALL enforcement point,
|
|
so a server outage / non-2xx / empty / malformed response must fail
|
|
CLOSED (deny) instead of "no opinion" — the bypass reported in #536.
|
|
"""
|
|
write_policy_hook_config(bridge_dir, ap_server_url="http://127.0.0.1:8787", ap_auth_headers={})
|
|
monkeypatch.setattr(native_policy_hook, "_EVALUATE_POLICY_RETRY_BUDGET_S", 0.0)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", make_failing_client(mode))
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{
|
|
"hook_event_name": "PreToolUse",
|
|
"tool_name": "Bash",
|
|
"tool_input": {"command": "rm -rf /"},
|
|
},
|
|
monkeypatch,
|
|
)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
result = json.loads(captured.out)
|
|
assert result["hookSpecificOutput"]["permissionDecision"] == "deny", result
|
|
assert result["hookSpecificOutput"]["permissionDecisionReason"]
|
|
|
|
|
|
def test_user_prompt_submit_fails_closed_on_error(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
A governed UserPromptSubmit blocks when no usable verdict is returned.
|
|
|
|
The request gate is the sole pre-turn enforcement point for native
|
|
sessions — a server outage must not let a blocked request proceed.
|
|
"""
|
|
write_policy_hook_config(bridge_dir, ap_server_url="http://127.0.0.1:8787", ap_auth_headers={})
|
|
monkeypatch.setattr(native_policy_hook, "_EVALUATE_POLICY_RETRY_BUDGET_S", 0.0)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", make_failing_client("connect_error"))
|
|
|
|
payload: dict[str, object] = {"hook_event_name": "UserPromptSubmit", "prompt": "hello"}
|
|
exit_code = _run_hook(bridge_dir, payload, monkeypatch)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
result = json.loads(captured.out)
|
|
assert result["decision"] == "block"
|
|
assert result["reason"]
|
|
|
|
|
|
def test_post_tool_use_fails_open_on_error(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
PostToolUse fails OPEN on a transport error — the tool already ran.
|
|
|
|
Mirroring the runner-side ``FAIL_CLOSED_PHASES``.
|
|
"""
|
|
write_policy_hook_config(bridge_dir, ap_server_url="http://127.0.0.1:8787", ap_auth_headers={})
|
|
monkeypatch.setattr(native_policy_hook, "_EVALUATE_POLICY_RETRY_BUDGET_S", 0.0)
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", make_failing_client("connect_error"))
|
|
payload: dict[str, object] = {
|
|
"hook_event_name": "PostToolUse",
|
|
"tool_name": "Bash",
|
|
"tool_input": {"command": "ls"},
|
|
"tool_output": "ok",
|
|
}
|
|
|
|
exit_code = _run_hook(bridge_dir, payload, monkeypatch)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert captured.out == ""
|
|
|
|
|
|
def test_pre_tool_use_uses_relay_when_tool_relay_json_has_session_id(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""Hook POSTs to relay /policies/evaluate when tool_relay.json has session_id."""
|
|
from omnigent.claude_native_bridge import _TOOL_RELAY_FILE
|
|
|
|
relay_token = "relay-tok-abc"
|
|
relay_url = "http://127.0.0.1:19999"
|
|
(bridge_dir / _TOOL_RELAY_FILE).write_text(
|
|
json.dumps({"url": relay_url, "token": relay_token, "session_id": "conv_active"})
|
|
)
|
|
_DenyHttpxClient.captured = {}
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
exit_code = _run_hook(
|
|
bridge_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {"command": "ls"}},
|
|
monkeypatch,
|
|
)
|
|
|
|
assert exit_code == 0
|
|
# Route is relay, not direct server.
|
|
assert _DenyHttpxClient.captured["url"] == f"{relay_url}/policies/evaluate"
|
|
# Auth is relay token, not a server bearer.
|
|
assert _DenyHttpxClient.captured["headers"] == {
|
|
"Content-Type": "application/json",
|
|
"Authorization": f"Bearer {relay_token}",
|
|
}
|
|
# Verdict still applied.
|
|
out = json.loads(capsys.readouterr().out)
|
|
assert out["hookSpecificOutput"]["permissionDecision"] == "deny"
|
|
|
|
|
|
def test_pre_tool_use_falls_back_to_policy_hook_json_when_relay_has_no_session_id(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""Hook falls back to policy_hook.json when tool_relay.json has no session_id."""
|
|
from omnigent.claude_native_bridge import _TOOL_RELAY_FILE
|
|
|
|
# Relay present but no session_id — not policy-capable.
|
|
(bridge_dir / _TOOL_RELAY_FILE).write_text(
|
|
json.dumps({"url": "http://127.0.0.1:19999", "token": "tok"})
|
|
)
|
|
write_policy_hook_config(
|
|
bridge_dir,
|
|
ap_server_url="http://127.0.0.1:8787",
|
|
ap_auth_headers={"Authorization": "Bearer direct-token"},
|
|
)
|
|
_DenyHttpxClient.captured = {}
|
|
monkeypatch.setattr(native_policy_hook.httpx, "Client", _DenyHttpxClient)
|
|
|
|
_run_hook(
|
|
bridge_dir,
|
|
{"hook_event_name": "PreToolUse", "tool_name": "Bash", "tool_input": {}},
|
|
monkeypatch,
|
|
)
|
|
|
|
# Falls back to direct server URL.
|
|
assert _DenyHttpxClient.captured["url"] == (
|
|
"http://127.0.0.1:8787/v1/sessions/conv_active/policies/evaluate"
|
|
)
|
|
assert _DenyHttpxClient.captured["headers"] == {"Authorization": "Bearer direct-token"}
|
|
|
|
|
|
# ── route-turn (first-message model routing) ────────────────────────
|
|
|
|
|
|
def _advertise_turn_router(bridge_dir: Path) -> None:
|
|
"""
|
|
Write a live ``turn_router.json`` advertisement into *bridge_dir*.
|
|
|
|
:param bridge_dir: The session's bridge directory.
|
|
:returns: None.
|
|
"""
|
|
import os
|
|
|
|
from omnigent.runner.turn_routing import ADVERTISEMENT_FILE
|
|
|
|
(bridge_dir / ADVERTISEMENT_FILE).write_text(
|
|
json.dumps(
|
|
{
|
|
"url": "http://127.0.0.1:54321",
|
|
"token": "turn-token",
|
|
"pid": os.getpid(),
|
|
"session_id": "conv_active",
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
|
|
|
|
def _run_route_turn(
|
|
bridge_dir: Path, payload: dict[str, object], monkeypatch: pytest.MonkeyPatch
|
|
) -> int:
|
|
"""
|
|
Feed *payload* on stdin and run the ``route-turn`` subcommand.
|
|
|
|
:param bridge_dir: The session's bridge directory.
|
|
:param payload: The codex ``UserPromptSubmit`` hook payload.
|
|
:param monkeypatch: pytest monkeypatch fixture.
|
|
:returns: The hook process exit code.
|
|
"""
|
|
monkeypatch.setattr(sys, "stdin", io.StringIO(json.dumps(payload)))
|
|
return codex_native_hook.main(
|
|
["route-turn", "--bridge-dir", str(bridge_dir), "--harness", "codex-native"]
|
|
)
|
|
|
|
|
|
def test_route_turn_fast_skips_on_the_marker(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
A consumed marker means no output and no network at all.
|
|
|
|
This is the re-entrancy guard the replayed prompt hits: the replay
|
|
re-fires ``UserPromptSubmit``, and a second block there would drop the
|
|
routed turn.
|
|
"""
|
|
from omnigent.runner.turn_routing import write_turn_routing_marker
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
write_turn_routing_marker(bridge_dir, session_id="conv_active", decision_id="d1")
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: pytest.fail("the marker must skip the network"),
|
|
)
|
|
|
|
exit_code = _run_route_turn(
|
|
bridge_dir, {"prompt": "hello", "model": "gpt-5.6-sol"}, monkeypatch
|
|
)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert captured.out == ""
|
|
|
|
|
|
def test_route_turn_blocks_and_switches_on_a_routed_verdict(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
A routed verdict switches the thread, writes the marker, then blocks.
|
|
|
|
Order is load-bearing: the marker is the runner's "I blocked it, you
|
|
owe it a replay" handshake, so it must be on disk before the block.
|
|
"""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
sent: dict[str, object] = {}
|
|
switched: list[str] = []
|
|
|
|
def _post(url: str, token: str, body: dict[str, object], timeout: float) -> dict[str, object]:
|
|
sent.update({"url": url, "token": token, "body": body, "timeout": timeout})
|
|
return {
|
|
"action": "route",
|
|
"model": "gpt-5.6-luna",
|
|
"rationale": "short lookup",
|
|
"terminal": True,
|
|
}
|
|
|
|
def _switch(bdir: Path, model: str) -> str | None:
|
|
# The marker is only written after the switch is accepted.
|
|
assert not (bdir / MARKER_FILE).exists()
|
|
switched.append(model)
|
|
return None
|
|
|
|
monkeypatch.setattr(codex_native_hook, "_post_json", _post)
|
|
monkeypatch.setattr(codex_native_hook, "_apply_thread_model", _switch)
|
|
|
|
exit_code = _run_route_turn(
|
|
bridge_dir,
|
|
{
|
|
"hook_event_name": "UserPromptSubmit",
|
|
"prompt": "what testing framework does this project use?",
|
|
"turn_id": "turn_1",
|
|
"model": "gpt-5.6-sol",
|
|
},
|
|
monkeypatch,
|
|
)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert sent["url"] == "http://127.0.0.1:54321/v1/sessions/conv_active/route-turn"
|
|
assert sent["token"] == "turn-token"
|
|
assert sent["body"] == {
|
|
"harness": "codex-native",
|
|
"prompt": "what testing framework does this project use?",
|
|
"turn_id": "turn_1",
|
|
# The live model comes from the payload; config.toml is stale.
|
|
"model": "gpt-5.6-sol",
|
|
}
|
|
assert switched == ["gpt-5.6-luna"]
|
|
assert (bridge_dir / MARKER_FILE).exists()
|
|
result = json.loads(captured.out)
|
|
assert result["decision"] == "block"
|
|
assert "gpt-5.6-luna" in result["reason"]
|
|
|
|
|
|
def test_route_turn_allows_and_marks_an_already_pinned_session(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""A terminal no-op writes the marker so later prompts skip the hop."""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {
|
|
"action": "allow",
|
|
"rationale": "already pinned",
|
|
"terminal": True,
|
|
},
|
|
)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_apply_thread_model",
|
|
lambda *args: pytest.fail("an allow verdict must not switch the model"),
|
|
)
|
|
|
|
exit_code = _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert captured.out == ""
|
|
assert (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_route_turn_keeps_asking_after_a_non_terminal_no_op(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""Routing can be toggled on mid-session, so no marker is written."""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {"action": "allow", "rationale": "routing off"},
|
|
)
|
|
|
|
exit_code = _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch)
|
|
|
|
assert exit_code == 0
|
|
assert capsys.readouterr().out == ""
|
|
assert not (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_route_turn_allows_the_prompt_when_the_switch_fails(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
No block without an applied switch — otherwise the prompt is lost.
|
|
|
|
The runner's replay only fires when the marker appears, so a hook that
|
|
blocked without switching would drop the user's message entirely.
|
|
"""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {
|
|
"action": "route",
|
|
"model": "gpt-5.6-luna",
|
|
"rationale": "x",
|
|
"terminal": True,
|
|
},
|
|
)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_apply_thread_model",
|
|
lambda *args: "could not switch to gpt-5.6-luna",
|
|
)
|
|
|
|
exit_code = _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
assert captured.out == ""
|
|
assert "could not switch" in captured.err
|
|
# No marker: it is the replay handshake, and this prompt is running.
|
|
assert not (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_route_turn_declines_visibly_when_the_pane_cannot_serve_the_pick(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
An unreachable pick is a recorded decline, not a silent drop.
|
|
|
|
The reason reaches both the routing trace and stderr, so "the pane never
|
|
moved" is answerable without reproducing the stale gateway map.
|
|
"""
|
|
from omnigent.runner.turn_routing import MARKER_FILE, TRACE_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {
|
|
"action": "route",
|
|
"model": "databricks-claude-opus-5",
|
|
"rationale": "deep refactor",
|
|
"terminal": True,
|
|
},
|
|
)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_apply_thread_model",
|
|
lambda *args: "routed model not in this pane's catalog (databricks-claude-opus-5)",
|
|
)
|
|
|
|
exit_code = _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch)
|
|
|
|
captured = capsys.readouterr()
|
|
assert exit_code == 0
|
|
# The prompt runs, unblocked, on the pane's own model.
|
|
assert captured.out == ""
|
|
assert "not in this pane's catalog" in captured.err
|
|
assert "not in this pane's catalog" in (bridge_dir / TRACE_FILE).read_text()
|
|
assert not (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_route_turn_no_ops_without_an_advertisement(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
exit_code = _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch)
|
|
assert exit_code == 0
|
|
assert capsys.readouterr().out == ""
|
|
|
|
|
|
def test_route_turn_no_ops_on_an_empty_prompt(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: pytest.fail("an empty prompt must not reach the router"),
|
|
)
|
|
assert _run_route_turn(bridge_dir, {"prompt": " "}, monkeypatch) == 0
|
|
assert capsys.readouterr().out == ""
|
|
|
|
|
|
def test_route_turn_no_ops_when_the_endpoint_is_unreachable(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""A routing outage must never block a user's turn."""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(codex_native_hook, "_post_json", lambda *args, **kwargs: None)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
assert capsys.readouterr().out == ""
|
|
assert not (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_route_turn_falls_open_on_the_ladders_own_request_budget(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
"""
|
|
Codex's hook waits ``HOOK_REQUEST_TIMEOUT_S`` for a verdict, and no longer.
|
|
|
|
Same hazard as claude's: the typed prompt is held in the TUI until this
|
|
expires. Asserted against the constant, not elapsed time.
|
|
"""
|
|
from omnigent.runner.turn_routing import HOOK_REQUEST_TIMEOUT_S, MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
seen: list[float] = []
|
|
|
|
def _timed_out(url: str, token: str, body: object, timeout: float) -> None:
|
|
del url, token, body
|
|
seen.append(timeout)
|
|
return
|
|
|
|
monkeypatch.setattr(codex_native_hook, "_post_json", _timed_out)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
assert seen == [HOOK_REQUEST_TIMEOUT_S]
|
|
# Must outlast a healthy route (catalog prep + router call), capped at the
|
|
# owner's 15s ceiling.
|
|
assert HOOK_REQUEST_TIMEOUT_S <= 15.0
|
|
assert capsys.readouterr().out == ""
|
|
assert not (bridge_dir / MARKER_FILE).exists()
|
|
|
|
|
|
def test_the_thread_switch_is_capped_inside_the_harness_hook_budget() -> None:
|
|
"""
|
|
Hop 2b sits inside hop 1 alongside hop 2a, with room to spare.
|
|
|
|
Codex's hook does two things after being invoked — ask for a verdict, then
|
|
switch the thread — and the harness kills it on one budget. If the two
|
|
inner budgets could together exceed the outer one, a slow switch would be
|
|
killed mid-``thread/settings/update``: the block marker is written after
|
|
the switch, so the harness would drop the prompt with nothing to replay it.
|
|
"""
|
|
from omnigent.runner.turn_routing import (
|
|
HARNESS_HOOK_TIMEOUT_S,
|
|
HOOK_REQUEST_TIMEOUT_S,
|
|
SETTINGS_UPDATE_TIMEOUT_S,
|
|
)
|
|
|
|
assert HARNESS_HOOK_TIMEOUT_S > HOOK_REQUEST_TIMEOUT_S + SETTINGS_UPDATE_TIMEOUT_S
|
|
# A local app-server RPC over a unix socket, so seconds is generous.
|
|
assert SETTINGS_UPDATE_TIMEOUT_S < 10.0
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"advertise",
|
|
[False, True],
|
|
ids=["no_advertisement", "unreachable_endpoint"],
|
|
)
|
|
def test_route_turn_traces_every_fall_open(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
advertise: bool,
|
|
) -> None:
|
|
"""A session that did not route always records which gate stopped it.
|
|
|
|
Without this the two ways first-message routing goes quiet — the hook
|
|
fell open, or the harness never fired it at all — are indistinguishable
|
|
from the logs, which is exactly how a reported "it never routed" ends
|
|
up unattributable.
|
|
"""
|
|
from omnigent.runner.turn_routing import TRACE_FILE
|
|
|
|
if advertise:
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(codex_native_hook, "_post_json", lambda *args, **kwargs: None)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
|
|
traced = [
|
|
json.loads(line)
|
|
for line in (bridge_dir / TRACE_FILE).read_text(encoding="utf-8").splitlines()
|
|
]
|
|
assert [entry["outcome"] for entry in traced] == ["fail-open"]
|
|
assert traced[0]["detail"]
|
|
|
|
|
|
def test_route_turn_traces_the_route_it_applied(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
capsys: pytest.CaptureFixture[str],
|
|
) -> None:
|
|
from omnigent.runner.turn_routing import TRACE_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {
|
|
"action": "route",
|
|
"model": "gpt-5.6-luna",
|
|
"terminal": True,
|
|
},
|
|
)
|
|
monkeypatch.setattr(codex_native_hook, "_apply_thread_model", lambda *args: None)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
assert json.loads(capsys.readouterr().out)["decision"] == "block"
|
|
traced = [
|
|
json.loads(line)
|
|
for line in (bridge_dir / TRACE_FILE).read_text(encoding="utf-8").splitlines()
|
|
]
|
|
assert [entry["outcome"] for entry in traced] == ["route"]
|
|
assert "gpt-5.6-luna" in traced[0]["detail"]
|
|
|
|
|
|
# ── the actuator: what spelling reaches thread/settings/update ───────
|
|
|
|
|
|
class _FakeAppServerClient:
|
|
"""
|
|
App-server client stub scripting ``model/list`` and recording requests.
|
|
|
|
:param catalog: ``model/list`` rows to serve, or ``None`` to make the
|
|
call raise (an unreadable catalog).
|
|
"""
|
|
|
|
def __init__(self, catalog: list[dict[str, object]] | None) -> None:
|
|
"""
|
|
Build the stub.
|
|
|
|
:param catalog: Rows to serve, or ``None`` to fail the call.
|
|
:returns: None.
|
|
"""
|
|
self._catalog = catalog
|
|
self.requests: list[tuple[str, dict[str, object]]] = []
|
|
self.closed = False
|
|
|
|
async def connect(self) -> None:
|
|
"""
|
|
Pretend to dial the app-server.
|
|
|
|
:returns: None.
|
|
"""
|
|
|
|
async def request(self, method: str, params: dict[str, object]) -> dict[str, object]:
|
|
"""
|
|
Record one request and answer it.
|
|
|
|
:param method: App-server method name.
|
|
:param params: Method parameters.
|
|
:returns: A JSON-RPC response envelope.
|
|
"""
|
|
self.requests.append((method, params))
|
|
if method == "model/list":
|
|
if self._catalog is None:
|
|
raise RuntimeError("app-server refused model/list")
|
|
return {"result": {"data": self._catalog, "nextCursor": None}}
|
|
return {"result": {}}
|
|
|
|
async def close(self) -> None:
|
|
"""
|
|
Record the close.
|
|
|
|
:returns: None.
|
|
"""
|
|
self.closed = True
|
|
|
|
|
|
def _install_fake_client(
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
client: _FakeAppServerClient,
|
|
) -> _FakeAppServerClient:
|
|
"""
|
|
Route the hook's app-server connect at a stub.
|
|
|
|
:param monkeypatch: pytest monkeypatch fixture.
|
|
:param client: Stub to hand back from ``client_for_transport``.
|
|
:returns: The stub the hook will use.
|
|
"""
|
|
monkeypatch.setattr(
|
|
"omnigent.codex_native_app_server.client_for_transport",
|
|
lambda *args, **kwargs: client,
|
|
)
|
|
return client
|
|
|
|
|
|
# A live databricks-gateway ``model/list`` response.
|
|
_LIVE_CATALOG: list[dict[str, object]] = [
|
|
{"id": "gpt-5.6-sol", "model": "gpt-5.6-sol", "isDefault": True},
|
|
{"id": "gpt-5.6-luna", "model": "gpt-5.6-luna"},
|
|
{"id": "system.ai.glm-5-2", "model": "system.ai.glm-5-2"},
|
|
]
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
("routed", "applied"),
|
|
[
|
|
# The routed arm arrives as a catalog id; codex only has metadata for
|
|
# its own dotted slug, and ``/model`` only highlights that one.
|
|
("databricks-gpt-5-6-luna", "gpt-5.6-luna"),
|
|
# Extended-catalog rows are listed under the catalog spelling, which
|
|
# IS codex's id for them — translating must not mangle it.
|
|
("system.ai.glm-5-2", "system.ai.glm-5-2"),
|
|
],
|
|
)
|
|
def test_apply_thread_model_switches_in_codex_spelling(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
routed: str,
|
|
applied: str,
|
|
) -> None:
|
|
"""The thread switch and the config.toml mirror both speak codex."""
|
|
from omnigent.codex_native_bridge import read_codex_config_model
|
|
|
|
codex_home_for_bridge_dir(bridge_dir).mkdir(parents=True, exist_ok=True)
|
|
client = _install_fake_client(monkeypatch, _FakeAppServerClient(_LIVE_CATALOG))
|
|
|
|
assert codex_native_hook._apply_thread_model(bridge_dir, routed) is None
|
|
|
|
assert client.requests == [
|
|
("model/list", {"includeHidden": True}),
|
|
("thread/settings/update", {"threadId": "thread_abc", "model": applied}),
|
|
]
|
|
assert client.closed is True
|
|
assert read_codex_config_model(bridge_dir) == applied
|
|
|
|
|
|
def test_apply_thread_model_declines_a_model_this_pane_cannot_serve(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""
|
|
A stale gateway map can route a pane onto a model its gateway will not run.
|
|
|
|
Switching anyway is a silent drop at the next turn, so the live catalog is
|
|
the authority: no row names the pick, no switch, and the reason is said out
|
|
loud rather than being swallowed.
|
|
"""
|
|
from omnigent.codex_native_bridge import read_codex_config_model
|
|
|
|
codex_home_for_bridge_dir(bridge_dir).mkdir(parents=True, exist_ok=True)
|
|
client = _install_fake_client(monkeypatch, _FakeAppServerClient(_LIVE_CATALOG))
|
|
|
|
declined = codex_native_hook._apply_thread_model(bridge_dir, "databricks-claude-opus-5")
|
|
|
|
assert declined is not None
|
|
assert "not in this pane's catalog" in declined
|
|
assert "databricks-claude-opus-5" in declined
|
|
# The catalog was read, and nothing was switched or mirrored.
|
|
assert client.requests == [("model/list", {"includeHidden": True})]
|
|
assert client.closed is True
|
|
assert read_codex_config_model(bridge_dir) is None
|
|
|
|
|
|
def test_apply_thread_model_declines_when_the_catalog_cannot_be_read(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""An unreadable catalog proves nothing, so it is not proof of reachability."""
|
|
from omnigent.codex_native_bridge import read_codex_config_model
|
|
|
|
codex_home_for_bridge_dir(bridge_dir).mkdir(parents=True, exist_ok=True)
|
|
client = _install_fake_client(monkeypatch, _FakeAppServerClient(None))
|
|
|
|
declined = codex_native_hook._apply_thread_model(bridge_dir, "databricks-gpt-5-6-luna")
|
|
|
|
assert declined is not None
|
|
assert "could not read this pane's model catalog" in declined
|
|
# Fail open: no switch, no crash, and the pane keeps its own model.
|
|
assert [method for method, _params in client.requests] == ["model/list"]
|
|
assert read_codex_config_model(bridge_dir) is None
|
|
|
|
|
|
def test_apply_thread_model_pages_the_catalog(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""A slug on a later page still translates."""
|
|
|
|
class _PagedClient(_FakeAppServerClient):
|
|
async def request(self, method: str, params: dict[str, object]) -> dict[str, object]:
|
|
"""
|
|
Serve ``model/list`` in two pages.
|
|
|
|
:param method: App-server method name.
|
|
:param params: Method parameters.
|
|
:returns: A JSON-RPC response envelope.
|
|
"""
|
|
self.requests.append((method, params))
|
|
if method != "model/list":
|
|
return {"result": {}}
|
|
if params.get("cursor") is None:
|
|
return {"result": {"data": [{"id": "gpt-5.5"}], "nextCursor": "p2"}}
|
|
return {"result": {"data": [{"id": "gpt-5.6-luna"}], "nextCursor": None}}
|
|
|
|
codex_home_for_bridge_dir(bridge_dir).mkdir(parents=True, exist_ok=True)
|
|
client = _install_fake_client(monkeypatch, _PagedClient(None))
|
|
|
|
assert codex_native_hook._apply_thread_model(bridge_dir, "databricks-gpt-5-6-luna") is None
|
|
|
|
assert client.requests[-1] == (
|
|
"thread/settings/update",
|
|
{"threadId": "thread_abc", "model": "gpt-5.6-luna"},
|
|
)
|
|
|
|
|
|
def test_route_turn_ignores_a_marker_another_session_left_in_the_dir(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""
|
|
A ``/clear`` rotation hands this dir to a NEW conversation.
|
|
|
|
The old session's marker must not make the new one fast-skip: it would
|
|
never route its first message, and the routing it never got would be
|
|
attributed to the superseded session id.
|
|
"""
|
|
from omnigent.runner.turn_routing import turn_routing_marker_session, write_turn_routing_marker
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
write_turn_routing_marker(bridge_dir, session_id="conv_superseded", decision_id="d0")
|
|
asked: list[str] = []
|
|
|
|
def _post(url: str, token: str, body: dict[str, object], timeout: float) -> dict[str, object]:
|
|
del token, body, timeout
|
|
asked.append(url)
|
|
return {"action": "allow", "rationale": "already pinned", "terminal": True}
|
|
|
|
monkeypatch.setattr(codex_native_hook, "_post_json", _post)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
# It asked, and the terminal answer re-keyed the marker onto THIS session.
|
|
assert asked == ["http://127.0.0.1:54321/v1/sessions/conv_active/route-turn"]
|
|
assert turn_routing_marker_session(bridge_dir) == "conv_active"
|
|
|
|
|
|
def test_route_turn_writes_a_session_scoped_marker_on_a_terminal_no_op(
|
|
bridge_dir: Path,
|
|
monkeypatch: pytest.MonkeyPatch,
|
|
) -> None:
|
|
"""The marker names the session and the decision it belongs to."""
|
|
from omnigent.runner.turn_routing import MARKER_FILE
|
|
|
|
_advertise_turn_router(bridge_dir)
|
|
monkeypatch.setattr(
|
|
codex_native_hook,
|
|
"_post_json",
|
|
lambda *args, **kwargs: {
|
|
"action": "allow",
|
|
"rationale": "smart routing is off for this session",
|
|
"terminal": True,
|
|
"decision_id": "decision-7",
|
|
},
|
|
)
|
|
|
|
assert _run_route_turn(bridge_dir, {"prompt": "hello"}, monkeypatch) == 0
|
|
|
|
marker = json.loads((bridge_dir / MARKER_FILE).read_text(encoding="utf-8"))
|
|
assert marker["session_id"] == "conv_active"
|
|
assert marker["decision_id"] == "decision-7"
|
|
|
|
|
|
def test_route_turn_is_not_registered_for_a_session_that_cannot_route(
|
|
tmp_path: Path,
|
|
) -> None:
|
|
"""
|
|
A routing-off session's ``hooks.json`` carries no route-turn command.
|
|
|
|
Registered unconditionally, every submit of every codex-native session paid
|
|
the hook's routing round trip (25s worst case on a degraded server) only to
|
|
be told the session does not route. The policy gate stays, unaffected.
|
|
"""
|
|
from omnigent.codex_native_app_server import _codex_policy_hooks_settings
|
|
|
|
off = _codex_policy_hooks_settings(tmp_path, sys.executable, turn_routing=False)
|
|
commands = [
|
|
hook["command"] for entry in off["hooks"]["UserPromptSubmit"] for hook in entry["hooks"]
|
|
]
|
|
assert not any("route-turn" in command for command in commands)
|
|
assert any("evaluate-policy" in command for command in commands)
|
|
|
|
on = _codex_policy_hooks_settings(tmp_path, sys.executable, turn_routing=True)
|
|
on_commands = [
|
|
hook["command"] for entry in on["hooks"]["UserPromptSubmit"] for hook in entry["hooks"]
|
|
]
|
|
assert sum("route-turn" in command for command in on_commands) == 1
|
|
assert any("evaluate-policy" in command for command in on_commands)
|
|
|
|
|
|
def test_the_turn_router_advertisement_is_the_switch_for_the_hook(tmp_path: Path) -> None:
|
|
"""The runner only advertises for a session that launched with routing on."""
|
|
import os
|
|
|
|
from omnigent.codex_native_app_server import _turn_router_advertised
|
|
from omnigent.runner.turn_routing import ADVERTISEMENT_FILE
|
|
|
|
assert _turn_router_advertised(tmp_path) is False
|
|
(tmp_path / ADVERTISEMENT_FILE).write_text(
|
|
json.dumps(
|
|
{
|
|
"url": "http://127.0.0.1:54321",
|
|
"token": "turn-token",
|
|
"pid": os.getpid(),
|
|
"session_id": "conv_active",
|
|
}
|
|
),
|
|
encoding="utf-8",
|
|
)
|
|
assert _turn_router_advertised(tmp_path) is True
|