Compare commits

..

8 Commits

Author SHA1 Message Date
Tomu Hirata cc21f7b27b Merge branch 'main' into ci/store-tests-mysql-postgres 2026-07-09 17:58:41 +09:00
Tomu Hirata 1c58760cf4 chore: update web/package-lock.json 2026-07-09 17:37:55 +09:00
Tomu Hirata ab9dca64db perf(ci): session-scoped DB per worker + mysqlclient for MySQL tests
- conftest: add session-scoped _worker_db_uri fixture that creates one
  database per xdist worker (not per test) and runs Alembic migrations
  once. The per-test db_uri fixture truncates tables between tests for
  isolation. This reduces migration runs from ~680 to 4.
- Remove FOREIGN_KEY_CHECKS toggles around TRUNCATE — all FKs were
  dropped in p1a2b3c4d5e6 so the toggles are pure overhead.
- CI: install libmysqlclient-dev + mysqlclient (C extension driver)
  instead of pure-Python pymysql, and switch dialect to mysql+mysqldb.
  mysqlclient is significantly faster per round-trip.
2026-07-09 17:27:57 +09:00
Tomu Hirata 560974fa36 style: ruff format 2026-07-09 17:19:30 +09:00
Tomu Hirata 960124fa31 fix(ci): all store+db tests pass on MySQL
- permission_store: add MySQL dialect branch in grant() and ensure_user()
  using ON DUPLICATE KEY UPDATE (mysql_insert) instead of PostgreSQL-
  specific OnConflictDoUpdate/OnConflictDoNothing
- conversation_store search: replace 'ci.data::text ILIKE' (Postgres-only)
  with CONVERT(ci.data USING utf8mb4) LIKE on MySQL
- test_db_models: CHECK constraint violations raise OperationalError on
  MySQL (code 3819) not IntegrityError; accept both in check constraint tests
- test_conversation_store: same fix for workspace CHECK constraint tests

682 passed, 3 skipped locally against MySQL.
2026-07-09 17:19:01 +09:00
Tomu Hirata 40ae18137f fix(ci): fix remaining MySQL test failures
- conversation_store search: add MySQL dialect branch using
  CONVERT(data USING utf8mb4) LIKE instead of the PostgreSQL-specific
  '::text ILIKE' cast
- test_db_models + test_conversation_store: CHECK constraint violations
  raise OperationalError on MySQL (code 3819), not IntegrityError;
  update test_check_constraint_* and workspace-check tests to accept
  both
2026-07-09 17:04:46 +09:00
Tomu Hirata 799123fbf5 fix(ci): MySQL PK fix for y1a2b3c4d5e6 widen_conversation_items_pk
MySQL PKs are unnamed; batch_alter_table can't drop then add without
erroring with 'Multiple primary key defined'. Use raw DDL for MySQL
matching the pattern from r1a2b3c4d5e6.
2026-07-09 16:24:55 +09:00
Tomu Hirata c8367d4904 ci: run store and db tests against PostgreSQL and MySQL
Adds two new CI jobs (stores-postgres, stores-mysql) that exercise
tests/stores and tests/db against real service containers, using a
fresh per-test database created via OMNIGENT_TEST_DB_URI. Updates the
db_uri fixture to support non-SQLite backends, adds pymysql to the
databricks extra, and fixes three SQLite-specific tests (PRAGMA
foreign_keys, FTS5 queries) to skip on incompatible backends plus one
SqlConversationItem insertion that used raw strings instead of encoded
SMALLINT values.
2026-07-09 15:53:12 +09:00
700 changed files with 8081 additions and 67864 deletions
-7
View File
@@ -1,9 +1,2 @@
# Treat the AppIcon bundle's contents as binary and never merge them.
web/electron/icons/AppIcon.icon/** binary -merge
# Protobuf bindings regenerated by scripts/gen_routing_pb2.py from the .proto
# schema. Mark them generated so review/code-quality tooling skips them (ruff
# and mypy already exclude them in pyproject.toml); the protoc output isn't
# hand-editable, so its unused-import/global artifacts are expected.
omnigent/api/**/*_pb2.py linguist-generated=true
omnigent/api/**/*_pb2.pyi linguist-generated=true
-1
View File
@@ -21,4 +21,3 @@ shivam5
TomeHirata
xq-yin
hzub
zhengwin
+2 -4
View File
@@ -24,7 +24,7 @@
" - 'web/' before 'web/electron/' and 'web/ios/'",
" - 'omnigent/inner/' before every 'omnigent/inner/<harness>_'.",
" owners - candidate reviewers/assignees. Must be maintainers in",
" .github/MAINTAINER. 2+ each incl. owners_paused. Edit these freely: the",
" .github/MAINTAINER. 2+ each. Edit these freely: the",
" reviewer-logic tests run against a frozen fixture",
" (auto-assign-reviewer.fixture.json), so ownership changes here",
" do not churn them. areas.test.js validates this file (every",
@@ -181,9 +181,7 @@
"omnigent/policies/"
],
"owners": [
"TomeHirata"
],
"owners_paused": [
"TomeHirata",
"ckcuslife-source"
]
},
-3
View File
@@ -8,7 +8,6 @@
REQUIRED=(
"Pre-commit checks"
"Docker build"
"Pytest (runtime-harnesses)"
"Pytest (runtime-policies)"
"Pytest (runtime-core)"
@@ -38,7 +37,6 @@ REQUIRED=(
)
ALLOW_SKIP=(
"Docker build"
"Pytest (runtime-harnesses)"
"Pytest (runtime-policies)"
"Pytest (runtime-core)"
@@ -75,7 +73,6 @@ is_allow_skip() { printf '%s\n' "${ALLOW_SKIP[@]}" | grep -qxF "$1"; }
# workflow is still queued or re-running.
workflow_for() {
case "$1" in
"Docker build") echo "Docker build" ;;
"Pytest ("*) echo "CI" ;;
"E2E Tests (shard "*) echo "E2E Tests" ;;
"E2E UI Tests (shard "*) echo "E2E UI Tests" ;;
-167
View File
@@ -1,167 +0,0 @@
#!/usr/bin/env python3
"""Daily Discord-watch rotation reminder.
Reads an explicit dated schedule (rotation_schedule.json) plus a name ->
slack_id/timezone roster (rotation_roster.json), finds today's assignee, and
pings them in Slack on the morning of *their* local timezone.
The GitHub Actions workflow wakes at a couple of fixed UTC times (one per
timezone's morning). On each run the day's assignee is pinged only if it's
currently morning where they live; if not, the run for their timezone's
morning handles them. Our timezones are far enough apart that only one is ever
in its morning at a time, so at most one person is pinged per run. Dates not
present in the schedule get no ping.
Set SLACK_WEBHOOK_URL to post for real. Leave it unset for a dry run that just
prints what it would do — handy for testing the schedule without Slack.
"""
from __future__ import annotations
import datetime
import json
import os
import pathlib
import urllib.error
import urllib.request
from dataclasses import dataclass
from zoneinfo import ZoneInfo
# Data files live alongside this script so they can be edited (swaps,
# holidays, extending the schedule) without touching the logic here.
ROSTER_PATH = pathlib.Path(__file__).with_name("rotation_roster.json")
SCHEDULE_PATH = pathlib.Path(__file__).with_name("rotation_schedule.json")
# Each cron run is one timezone's morning scan: we ping today's assignee only
# if it's currently morning where they are. A run that's morning in SF is night
# in Singapore and vice versa, so at most one timezone matches per run. Morning
# is a band rather than an exact hour, which absorbs both daylight saving and
# GitHub's frequently-delayed cron schedule — a run that fires a few hours late
# still counts as that person's morning. The band starts at 05:00 (not
# midnight) so a delayed *other* timezone's cron spilling past local midnight
# isn't mistaken for this timezone's morning, which would double-ping.
MORNING_START_HOUR = 5
MORNING_END_HOUR = 12
@dataclass(frozen=True)
class Person:
name: str # display name; matches the names used in the schedule
slack_id: str # Slack member ID, e.g. "U01ABC2DEF" (NOT the display name)
tz: str # IANA timezone name, e.g. "America/Los_Angeles"
def load_roster(roster_path: pathlib.Path = ROSTER_PATH) -> dict[str, Person]:
"""Load the name -> Person mapping from JSON."""
roster = json.loads(roster_path.read_text())
return {
name: Person(name=name, slack_id=entry["slack_id"], tz=entry["tz"])
for name, entry in roster["people"].items()
}
def load_schedule(
schedule_path: pathlib.Path = SCHEDULE_PATH,
) -> dict[datetime.date, str]:
"""Load the date -> assignee-name mapping from JSON."""
doc = json.loads(schedule_path.read_text())
return {datetime.date.fromisoformat(row["date"]): row["name"] for row in doc["schedule"]}
ROSTER: dict[str, Person] = load_roster()
SCHEDULE: dict[datetime.date, str] = load_schedule()
def assignee_for(local_date: datetime.date) -> Person | None:
"""The person scheduled for a given date, or None if the date isn't listed."""
name = SCHEDULE.get(local_date)
if name is None:
return None
return ROSTER.get(name)
def whose_turn_now(now_utc: datetime.datetime) -> Person | None:
"""Return the person to ping right now, or None if it isn't anyone's morning.
Each person is evaluated in their own timezone: it must currently be morning
(05:0011:59) there, and today's schedule entry must name them. Since our
timezones are far enough apart that only one is ever in its morning at a
time, at most one person matches. A person missed by a late/early run is
picked up by the next run that lands in their morning.
"""
for person in ROSTER.values():
local = now_utc.astimezone(ZoneInfo(person.tz))
if not (MORNING_START_HOUR <= local.hour < MORNING_END_HOUR):
continue
if assignee_for(local.date()) == person:
return person
return None
class SlackPostError(RuntimeError):
"""Raised when the Slack POST fails, without exposing the webhook URL."""
def post_to_slack(webhook_url: str, person: Person) -> None:
text = (
f"<@{person.slack_id}> you're on *Discord watch* today \U0001f440 "
f"— please keep an eye on the channel."
)
payload = json.dumps({"text": text}).encode()
req = urllib.request.Request(
webhook_url,
data=payload,
headers={"Content-Type": "application/json"},
)
# Catch and re-raise without the URL: urllib errors stringify the full
# webhook URL, which must never reach the Actions log or error output.
try:
with urllib.request.urlopen(req, timeout=30) as resp:
resp.read()
except urllib.error.HTTPError as exc:
raise SlackPostError(f"Slack returned HTTP {exc.code} {exc.reason}") from None
except urllib.error.URLError as exc:
raise SlackPostError(f"could not reach Slack: {exc.reason}") from None
def _report_todays_assignees(now_utc: datetime.datetime) -> None:
"""Log who's on watch for each timezone's current local date.
Runs regardless of the morning window so a manual run is always
informative, even outside anyone's ping window.
"""
for tz in sorted({p.tz for p in ROSTER.values()}):
local = now_utc.astimezone(ZoneInfo(tz))
person = assignee_for(local.date())
who = person.name if person else "nobody (no schedule entry)"
print(f" {tz}: {local:%Y-%m-%d %a} -> {who}")
def main() -> None:
now_utc = datetime.datetime.now(datetime.timezone.utc)
print(f"Today's watch by timezone (as of {now_utc:%Y-%m-%d %H:%M UTC}):")
_report_todays_assignees(now_utc)
person = whose_turn_now(now_utc)
if person is None:
print(f"{now_utc:%Y-%m-%d %H:%M UTC}: nobody's on watch right now, nothing to do.")
return
local = now_utc.astimezone(ZoneInfo(person.tz))
webhook_url = os.environ.get("SLACK_WEBHOOK_URL")
if not webhook_url:
print(
f"[dry run] Would ping {person.name} ({person.slack_id}) "
f"— it's {local:%Y-%m-%d %H:%M} in {person.tz}. "
f"Set SLACK_WEBHOOK_URL to post for real."
)
return
post_to_slack(webhook_url, person)
print(f"Pinged {person.name} ({person.slack_id}) at {local:%Y-%m-%d %H:%M %Z}.")
if __name__ == "__main__":
main()
-128
View File
@@ -1,128 +0,0 @@
#!/usr/bin/env python3
"""Maintain the Discord-watch schedule: prune elapsed dates, extend the horizon.
Keeps rotation_schedule.json a rolling window of upcoming weekdays. On each run
it drops rows before today and appends new weekday rows — continuing the
rotation order from wherever the schedule currently ends — until the schedule
reaches HORIZON_DAYS ahead. Idempotent: running it twice in a row is a no-op
once the horizon is full, and a missed run just gets caught up on the next one.
Manual edits (swaps, holiday coverage) on future dates are preserved — pruning
only removes past dates, and extension only appends beyond the current last
date, so it never rewrites a row a human changed.
Run with --check to exit non-zero when the file would change (no write), for a
dry run in CI. Otherwise it rewrites the file in place.
"""
from __future__ import annotations
import argparse
import datetime
import json
import pathlib
ROSTER_PATH = pathlib.Path(__file__).with_name("rotation_roster.json")
SCHEDULE_PATH = pathlib.Path(__file__).with_name("rotation_schedule.json")
# Keep the schedule filled this many days into the future.
HORIZON_DAYS = 90
def _roster_order(roster_path: pathlib.Path) -> list[str]:
"""Rotation order = the order names appear in the roster JSON."""
roster = json.loads(roster_path.read_text())
return list(roster["people"].keys())
def _next_weekday(date: datetime.date) -> datetime.date:
"""The next MonFri strictly after date."""
nxt = date + datetime.timedelta(days=1)
while nxt.weekday() >= 5: # 5=Sat, 6=Sun
nxt += datetime.timedelta(days=1)
return nxt
def maintain(
schedule_doc: dict,
order: list[str],
today: datetime.date,
horizon_days: int = HORIZON_DAYS,
) -> dict:
"""Return a new schedule doc with past dates pruned and horizon extended."""
rows = schedule_doc.get("schedule", [])
# Prune elapsed dates (keep today onward).
kept = [r for r in rows if datetime.date.fromisoformat(r["date"]) >= today]
kept.sort(key=lambda r: r["date"])
# Figure out where to resume the rotation.
if kept:
last_date = datetime.date.fromisoformat(kept[-1]["date"])
last_idx = order.index(kept[-1]["name"]) if kept[-1]["name"] in order else -1
else:
# Empty (or fully elapsed) schedule: start today, at the top of the order.
last_date = today - datetime.timedelta(days=1)
last_idx = -1
horizon = today + datetime.timedelta(days=horizon_days)
date = _next_weekday(last_date) if kept else _first_weekday_on_or_after(today)
idx = last_idx
while date <= horizon:
idx = (idx + 1) % len(order)
kept.append({"date": date.isoformat(), "name": order[idx]})
date = _next_weekday(date)
new_doc = dict(schedule_doc)
new_doc["schedule"] = kept
return new_doc
def _first_weekday_on_or_after(date: datetime.date) -> datetime.date:
while date.weekday() >= 5:
date += datetime.timedelta(days=1)
return date
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"--check",
action="store_true",
help="exit non-zero if the file would change; do not write",
)
parser.add_argument(
"--today",
type=datetime.date.fromisoformat,
default=datetime.date.today(),
help="override today's date (ISO), for testing",
)
args = parser.parse_args()
doc = json.loads(SCHEDULE_PATH.read_text())
order = _roster_order(ROSTER_PATH)
new_doc = maintain(doc, order, args.today)
old_text = SCHEDULE_PATH.read_text()
new_text = json.dumps(new_doc, indent=2) + "\n"
if old_text == new_text:
print("Schedule already current; no change.")
return 0
old_n = len(doc.get("schedule", []))
new_n = len(new_doc["schedule"])
print(
f"Schedule updated: {old_n} -> {new_n} rows (through {new_doc['schedule'][-1]['date']})."
)
if args.check:
print("(--check) not writing.")
return 1
SCHEDULE_PATH.write_text(new_text)
return 0
if __name__ == "__main__":
raise SystemExit(main())
-30
View File
@@ -1,30 +0,0 @@
{
"_readme": [
"Discord-watch roster: the name -> Slack member ID + timezone mapping.",
"Read by .github/scripts/rotation.py; the day-to-day schedule lives",
"separately in rotation_schedule.json (a flat list of {date, name}).",
"",
"Fields per person (keyed by display name, which the schedule references):",
" slack_id - Slack member ID (profile -> More -> Copy member ID), e.g.",
" 'U01ABC2DEF'. NOT the @display-name; only the member ID",
" actually notifies the person.",
" tz - IANA timezone; the person is pinged on the morning of this",
" zone. Currently 'America/Los_Angeles' or 'Asia/Singapore'.",
"",
"It is .json (not .yaml) on purpose: the CI runner has no PyYAML, so JSON",
"is read natively by the stdlib (matches .github/areas.json)."
],
"people": {
"Aravind Segu": { "slack_id": "U01A12R8NUR", "tz": "America/Los_Angeles" },
"Bryan Qiu": { "slack_id": "U05KA5T983Y", "tz": "America/Los_Angeles" },
"Daniel Lok": { "slack_id": "U060CNWNHSQ", "tz": "Asia/Singapore" },
"Dhruv Gupta": { "slack_id": "U0A76097E1F", "tz": "America/Los_Angeles" },
"Edwin He": { "slack_id": "U077B1V6WQJ", "tz": "America/Los_Angeles" },
"Pat Sukprasert": { "slack_id": "U05HRKWFY81", "tz": "Asia/Singapore" },
"Sabhya Chhabria": { "slack_id": "U07A1KQDXAB", "tz": "America/Los_Angeles" },
"Serena Ruan": { "slack_id": "U0571L5KNLR", "tz": "Asia/Singapore" },
"Shivam Mittal": { "slack_id": "U09FZKX9S6B", "tz": "America/Los_Angeles" },
"Tomu Hirata": { "slack_id": "U07TX4PR5MZ", "tz": "Asia/Singapore" },
"Zeyi (Rice) Fan": { "slack_id": "U09L5HT4CH0", "tz": "America/Los_Angeles" }
}
}
-292
View File
@@ -1,292 +0,0 @@
{
"_readme": [
"Discord-watch schedule. Read by .github/scripts/rotation.py.",
"",
"One row per assigned weekday, in date order. On each run the bot finds the",
"row whose date is today (in the assignee timezone) and pings that person on",
"the morning of their timezone. Dates not listed here get no ping, so keep",
"this topped up \u2014 extend it before it runs out.",
"",
"To swap or cover a holiday, just edit the name on the affected date(s).",
"name must match an entry in rotation_roster.json (which holds the",
"name -> slack_id + timezone mapping)."
],
"schedule": [
{
"date": "2026-07-14",
"name": "Edwin He"
},
{
"date": "2026-07-15",
"name": "Pat Sukprasert"
},
{
"date": "2026-07-16",
"name": "Sabhya Chhabria"
},
{
"date": "2026-07-17",
"name": "Serena Ruan"
},
{
"date": "2026-07-20",
"name": "Shivam Mittal"
},
{
"date": "2026-07-21",
"name": "Tomu Hirata"
},
{
"date": "2026-07-22",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-07-23",
"name": "Aravind Segu"
},
{
"date": "2026-07-24",
"name": "Bryan Qiu"
},
{
"date": "2026-07-27",
"name": "Daniel Lok"
},
{
"date": "2026-07-28",
"name": "Dhruv Gupta"
},
{
"date": "2026-07-29",
"name": "Edwin He"
},
{
"date": "2026-07-30",
"name": "Pat Sukprasert"
},
{
"date": "2026-07-31",
"name": "Sabhya Chhabria"
},
{
"date": "2026-08-03",
"name": "Serena Ruan"
},
{
"date": "2026-08-04",
"name": "Shivam Mittal"
},
{
"date": "2026-08-05",
"name": "Tomu Hirata"
},
{
"date": "2026-08-06",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-08-07",
"name": "Aravind Segu"
},
{
"date": "2026-08-10",
"name": "Bryan Qiu"
},
{
"date": "2026-08-11",
"name": "Daniel Lok"
},
{
"date": "2026-08-12",
"name": "Dhruv Gupta"
},
{
"date": "2026-08-13",
"name": "Edwin He"
},
{
"date": "2026-08-14",
"name": "Pat Sukprasert"
},
{
"date": "2026-08-17",
"name": "Sabhya Chhabria"
},
{
"date": "2026-08-18",
"name": "Serena Ruan"
},
{
"date": "2026-08-19",
"name": "Shivam Mittal"
},
{
"date": "2026-08-20",
"name": "Tomu Hirata"
},
{
"date": "2026-08-21",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-08-24",
"name": "Aravind Segu"
},
{
"date": "2026-08-25",
"name": "Bryan Qiu"
},
{
"date": "2026-08-26",
"name": "Daniel Lok"
},
{
"date": "2026-08-27",
"name": "Dhruv Gupta"
},
{
"date": "2026-08-28",
"name": "Edwin He"
},
{
"date": "2026-08-31",
"name": "Pat Sukprasert"
},
{
"date": "2026-09-01",
"name": "Sabhya Chhabria"
},
{
"date": "2026-09-02",
"name": "Serena Ruan"
},
{
"date": "2026-09-03",
"name": "Shivam Mittal"
},
{
"date": "2026-09-04",
"name": "Tomu Hirata"
},
{
"date": "2026-09-07",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-09-08",
"name": "Aravind Segu"
},
{
"date": "2026-09-09",
"name": "Bryan Qiu"
},
{
"date": "2026-09-10",
"name": "Daniel Lok"
},
{
"date": "2026-09-11",
"name": "Dhruv Gupta"
},
{
"date": "2026-09-14",
"name": "Edwin He"
},
{
"date": "2026-09-15",
"name": "Pat Sukprasert"
},
{
"date": "2026-09-16",
"name": "Sabhya Chhabria"
},
{
"date": "2026-09-17",
"name": "Serena Ruan"
},
{
"date": "2026-09-18",
"name": "Shivam Mittal"
},
{
"date": "2026-09-21",
"name": "Tomu Hirata"
},
{
"date": "2026-09-22",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-09-23",
"name": "Aravind Segu"
},
{
"date": "2026-09-24",
"name": "Bryan Qiu"
},
{
"date": "2026-09-25",
"name": "Daniel Lok"
},
{
"date": "2026-09-28",
"name": "Dhruv Gupta"
},
{
"date": "2026-09-29",
"name": "Edwin He"
},
{
"date": "2026-09-30",
"name": "Pat Sukprasert"
},
{
"date": "2026-10-01",
"name": "Sabhya Chhabria"
},
{
"date": "2026-10-02",
"name": "Serena Ruan"
},
{
"date": "2026-10-05",
"name": "Shivam Mittal"
},
{
"date": "2026-10-06",
"name": "Tomu Hirata"
},
{
"date": "2026-10-07",
"name": "Zeyi (Rice) Fan"
},
{
"date": "2026-10-08",
"name": "Aravind Segu"
},
{
"date": "2026-10-09",
"name": "Bryan Qiu"
},
{
"date": "2026-10-12",
"name": "Daniel Lok"
},
{
"date": "2026-10-13",
"name": "Dhruv Gupta"
},
{
"date": "2026-10-14",
"name": "Edwin He"
},
{
"date": "2026-10-15",
"name": "Pat Sukprasert"
},
{
"date": "2026-10-16",
"name": "Sabhya Chhabria"
}
]
}
+2 -3
View File
@@ -32,10 +32,9 @@ for (const a of areas)
for (const a of areas)
assert(`area ${a.key} label ${a.label} is a real comp:*`, ALLOWED_LABELS.has(a.label));
// Every area has >= 2 owners (the 2+ codeowner requirement). Paused owners
// still count -- pausing someone must not force adding a new active owner.
// Every area has >= 2 owners (the 2+ codeowner requirement).
for (const a of areas) {
const n = (a.owners || []).length + (a.owners_paused || []).length;
const n = (a.owners || []).length;
assert(`area ${a.key} has >= 2 owners`, n >= 2, `${n} owner(s)`);
}
+10 -36
View File
@@ -63,12 +63,12 @@ jobs:
strategy:
fail-fast: false
matrix:
backend: [sqlite, postgres, mysql]
backend: [sqlite, postgres]
services:
# The Postgres and MySQL services are defined unconditionally (GitHub
# Actions has no per-matrix-value service gating); each leg connects only
# to its own backend and ignores the others. postgres:16 mirrors
# Lakebase's major version; mysql:8.0 matches the stores-mysql CI lane.
# A Postgres service is defined unconditionally (GitHub Actions has no
# per-matrix-value service gating), but only the postgres leg connects to
# it — the sqlite leg simply ignores it. postgres:16 mirrors Lakebase's
# major version.
postgres:
image: postgres:16
env:
@@ -81,18 +81,6 @@ jobs:
--health-interval 5s
--health-timeout 5s
--health-retries 10
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: bench
MYSQL_DATABASE: benchdb
ports:
- 3306:3306
options: >-
--health-cmd "mysqladmin ping -h 127.0.0.1 -u root -pbench"
--health-interval 5s
--health-timeout 5s
--health-retries 10
steps:
- name: Check out repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
@@ -113,21 +101,11 @@ jobs:
# `databricks` extra carries psycopg[binary] for the Postgres backend.
run: uv sync --extra dev --extra databricks
- name: Install MySQL driver
# mysqlclient (mysql+mysqldb://) needs the system client library and is
# not in any extra, so install it only on the mysql leg. Matches the
# stores-mysql lane in ci.yml.
if: matrix.backend == 'mysql'
run: |
sudo apt-get update -qq && sudo apt-get install -y -q libmysqlclient-dev
uv pip install mysqlclient
# Resolve the DB URI + a stable seed-cache key for this backend. The
# cache key binds the DB schema head + seed.py contents + corpus config,
# so a schema change or seed edit busts the cache and forces a reseed —
# the "you changed the schema, refresh the seed" contract (SQLite only;
# the Postgres/MySQL services are fresh each run so their DB is never
# cached).
# the Postgres service is fresh each run so its DB is never cached).
- name: Resolve DB target
id: db
run: |
@@ -135,9 +113,6 @@ jobs:
if [[ "${{ matrix.backend }}" == "postgres" ]]; then
echo "uri=postgresql+psycopg://postgres:bench@localhost:5432/benchdb" >> "$GITHUB_OUTPUT"
echo "cache_path=" >> "$GITHUB_OUTPUT"
elif [[ "${{ matrix.backend }}" == "mysql" ]]; then
echo "uri=mysql+mysqldb://root:bench@127.0.0.1:3306/benchdb" >> "$GITHUB_OUTPUT"
echo "cache_path=" >> "$GITHUB_OUTPUT"
else
echo "uri=sqlite:///$PWD/bench.db" >> "$GITHUB_OUTPUT"
echo "cache_path=bench.db" >> "$GITHUB_OUTPUT"
@@ -145,7 +120,7 @@ jobs:
echo "cache_key=benchdb-${{ matrix.backend }}-$HEAD-${SESSIONS}x${ITEMS}-${{ hashFiles('dev/benchmarks/omnigent/seed.py') }}" >> "$GITHUB_OUTPUT"
# Reuse a previously-seeded SQLite corpus when schema + seed + config are
# unchanged. No-op for the server-backed legs (empty path).
# unchanged. No-op for the postgres leg (empty path).
- name: Restore seeded SQLite corpus
if: matrix.backend == 'sqlite'
id: seedcache
@@ -155,10 +130,9 @@ jobs:
key: ${{ steps.db.outputs.cache_key }}
- name: Seed corpus
# The fresh-service backends (postgres, mysql) always seed; SQLite seeds
# only on a cache miss. seed.py is itself idempotent, so a stray hit is
# harmless.
if: matrix.backend != 'sqlite' || steps.seedcache.outputs.cache-hit != 'true'
# Postgres always seeds (fresh service each run); SQLite seeds only on a
# cache miss. seed.py is itself idempotent, so a stray hit is harmless.
if: matrix.backend == 'postgres' || steps.seedcache.outputs.cache-hit != 'true'
run: |
uv run --no-sync dev/benchmarks/omnigent/seed.py \
--database-uri "${{ steps.db.outputs.uri }}" \
+5 -22
View File
@@ -12,11 +12,9 @@ name: Bump Version
# this workflow wraps it with `uv lock`, a consistency check, and an
# auto-opened PR.
#
# NOTE: when the omnigent-ci App is configured (vars.OMNIGENT_BOT_APP_ID),
# the branch is pushed and the PR opened with a short-lived App token, so CI
# runs on the bump PR automatically. Without it (e.g. in forks) the
# GITHUB_TOKEN fallback applies and, by GitHub policy, CI does NOT auto-run —
# re-open the PR or push to it to kick CI.
# NOTE: the PR is created with GITHUB_TOKEN, so by GitHub policy it does
# NOT trigger other workflows (CI won't auto-run on it). Push an empty
# commit or re-open the PR to kick CI, or swap in a PAT if that matters.
on:
workflow_dispatch:
@@ -89,21 +87,9 @@ jobs:
- name: Verify all locations agree
run: uv run --no-project --python 3.12 --with packaging python scripts/update_versions.py check
# A bump PR pushed by the App identity gets CI runs; a GITHUB_TOKEN push
# would not (GitHub suppresses events from GITHUB_TOKEN-authored pushes).
- name: Mint App token (omnigent)
id: app-token
if: vars.OMNIGENT_BOT_APP_ID != ''
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.OMNIGENT_BOT_APP_ID }}
private-key: ${{ secrets.OMNIGENT_BOT_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: omnigent
- name: Open bump PR
env:
GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }}
GH_TOKEN: ${{ github.token }}
MODE: ${{ github.event.inputs.mode }}
NEW_VERSION: ${{ github.event.inputs.new_version }}
BASE: ${{ github.event.inputs.base_branch }}
@@ -123,9 +109,6 @@ jobs:
exit 0
fi
git commit -s -m "Bump version to ${resolved}"
# Push with the same token that opens the PR (see the App-token
# note above); the checkout's persisted credential is GITHUB_TOKEN.
git remote set-url origin "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
git push --force-with-lease origin "$branch"
existing="$(gh pr list --head "$branch" --base "$BASE" --json number --jq '.[0].number')"
@@ -141,4 +124,4 @@ jobs:
Rewrote \`[project].version\` and sibling \`==\` pins across all three packages (\`pyproject.toml\`, \`sdks/python-client\`, \`sdks/ui\`), the runtime \`VERSION\` constant in \`omnigent/version.py\`, and regenerated \`uv.lock\`.
Generated by \`scripts/update_versions.py\`. Opened via the omnigent-ci App when configured (CI runs automatically); on the GITHUB_TOKEN fallback, re-open or push to kick CI."
Generated by \`scripts/update_versions.py\`. CI does not auto-trigger on GITHUB_TOKEN PRs — re-open or push to run it."
+2 -5
View File
@@ -12,14 +12,11 @@ name: CI
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
paths-ignore: ['web/**', 'tests/e2e_ui/**', 'CHANGELOG.md']
paths-ignore: ['web/**', 'tests/e2e_ui/**']
push:
branches:
- main
# Release branches: release.yml's green-CI gate reads check runs off the
# branch head, so cherry-picks and release-bump commits must run CI.
- 'branch-[0-9]*'
paths-ignore: ['web/**', 'tests/e2e_ui/**', 'CHANGELOG.md']
paths-ignore: ['web/**', 'tests/e2e_ui/**']
permissions:
contents: read
@@ -1,54 +0,0 @@
name: Discord watch rotation - maintain schedule
# Monthly housekeeping for rotation_schedule.json: prune elapsed dates and
# extend the horizon ~3 months out. Opens a PR rather than pushing to main, so
# the change is reviewable and no write to a protected branch is needed.
on:
schedule:
- cron: "0 8 1 * *" # 08:00 UTC on the 1st of each month
workflow_dispatch: {} # manual "Run workflow" button
# Needs to push a branch and open a PR; no other write scope.
permissions:
contents: write
pull-requests: write
concurrency:
group: discord-watch-rotation-maintain
cancel-in-progress: false
jobs:
extend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Update schedule
id: update
run: |
if python3 .github/scripts/rotation_maintain.py; then
if git diff --quiet -- .github/scripts/rotation_schedule.json; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
else
echo "Schedule maintenance failed" >&2
exit 1
fi
- name: Open PR
if: steps.update.outputs.changed == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
branch="rotation-schedule-$(date -u +%Y%m%d)"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -b "$branch"
git add .github/scripts/rotation_schedule.json
git commit -m "chore(ci): extend Discord watch rotation schedule"
git push -u origin "$branch"
gh pr create \
--base main \
--head "$branch" \
--title "chore(ci): extend Discord watch rotation schedule" \
--body "Automated monthly housekeeping: pruned elapsed dates and extended \`rotation_schedule.json\` ~3 months out. Generated by the discord-watch-rotation-maintain workflow."
@@ -1,32 +0,0 @@
name: Discord watch rotation
# Wakes up only at the UTC times that are ~08:00 in an assignee's timezone.
# Note: a single fixed UTC time can't track San Francisco's daylight saving,
# so the SF ping lands at 08:00 in summer (PDT) and 07:00 in winter (PST).
on:
schedule:
- cron: "0 0 * * *" # 08:00 Asia/Singapore (UTC+8, no daylight saving)
- cron: "0 15 * * *" # 08:00 SF in summer (PDT); 07:00 in winter (PST)
workflow_dispatch: {} # manual "Run workflow" button for testing
# Only needs to check out the repo; nothing is written back.
permissions:
contents: read
# Avoid overlapping runs if one is slow.
concurrency:
group: discord-watch-rotation
cancel-in-progress: false
jobs:
ping:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.12" # for zoneinfo in the stdlib
- name: Send rotation ping
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
run: python .github/scripts/rotation.py
-81
View File
@@ -1,81 +0,0 @@
# Build-only Docker check for PRs. Compensates for retiring per-commit main
# publishes (oss-publish-images.yml now builds on tags + nightly only): a broken
# Dockerfile / lockfile / frontend build would otherwise not surface until the
# nightly rebuild or a release. Builds the server image single-arch (linux/amd64)
# with the GHA layer cache and runs a `omnigent --help` CLI smoke. It never pushes.
#
# Scope: the server target exercises the shared builder stage (Python deps +
# web SPA build) that all four published variants inherit, so it catches the
# common breakage without paying for the host/openshell/kubernetes variants or
# the emulated arm64 leg.
#
# Blocking merge-gate check: "Docker build" is in the REQUIRED list in
# .github/scripts/merge-ready/required.sh. Because of the paths filter below it
# can legitimately be absent (a PR touching nothing in the image), so it is also
# in ALLOW_SKIP with a workflow_for() arm, and this workflow's name is in
# merge-ready.yml's workflow_run list so the gate re-evaluates when it completes.
name: Docker build
on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
# Only build when something that lands in the image changes. Mirrors the
# publish workflow's former push paths (web/** IS included here — the image
# bakes the SPA, so a web-only PR can still break the build).
paths:
- 'deploy/docker/Dockerfile'
- 'deploy/docker/entrypoint.py'
- 'omnigent/**'
- 'web/**'
- 'sdks/**'
- 'pyproject.toml'
- 'setup.py'
- 'uv.lock'
- 'web/package-lock.json'
- '.github/workflows/docker-build.yml'
permissions:
contents: read
concurrency:
group: docker-build-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true
jobs:
# Security precondition gate (security-gate.yml): untrusted PRs wait for the
# scan before the build runs on their code; trusted authors pass through.
gate:
uses: ./.github/workflows/security-gate.yml
build:
name: Docker build
needs: gate
# Draft PRs skip the build (ready_for_review re-fires the workflow), matching
# the pytest job in ci.yml.
if: ${{ !github.event.pull_request.draft }}
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up Buildx
uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0
# Single-arch (amd64) build, no push. load: true imports the result into
# the runner's Docker so the smoke step below can run it. Shares the same
# type=gha cache the publish workflow writes, so warm PRs reuse layers.
- name: Build server image (amd64, no push)
uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0
with:
context: .
file: deploy/docker/Dockerfile
push: false
load: true
tags: omnigent-server:pr-${{ github.event.pull_request.number || github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
provenance: false
- name: CLI smoke
run: docker run --rm omnigent-server:pr-${{ github.event.pull_request.number || github.sha }} omnigent --help
+1 -26
View File
@@ -409,38 +409,13 @@ jobs:
RELEASE_ID: ${{ steps.release.outputs.release_id }}
run: |
set -euo pipefail
# Always end the notes with the community thanks. The AI drafter curates
# freely (and can drop a hand-added line), so this is appended here rather
# than via the prompt — every release, AI-drafted or mechanical fallback,
# gets it. Idempotent, and placed just before the trailing "Full Changelog:"
# link to match the layout of prior releases.
python3 - <<'PYEOF'
import pathlib
NOTE = (
"### 💜 Thanks to our community\n\n"
"This release was shaped by the people who filed issues, opened PRs, and "
"talked through feature requests with us on our Discord! Thank you for "
"building omnigent with us, keep the bug reports, ideas and contributions "
"coming :)"
)
path = pathlib.Path("/tmp/release_notes.md")
text = path.read_text(encoding="utf-8").rstrip("\n")
if "Thanks to our community" not in text:
idx = text.find("\nFull Changelog:")
if idx != -1:
head, tail = text[:idx].rstrip("\n"), text[idx:].lstrip("\n")
text = f"{head}\n\n{NOTE}\n\n{tail}"
else:
text = f"{text}\n\n{NOTE}"
path.write_text(text + "\n", encoding="utf-8")
PYEOF
# github-release.yml seeds only a short placeholder body (no
# auto-generated notes), so replace it wholesale with the curated notes.
# Edit by release ID: a draft release can't be addressed by tag (the
# get/edit-by-tag REST endpoint 404s until the release is published).
gh api --method PATCH "repos/${SOURCE_REPO}/releases/${RELEASE_ID}" \
--field body=@/tmp/release_notes.md > /dev/null
echo "Enriched the ${TAG} release draft with curated notes + community note." \
echo "Enriched the ${TAG} release draft with curated notes." \
| tee -a "$GITHUB_STEP_SUMMARY"
# ::add-mask:: redacts rendered logs, not artifact files — scrub the key
-1
View File
@@ -18,7 +18,6 @@ on:
# Security Gate via rerun-security-gate.yml, so label churn need not re-run
# the heavy Playwright suite. (#399 added these for the gate; superseded.)
types: [opened, synchronize, reopened, ready_for_review]
paths-ignore: ['CHANGELOG.md']
schedule:
- cron: "0 9 * * *"
workflow_dispatch:
+1 -1
View File
@@ -23,7 +23,7 @@ on:
# (rerun-security-gate-run.yml falls back to this trigger). The concurrency
# group key isolates label events so they never cancel a code-push run.
types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled]
paths-ignore: ['web/**', 'tests/e2e_ui/**', 'CHANGELOG.md']
paths-ignore: ['web/**', 'tests/e2e_ui/**']
workflow_dispatch:
inputs:
branch:
-206
View File
@@ -1,206 +0,0 @@
# Publish a FINAL release's GitHub draft as Latest — the last release step,
# run after the prod PyPI publish succeeded and the draft notes are curated
# (designs/RELEASE-AUTOMATION.md).
#
# Deterministic gates first (all fail with actionable links):
# * the tag is a final vX.Y.Z with an unpublished draft release,
# * PyPI serves all three lockstep packages at the version (never advertise
# a release that isn't installable),
# * the auto/changelog/vX.Y.Z CHANGELOG PR isn't sitting open,
# * the docs sweep: no open PRs against omnigent-site's X.Y-docs staging
# branch (every doc staged this cycle is reviewed + merged/closed).
#
# The publish job binds the `publish-release` environment (one-time setup:
# create it in repo settings with required reviewers). Approving it is the
# human attestation "I reviewed the draft notes". The publish itself uses the
# App token — GITHUB_TOKEN-published releases emit no `release: published`
# event, and publish-changelog.yml + update-homebrew.yml hang off it — and
# sets make_latest explicitly, which API publishes don't do on their own.
#
# rc tags never finalize: their drafts deliberately stay unpublished.
name: Finalize release
on:
workflow_dispatch:
inputs:
tag:
description: "Final release tag to publish as Latest, e.g. v0.6.0."
required: true
type: string
permissions:
contents: read
concurrency:
group: finalize-release-${{ inputs.tag }}
cancel-in-progress: false
jobs:
# Maintainer-only, same gate as release.yml.
authorize:
if: github.repository == 'omnigent-ai/omnigent'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Require admin/maintain role
env:
GH_TOKEN: ${{ github.token }}
ACTOR: ${{ github.actor }}
run: |
set -euo pipefail
role="$(gh api "repos/${GITHUB_REPOSITORY}/collaborators/${ACTOR}/permission" --jq .role_name)"
case "$role" in
admin|maintain)
echo "Dispatcher ${ACTOR} has role ${role} — authorized." | tee -a "$GITHUB_STEP_SUMMARY" ;;
*)
echo "::error::Release workflows require the admin or maintain role (dispatcher ${ACTOR} has '${role}')."
exit 1 ;;
esac
checks:
needs: authorize
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
release_id: ${{ steps.draft.outputs.release_id }}
already_published: ${{ steps.draft.outputs.already_published }}
steps:
- name: Require a final vX.Y.Z tag
env:
TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
if ! [[ "$TAG" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::${TAG} is not a final vX.Y.Z tag — rc/dev/alpha/beta releases never finalize."
exit 1
fi
# Drafts are invisible to read-only tokens and unaddressable by tag
# (the get-by-tag endpoint 404s on drafts) — resolve by listing with the
# App token, same as draft-release-notes.yml. Scoped to BOTH repos: an
# installation token cannot reach outside its grant, and the docs sweep
# below queries omnigent-site.
- name: Mint App token (omnigent + omnigent-site)
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.OMNIGENT_BOT_APP_ID }}
private-key: ${{ secrets.OMNIGENT_BOT_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: omnigent,omnigent-site
- name: Resolve the draft release
id: draft
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
match="$(gh api "repos/${GITHUB_REPOSITORY}/releases" --paginate \
--jq 'map(select(.tag_name == env.TAG)) | first // empty')"
if [ -z "$match" ]; then
echo "::error::No GitHub release found for ${TAG}. Did the tag push run github-release.yml?"
exit 1
fi
is_draft="$(printf '%s' "$match" | jq -r '.draft')"
release_id="$(printf '%s' "$match" | jq -r '.id')"
already_published=false
if [ "$is_draft" != "true" ]; then
already_published=true
echo "Release ${TAG} is already published — nothing to do (idempotent no-op)." \
| tee -a "$GITHUB_STEP_SUMMARY"
fi
{
echo "release_id=${release_id}"
echo "already_published=${already_published}"
} >> "$GITHUB_OUTPUT"
- name: Assert PyPI serves all three packages
if: steps.draft.outputs.already_published != 'true'
env:
TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
version="${TAG#v}"
for pkg in omnigent omnigent-client omnigent-ui-sdk; do
if ! curl -fsS "https://pypi.org/pypi/${pkg}/${version}/json" >/dev/null; then
echo "::error::${pkg}==${version} is not on PyPI — run the secure-repo publish first (never advertise an uninstallable release)."
exit 1
fi
echo "PyPI OK: ${pkg}==${version}"
done
- name: Assert the CHANGELOG PR is not open
if: steps.draft.outputs.already_published != 'true'
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
open_pr="$(gh pr list --repo "$GITHUB_REPOSITORY" --head "auto/changelog/${TAG}" \
--state open --json url --jq '.[0].url // empty')"
if [ -n "$open_pr" ]; then
echo "::error::The CHANGELOG PR for ${TAG} is still open — merge it first: ${open_pr}"
exit 1
fi
echo "CHANGELOG PR for ${TAG}: merged or not needed."
- name: Docs sweep — no open PRs against the X.Y-docs staging branch
if: steps.draft.outputs.already_published != 'true'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
TAG: ${{ inputs.tag }}
SITE_REPO: ${{ github.repository_owner }}/omnigent-site
run: |
set -euo pipefail
version="${TAG#v}"
docs_branch="${version%.*}-docs"
open="$(gh pr list --repo "$SITE_REPO" --base "$docs_branch" --state open \
--json url,title --jq '.[] | "- \(.url) \(.title)"')"
if [ -n "$open" ]; then
{
echo "## Docs sweep failed for ${TAG}"
echo ""
echo "Open PRs still target \`${docs_branch}\` on ${SITE_REPO} — review and merge/close them, then re-dispatch:"
echo "$open"
} | tee -a "$GITHUB_STEP_SUMMARY"
echo "::error::Open doc PRs still target ${docs_branch} — see the run summary."
exit 1
fi
echo "Docs sweep clean: no open PRs against ${docs_branch}." | tee -a "$GITHUB_STEP_SUMMARY"
# Approving this environment attests "I reviewed the curated draft notes".
publish:
needs: [authorize, checks]
if: needs.checks.outputs.already_published != 'true'
runs-on: ubuntu-latest
timeout-minutes: 5
environment: publish-release
steps:
- name: Mint App token (omnigent)
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.OMNIGENT_BOT_APP_ID }}
private-key: ${{ secrets.OMNIGENT_BOT_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: omnigent
- name: Publish the draft as Latest
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
TAG: ${{ inputs.tag }}
RELEASE_ID: ${{ needs.checks.outputs.release_id }}
run: |
set -euo pipefail
# Edit by id (drafts 404 by tag). -F sends real booleans; make_latest
# must be explicit — API publishes don't set it.
gh api --method PATCH "repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}" \
-F draft=false -f make_latest=true > /dev/null
{
echo "## Published ${TAG} as Latest"
echo ""
echo "The \`release: published\` event now fires (App-token publish):"
echo "- **publish-changelog.yml** opens the omnigent-site release-post PR and the docs-publish PR — review and merge both."
echo "- **update-homebrew.yml** opens the homebrew-tap bump PR — review the resource diff, then apply the \`pr-pull\` label."
} >> "$GITHUB_STEP_SUMMARY"
-26
View File
@@ -11,9 +11,6 @@ on:
push:
branches:
- main
# Release branches: release.yml's green-CI gate reads check runs off the
# branch head, so cherry-picks and release-bump commits must run checks.
- 'branch-[0-9]*'
permissions:
contents: read
@@ -128,26 +125,3 @@ jobs:
- name: Type-check web
working-directory: web
run: npm run type-check
# The three packages release in lockstep (identical versions + `==` sibling
# pins). Assert agreement on every change so drift from a bad merge or
# cherry-pick — however it happened — is caught before it reaches a release.
version-lockstep:
name: Version lockstep check
needs: gate
if: ${{ !github.event.pull_request.draft }}
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Check out repo
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version-file: ".python-version"
- name: All version locations agree
run: |
python -m pip install --quiet --disable-pip-version-check packaging
python scripts/update_versions.py check
+1 -1
View File
@@ -27,7 +27,7 @@ on:
pull_request_target:
types: [labeled]
workflow_run:
workflows: [PR Template, CI, Lint, Docker build, E2E UI Tests, E2E Tests, Integration Tests]
workflows: [PR Template, CI, Lint, E2E UI Tests, E2E Tests, Integration Tests]
types: [completed]
issue_comment:
types: [created]
+66 -16
View File
@@ -12,8 +12,10 @@
# `pip install omnigent` resolves to. Pre-releases never move it.
# :latest-rc the highest version OVERALL, max(release, rc) — the newest
# thing tagged, pre-release or not.
# :latest-nightly the most recent nightly main build (bleeding edge); moves
# once a day when the scheduled build rebuilds main HEAD.
# :latest-dev the most recent main build (bleeding edge); moves on every
# qualifying main commit.
# :latest-nightly the most recent main build as of the daily cron; retagged
# from :latest-dev once a day (no rebuild).
# Ordering for :latest / :latest-rc uses PEP 440 (1.2.3rc1 < 1.2.3), which
# `sort -V` gets wrong, so the max is computed with .github/scripts/
# oss-publish-images/maxver.py (Python `packaging`).
@@ -23,15 +25,23 @@
name: Publish images (public)
on:
# Release builds only — every v* tag push publishes the immutable version pin
# and moves the floating release tags. Per-commit main builds were retired in
# favour of the nightly rebuild below; PRs get a build-only check (docker-build.yml)
# so a broken image is caught before merge without a push.
push:
branches: [main]
tags: ['v*']
# Nightly rebuild of main HEAD (07:00 UTC): the build-and-push job publishes
# :sha-<short> + :latest-nightly. This is what keeps bleeding-edge ~1 day
# fresh now that main commits no longer each trigger a build.
# Only rebuild when something that lands in the image changes.
paths:
- 'deploy/docker/Dockerfile'
- 'deploy/docker/entrypoint.py'
- 'omnigent/**'
- 'web/**'
- 'sdks/**'
- 'pyproject.toml'
- 'setup.py'
- 'uv.lock'
- 'web/package-lock.json'
- '.github/workflows/oss-publish-images.yml'
# Daily nightly promotion (07:00 UTC). Retags the current :latest-dev as
# :latest-nightly — handled by promote-nightly, not a rebuild.
schedule:
- cron: '0 7 * * *'
workflow_dispatch:
@@ -40,6 +50,10 @@ on:
description: 'Also move :latest to this build (manual release of latest). Off by default.'
type: boolean
default: false
force_nightly:
description: 'Promote :latest-dev -> :latest-nightly now (runs only the nightly job). Off by default.'
type: boolean
default: false
reconcile_floating:
description: 'Repoint :latest and :latest-rc onto the correct existing version images (no rebuild). Runs only the reconcile job. Off by default.'
type: boolean
@@ -59,11 +73,10 @@ jobs:
permissions:
contents: read
packages: write # push the image to GHCR via GITHUB_TOKEN
# Gated to this repository; inert in forks and mirrors. Runs on tag pushes,
# the nightly schedule (rebuild of main HEAD), and bump_latest dispatches.
# Skipped on reconcile_floating dispatches — that only drives the
# reconcile-floating retag job.
if: github.repository == 'omnigent-ai/omnigent' && !inputs.reconcile_floating
# Gated to this repository; inert in forks and mirrors. Skip the (re)build
# on schedule, force_nightly, and reconcile_floating dispatches — those only
# drive the promote-nightly / reconcile-floating jobs.
if: github.repository == 'omnigent-ai/omnigent' && github.event_name != 'schedule' && !inputs.force_nightly && !inputs.reconcile_floating
runs-on: ubuntu-latest
# Multi-arch: the linux/arm64 leg cross-builds under QEMU emulation on this
# amd64 runner, which roughly doubles the host-image build time (emulated
@@ -128,9 +141,9 @@ jobs:
KUBERNETES_TAGS="${KUBERNETES_TAGS},${KUBERNETES_IMAGE}:$1"
}
# The nightly rebuild of main moves :latest-nightly (bleeding edge).
# Every qualifying main commit moves :latest-dev (bleeding edge).
if [ "${GH_REF}" = "refs/heads/main" ]; then
add_tag "latest-nightly"
add_tag "latest-dev"
fi
if [[ "${GH_REF}" == refs/tags/v* ]]; then
@@ -316,6 +329,43 @@ jobs:
kubernetes-sbom.spdx.json
retention-days: 90
promote-nightly:
# Daily cron (or a manual force_nightly dispatch): move :latest-nightly to
# the current main build by retagging :latest-dev with `crane tag`
# (digest-preserving, no rebuild).
if: github.repository == 'omnigent-ai/omnigent' && (github.event_name == 'schedule' || inputs.force_nightly)
permissions:
contents: read
packages: write # retag within GHCR via GITHUB_TOKEN
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Set up crane
uses: imjasonh/setup-crane@59c71e96a00b28651f10369ba3359a6d730740a0 # v0.6
with:
version: v0.21.6
- name: Log in to GHCR
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Promote latest-dev -> latest-nightly
run: |
set -euo pipefail
# crane tag points a new tag at an EXISTING manifest digest without
# re-serializing it, so :latest-nightly keeps :latest-dev's exact digest.
for img in ghcr.io/omnigent-ai/omnigent-server ghcr.io/omnigent-ai/omnigent-host ghcr.io/omnigent-ai/omnigent-server-openshell ghcr.io/omnigent-ai/omnigent-server-kubernetes; do
if crane digest "${img}:latest-dev" >/dev/null 2>&1; then
crane tag "${img}:latest-dev" latest-nightly
echo "promoted ${img}:latest-dev -> :latest-nightly ($(crane digest "${img}:latest-nightly"))"
else
echo "::warning::${img}:latest-dev not found yet; skipping nightly promotion"
fi
done
reconcile-floating:
# Manual reconcile (workflow_dispatch with reconcile_floating=true): repoint
# :latest and :latest-rc onto the correct EXISTING version images, computed
-364
View File
@@ -1,364 +0,0 @@
# Cut or advance a release deterministically (designs/RELEASE-AUTOMATION.md):
#
# dispatch with version=0.6.0rc1 -> create branch-0.6 from `ref`, stamp the
# lockstep version (scripts/update_versions.py + `uv lock`), tag v0.6.0rc1,
# push branch + tag. Later dispatches (0.6.0rc2, 0.6.0, 0.6.1) reuse the
# existing branch-0.6 head and ignore `ref`.
#
# The branch + tag are pushed with the omnigent-ci App token, NOT GITHUB_TOKEN:
# GITHUB_TOKEN-pushed tags trigger no workflows by GitHub policy, and the whole
# release chain (github-release.yml -> draft-release-notes.yml, and
# oss-publish-images.yml) hangs off the tag push.
#
# PyPI publishing does NOT happen here — after this run, dispatch the secure
# release repo on the tag (see RELEASING.md). Everything here is idempotent:
# re-dispatch with identical inputs after any failure and it converges
# (branch exists -> reused; version stamped -> no new commit; tag at the
# converged commit -> no-op; tag anywhere else -> loud failure).
#
# `dry_run` defaults TRUE (repo convention, same as the vscode release
# workflows): the plan job prints exactly what would happen; nothing is pushed.
name: Release
on:
workflow_dispatch:
inputs:
version:
description: "Version to release, e.g. 0.6.0rc1 or 0.6.0 (no leading v)."
required: true
type: string
ref:
description: "Branch/tag/SHA to cut branch-X.Y from. Only consulted when the branch does not exist yet (rc1); later phases build from the existing branch head."
required: false
default: main
type: string
dry_run:
description: "Plan only: validate + print what would happen, push nothing."
required: false
type: boolean
default: true
skip_ci_check:
description: "Skip the green-CI assertion on the base commit (flaky-check escape hatch — use deliberately)."
required: false
type: boolean
default: false
# Nothing here writes with GITHUB_TOKEN; pushes use the App token.
permissions:
contents: read
# Serialize all release runs: two concurrent cuts (even of different versions)
# could race the same branch-X.Y head.
concurrency:
group: release
cancel-in-progress: false
jobs:
# Releases are maintainer-only. `workflow_dispatch` is open to anyone with
# write access, so gate on the dispatcher's actual repo role instead of a
# hand-kept list. `github.actor` on a dispatch is the dispatcher.
authorize:
if: github.repository == 'omnigent-ai/omnigent'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Require admin/maintain role
env:
GH_TOKEN: ${{ github.token }}
ACTOR: ${{ github.actor }}
run: |
set -euo pipefail
role="$(gh api "repos/${GITHUB_REPOSITORY}/collaborators/${ACTOR}/permission" --jq .role_name)"
case "$role" in
admin|maintain)
echo "Dispatcher ${ACTOR} has role ${role} — authorized." | tee -a "$GITHUB_STEP_SUMMARY" ;;
*)
echo "::error::Release workflows require the admin or maintain role (dispatcher ${ACTOR} has '${role}')."
exit 1 ;;
esac
# Resolve everything and validate BEFORE mutating anything. Runs checkout-free
# (pure API reads) and also serves as the whole dry run.
plan:
needs: authorize
runs-on: ubuntu-latest
timeout-minutes: 10
outputs:
version: ${{ steps.derive.outputs.version }}
tag: ${{ steps.derive.outputs.tag }}
branch: ${{ steps.derive.outputs.branch }}
prerelease: ${{ steps.derive.outputs.prerelease }}
branch_exists: ${{ steps.state.outputs.branch_exists }}
base_sha: ${{ steps.state.outputs.base_sha }}
already_done: ${{ steps.state.outputs.already_done }}
steps:
- name: Validate version and derive names
id: derive
env:
VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
# Final X.Y.Z or a PEP 440 pre-release (a/b/rc). No dev/post here.
if ! [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+((a|b|rc)[0-9]+)?$ ]]; then
echo "::error::Invalid release version: ${VERSION} (expect 0.6.0 or 0.6.0rc1)"; exit 1
fi
major="${VERSION%%.*}"; rest="${VERSION#*.}"; minor="${rest%%.*}"
prerelease=false
case "$VERSION" in *a[0-9]*|*b[0-9]*|*rc[0-9]*) prerelease=true ;; esac
{
echo "version=${VERSION}"
echo "tag=v${VERSION}"
echo "branch=branch-${major}.${minor}"
echo "prerelease=${prerelease}"
} >> "$GITHUB_OUTPUT"
- name: Resolve branch, base commit, and tag state
id: state
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ steps.derive.outputs.version }}
TAG: ${{ steps.derive.outputs.tag }}
BRANCH: ${{ steps.derive.outputs.branch }}
REF: ${{ inputs.ref }}
run: |
set -euo pipefail
# `gh api` prints the error body to STDOUT on 404, so capturing with
# `|| true` would treat the "Not Found" JSON as an existing ref —
# gate on the exit code instead.
if branch_sha="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/heads/${BRANCH}" --jq .object.sha 2>/dev/null)"; then
branch_exists=true
base_sha="$branch_sha"
# `ref` only applies at branch creation. An explicit non-default ref
# that disagrees with the branch head is a mistake, not a retarget.
if [ "$REF" != "main" ]; then
ref_sha="$(gh api "repos/${GITHUB_REPOSITORY}/commits/${REF}" --jq .sha)"
if [ "$ref_sha" != "$branch_sha" ]; then
echo "::error::${BRANCH} already exists at ${branch_sha}; ref=${REF} (${ref_sha}) would not be used. Re-dispatch without ref, or delete the branch if this is recovery."
exit 1
fi
fi
else
branch_exists=false
base_sha="$(gh api "repos/${GITHUB_REPOSITORY}/commits/${REF}" --jq .sha)"
fi
# Tag state: absent -> normal; at the converged release commit ->
# no-op; anywhere else -> refuse (never silently move a tag).
already_done=false
if tag_sha="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/tags/${TAG}" --jq .object.sha 2>/dev/null)"; then
tag_type="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/tags/${TAG}" --jq .object.type)"
if [ "$tag_type" = "tag" ]; then
tag_sha="$(gh api "repos/${GITHUB_REPOSITORY}/git/tags/${tag_sha}" --jq .object.sha)"
fi
stamped="$(gh api -H "Accept: application/vnd.github.raw+json" \
"repos/${GITHUB_REPOSITORY}/contents/pyproject.toml?ref=${TAG}" \
| sed -n 's/^version = "\(.*\)"$/\1/p' | head -1)"
if [ "$tag_sha" = "$base_sha" ] && [ "$stamped" = "$VERSION" ]; then
already_done=true
echo "Tag ${TAG} already at the converged release commit ${base_sha} — nothing to do." \
| tee -a "$GITHUB_STEP_SUMMARY"
else
echo "::error::Tag ${TAG} already exists at ${tag_sha} (stamped version: ${stamped:-unknown}), which is not the converged branch head ${base_sha}. Delete the tag first if this is recovery (see RELEASING.md)."
exit 1
fi
fi
{
echo "branch_exists=${branch_exists}"
echo "base_sha=${base_sha}"
echo "already_done=${already_done}"
} >> "$GITHUB_OUTPUT"
- name: Assert green CI on the base commit
if: steps.state.outputs.already_done != 'true' && !inputs.skip_ci_check
env:
GH_TOKEN: ${{ github.token }}
BASE_SHA: ${{ steps.state.outputs.base_sha }}
run: |
set -euo pipefail
runs="$(gh api "repos/${GITHUB_REPOSITORY}/commits/${BASE_SHA}/check-runs?per_page=100" \
--paginate --jq '.check_runs[] | [.name, .status, .conclusion // "-"] | @tsv')"
total="$(printf '%s' "$runs" | grep -c . || true)"
pending="$(printf '%s' "$runs" | awk -F'\t' '$2 != "completed"' || true)"
# Cancelled runs are chronically present on main (superseded
# benchmark/eval runs) — warn, don't block; real failures still gate.
bad="$(printf '%s' "$runs" | awk -F'\t' '$3 ~ /^(failure|timed_out|action_required|startup_failure)$/' || true)"
cancelled="$(printf '%s' "$runs" | awk -F'\t' '$3 == "cancelled"' || true)"
if [ -n "$bad" ]; then
echo "::error::Failing check runs on ${BASE_SHA}:"; printf '%s\n' "$bad"; exit 1
fi
if [ -n "$pending" ]; then
echo "::error::Check runs still running on ${BASE_SHA} — wait for CI:"; printf '%s\n' "$pending"; exit 1
fi
if [ "$total" -eq 0 ]; then
echo "::error::No check runs found on ${BASE_SHA}. Wait for CI on that commit, or re-dispatch with skip_ci_check=true if you are sure."
exit 1
fi
if [ -n "$cancelled" ]; then
echo "::warning::Cancelled (superseded) check runs on ${BASE_SHA} — not blocking:"
printf '%s\n' "$cancelled"
fi
echo "CI green on ${BASE_SHA} (${total} completed check runs, none failing)." \
| tee -a "$GITHUB_STEP_SUMMARY"
- name: Write the plan
env:
DRY_RUN: ${{ inputs.dry_run }}
VERSION: ${{ steps.derive.outputs.version }}
TAG: ${{ steps.derive.outputs.tag }}
BRANCH: ${{ steps.derive.outputs.branch }}
BRANCH_EXISTS: ${{ steps.state.outputs.branch_exists }}
BASE_SHA: ${{ steps.state.outputs.base_sha }}
ALREADY_DONE: ${{ steps.state.outputs.already_done }}
run: |
set -euo pipefail
{
echo "## Release plan for ${TAG}"
echo ""
echo "| | |"
echo "| --- | --- |"
echo "| Version | \`${VERSION}\` |"
echo "| Branch | \`${BRANCH}\` ($([ "$BRANCH_EXISTS" = "true" ] && echo "exists — reused" || echo "will be created")) |"
echo "| Base commit | \`${BASE_SHA}\` |"
echo "| Converged already | ${ALREADY_DONE} |"
echo "| Mode | $([ "$DRY_RUN" = "true" ] && echo "DRY RUN — nothing pushed" || echo "EXECUTE") |"
} >> "$GITHUB_STEP_SUMMARY"
# Stamp + tag + push. Only reached on a real run that isn't already converged.
cut:
needs: [authorize, plan]
if: ${{ !inputs.dry_run && needs.plan.outputs.already_done != 'true' }}
runs-on: ubuntu-latest
timeout-minutes: 15
env:
# Clean public resolution for `uv lock` — the committed lockfile must
# reference https://pypi.org/simple (never a proxy).
UV_INDEX_URL: https://pypi.org/simple
PIP_INDEX_URL: https://pypi.org/simple
steps:
- name: Mint App token (omnigent)
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.OMNIGENT_BOT_APP_ID }}
private-key: ${{ secrets.OMNIGENT_BOT_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: omnigent
- name: Checkout base
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
# Existing branch: its head. New branch: the resolved base commit.
ref: ${{ needs.plan.outputs.branch_exists == 'true' && needs.plan.outputs.branch || needs.plan.outputs.base_sha }}
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version-file: ".python-version"
- name: Install uv
uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: false
- name: Stamp the lockstep version
env:
VERSION: ${{ needs.plan.outputs.version }}
run: |
set -euo pipefail
current="$(uv run --no-project --python 3.12 --with packaging \
python scripts/update_versions.py check 2>/dev/null || true)"
if [ "$current" = "$VERSION" ]; then
echo "Already stamped at ${VERSION} — skipping bump (idempotent re-run)."
else
uv run --no-project --python 3.12 --with packaging \
python scripts/update_versions.py pre-release --new-version "$VERSION"
uv lock
fi
uv run --no-project --python 3.12 --with packaging \
python scripts/update_versions.py check --expect "$VERSION"
- name: Commit, tag, and push
env:
PUSH_TOKEN: ${{ steps.app-token.outputs.token }}
VERSION: ${{ needs.plan.outputs.version }}
TAG: ${{ needs.plan.outputs.tag }}
BRANCH: ${{ needs.plan.outputs.branch }}
run: |
set -euo pipefail
git config user.name "omnigent-ci[bot]"
git config user.email "294685417+omnigent-ci[bot]@users.noreply.github.com"
git add pyproject.toml sdks/python-client/pyproject.toml sdks/ui/pyproject.toml \
omnigent/version.py uv.lock
if git diff --cached --quiet; then
echo "No version changes to commit (already stamped)."
else
git commit -s -m "release: ${TAG}"
fi
git tag "$TAG"
# One push for branch + tag, via the App token so the tag-push
# workflows fire. Non-fast-forward on the branch fails loudly.
push_url="https://x-access-token:${PUSH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git"
git push "$push_url" "HEAD:refs/heads/${BRANCH}" "refs/tags/${TAG}"
echo "Pushed ${BRANCH} + ${TAG} at $(git rev-parse HEAD)." | tee -a "$GITHUB_STEP_SUMMARY"
- name: Next steps
env:
TAG: ${{ needs.plan.outputs.tag }}
PRERELEASE: ${{ needs.plan.outputs.prerelease }}
run: |
set -euo pipefail
{
echo "## Next steps"
echo ""
echo "1. Dispatch the secure-release repo on this tag:"
echo ' ```'
echo " gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \\"
echo " -f ref=${TAG} -f destination=pypi -f dry-run=true # gates rehearsal"
echo " gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \\"
echo " -f ref=${TAG} -f destination=pypi -f dry-run=false # real publish"
echo ' ```'
if [ "$PRERELEASE" = "true" ]; then
echo "2. Validate the rc from PyPI (see RELEASING.md). The GitHub draft for ${TAG} stays unpublished."
else
echo "2. Merge the CHANGELOG PR, curate the ${TAG} draft notes, then dispatch finalize-release.yml (tag=${TAG})."
fi
} >> "$GITHUB_STEP_SUMMARY"
# First cut of a cycle (rc1) immediately moves main to the next .dev0 so main
# never re-freezes and doc-sync keeps deriving the right X.Y-docs branch.
bump-main:
needs: [authorize, plan, cut]
if: ${{ !inputs.dry_run && needs.plan.outputs.branch_exists == 'false' }}
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
actions: write
steps:
- name: Dispatch the post-release main bump
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ needs.plan.outputs.version }}
run: |
set -euo pipefail
# A cut below main's current line (a throwaway rehearsal rc, or
# resurrecting an old series for a backport) must not walk main's
# version backwards.
MAIN_VERSION="$(gh api -H "Accept: application/vnd.github.raw+json" \
"repos/${GITHUB_REPOSITORY}/contents/pyproject.toml?ref=main" \
| sed -n 's/^version = "\(.*\)"$/\1/p' | head -1)"
export MAIN_VERSION
python3 -m pip install --quiet --disable-pip-version-check packaging
if ! python3 -c 'import os, sys; from packaging.version import Version; sys.exit(0 if Version(os.environ["VERSION"]) > Version(os.environ["MAIN_VERSION"]) else 1)'; then
echo "Released ${VERSION} sorts below main's ${MAIN_VERSION} — skipping the main bump." \
| tee -a "$GITHUB_STEP_SUMMARY"
exit 0
fi
gh workflow run bump-version.yml --repo "$GITHUB_REPOSITORY" \
-f mode=post-release -f "new_version=${VERSION}" -f base_branch=main
echo "Dispatched bump-version.yml (post-release ${VERSION}) — review and merge the main bump PR." \
| tee -a "$GITHUB_STEP_SUMMARY"
-247
View File
@@ -1,247 +0,0 @@
# Open the omnigent-ai/homebrew-tap version-bump PR when a FINAL release is
# published (designs/RELEASE-AUTOMATION.md). This is the missing link that let
# the tap freeze while PyPI moved on: the tap already builds bottles on every
# PR (brew test-bot) and publishes them on the `pr-pull` label — nobody was
# opening the bump PR.
#
# What it does: wait for the new sdist on PyPI, rewrite the formula's
# url/sha256 (dropping any bottle `revision`), regenerate the pinned Python
# resources with `brew update-python-resources`, sanity-check that the
# hand-maintained sections survived, and open the tap PR. A human reviews the
# resource diff and applies `pr-pull`; the tap's own automation bottles and
# merges. The omnigent-desktop cask is `version :latest` and needs nothing.
#
# Pre-releases never reach the tap. The `release: published` trigger fires
# from finalize-release.yml's App-token publish; `workflow_dispatch` covers
# retries and catch-up (e.g. jumping the formula straight to the newest
# version after a missed cycle).
name: Update Homebrew tap
on:
release:
types: [published]
workflow_dispatch:
inputs:
tag:
description: "Final release tag to bump the tap to, e.g. v0.6.0."
required: true
type: string
permissions:
contents: read
concurrency:
group: update-homebrew-${{ github.event.release.tag_name || inputs.tag }}
cancel-in-progress: false
jobs:
resolve:
name: Resolve release tag
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
tag: ${{ steps.r.outputs.tag }}
is_final: ${{ steps.r.outputs.is_final }}
steps:
- name: Resolve tag and finality
id: r
env:
EVENT_TAG: ${{ github.event.release.tag_name }}
PRERELEASE: ${{ github.event.release.prerelease }}
INPUT_TAG: ${{ inputs.tag }}
run: |
set -euo pipefail
tag="${INPUT_TAG:-$EVENT_TAG}"
is_final=true
case "$tag" in
v[0-9]*.[0-9]*.[0-9]*) ;;
*) is_final=false ;;
esac
case "$tag" in
*rc*|*dev*|*a[0-9]*|*b[0-9]*) is_final=false ;;
esac
if [ "${PRERELEASE}" = "true" ]; then
is_final=false
fi
{
echo "tag=${tag}"
echo "is_final=${is_final}"
} >> "$GITHUB_OUTPUT"
echo "Resolved tag=${tag} is_final=${is_final}" | tee -a "$GITHUB_STEP_SUMMARY"
bump:
name: Open tap bump PR
needs: resolve
# Canonical repo only; skip cleanly where the App isn't configured. The
# release-event path is already gated by finalize-release's environment
# approval; only manual dispatches need the role check below.
if: >-
needs.resolve.outputs.is_final == 'true' &&
github.repository == 'omnigent-ai/omnigent' &&
vars.OMNIGENT_BOT_APP_ID != ''
# macOS: `brew update-python-resources` evaluates the formula (with its
# on_macos blocks) in a real Homebrew.
runs-on: macos-latest
timeout-minutes: 30
env:
TAG: ${{ needs.resolve.outputs.tag }}
TAP_REPO: ${{ github.repository_owner }}/homebrew-tap
steps:
- name: Require admin/maintain role (manual dispatches)
if: github.event_name == 'workflow_dispatch'
env:
GH_TOKEN: ${{ github.token }}
ACTOR: ${{ github.actor }}
run: |
set -euo pipefail
role="$(gh api "repos/${GITHUB_REPOSITORY}/collaborators/${ACTOR}/permission" --jq .role_name)"
case "$role" in
admin|maintain)
echo "Dispatcher ${ACTOR} has role ${role} — authorized." ;;
*)
echo "::error::Release workflows require the admin or maintain role (dispatcher ${ACTOR} has '${role}')."
exit 1 ;;
esac
- name: Wait for the sdist on PyPI
id: sdist
run: |
set -euo pipefail
version="${TAG#v}"
echo "version=${version}" >> "$GITHUB_OUTPUT"
for _ in $(seq 1 30); do
if json="$(curl -fsS "https://pypi.org/pypi/omnigent/${version}/json" 2>/dev/null)"; then
url="$(printf '%s' "$json" | jq -r '.urls[] | select(.packagetype == "sdist") | .url')"
sha="$(printf '%s' "$json" | jq -r '.urls[] | select(.packagetype == "sdist") | .digests.sha256')"
if [ -n "$url" ] && [ -n "$sha" ]; then
{
echo "url=${url}"
echo "sha=${sha}"
} >> "$GITHUB_OUTPUT"
echo "sdist for ${version}: ${url}"
exit 0
fi
fi
echo "omnigent==${version} not visible on PyPI yet — retrying in 20s…"
sleep 20
done
echo "::error::omnigent==${version} never appeared on PyPI (is the secure-repo publish done?)."
exit 1
- name: Mint App token (homebrew-tap)
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.OMNIGENT_BOT_APP_ID }}
private-key: ${{ secrets.OMNIGENT_BOT_APP_KEY }}
owner: ${{ github.repository_owner }}
repositories: homebrew-tap
- name: Checkout the tap
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
repository: ${{ env.TAP_REPO }}
token: ${{ steps.app-token.outputs.token }}
path: tap
persist-credentials: false
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@18fcb8e3e06b4247c676c506750dc95ea7226479 # 2026-07-10
with:
token: ${{ github.token }}
- name: Rewrite the formula's stable url/sha256
working-directory: tap
env:
SDIST_URL: ${{ steps.sdist.outputs.url }}
SDIST_SHA: ${{ steps.sdist.outputs.sha }}
run: |
set -euo pipefail
python3 - <<'PYEOF'
import os, pathlib, re
path = pathlib.Path("Formula/omnigent.rb")
text = path.read_text(encoding="utf-8")
# The formula's own url/sha256 sit at 2-space indent; resource and
# bottle entries are deeper, so first-match at this indent is safe.
text, n_url = re.subn(r'(?m)^ url ".*"$', f' url "{os.environ["SDIST_URL"]}"', text, count=1)
text, n_sha = re.subn(r'(?m)^ sha256 ".*"$', f' sha256 "{os.environ["SDIST_SHA"]}"', text, count=1)
text, _ = re.subn(r'(?m)^ revision \d+\n', "", text, count=1)
assert n_url == 1 and n_sha == 1, f"unexpected formula shape (url={n_url}, sha={n_sha})"
path.write_text(text, encoding="utf-8")
PYEOF
git diff --stat
- name: Regenerate the pinned Python resources
env:
HOMEBREW_NO_AUTO_UPDATE: "1"
HOMEBREW_NO_INSTALL_FROM_API: "1"
run: |
set -euo pipefail
# Make the checkout visible to brew as the real tap.
tap_root="$(brew --repository)/Library/Taps/omnigent-ai"
mkdir -p "$tap_root"
ln -sfn "${GITHUB_WORKSPACE}/tap" "${tap_root}/homebrew-tap"
# Excluded packages stay hand-maintained in the formula: the brewed
# deps (certifi/cryptography/pydantic/rpds-py and their transitive
# cffi/pycparser) and the platform-conditional google-antigravity
# wheel stanzas.
brew update-python-resources \
--exclude-packages=certifi,cryptography,pydantic,rpds-py,cffi,pycparser,google-antigravity \
omnigent-ai/tap/omnigent
brew style omnigent-ai/tap/omnigent
- name: Assert the hand-maintained sections survived
working-directory: tap
run: |
set -euo pipefail
fail=0
for needle in 'resource "google-antigravity"' 'depends_on "pydantic"' 'depends_on "cryptography"'; do
if ! grep -qF "$needle" Formula/omnigent.rb; then
echo "::error::update-python-resources dropped: ${needle} — fix the formula by hand this cycle."
fail=1
fi
done
[ "$fail" -eq 0 ]
# The lockstep siblings must have moved with the release. Match the
# sdist filename (PEP 503-normalized name + version) in the resource
# url, not a bare version substring.
version="${TAG#v}"
for sib in omnigent-client omnigent-ui-sdk; do
if ! grep -A2 "resource \"${sib}\"" Formula/omnigent.rb | grep -q "${sib//-/_}-${version}"; then
echo "::error::resource ${sib} did not update to ${version}."
exit 1
fi
done
- name: Open or update the tap bump PR
working-directory: tap
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
VERSION: ${{ steps.sdist.outputs.version }}
run: |
set -euo pipefail
if [ -z "$(git status --porcelain -- Formula/omnigent.rb)" ]; then
echo "Formula already at ${VERSION} — nothing to do." | tee -a "$GITHUB_STEP_SUMMARY"
exit 0
fi
BRANCH="bump-omnigent-${VERSION}"
git config user.name "omnigent-ci[bot]"
git config user.email "294685417+omnigent-ci[bot]@users.noreply.github.com"
PUSH_URL="https://x-access-token:${GH_TOKEN}@github.com/${TAP_REPO}.git"
git switch -C "$BRANCH"
git add Formula/omnigent.rb
git commit -m "omnigent ${VERSION}"
git push --force "$PUSH_URL" "$BRANCH"
if [ -n "$(gh pr list --repo "$TAP_REPO" --head "$BRANCH" --state open --json number --jq '.[].number')" ]; then
echo "Bump PR already open for ${BRANCH} — force-push updated it." | tee -a "$GITHUB_STEP_SUMMARY"
exit 0
fi
body="$(printf 'Bumps the omnigent formula to **%s** (new sdist url/sha256, resources regenerated via `brew update-python-resources`).\n\ntest-bot builds the bottles on this PR. Review the resource diff — especially that the extras'"'"' deps survived — then apply the `pr-pull` label to publish bottles and merge.\n\nOpened by omnigent `.github/workflows/update-homebrew.yml`.' "$VERSION")"
gh pr create \
--repo "$TAP_REPO" \
--base main \
--head "$BRANCH" \
--title "omnigent ${VERSION}" \
--body "$body"
echo "Opened tap bump PR for omnigent ${VERSION}." | tee -a "$GITHUB_STEP_SUMMARY"
-10
View File
@@ -105,16 +105,6 @@ repos:
files: ^uv\.lock$
pass_filenames: true
# Fail if routing.proto changed without regenerating the committed
# bindings (or vice versa). Verify-only, not a fixer: regen needs
# grpcio-tools, so CI's `uv sync --extra dev` enforces it (like ktlint).
- id: routing-pb2-fresh
name: routing protobuf bindings are up to date
language: system
entry: .venv/bin/python scripts/gen_routing_pb2.py --check
files: ^omnigent/api/routing/v1/routing(\.proto|_pb2\.pyi?)$
pass_filenames: false
# ── File hygiene ────────────────────────────────────────────────
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
-91
View File
@@ -5,97 +5,6 @@ generated at release time from each PR's `## Changelog` section, tagged by the
PR's `Type of change` (e.g. `[UI]`); the concise, curated highlights live on the
website under `/releases`.
## [Unreleased]
### Features
- [UI / Feature] Added a Nord color theme (arctic frost-blue palette) to the Appearance settings palette picker.
## [v0.5.0] — 2026-07-10
- [Bug fix] Messaging a long-idle session no longer risks the new turn being killed mid-flight by the idle reaper (#1834)
- [UI / Feature] Introduce more secure sharing modes and the ability to toggle public chats on/off. (#1835)
- [UI / Feature] Added: `.ipynb` notebooks render as read-only previews in the workspace file viewer (raw JSON still available via the source view) (#1848)
- [Feature] `OMNIGENT_OIDC_SKIP_EMAIL_VERIFICATION=1` lets OIDC logins through when the IdP omits the `email_verified` claim (e.g. standard-tier Okta with directory-provisioned users) (#1859)
- [UI / Feature] User message bubbles now have a copy button, matching assistant responses (#1900)
- [UI] Renamed the sidebar's "Chats" section to "Sessions" to match the "New session" button (#1903)
- [UI / Bug fix] Brain-harness override (e.g. claude-sdk vs openai-agents) is now remembered across sessions per agent (#1904)
- [UI / Bug fix] "Back to Omnigent" from Settings now returns you to the conversation you were viewing instead of the home page (#1905)
- [Bug fix] Release notes now list only user-facing bug fixes and call out breaking changes in their own section (#1909)
- [Test/CI] Auto-drafted docs now stage on a per-minor `X.Y-docs` branch and publish to the live site at release, instead of deploying on merge. (#1915)
- [UI] Removed the collapse toggle from the Files panel "Working folder" header — the file list is always visible (#1916)
- [UI / Bug fix] Opencode agents addressed as `native-opencode` now render with their native terminal UI instead of falling back to plain chat. (#1929)
- [Bug fix / Chore] Fixed harness workers (claude, codex, etc.) failing to start when omnigent is launched from a macOS or Linux GUI client due to a stripped PATH. Fix now lives in the Electron launcher (web/electron/src/main.js) per reviewer guidance. (#1935)
- [Feature] Child-session lookup by `(agent, title)` now filters server-side instead of fetching all children and scanning in Python. (#1944)
- [Bug fix] Sandboxed claude-sdk harnesses now authenticate from an existing host Claude login (`~/.claude/.credentials.json` is bound into the sandbox). (#1946)
- [Chore / Test/CI] Runner MCP servers are shared across matching agent specs and started lazily to reduce local memory use. (#1948)
- [Bug fix] Fixed: resumed claude-native sessions no longer crash on compaction ("Cannot destructure property 'cumulativeDroppedTokens'") (#1957)
- [UI / Feature] The Claude model picker now offers Fable and both Sonnet generations (Sonnet 5 and Sonnet 4.6) as separate selections (#1981)
- [Bug fix] Stop rendering a false "terminal did not become ready" error when sending a message to Claude Code mid-turn (#2001)
- [UI] [UI] The "Working…" indicator now stays visible for the whole turn and rotates through a few different labels. (#2006)
- [Bug fix] Members page now shows a clear "not available in single-user mode" message instead of a confusing auth error when running without accounts or OIDC. (#2013)
- [UI / Bug fix] Global Policies settings page now appears correctly in single-user/header auth mode instead of showing a "no permission" error. (#2017)
- [Feature] `intent_gate` policy now prompts for user approval (`ASK`) instead of hard-blocking (`DENY`) tool calls that don't match the session's original intent. (#2024)
- [UI / Bug fix] Submitting the Codex goal dialog no longer shifts the footer buttons — the loading spinner replaces the button label in place instead of widening the button (#2032)
- [UI / Feature] Add a UI font size setting in Appearance to scale the interface (#2040)
- [Bug fix] `/compact` on a `claude-sdk` agent with a pinned Anthropic model no longer 500s — the compaction summarizer was routing bare `claude-*` ids to OpenAI instead of Anthropic. (#2043)
- [UI / Feature] Set a custom UI font family in Settings → Appearance (type any installed font; blank = system default). (#2047)
- [UI / Bug fix] Fix the Appearance font-size input so you can clear and retype a value instead of it clamping mid-edit (#2053)
- [Bug fix] Native Claude sessions no longer get stuck showing "Stop" after switching models in the terminal with `/model` (#2082)
- [UI / Feature] The sidebar "Search" now opens the command palette (⌘K) to search sessions by title and chat content, with a keyboard-shortcut hint on hover (#2086)
- [UI / Feature] Start a new session directly in an existing git worktree by picking it from the worktree field. (#2088)
- [Bug fix] Stop rendering a false "terminal did not become ready" error when sending a message to Claude Code mid-turn with many subagents running (#2089)
- [UI / Feature] Generate a unique worktree branch name from the new-session composer. (#2094)
- [Feature] The harness capability bench now observes native harness tool calls (Tool (#2096)
- [Bug fix] Report missing bubblewrap when building a `web_fetch` researcher instead of failing during spawn (#2097)
- [UI / Feature] Sessions started in an existing git worktree now show the branch in the sidebar and can delete the worktree + branch from the session delete dialog. (#2098)
- [Bug fix] Fixed OpenShell k8s managed sandboxes failing due to Landlock LSM denying `/home/sandbox`; changed home path to `/sandbox` (#2106)
- [UI / Bug fix] The share dialog no longer overflows when a grantee's email is long — the name truncates and the domain stays visible. (#2108)
- [Bug fix / Test/CI] Keep claude-native model, permission mode, and effort overrides stable across wrapped Claude Code restarts that preserve the settings sidecar. (#2116)
- [Feature] Kubernetes sandbox runner Pods can now schedule on arm64 nodes: set `sandbox.kubernetes.node_selector: {kubernetes.io/arch: arm64}` (amd64 remains the default). (#2123)
- [Feature / Test/CI] New official `omnigent-server-kubernetes` image ships the kubernetes sandbox provider SDK — the `sandbox-runners` overlay now works against published images, no custom build needed. (#2124)
- [UI / Bug fix] codex-native sessions now show MCP server startup progress in the chat, name servers that failed or were cancelled, and Stop can abort a slow MCP startup (#2128)
- [Bug fix] Host-spawned runners now inherit `DATABRICKS_AUTH_STORAGE`, so a runner authenticates against the same Databricks token store as the host (fixes a runner tunnel 401 when the store is selected via env var rather than `~/.databrickscfg`). (#2132)
- [UI / Feature] Set the code editor and terminal font size and family from Settings → Appearance (#2135)
- [Bug fix] Intelligent routing now correctly routes claude sessions instead of leaving them (#2136)
- [Bug fix] Fixed inbox approvals not resuming the gated tool call. (#2142)
- [UI / Feature] Pick a color theme (Omnigent, Dracula, GitHub, Catppuccin, or Gruvbox) in Appearance settings, independent of light/dark mode. (#2147)
- [UI / Feature] Choose a terminal theme (light or dark) independent of the app theme in Settings, Appearance (#2154)
- [UI / Feature] Sessions shared with you now live in a dedicated "Shared with me" sidebar tab (multi-user servers only) (#2156)
- [Feature] Tightened `conversations.title` DB column to NOT NULL; untitled conversations are now stored as `''` instead of `NULL`. (#2158)
- [Feature / Test/CI] Add a performance-benchmark harness for HTTP user journeys, with a seeded corpus, a SQLite+Postgres backend matrix, and a nightly workflow (`uv run dev/benchmarks/omnigent/run.py`) (#2159)
- [Bug fix] Sub-agent hermes sessions no longer wake their parent orchestrator before the turn's final answer is mirrored into the transcript (#2161)
- [UI / Feature] Session search now shows a preview of the matching message so you can see why a session matched, with the search term highlighted (#2162)
- [Feature / Test/CI] Host runner start logs now include the `conv_*` conversation ID alongside the runner token and log path. (#2170)
- [Bug fix] The harness capability bench now reports a real native Policy DENY verdict (#2171)
- [UI / Bug fix] Cancel in the add-policy dialog now returns to the policy list instead of closing it (#2183)
- [UI / Feature] Users can now edit the policy name in the Add Policy dialog before submitting. (#2196)
- [Feature / Test/CI] Add a performance-benchmark harness for HTTP + full-turn user journeys (`uv run dev/benchmarks/omnigent/run.py`), with a seeded corpus and SQLite+Postgres backend matrix (#2202)
- [UI / Bug fix] The new-session picker now remembers the host you last picked instead of resetting to the default. (#2218)
- [Bug fix] Fixed the Hermes `pre_tool_call` hook double-gating Omnigent relay tools, which parked a (#2220)
- [UI / Chore] Redesigned Appearance settings: separate Mode and Color theme sections, app-preview Mode tiles, and a color-theme dropdown. (#2225)
- [UI / Feature] Added: auto-routing decisions now show as a collapsible card (model pill, tier, rationale, expandable raw verdict) matching the SmartRoutingCard style (#2246)
- [Bug fix] Sessions shared with you no longer appear under "My sessions" when they belong to a project — they stay under "Shared with me" (#2249)
- [Test/CI] Doc-sync site PRs are now titled after the documentation change instead of the source PR number. (#2250)
- [UI / Bug fix] Stop-session dialog now shows the actual server error instead of a generic message. (#2252)
- [UI / Bug fix] Project picker menu rows now align on the left and share a consistent height (#2260)
- [Feature] The harness bench can now probe any registered harness by name — including the (#2265)
- [UI / Feature] A default base branch can be set in Settings Git to auto-fill the base when naming a new worktree branch (#2267)
- [Feature] `omnigent debug logs` tails runner, server, or CLI diagnostic logs; `--session` scopes runner logs to a specific session across relaunches (#2273)
- [Bug fix] `omni run --harness acp:<slug>` now launches a configured ACP agent instead of failing on the colon in the synthesized agent name. (#2280)
- [UI / Bug fix] [UI] Fix iOS crash when granting camera or voice-dictation permission in the app (#2282)
- [Test/CI] DELETE THIS WHOLE SECTION — CI-only change, not user-facing. (#2288)
- [Bug fix / Feature] Fixed: intelligent routing now overrides any model the orchestrator specified in `sys_session_send` when the parent session has the routing toggle on (#2291)
- [Bug fix] Fixed a crash when resuming a Claude-native session whose history contained a `TaskOutput` (or similar) result, so resume no longer times out with a terminal-not-ready error. (#2293)
- [Test/CI] DELETE THIS WHOLE SECTION — CI-only change, not user-facing. (#2295)
- [UI / Bug fix] "Select all" in bulk selection mode now only selects sessions in expanded sidebar sections, not hidden or archived ones. (#2311)
- [Bug fix] Fix pi (and opencode policy) losing live web-UI updates on multi-instance deployments by sending their out-of-process callbacks to the same server instance as the runner. (#2328)
- [Bug fix] Default policies created via the API (`POST /v1/policies`) now take effect on sessions. (#2333)
- [Feature] omnidev dev pods now get their own isolated `config.yaml` (seeded from `~/.omnigent/config.yaml`), so server-config edits while testing in a pod no longer touch your real config (#2360)
- [Bug fix] Session search returns matched-content previews faster on large histories. (#2365)
- [Feature / Docs / Test/CI] Harness Bench now measures Policy ALLOW and ASK through native CLI policy hooks. (#2370)
- [Bug fix] Managed claude-native sessions against an Anthropic-compatible gateway (e.g. LiteLLM or Databricks) now pass through the gateway model and don't stall on Claude Code's custom-API-key menu. (#2371)
## [v0.4.0] — 2026-07-03
Highlights and full notes: <https://github.com/omnigent-ai/omnigent/releases/tag/v0.4.0>
+1 -65
View File
@@ -67,26 +67,6 @@ One command installs Omnigent and everything it needs:
curl -fsSL https://raw.githubusercontent.com/omnigent-ai/omnigent/main/scripts/install_oss.sh | sh
```
<details>
<summary>Optional integrations and extras</summary>
Need an optional integration? Pass one or more extras to the installer:
```bash
curl -fsSL https://raw.githubusercontent.com/omnigent-ai/omnigent/main/scripts/install_oss.sh | sh -s -- --extra databricks
curl -fsSL https://raw.githubusercontent.com/omnigent-ai/omnigent/main/scripts/install_oss.sh | sh -s -- --extra modal,e2b
```
Available user-facing extras include:
- **Model providers:** `databricks`, `bedrock`, `vertex`
- **Sandbox providers:** `modal`, `daytona`, `boxlite`, `cwsandbox`, `e2b`,
`openshell`, `kubernetes`
- **SDK harnesses:** `antigravity`, `copilot`, `cursor`, `agents-sdk`
- **Storage and memory:** `s3`, `hindsight`
</details>
<details>
<summary>Prefer to install manually?</summary>
@@ -96,12 +76,6 @@ Omnigent needs **Python 3.12+**. Install the `omnigent` package:
uv tool install omnigent # or: pip install "omnigent"
```
Manual installs use the same extras syntax, for example:
```bash
uv tool install "omnigent[databricks,modal]"
```
Or with [Homebrew](https://github.com/omnigent-ai/homebrew-tap):
```bash
@@ -199,41 +173,6 @@ mirrors work out of the box; override with `OMNIGENT_INDEX_URL` if needed.
</details>
<details>
<summary>Uninstalling Omnigent</summary>
Preview the CLI/profile cleanup that would run by default:
```bash
omnigent uninstall
```
Remove the CLI and installer-managed PATH entries while keeping your local
history, credentials, and projects:
```bash
omnigent uninstall --yes
```
To also remove Omnigent state under `~/.omnigent`, pass `--purge`; Omnigent
backs it up outside the target before deletion. Your `~/omnigent` workspace is
kept unless you explicitly add `--purge-workspace`.
```bash
omnigent uninstall --purge --yes
```
If the installed wheel is broken or `omnigent` is not on `PATH`, run the
standalone script instead:
```bash
curl -fsSL https://raw.githubusercontent.com/omnigent-ai/omnigent/main/scripts/uninstall_oss.sh | sh
```
Add `--yes` to the standalone script to perform the previewed CLI cleanup.
</details>
### 2. Start your first agent
`omnigent` picks a model with you and starts a session in your terminal. It
@@ -512,10 +451,6 @@ Polly at [`examples/polly/`](https://github.com/omnigent-ai/omnigent/tree/main/e
Contributions are welcome. See [CONTRIBUTING.md](https://github.com/omnigent-ai/omnigent/blob/main/CONTRIBUTING.md) for how to set up your environment, run the checks, and open a pull request.
Adding or changing support for a harness (Claude, Codex, Cursor, OpenCode,
Hermes, Pi, ...)? Run the [harness test bench](https://github.com/omnigent-ai/omnigent/tree/main/tests/harness_bench)
to check its capability matrix against observed behavior.
### Contributors
@@ -524,3 +459,4 @@ Thanks to all of our amazing contributors!
<a href="https://github.com/omnigent-ai/omnigent/graphs/contributors">
<img src="https://contrib.rocks/image?repo=omnigent-ai/omnigent" />
</a>
+187 -196
View File
@@ -13,11 +13,6 @@ omnigent ships **three PyPI packages that version-lock together**:
pin each other with `==`), so every release builds and publishes **all three at
one identical version**.
Releases are driven by **workflow dispatches, not by hand** (design:
`designs/RELEASE-AUTOMATION.md`). Every workflow below is idempotent —
re-dispatch with identical inputs after any failure and it converges — and
every dispatch requires the **admin or maintain** role on this repo.
## Where things run
- **Source of truth** (versions, tags, GitHub Releases): **`omnigent-ai/omnigent`**
@@ -25,31 +20,31 @@ every dispatch requires the **admin or maintain** role on this repo.
on the public repo).
- **Publishing to PyPI**: the central **secure-release repo**
**`databricks/secure-public-registry-releases-eng`**, `omnigent` workflow —
use whichever account has access to that repo. Publishing runs on hardened runner
use the **Databricks EMU account**. Publishing runs on hardened runner
groups with **OIDC Trusted Publishing (no stored secrets)** and a **mandatory
dependency scan**. This is why we don't publish from `omnigent-ai/omnigent`,
and why the pipeline is two dispatches per phase rather than one.
dependency scan**. This is why we don't publish from `omnigent-ai/omnigent`.
> The exact account handles — and how to request publish access — live in the
> internal release wiki; this public runbook refers to them only by role.
> Substitute your own handles for `<oss-account>` / `<emu-account>` in the
> `gh auth switch --user …` commands below.
The legacy `.github/workflows/release-omnigent.yml` in this repo is a
**deprecated manual fallback only** — its tag-push trigger was removed so a tag
never double-publishes. Use the secure repo for real releases.
> The secure `omnigent` workflow is **manual `workflow_dispatch`** — it can't see
> this repo's tag pushes. You bump + tag here, then dispatch it with that tag.
## Versioning model
- `main` always carries the **next** version with a `.dev0` suffix
(e.g. `0.6.0.dev0`) — never a clean released number. This matches
(e.g. `0.2.0.dev0`) — never a clean released number. This matches
MLflow / Delta / Unity Catalog and keeps every `main` build PEP 440-ordered as
"ahead of the last release, not yet the next one".
- Releases are cut on **per-minor release branches** (`branch-X.Y`) and tagged
there (`vX.Y.Z`, rc tags `vX.Y.ZrcN`); patches (`vX.Y.1`, `vX.Y.2`, …) are
cherry-picked onto the same `branch-X.Y`. `main` is never tagged.
- Every release ships as an **rc first** (`0.6.0rc1` → … → `0.6.0`). rcs go to
**real PyPI** as PEP 440 pre-releases — a default `pip install omnigent`
never resolves them, and testers install with exact pins. TestPyPI is no
longer part of the standard flow.
there (`vX.Y.Z`); patches (`vX.Y.1`, `vX.Y.2`, …) are cherry-picked onto the
same `branch-X.Y`. `main` is never tagged.
## Docs staging
@@ -61,225 +56,221 @@ branch** on `omnigent-site` instead of `main`:
- **`doc-sync.yml`** — drafts prose docs for each merged PR that needs them.
- **`sync-openapi-to-site.yml`** — syncs the API reference (`openapi.json`).
Both derive the branch name from `omnigent/version.py` (`0.6.0.dev0``0.6-docs`)
Both derive the branch name from `omnigent/version.py` (`0.5.0.dev0``0.5-docs`)
and create it off site `main` the first time a doc PR lands in the cycle. All docs
for the `0.6` line — including patches — accumulate on `0.6-docs`. Each PR still
for the `0.5` line — including patches — accumulate on `0.5-docs`. Each PR still
gets its own review, but merging one only lands it on the staging branch, not the
live site. At finalize time, the whole batch goes live at once (step 4 below).
live site.
At release, publishing the GitHub Release fires `publish-changelog.yml`, which
opens the **`0.5-docs → main`** PR (see step 5). Merging that publishes the whole
cycle's docs at once. Nothing to create or retarget by hand — the branch name
tracks `main`'s version automatically.
---
## Standard flow
## Release steps (example: `v0.2.0`)
### rc phase (example: `0.6.0rc1`)
### 1. Cut the release branch + tag — `omnigent-ai/omnigent` (OSS account)
**1. Cut + tag — dispatch `Release` (`release.yml`), OSS account.**
Only tag a commit that already has **green CI** — verify `main` is green before
branching:
```bash
gh workflow run release.yml --repo omnigent-ai/omnigent \
-f version=0.6.0rc1 -f dry_run=false
# optional: -f ref=<sha> to cut branch-0.6 from a specific commit (rc1 only);
# dry_run defaults to true — run once without -f dry_run to preview the plan.
gh auth switch --user <oss-account>
git fetch origin
gh run list --repo omnigent-ai/omnigent --branch main --status success --limit 1
git checkout -b branch-0.2 origin/main
```
What it does (all idempotent):
Set the release version in **all three** `pyproject.toml` files — the
`version` field **and** the cross-package `==` pins — plus `uv.lock`
(`0.2.0.dev0``0.2.0`):
- asserts green CI on the base commit (escape hatch: `-f skip_ci_check=true`,
use deliberately — needed for a flaky check, or when the base commit ran no
checks at all, e.g. a cherry-pick that only touched `paths-ignore`d files);
- creates `branch-0.6` from `ref` (rc1) or reuses the existing branch head
(rc2+, final, patches — `ref` is ignored then);
- stamps the lockstep version via `scripts/update_versions.py` and regenerates
`uv.lock` with a clean public-PyPI resolution — **never hand-edit `uv.lock`
or run `uv lock` behind a proxy**; the workflow owns this now;
- commits `release: v0.6.0rc1`, tags, and pushes branch + tag with the
omnigent-ci App token, which fires the downstream automation:
`github-release.yml` (draft GH release, pre-release flagged),
`draft-release-notes.yml`, and `oss-publish-images.yml` (Docker);
- on the **first** cut of a cycle (rc1), dispatches `bump-version.yml`
(post-release) — **review and merge the `main → 0.7.0.dev0` bump PR
promptly**, so `doc-sync` keeps staging to the right docs branch.
- `pyproject.toml` (`version`, `omnigent-client==`, `omnigent-ui-sdk==`)
- `sdks/python-client/pyproject.toml` (`version`, `omnigent==`)
- `sdks/ui/pyproject.toml` (`version`, `omnigent-client==`)
- `uv.lock`**hand-edit** the three `version = "…"` lines (omnigent,
omnigent-client, omnigent-ui-sdk) and the one cross-pin `specifier = "==…"`
(`omnigent-ui-sdk`'s dep on `omnigent-client`). The three packages are
**editable workspace members** (`source = { editable = … }`), so uv records
**no wheel `hash` entries** for them, and the other two cross-deps appear as
`editable = "…"` with no `==` specifier — so only those version/specifier
strings change, nothing else (no hashes to touch).
**Do not run `uv lock`** locally: it rewrites every registry URL to the
internal proxy and that leaks into the lockfile (breaks CI). The published
lock must use `https://pypi.org/simple`.
**2. Publish to PyPI — dispatch the secure repo (EMU account).**
Stage exactly the version files (don't `-a`, which would sweep in any stray
local edits), then commit, tag, and push **the branch + only this tag**:
```bash
gh auth switch --user <secure-repo-account>
git add pyproject.toml sdks/python-client/pyproject.toml sdks/ui/pyproject.toml uv.lock
git commit -m "release: v0.2.0"
git tag v0.2.0
git push -u origin branch-0.2 v0.2.0 # explicit tag, NOT --tags; pushing the tag drafts the GitHub Release (step 5)
```
> Pushing the tag also kicks off the **changelog automation** (see step 5):
> `github-release.yml` drafts the Release, then `draft-release-notes.yml` opens a
> `CHANGELOG.md` PR and fills the draft with curated notes — both ready by the time
> you get to step 5.
Keep `main` from re-freezing — bump it to the next dev marker and push:
```bash
git checkout main
# set 0.2.0.dev0 -> 0.3.0.dev0 in the 3 pyprojects (+ pins) and uv.lock.
# Hand-edit uv.lock here too — same rule, do NOT run `uv lock` (it leaks the proxy URL).
git add pyproject.toml sdks/python-client/pyproject.toml sdks/ui/pyproject.toml uv.lock
git commit -m "chore: bump main to 0.3.0.dev0"
git push
```
### 2. Dry-run the gates — secure repo (EMU account)
```bash
gh auth switch --user <emu-account>
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.6.0rc1 -f destination=pypi -f dry-run=true # gates rehearsal
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.6.0rc1 -f destination=pypi -f dry-run=false # real publish
-f ref=v0.2.0 -f destination=test-pypi -f dry-run=true
```
The dry run exercises build + dependency scan + the gates (lockstep
version/pins, web-UI-in-wheel, `twine check`, smoke-install) and the OIDC
token exchange without uploading. The real run binds the per-package
Trusted-Publisher environments (may gate on reviewer approval) and re-verifies
that `ref` is exactly the tag and points at the built commit.
Runs build + dependency scan + the gates (lockstep version/pins, web-UI-in-wheel,
`twine check`, smoke-install) and the OIDC token exchange — **without uploading**.
**3. Validate from PyPI** (clean venv; exact pins resolve pre-releases):
### 3. Publish to TestPyPI + validate
```bash
python -m venv /tmp/omni-rc && /tmp/omni-rc/bin/pip install \
--index-url https://pypi.org/simple/ \
omnigent==0.6.0rc1 omnigent-client==0.6.0rc1 omnigent-ui-sdk==0.6.0rc1
/tmp/omni-rc/bin/omnigent --version # expect 0.6.0rc1
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.2.0 -f destination=test-pypi -f dry-run=false
```
The rc's GitHub draft stays **unpublished** — rc drafts are never published.
Need another candidate? Repeat with `0.6.0rc2` (fixes land on `branch-0.6`
first, via cherry-pick PRs or direct pushes; CI runs on `branch-*` pushes).
Validate in a clean venv. **Don't** use `--extra-index-url` with TestPyPI: pip
resolves each name across *both* indexes and picks the highest version, so anyone
squatting `omnigent` / `omnigent-client` / `omnigent-ui-sdk` on real PyPI at a
higher version wins the resolution (dependency confusion). Instead, take **deps
from real PyPI only** and the **candidates from TestPyPI only**, exact-pinned with
`--no-deps`:
### Final phase (example: `0.6.0`)
```bash
python -m venv /tmp/omni-rc
# 1) seed the dependency closure from REAL PyPI (the last released omnigent):
/tmp/omni-rc/bin/pip install --index-url https://pypi.org/simple/ omnigent
# 2) overlay the candidates from TestPyPI ONLY, exact-pinned, no deps:
/tmp/omni-rc/bin/pip install --index-url https://test.pypi.org/simple/ --no-deps \
omnigent==0.2.0 omnigent-client==0.2.0 omnigent-ui-sdk==0.2.0
/tmp/omni-rc/bin/omnigent --version # expect 0.2.0
```
1. **Cut + tag**: `gh workflow run release.yml -f version=0.6.0 -f dry_run=false`
— same as above; builds from the `branch-0.6` head.
2. **Publish to PyPI**: same secure-repo dispatches on `ref=v0.6.0`.
3. **Curate**: merge the `CHANGELOG.md` PR that `draft-release-notes.yml`
opened, and review/trim the curated notes in the `v0.6.0` draft on the
Releases page — whatever you leave becomes the website post.
4. **Finalize — dispatch `Finalize release` (`finalize-release.yml`)**:
> If this release **adds a new runtime dependency** the previous release didn't
> have, install it explicitly from real PyPI first
> (`/tmp/omni-rc/bin/pip install --index-url https://pypi.org/simple/ <dep>`) —
> never let a `--no-deps` TestPyPI install pull third-party deps from TestPyPI.
```bash
gh workflow run finalize-release.yml --repo omnigent-ai/omnigent -f tag=v0.6.0
```
### 4. Publish to PyPI (prod)
It verifies PyPI serves all three packages, the CHANGELOG PR isn't open,
and the **docs sweep**: no open PRs against `0.6-docs` on `omnigent-site`
(it lists any stragglers — get them reviewed and merged/closed, then
re-dispatch). Then it pauses on the **`publish-release` environment**;
approving it attests "I reviewed the draft notes". It publishes the release
as **Latest**, which fires:
- `publish-changelog.yml` → the site **release-post PR** and the
**`0.6-docs → main` docs-publish PR** — review and merge both;
- `update-homebrew.yml` → the **homebrew-tap bump PR** (new sdist pin +
regenerated resources; test-bot builds the bottles on it) — review the
resource diff, then apply the **`pr-pull`** label to bottle + merge.
Requires **admin/maintain** on the secure repo (if you hit a 403, request access
via the secure-release owning team / internal release wiki before proceeding);
binds the per-package `pypi-omnigent`, `pypi-omnigent-client`,
`pypi-omnigent-ui-sdk` Trusted-Publisher environments (may gate on reviewer
approval). The prod path also re-verifies that
`ref` is exactly the `vX.Y.Z` tag and that the tag points at the built commit.
### Patch release (example: `0.6.1`)
```bash
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.2.0 -f destination=pypi -f dry-run=false
Cherry-pick the fixes onto `branch-0.6` (CI runs on the push), then run the
same flow with `version=0.6.1` — an rc first if the patch warrants one. `main`
does not change for a patch, and a patch never needs a new branch.
uv tool install omnigent==0.2.0 # final sanity from real PyPI
```
> Note: the dispatch's `-f ref=v0.2.0` is the **omnigent source ref**; it is
> distinct from `gh workflow run --ref`, which selects the branch the *workflow
> definition* runs from (the secure repo's default).
### 5. Publish the GitHub Release — `omnigent-ai/omnigent` (OSS account)
Pushing the `v0.2.0` tag (step 1) set the **changelog automation** in motion —
two workflows have already done the prep for you:
- `github-release.yml` created a **draft** release.
- `draft-release-notes.yml` (fires right after) then:
1. opened a **`CHANGELOG.md` PR to `main`** — the granular, feature-level log,
harvested mechanically from each merged PR's `## Changelog` section; and
2. **filled the draft's body** with concise, curated notes (Major new features /
Breaking changes / Bug fixes — user-facing only), synthesized by an agent from
the merged PRs, with the original auto-notes tucked into a collapsed
`<details>` for reference. Security and CI/internal fixes are deliberately left
out of the highlights.
Now:
1. **Merge the `CHANGELOG.md` PR** as part of cutting the release, so the draft's
`Full Changelog` link (which points at `CHANGELOG.md` on `main`) resolves.
2. Open <https://github.com/omnigent-ai/omnigent/releases>, find the `v0.2.0`
draft, and **review/trim the curated notes** — they're a strong starting point,
not the final word. Lead with user-facing highlights; call out breaking changes.
Whatever you leave here becomes the website post, so curate it well.
3. **Publish the release** (ideally only after the prod PyPI publish in step 4 has
succeeded, so you never advertise a version that isn't installable).
Publishing a **final** release fires `.github/workflows/publish-changelog.yml`,
which opens **two** PRs to review and merge (pre-releases are skipped):
- **`omnigent-site` `/releases/<version>`** — a per-version post mirroring the
notes you just curated (PR refs and angle/brace characters are made MDX-safe for
you). Targets `main`.
- **`omnigent-site` `X.Y-docs → main`** — publishes the docs staged this cycle
(see [Docs staging](#docs-staging) below). Skipped if that branch doesn't exist
or has nothing beyond `main`. Review the batch and merge to take the version's
docs live.
To re-run either half for an already-cut tag: dispatch `draft-release-notes.yml`
with the `tag` (re-opens the CHANGELOG PR; it leaves the notes alone once the
release is published), or `publish-changelog.yml` with the `tag` (re-opens the
site post PR).
If the draft wasn't created (e.g. the workflow was disabled), do it manually:
```bash
gh auth switch --user <oss-account>
gh release create v0.2.0 --repo omnigent-ai/omnigent \
--draft --verify-tag --generate-notes --title "v0.2.0"
# review/edit, then publish from the Releases page (or `gh release edit v0.2.0 --draft=false`)
```
---
## One-time setup (repo admin)
## Patch release (e.g. `v0.2.1`)
- **`publish-release` environment** on `omnigent-ai/omnigent` with required
reviewers = the release managers. Without it the finalize publish job runs
ungated.
- **omnigent-ci App** installed on `omnigent-ai/homebrew-tap` (it already
covers `omnigent` and `omnigent-site`).
- **Tag ruleset** (recommended): restrict `v[0-9]*` create/update/delete to
the omnigent-ci App + admins, so no write-access account can start the
tag-push automation by hand.
Cherry-pick the fix onto the existing `branch-0.2`, then:
1. Confirm CI is green on `branch-0.2` after the cherry-pick
(`gh run list --repo omnigent-ai/omnigent --branch branch-0.2 --status success --limit 1`).
2. Bump the three versions/pins + `uv.lock` to `0.2.1` (same hand-edit rules as above).
3. Stage explicitly, commit, and tag **on `branch-0.2`**:
`git add <version files> && git commit -m "release: v0.2.1" && git tag v0.2.1 && git push origin branch-0.2 v0.2.1`.
4. Repeat steps 25.
`main` does **not** change for a patch, and a patch never needs a new
`branch-0.Y` — patches always ship from the existing minor branch.
---
## If a publish goes wrong (recovery)
**PyPI releases can't be deleted, only _yanked_**, and a version number once used
can never be reused. So:
- **Any workflow failed mid-run:** fix the cause and **re-dispatch with the
same inputs** — every step converges (branch exists → reused; version
stamped → no new commit; tag at the converged commit → no-op) or fails
loudly (tag elsewhere) rather than duplicating work.
- **Wrong commit tagged, nothing published yet:** delete the tag and draft
(`gh release delete vX.Y.Z`, `git push origin :refs/tags/vX.Y.Z`), then
re-dispatch `release.yml`.
- **rc is bad:** just cut the next rc — rcs are cheap and invisible to
default installs.
- **TestPyPI failed / candidate is bad:** bump to the next number (don't reuse the
version) and re-run — TestPyPI is disposable.
- **Prod publish partially succeeded** (e.g. two of three packages uploaded):
**yank** the published version(s) on PyPI (each affected project → *Manage*
*Releases* → *Yank*) so installs don't resolve a half-published set, then cut
the next version with the fix. Don't try to overwrite — Trusted Publishing /
`twine` rejects re-uploading an existing version.
- Publishing uses **OIDC Trusted Publishing (no stored secrets)**, so a failed
run leaks nothing — fix forward to the next version.
---
## Rehearsing the pipeline (throwaway release to TestPyPI)
To exercise the whole flow end to end without touching users, release a
deliberately **below-latest** rc (e.g. `0.0.1rc1`) and publish it to
**TestPyPI**. A below-latest rc is inert everywhere that matters: the GitHub
draft stays unpublished, Docker publishes only the immutable `:v0.0.1rc1`
image tag (`:latest` / `:latest-rc` only move for the highest version), the
notes/site/homebrew workflows ignore rc tags, `bump-main` skips itself (the
version sorts below main's), and nothing on TestPyPI is ever resolved by a
default `pip install`.
1. **Plan (read-only)** — dry run is the default:
```bash
gh workflow run release.yml --repo omnigent-ai/omnigent -f version=0.0.1rc1
```
2. **Execute**: re-run with `-f dry_run=false`. Expect `branch-0.0` + tag
`v0.0.1rc1` pushed, the tag firing the draft-release and image workflows,
and CI running on the branch push. If the CI gate rejects main's head
(failing or still-pending checks), that's the gate working — wait, or
re-dispatch with `-f ref=<green sha>` / `-f skip_ci_check=true`.
Cancelled (superseded) runs only warn.
3. **Idempotency**: dispatch the exact same command again — it must no-op
("already at the converged release commit").
4. **Secure-repo publish**, pointed at TestPyPI instead of PyPI:
```bash
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.0.1rc1 -f destination=test-pypi -f dry-run=true # gates only
gh workflow run omnigent.yml --repo databricks/secure-public-registry-releases-eng \
-f ref=v0.0.1rc1 -f destination=test-pypi -f dry-run=false # real TestPyPI upload
```
test-pypi runs skip the tag/version prod gate and the post-publish
`validate` job (TestPyPI lacks the dependency closure) and bind the shared
`test-pypi` environment. If an upload leg fails with an invalid-publisher
error, add the missing TestPyPI Trusted Publisher for that package and
re-dispatch — already-uploaded legs are skipped.
5. **Publish idempotency**: re-dispatch step 4's second command — all three
packages must skip as already published.
6. **Finalize gates (no side effects)**:
`gh workflow run finalize-release.yml -f tag=v0.0.1rc1` must fail fast
("not a final tag"), and `-f tag=v0.5.1` (any already-published release)
must no-op as already published.
Cleanup — delete everything the rehearsal minted:
```bash
gh release delete v0.0.1rc1 --repo omnigent-ai/omnigent --cleanup-tag --yes
gh api -X DELETE repos/omnigent-ai/omnigent/git/refs/heads/branch-0.0
```
Optionally delete the `v0.0.1rc1` image versions from GHCR. TestPyPI needs no
cleanup — the version number is burned there only, which is what TestPyPI is
for.
---
## Break-glass appendix (manual fallback)
If the workflows are unavailable, the flow can be driven by hand — but keep two
rules even then:
1. **Never hand-edit `uv.lock` and never run `uv lock` behind a proxy.** Use
`bump-version.yml` (mode `pre-release`, `base_branch=branch-X.Y`) to
produce the bump as a PR with a cleanly regenerated lockfile, and merge it.
2. **Push tags from an account, not automation you improvised** — the tag push
must fire `github-release.yml` et al., which a `GITHUB_TOKEN`-authored push
would not.
```bash
gh auth switch --user <oss-account>
git fetch origin && git checkout -b branch-0.6 origin/main # rc1 only
gh workflow run bump-version.yml -f mode=pre-release -f new_version=0.6.0rc1 \
-f base_branch=branch-0.6 # then merge the PR
git fetch origin && git checkout branch-0.6 && git pull
git tag v0.6.0rc1 && git push origin branch-0.6 v0.6.0rc1 # explicit tag, NOT --tags
```
Then continue from step 2 of the standard flow (secure-repo dispatches). If the
GH draft wasn't created, `gh release create vX.Y.Z --draft --verify-tag
--title vX.Y.Z` recreates it. To re-run the notes/site halves for an existing
tag, dispatch `draft-release-notes.yml` or `publish-changelog.yml` with the
`tag` input; for the tap, dispatch `update-homebrew.yml`.
*Releases**Yank*) so installs don't resolve a half-published set, then cut the
next patch with the fix. Don't try to overwrite — Trusted Publishing / `twine`
rejects re-uploading an existing version.
- **GitHub Release** for a version you abandoned:
`gh release delete vX.Y.Z --repo omnigent-ai/omnigent`, and drop the tag if it
shouldn't exist (`git push origin :refs/tags/vX.Y.Z`); re-tag only the corrected
commit.
- Publishing uses **OIDC Trusted Publishing (no stored secrets)**, so a failed run
leaks nothing — just fix forward to the next version.
+1 -2
View File
@@ -334,8 +334,7 @@ RUN set -eu; \
# site-packages and imports no longer require /build at runtime.
COPY --from=builder /opt/venv /opt/venv
COPY --from=builder /build /build
RUN pip install --no-cache-dir /build /build/sdks/python-client /build/sdks/ui \
&& ! grep -R --include='*.pth' --include='*.egg-link' -nE '/build(/|$)' /opt/venv/lib/python*/site-packages
RUN pip install --no-cache-dir /build
# Sandbox launchers exec commands through `bash -lc`, and Debian's
# /etc/profile unconditionally resets PATH for login shells — the ENV
+14 -61
View File
@@ -10,9 +10,10 @@ running Omnigent hosts, two ways:
a session is created with `"host_type": "managed"` and terminates it
when the session is deleted.
Sandboxes boot from the official prebaked host image. The Islo launcher
uses the Islo Python SDK, installed with the optional `omnigent[islo]`
extra, and authenticates with an API key.
Sandboxes boot from the official prebaked host image, so startup is
seconds. Unlike Modal and Daytona, the Islo launcher talks to the Islo
HTTP API directly through `httpx` (already an Omnigent dependency), so
there is **no provider SDK extra to install** — just an API key.
What makes Islo different from the other providers, and shapes the rest
of this guide:
@@ -30,13 +31,11 @@ of this guide:
## Prerequisites
Install Omnigent with the Islo extra, install the
[Islo CLI](https://docs.islo.dev), and create an API key. Make the key
available where the launcher runs — your shell for the CLI flow, the
**server** process for managed sandboxes:
Install the [Islo CLI](https://docs.islo.dev) and create an API key, then
make it available where the launcher runs — your shell for the CLI flow,
the **server** process for managed sandboxes:
```bash
pip install 'omnigent[islo]' # or: uv tool install 'omnigent[islo]'
curl -fsSL https://islo.dev/install.sh | sh # install the islo CLI
islo login # browser OAuth (one-time)
islo api-key create omnigent --show # prints an islo_key_… value
@@ -45,9 +44,9 @@ export ISLO_API_KEY=islo_key_…
# export ISLO_BASE_URL=https://api.islo.dev
```
`ISLO_API_KEY` is exchanged by the SDK for short-lived session tokens and
refreshed automatically. The key is the only required runtime credential;
no `~/.config` file is needed where the launcher runs.
`ISLO_API_KEY` is exchanged for a short-lived session token at
`POST /auth/token`; the token is cached until shortly before expiry. The
key is the only required credential — no SDK, no `~/.config` file.
> [!NOTE]
> **Islo cannot forward a local callback port into the sandbox.** The
@@ -96,7 +95,7 @@ pulls the image, not Omnigent).
Provision a sandbox and ship your local checkout into it:
```bash
omnigent sandbox create --provider islo --server https://your-host
omnigent sandbox create --provider islo
```
This pulls the host image, builds wheels from your local checkout, and
@@ -122,31 +121,6 @@ delete the old one (Islo sandboxes have no lifetime cap, so an abandoned
sandbox keeps billing until removed via `islo rm <id>` or the
[dashboard](https://app.islo.dev)).
### Live smoke checklist
Use this checklist before opening a provider-change PR, or when validating
a new Islo account/key. It assumes your Omnigent server is reachable from
Islo's cloud at `https://your-host` (for local testing, expose it with a
tunnel and use the public URL).
```bash
islo login
islo api-key create omnigent-smoke --show
export ISLO_API_KEY=islo_key_...
omnigent sandbox create --provider islo --server https://your-host
omnigent sandbox connect --provider islo \
--sandbox-id <id-printed-by-create> \
--server https://your-host
islo ls
islo rm <id-printed-by-create>
```
Expected result: `create` provisions the sandbox and ships wheels,
`connect` registers the host with the Omnigent server, `islo ls` shows the
sandbox while it exists, and `islo rm` deletes it. If `connect` cannot
reach the server, first verify the `--server` URL from a machine outside
your laptop network.
To inject LLM/git credentials into a CLI-launched sandbox, set
`OMNIGENT_ISLO_SANDBOX_ENV` in your shell to a comma-separated list of
variable names (e.g. `ANTHROPIC_API_KEY,GIT_TOKEN`) before running
@@ -210,12 +184,6 @@ Each managed sandbox authenticates back with a server-minted, per-launch
token (7-day TTL — see [Lifecycle](#lifecycle-notes)); no user
credentials enter the sandbox for the server connection.
Managed Islo sandboxes pause after 15 idle minutes by default. When a new
message arrives for a session bound to an offline Islo-managed host,
Omnigent resumes the same sandbox id, mints a fresh launch token, and
restarts `omnigent host` against the existing workspace. Deleting the
session still deletes the sandbox.
### Managed hosts and server auth
How the dial-back authenticates depends on how the **server** does auth,
@@ -266,12 +234,11 @@ sandbox:
env: [OPENAI_API_KEY, GIT_TOKEN] # copy from server env
base_url: https://api.islo.dev # non-default API endpoint
gateway_profile: default # Islo gateway for egress + credential injection
snapshot_name: omnigent-host-snapshot # optional named Islo snapshot
snapshot_name: warm-host # boot from a prebaked snapshot
workdir: /root/workspace # sandbox working directory
vcpus: 2
memory_mb: 4096
disk_gb: 20
idle_pause_after_s: 900 # null disables idle pause
```
## Model credentials (LLM keys)
@@ -474,21 +441,8 @@ guide](../modal/README.md#git-credentials-private-repositories).
yourself (`islo rm <id>`).
- **Resources.** Sandboxes default to 2 vCPUs and 4 GiB of memory;
override per managed launch with `vcpus` / `memory_mb` / `disk_gb`.
- **Snapshots.** Set `sandbox.islo.snapshot_name` to boot from a named
Islo snapshot instead of the configured image.
- **Idle pause.** Server-managed Islo sandboxes pause after 15 idle
minutes by default (`idle_pause_after_s: 900`). Set
`idle_pause_after_s: null` to opt out and manage sandbox lifetime
yourself. The policy is set when the sandbox is created, so changing it
affects new managed sandboxes, not existing ones. This uses Islo's
pause/resume lifecycle because the workspace survives and Omnigent can
wake it on the next message. Daytona's 15-minute provider default is
disabled in Omnigent instead, because Daytona auto-stop would otherwise
kill the host between turns.
- **Managed resume.** Paused or stopped server-managed Islo sandboxes can
resume in place under the same sandbox id and workspace. Session delete
still deletes the sandbox. This resume path is what wakes a 15-minute
idle-paused host on the next message.
- **Warm starts.** Set `sandbox.islo.snapshot_name` to boot from a
prebaked Islo snapshot instead of a cold image pull.
- **Provider-side lifecycle** (list / status / delete / stop) — use the
`islo` CLI (`islo ls`, `islo rm <id>`) or the
[dashboard](https://app.islo.dev) directly.
@@ -531,7 +485,6 @@ free credits. Rates: [islo.dev](https://islo.dev).
|---|---|---|
| `ISLO_API_KEY` | CLI machine / server | Islo API credentials (required) |
| `ISLO_BASE_URL` | CLI machine / server | Non-default Islo API endpoint (default `https://api.islo.dev`) |
| `ISLO_COMPUTE_URL` | CLI machine / server | Non-default Islo compute endpoint (SDK default is production compute) |
| `OMNIGENT_ISLO_HOST_IMAGE` | CLI machine / server | Override the host image ref (`sandbox.islo.image` takes precedence for managed) |
| `OMNIGENT_ISLO_SANDBOX_ENV` | CLI machine / server | Comma-separated launcher-side env var names to inject (`sandbox.islo.env` takes precedence for managed) |
| `OMNIGENT_RUNNER_ENV_PASSTHROUGH` | inside the sandbox (injected) | Extra env var names the host forwards to runners |
-475
View File
@@ -1,475 +0,0 @@
# Deterministic release pipeline
Status: accepted 2026-07-14; implemented in this repo 2026-07-15 (release.yml,
finalize-release.yml, update-homebrew.yml, bump-version App token, branch-CI
triggers, lockstep CI check, RELEASING.md rewrite). Secure-repo restructure and
the tag ruleset are follow-ups. Owner: @dhruv0811.
Today a release is an LLM agent (or human) walking `RELEASING.md` step by step:
~15 CLI commands across two GitHub accounts, two repos, a hand-edited lockfile,
and judgment calls interleaved with mechanical steps. Every step of that runbook
is either already a workflow or trivially expressible as one. This doc proposes
collapsing the mechanical 90% into **two `workflow_dispatch` runs per release
phase** (rc, then final), parameterized by `version` + `ref`, while keeping every
human-judgment point (publish approval, notes curation, docs review) as an
explicit gate rather than an implicit runbook step.
## What exists today (verified against the repo, 2026-07-14)
The pipeline is already more automated than RELEASING.md's manual framing
suggests. Per release step:
| Step | Mechanism today | Deterministic? |
| --- | --- | --- |
| Cut `branch-X.Y` from green main/SHA | human CLI | ❌ manual |
| Lockstep bump (3 `pyproject.toml` + `omnigent/version.py` + `uv.lock`) | `scripts/update_versions.py` (+ `bump-version.yml` wrapper) | ✅ exists, but human-invoked; RELEASING.md still says "hand-edit `uv.lock`" (CI `uv lock` has no proxy problem) |
| Tag `vX.Y.Z[rcN]` + push | human CLI | ❌ manual |
| Bump main to next `.dev0` | human CLI (or `bump-version.yml` post-release) | 🟡 semi |
| Draft GH release (prerelease flag for rc, rerun-safe) | `github-release.yml` on tag push | ✅ |
| CHANGELOG PR + LLM-curated draft notes | `draft-release-notes.yml` via `workflow_run` (final tags only) | ✅ |
| Secure-repo gates + PyPI publish | manual `gh workflow run omnigent.yml` ×23 (dry-run, [test-pypi], pypi) in `databricks/secure-public-registry-releases-eng` | ❌ manual dispatches |
| Post-publish validation (clean venv install + `--version`) | human CLI recipe | ❌ manual |
| Publish GH release as Latest | human UI click | ❌ manual (and API publish does **not** set `make_latest` unless told to) |
| Site release post + `X.Y-docs → main` PR | `publish-changelog.yml` on `release: published` | ✅ |
| Sweep open doc PRs against `X.Y-docs` before docs go live | nobody | ❌ missing |
| Docker images (`:vX.Y.Z`, `:latest`, `:latest-rc`) | `oss-publish-images.yml` on tag push, PEP 440-ordered moving tags | ✅ |
| Homebrew formula bump (`omnigent-ai/homebrew-tap`) | nobody — tap frozen at **0.2.0** while PyPI is at 0.5.1 | ❌ missing |
Internal precedent: the VS Code extension track already ships the exact target
shape — `vscode-release-pr.yml` (`version`, `dry_run` → bump PR) +
`vscode-extension-release.yml` (`version`, `dry_run` → build + draft release).
This proposal is the same pattern applied to the Python release.
Actual release history confirms the rc-then-final model this automates:
`v0.4.0rc1 → rc2 → v0.4.0`, `v0.5.0rc1 → rc2 → v0.5.0 → v0.5.1` (patch), with rc
GitHub releases left as prerelease drafts.
## Target model
Per phase (rc or final), the human does:
```
rc: dispatch release.yml (version=0.6.0rc1) # cut/bump/tag — one run
dispatch secure omnigent.yml (ref=v0.6.0rc1) # gates → [approve] → publish → validate
final: dispatch release.yml (version=0.6.0)
dispatch secure omnigent.yml (ref=v0.6.0)
…curate the draft notes, merge the CHANGELOG PR…
dispatch finalize-release.yml (tag=v0.6.0) # checks → [approve] → publish-as-Latest
…merge the two site PRs it triggers…
…review the auto-opened homebrew-tap bump PR, apply the pr-pull label…
```
Two runs per phase (finalize is the third, final-only, and exists to *gate*
judgment, not do work). Everything inside a run is deterministic, idempotent,
and re-dispatchable after a failure with the same inputs.
Deliberately **not** one run: the secure-repo dispatch stays separate because it
crosses the org/account boundary that repo exists to enforce. Auto-dispatching
it from the public repo would require storing a Databricks-account PAT in
`omnigent-ai/omnigent` — weakening the isolation for the sake of one saved
click. Rejected.
## Workflow 1 — `release.yml` (new, omnigent-ai/omnigent)
`workflow_dispatch` inputs:
- `version``0.6.0rc1` | `0.6.0` | `0.6.1` (no leading `v`; `.dev` rejected)
- `ref` — default `main`; branch/tag/SHA to cut from. **Only consulted when
`branch-X.Y` does not exist yet** (i.e. at rc1). Later rcs, the final, and
patches always build from the existing `branch-X.Y` head; passing a `ref` that
disagrees with it fails loudly instead of silently retargeting.
- `dry_run` — default `true` (repo convention, matches the vscode workflows):
run the whole plan, print it, push nothing.
Jobs:
1. **plan** (always): validate version shape (reuse `bump-version.yml`'s PEP 440
regex minus `.dev`); derive `branch-X.Y` + `vX.Y.Z[rcN]`; resolve the base SHA
(existing branch head, else `ref`); assert the tag doesn't exist (or already
points at the fully-converged state → declare no-op); assert the resolved
SHA's check suites are green (not just "some run on main succeeded"); for a
final, warn if no `vX.Y.*rc*` tag exists on the branch. Write the plan to the
step summary.
2. **execute** (`dry_run == false`): mint the omnigent-ci App token; create
`branch-X.Y` at the base SHA if missing; `update_versions.py pre-release
--new-version $VERSION`; `uv lock` (runner resolves against real PyPI — this
*retires the hand-edit-uv.lock ritual entirely*); `update_versions.py check`;
commit `release: vX.Y.Z` (skip when already stamped); tag; push branch + tag
**with the App token**. Pushing with the App token (not `GITHUB_TOKEN`) is
load-bearing: `GITHUB_TOKEN`-pushed tags do not trigger workflows, and the
whole downstream chain (`github-release.yml``draft-release-notes.yml`,
`oss-publish-images.yml`) hangs off that tag push.
3. **bump-main** (only when the branch was created in this run, i.e. rc1):
`gh workflow run bump-version.yml -f mode=post-release …` — opens the
`main → next .dev0` PR immediately at branch cut, exactly as RELEASING.md
step 1 prescribes ("keep main from re-freezing"). Merging it promptly also
matters for docs: `doc-sync.yml` derives the `X.Y-docs` staging branch from
main's version. **Decided:** `bump-version.yml` switches its PR-creation
push to the omnigent-ci App token (falling back to `GITHUB_TOKEN` where the
App vars are absent, e.g. forks) so CI runs on bump PRs — retiring the
documented "push an empty commit to kick CI" workaround.
4. **summary**: print the exact secure-repo dispatch command for this tag.
Idempotency contract: branch exists → reuse; version already stamped → no
commit; tag exists at the converged commit → no-op; tag exists elsewhere →
fail. A half-failed run is always safe to re-dispatch verbatim.
Security posture: this executes repo scripts from a maintainer-chosen,
CI-green commit under `workflow_dispatch` — the same trust level as the
existing `bump-version.yml`. The no-code-exec guarantee of `github-release.yml`
(which is *tag-triggered*, attacker-influenceable) is unaffected.
## Workflow 2 — secure repo `omnigent.yml` restructure
Today: 23 dispatches (dry-run=true, optional test-pypi, then pypi) with manual
validation between. Proposal — same file, split into three chained jobs so one
dispatch covers the user flow "dry-run, then real publish, then validate":
1. **gates** (always): build all three distributions once; dependency scan;
lockstep/pin verification; web-UI-in-wheel; `twine check`; smoke-install.
Upload the built artifacts as run artifacts. This *is* the dry run.
2. **publish**: `needs: gates`, bound to the protected Trusted-Publisher
environments (required reviewer = the human authorization click). Downloads
the **same artifacts** — never rebuilds, so what was scanned is what ships.
Before each upload, probe `https://pypi.org/pypi/<pkg>/<ver>/json` and skip
already-published packages (`skip-existing` semantics): a partially-failed
publish is healed by re-running instead of yanking, because the remaining
identical artifacts complete the set.
3. **validate**: `needs: publish`. Clean venv; poll the real index until all
three resolve (propagation lag, bounded ~10 min); `pip install
omnigent==X omnigent-client==X omnigent-ui-sdk==X` (exact rc pins resolve
without `--pre`); assert `omnigent --version` == X; import smoke. Replaces
the manual venv recipe.
`destination=test-pypi` and `dry-run=true` inputs stay for rehearsals, but the
standard flow no longer uses TestPyPI (per new policy: rc goes to real PyPI as a
PEP 440 prerelease, which default `pip install omnigent` never resolves — safer
than the TestPyPI dependency-confusion dance RELEASING.md currently documents).
Net: one dispatch, one approval click, per phase.
## Workflow 3 — `finalize-release.yml` (new, final releases only)
`workflow_dispatch` input: `tag` (e.g. `v0.6.0`).
1. **checks** (all fail with actionable links):
- tag is a final `vX.Y.Z`; a *draft* GH release exists for it;
- PyPI serves all three packages at the version (JSON API) — never publish
release notes for something uninstallable;
- the `auto/changelog/vX.Y.Z` CHANGELOG PR is merged;
- **docs sweep**: zero open PRs in `omnigent-site` with base `X.Y-docs`
the deterministic form of "all release docs PRs reviewed + merged/closed".
Each open PR is listed in the summary; resolving them stays human work.
2. **publish** behind a `publish-release` environment (required reviewer).
Approving *is* the attestation "I reviewed/curated the draft notes."
Then, with the App token: `gh release edit vX.Y.Z --draft=false --latest`.
Two footguns handled here that have bitten before: `--latest` must be
explicit (API publishes don't set `make_latest`), and the App token (not
`GITHUB_TOKEN`) ensures the `release: published` event actually fires
`publish-changelog.yml`, which opens the site release-post PR and the
`X.Y-docs → main` docs-publish PR.
3. **summary**: links to the two site PRs awaiting merge.
rc releases never finalize: their GH drafts stay unpublished prerelease drafts
(**decided**: keep exactly today's pattern — rc drafts are never published on
GitHub).
## Workflow 4 — `update-homebrew.yml` (new, final releases only)
Current state of `omnigent-ai/homebrew-tap`: a homebrew-core-style tap that is
already 2/3 automated —
- `Formula/omnigent.rb`: `Language::Python::Virtualenv` formula; stable
installs the **PyPI sdist** (url + sha256) with **94 pinned Python
resources**; a few deps come from brewed formulae instead
(`certifi`/`cryptography`/`pydantic`/`rpds-py` as `:no_linkage`, plus
`python@3.14`, `libyaml`, `tmux`, Rust build deps); hand-maintained
platform-conditional `google-antigravity` wheel stanzas; bottles hosted on
the tap's GitHub releases.
- `tests.yml`: `brew test-bot` on 3 macOS runners — on every PR it builds the
formula (i.e. builds the bottles) and uploads them as artifacts.
- `publish.yml`: on the `pr-pull` label, `brew pr-pull` publishes the bottles
to a tap release, rewrites the bottle block, merges to main.
The **only missing link is the bump PR** — nobody opens it, which is exactly
why the tap froze at 0.2.0 (2026-06-23) while PyPI moved to 0.5.1. The
`omnigent-desktop` cask needs nothing: it is `version :latest` /
`sha256 :no_check` against `omnigent.ai/download/mac`, i.e. evergreen.
New workflow in omnigent-ai/omnigent, shaped exactly like
`publish-changelog.yml` (event + dispatch fallback, App token, idempotent
PR-opening):
- Triggers: `release: types: [published]` (fires automatically from
finalize's App-token publish; guarded to final `vX.Y.Z` like
publish-changelog) + `workflow_dispatch(tag)` for retries and catch-up.
- Steps: bounded-poll the PyPI JSON API until the new sdist is visible; on a
macOS runner with `Homebrew/actions/setup-homebrew`, check out the tap via
an App token (App installed on `homebrew-tap`); rewrite `url`/`sha256` from
the PyPI metadata and drop any `revision`; regenerate the resource pins with
`brew update-python-resources` (excluding the brewed-formula deps and the
hand-maintained `google-antigravity` stanzas so they're preserved); run
`brew style`/`brew audit` as a sanity gate; push `bump-omnigent-<version>`
and open (or update) the tap PR.
- From there the tap's own machinery takes over: test-bot builds the bottles
on the PR; a human reviews the resource diff and applies `pr-pull`; the
existing publish workflow bottles + merges. One review + one label click per
final release — the human gate the tap already has, kept.
First run doubles as the **catch-up**: dispatch with `tag=v0.5.1` to jump the
formula 0.2.0 → 0.5.1 (expect that one resource diff to be large).
## Who can trigger a release (maintainer-only)
`workflow_dispatch` is runnable by anyone with write access, which is too
broad. Every release workflow (`release.yml`, `finalize-release.yml`,
`update-homebrew.yml`'s dispatch path) gets a first `authorize` job that all
other jobs `need`:
```
role=$(gh api "repos/$GITHUB_REPOSITORY/collaborators/${GITHUB_ACTOR}/permission" --jq .role_name)
case "$role" in admin|maintain) ;; *) fail "release workflows require maintain/admin" ;; esac
```
`github.actor` on a dispatch is the dispatcher and can't be spoofed; roles
come from repo settings, so there's no hand-kept allowlist to rot. Defense in
depth stacks three independent layers: this actor gate (highest repo
privilege to start anything), the `v[0-9]*` **tag ruleset** (create/update/
delete restricted to the omnigent-ci App + admins — even a bypassed workflow
can't tag; goose's primary gate), and the secure repo's own access model
(admin/maintain to dispatch, environment reviewers on the upload). The
alternative — a required-reviewer environment on the first job — adds an
approval click and a separately-maintained reviewer list for no additional
precision; rejected.
## What stays human, on purpose
1. Choosing version/timing/base commit (the dispatches).
2. Secure-repo environment approval — publish authorization.
3. Release-notes curation + the finalize approval that attests to it.
4. Content review merges: CHANGELOG PR, bump-main PR, doc PRs on `X.Y-docs`,
the release-post PR, the docs-publish PR.
4a. The homebrew-tap bump PR: review the resource diff, apply `pr-pull`.
5. Yank decisions when something shipped broken (policy unchanged: never reuse
a version; `skip-existing` re-runs heal *partial* publishes, yank handles
*bad* ones).
## Recovery model
Any run can be re-dispatched with identical inputs after any failure; every
step converges or fails loudly rather than duplicating. Pre-publish mistakes
(wrong commit tagged): delete tag + draft, re-dispatch — unchanged from
RELEASING.md. Post-publish: fix forward to the next version.
## Cleanups this unlocks
- **Delete `release-omnigent.yml`** — its own header says "to be deleted once
the secure path has done a prod release", which has now happened repeatedly.
Also retire its `pypi`/`test-pypi` Trusted Publishers on PyPI: a live trusted
publisher pointing at the public repo is standing attack surface.
- Rewrite `RELEASING.md` around the dispatches, demoting today's CLI runbook to
a break-glass appendix. The `uv.lock` hand-edit instructions disappear.
## What peer projects do (survey, 2026-07)
### pi (`earendil-works/pi`)
Lean solo-maintainer automation, no release branches, no rc channel — cadence
(a release every 12 days) substitutes for candidates. Mechanics worth noting:
- **Draft-then-flip**: binaries staged on a *draft* GH release; the release is
made public only after npm publish succeeds; any failure deletes the draft;
the workflow *refuses to mutate an already-published release*.
- **Idempotent publish**: `npm view <pkg>@<ver>` before every upload, skip if
present — re-running a tag workflow after a partial failure heals it.
(The direct inspiration for the `skip-existing` PyPI probe above.)
- **Recovery dispatch**: the tag-triggered build workflow has a
`workflow_dispatch` twin with `tag` + `source_ref`, labeled "release
recovery only".
- Lockstep versions across 4 npm packages enforced by one sync script with a
check mode (their `sync-versions.js` ≈ our `update_versions.py`).
- Release notes: maintainer runs pi's own `/cl` prompt to audit CHANGELOG
entries with a human-confirm step — the same posture as our
`draft-release-notes.yml` + human curation.
- Pre-publish smoke is a *manual* isolated-install checklist in AGENTS.md;
**no automated post-publish validation exists** in their CI.
### opencode (`anomalyco/opencode`)
Continuous-publish machine: every push to `dev` ships an npm prerelease under
a branch-named dist-tag; an hourly bot assembles a `beta` branch (with their
own agent resolving merge conflicts); a real "latest" release is **one
`workflow_dispatch` click** (bump dropdown) — build, sign, notarize, npm,
Docker, AUR, Homebrew, LLM-authored release notes, Discord announce, all
unattended. Relevant mechanics:
- Bot pushes via a **GitHub App token** (`create-github-app-token`), never a
PAT — same identity pattern as our omnigent-ci App.
- Same idempotent already-published-skip before every npm publish.
- npm auth is OIDC trusted publishing, zero registry tokens in CI.
- Fully autonomous LLM changelog with *no* human review gate, and no
environment protection on the publish job at all — a rigor level below what
a Databricks-governed project should copy.
- Docs are evergreen/unversioned, deployed on push, fully decoupled from
releases.
### Cross-cutting (both)
- **Neither peer automates post-publish validation** (clean-env install of
the just-published artifact + run it). The `validate` job in the secure repo
puts omnigent ahead of both, not just at parity.
- **Neither has an rc→final concept** — both rebuild rather than promote.
Rebuilding the final from the same `branch-X.Y` (rather than promoting rc
artifacts) is also what our model does; PyPI's no-reupload rule makes
rebuild-and-restamp the pragmatic norm.
- Both decouple docs publishing from the release pipeline structurally — which
supports keeping our site PRs as separate human-reviewed merges rather than
folding them into `release.yml`.
### cline (`cline/cline`)
Three independent release trains (VS Code extension, CLI, SDK), all
`workflow_dispatch`, all preconditioned on a *human-authored* version-bump +
changelog PR — despite appearances, no bot writes their bumps. Worth stealing:
- **Tag/SHA idempotency guard** (`ext-vscode-publish-stable.yml`, "Resolve
Release Tag"): tag exists → assert it points at the tested SHA (no-op on
match, hard-fail on mismatch); tag absent → create it from the tested SHA
after asserting that SHA is an ancestor of `main`. Verbatim the semantics
`release.yml`'s plan/execute jobs adopt.
- **Gate placement**: the named-required-reviewer GitHub Environment guards
*only* the VS Code Marketplace publish (highest blast radius); CLI/SDK get a
typed `confirm_publish: "publish"` string. Principle: spend the heavyweight
second-person gate on the irreversible step only — for omnigent, that is the
secure-repo PyPI upload, which already has exactly such an environment.
- **Changelog-as-gate**: publish hard-fails if the changelog's top entry ≠ the
version, then reuses that section as the release body (and a Slack post).
Our equivalent is finalize's "CHANGELOG PR merged" check.
- No release branches, no rc versions (marketplace "pre-release" is a flag on
a normal version), no post-publish validation, no rollback story.
### kilocode (`Kilo-Org/kilocode`)
Product forked from cline, but the *release pipeline* is forked from opencode
(they even poll `anomalyco/opencode` releases to sync). Main train: **one
dispatch** (`bump` dropdown, `pre_release` defaults true) → version → build →
**validate matrix** (executes the built binary on macOS/Linux/Windows/Alpine)
**smoke-test** (real eval tasks against the *draft release's* assets) →
unattended publish to npm/Marketplace/GHCR/AUR/brew. No environment gate at
all on that train — below the rigor a Databricks-governed project should copy.
The interesting part is the **JetBrains train**, the only peer flow with true
rc→stable promotion: `prepare-jetbrains-release.yml` (`kind: rc|stable`,
`version`, `from_tag`) opens a release branch + PR; the human *merge* of that
PR is the approval gate; `publish-jetbrains.yml` fires on the merge, with a
dispatch fallback for re-runs; rc tags chain `-rc.1 … -rc.15 → stable`.
**Considered variant for omnigent** (from the JetBrains pattern): have
`release.yml` open a bump *PR* onto `branch-X.Y` instead of pushing directly,
making the merge a second-person cut-approval and running CI on the bump
commit. Rejected as the default: the bump is deterministic robot output
(`update_versions.py` + `check`), the cut is fully reversible, the secure
repo's gates re-verify everything against the tag before anything publishes,
and the extra merge per rc works against the 12-runs goal. Easy to switch to
later if a second-person cut gate is ever wanted.
### goose (`block/goose` → now `aaif-goose/goose`)
The closest org-shape analogue (big-company compliance, busy monorepo,
canary + stable channels, release branches). Minor release = weekly scheduled
bump PR → human merge → auto-cut `release/X.Y.0` + release PR → human runs two
copy-pasted `git tag && git push` commands → everything downstream (10-platform
build, signing, GHCR + SLSA, LLM release notes, Discord, auto-created next
hotfix branch) is automatic. ~5 human actions per minor. Findings that matter:
- **Their gate is a repo-wide tag-protection ruleset** (create/update/delete
blocked on *all* tags without bypass privilege), not environment reviewers —
environments are used only to scope secrets. Cheap, auditable.
- **They hit the `GITHUB_TOKEN` event-suppression gotcha in production**:
their LLM release-notes workflow runs on `workflow_run` *specifically*
because `release: published` doesn't fire for token-authored releases — the
same trap our App-token choices are designed around (and that
`draft-release-notes.yml` already dodges the same way).
- Their SDK packages **silently drifted out of lockstep** because nothing
asserts it — the failure mode our `update_versions.py check` prevents, and
an argument for running it in CI permanently (see hardening below).
- Canary = a single floating GH release overwritten in place; promotion is
always rebuild-from-source, never relabel.
- No dry-run, no post-publish validation, dependency scan *not* wired as a
publish gate, idempotency uneven, no rollback runbook.
### hermes (`NousResearch/hermes-agent`)
Real and public. CalVer tags (`v2026.7.7.2`), no release branches, no rc
channel, weekly cadence with same-day suffixed hotfixes; releasing is a local
`release.py` a maintainer runs (~3 actions), with GH Actions as reactive side
effects. Worth stealing:
- **Lockstep-as-a-test**: a real CI test asserts their four version locations
agree — drift is caught structurally no matter how it happened (bad merge,
cherry-pick, manual edit), not just when the bump script runs.
- **PyPI publish uses `skip-existing: true`** (pypa action) — direct precedent
for the partial-publish healing proposed for the secure repo.
- **Re-publish escape hatch**: `upload_to_pypi.yml` has a dispatch with a
`confirm_tag` input documented as "re-publish an existing tag" — the
idempotent-retry shape our secure-repo dispatch already has via `ref`.
- Bounded poll-with-warning (not hard-fail) when reading back a just-created
release/tag that may lag — adopted in the `validate` job's PyPI polling.
- Cautionary tale: their dependency-manifest review ruleset was empirically
self-merged around on a real release PR — review gates that the same person
can approve are decoration. (The secure repo's separate-org reviewer set
doesn't have this hole; keep it that way.)
### Cross-cutting (all six)
- **Nobody automates post-publish validation** — the secure repo `validate`
job is ahead of every peer surveyed.
- **Nobody has versioned docs** — all continuous-deploy latest-only. The
`X.Y-docs` staging design has no prior art to borrow; it's already built and
just needs the sweep gate.
- **Nobody has a backport/patch-branch story** as good as `branch-X.Y` +
cherry-pick; cline maintains one frozen legacy branch, kilocode has nothing.
- Pre-publish smoke against built artifacts (kilocode) ≈ the secure repo's
existing smoke-install gate. Parity, not a gap.
- **Nobody documents rollback/yank** — RELEASING.md's recovery section is
ahead of all six; the new workflows keep it (and make partial-publish
recovery automatic via skip-existing).
- rc→final promotion is rebuild-from-the-pinned-ref everywhere it exists at
all (goose canary→stable, kilocode JetBrains) — never artifact relabeling.
Validates our model: the final independently re-runs build+scan+publish
from `branch-X.Y`, which the mandatory dependency scan requires anyway.
- omnigent's mandatory scan-gates-publish + separate-org publisher is
**stricter than every peer surveyed** (goose's scan isn't a gate; hermes's
review gate was self-merged around; opencode/kilocode publish unattended).
## Hardening extras (cheap, independent of the workflows)
- **Run `update_versions.py check` in CI permanently** (a test or `ci.yml`
step), not just inside bump/release workflows — goose's SDKs silently
drifted out of lockstep for lack of exactly this assertion (hermes has it
and it works).
- **Tag ruleset on `v[0-9]*`**: restrict create/update/delete to maintainers +
the omnigent-ci App. Today any write-access account can push a version tag
and set off the draft-release + docker-publish chain; goose treats tag
protection as their primary release gate.
## Decisions (2026-07-14)
1. **Secure-repo restructure: approved direction** — gates → env-approval →
publish (skip-existing) → validate, one dispatch per phase.
2. **rc GH drafts are never published** — keep today's pattern exactly.
3. **bump PRs move to the App token** so CI runs on them (empty-commit
workaround retired).
4. **Release workflows are maintainer-only**: `authorize` actor-role gate
(admin/maintain) + the `v[0-9]*` tag ruleset as backstop.
5. **Homebrew joins the pipeline** via `update-homebrew.yml` on
`release: published`; tap-side human gate (`pr-pull` label) kept.
## Open questions
1. Environment `publish-release` reviewer set = who may finalize a release.
2. `brew update-python-resources` vs. the hand-maintained formula sections:
confirm on the catch-up run that the exclusion flags preserve the
`google-antigravity` platform stanzas and the brewed-dep comments, or keep
those sections behind guard comments the updater skips.
3. Tap bottle coverage (currently arm64 macOS only) — widen the test-bot
matrix? Orthogonal to this pipeline; tracked here so it isn't forgotten.
+8 -28
View File
@@ -81,24 +81,12 @@ drift negligible (~2 ms/turn).
| Journey | Operation timed |
| --- | --- |
| `session_cold_start` | Spawn a **fresh runner process**, wait for its tunnel, bind a session, and drive the first turn to `idle` — the full new-conversation cold path |
| `session_cold_start` | Create+bind a fresh session and drive its first turn to `idle` (runner spawn + executor construction + turn) |
| `warm_turn` | Drive a turn on an already-warm session — steady-state dispatch overhead |
| `time_to_first_token` | Post a turn; time to the first streamed `output_text` delta |
| `interrupt` | Interrupt a running (gated) turn; time to cancellation |
| `read_runner_file` | `GET .../environments/default/filesystem/{path}` — server → runner filesystem read proxy |
**`session_cold_start` spawns a real runner.** The env spawns one runner at
boot, but the warm journeys reuse it — so `session_cold_start` instead spawns a
*fresh* runner subprocess per iteration and waits for its reverse tunnel to
register before binding and driving the turn. That captures the runner process
start + tunnel handshake that a real new conversation always pays (and that a
host-launched session pays on its first message), not just the sub-second
executor-construction + first-turn overhead. Each iteration terminates its
runner afterward, so at most one extra runner is ever live. Each spawned runner
mints its own binding token and derives its `runner_id` from it (so tunnel,
mint, and session binding all agree on one id) and registers over loopback,
exactly like the boot runner — a fully independent runner.
`read_runner_file` needs a runner but does **not** drive a turn or call the LLM:
its setup plants a file via `PUT`, and the timed op is the proxied read (a
localhost round-trip). Being far cheaper than a turn, it uses a higher iteration
@@ -140,8 +128,7 @@ that a schema change hasn't broken seeding.
## Backends
`--database-uri` selects the DB; the report's `backend` field (`sqlite` /
`postgres` / `mysql`) is derived from the URI scheme so results group by
backend.
`postgres`) is derived from the URI scheme so results group by backend.
- **SQLite** (default) — in-process; fast, but not prod-representative.
- **Postgres** — `postgresql+psycopg://user@host:5432/db` (the fully-qualified
@@ -149,11 +136,6 @@ backend.
Requires `psycopg[binary]` (the `databricks` extra). Matches prod's
round-trip/pooling profile. Stand up a local one with
`docker run -e POSTGRES_PASSWORD=… -p 5432:5432 postgres:16`.
- **MySQL** — `mysql+mysqldb://user@host:3306/db`. Requires the `mysqlclient`
driver (`pip install mysqlclient`, which needs the `libmysqlclient-dev`
system library) — it is not in any extra. A supported backend, though prod
runs on Postgres. Stand up a local one with
`docker run -e MYSQL_ROOT_PASSWORD=… -e MYSQL_DATABASE=benchdb -p 3306:3306 mysql:8.0`.
## Output → Databricks → dashboard
@@ -175,7 +157,7 @@ document without running the harness.
```jsonc
{
"schema_version": 2,
"schema_version": 1,
"generated_at": "<ISO-8601 UTC>",
"git_sha": "<HEAD sha>",
"git_branch": "<branch>",
@@ -187,8 +169,7 @@ document without running the harness.
"journeys": {
"<journey name>": {
"kind": "latency" | "throughput",
"backend": "sqlite" | "postgres" | "mysql",
"needs_runner": false, // hardcoded per journey: HTTP=false, full-turn=true
"backend": "sqlite" | "postgres",
"runs": [ // one per --runs
{"n_success": N, "n_failures": N, "failures": {"HTTP 500": 1},
"wall_time_s": , "mean_ms": , "p50_ms": , "p95_ms": ,
@@ -224,11 +205,10 @@ creds).
## CI
`.github/workflows/benchmark.yml` runs nightly (and on dispatch) as a backend
matrix — `sqlite`, `postgres` (a `postgres:16` service container), and `mysql`
(a `mysql:8.0` service container; the `mysqlclient` driver is installed on that
leg only). Each leg seeds a corpus (SQLite reuses a cache keyed on the schema
head + `seed.py` + corpus config, so a migration busts the cache and forces a
reseed; Postgres and MySQL are fresh per run), runs the benchmark, and uploads
matrix — `sqlite` and `postgres` (a `postgres:16` service container). Each leg
seeds a corpus (SQLite reuses a cache keyed on the schema head + `seed.py` +
corpus config, so a migration busts the cache and forces a reseed; Postgres is
fresh per run), runs the benchmark, and uploads
`benchmark-results-<backend>-<run_id>.json`. The workspace notebook pulls those
artifacts.
+36 -279
View File
@@ -37,7 +37,6 @@ from typing import IO
import httpx
import yaml
from omnigent.host.identity import HOST_ID_ENV_VAR, HOST_NAME_ENV_VAR
from omnigent.runner.identity import OMNIGENT_INTERNAL_WS_ORIGIN, token_bound_runner_id
from tests._helpers.compat import (
apply_runner_env,
@@ -55,10 +54,6 @@ _HEALTH_TIMEOUT_S = 90.0
_MOCK_TIMEOUT_S = 15.0
_POLL_INTERVAL_S = 0.2
_TURN_TIMEOUT_S = 180.0
# Budget for the host daemon (session_cold_start journey, with_host) to connect
# its tunnel and register in the hosts table after being spawned. Covers
# interpreter start + imports + the reverse-tunnel handshake.
_HOST_ONLINE_TIMEOUT_S = 60.0
# Terminal SSE events — if one arrives before any delta, the turn produced no
# streamed text (a failure for the TTFT journey).
@@ -89,19 +84,6 @@ def _find_free_port() -> int:
return int(sock.getsockname()[1])
def _omni_executable() -> str:
"""The ``omni`` console script beside the (compat-aware) interpreter.
``server_executable()`` returns the interpreter the server/runner subprocess
should run under — ``sys.executable`` normally, or a pinned older build's
python in cross-version compat mode. The ``omni`` console script is
installed next to that interpreter (``[project.scripts]`` in pyproject), so
deriving it from the same directory launches the real user-facing command
(``omni server`` / ``omni host``) while still honoring the compat pin.
"""
return str(Path(server_executable()).with_name("omni"))
class BenchEnvironment:
"""Async context manager owning the benchmark's server (± runner + mock).
@@ -109,12 +91,6 @@ class BenchEnvironment:
v1 HTTP-journey path. When ``True``, also spawn the mock LLM and a
runner and wire the policy classifier at the mock — the phase-2
full-turn path.
:param with_host: When ``True`` (implies ``with_runner``), additionally
spawn a real ``omnigent host`` daemon. Additive over ``with_runner``:
the boot runner still serves the warm journeys, while the daemon lets
the ``session_cold_start`` journey create host-bound sessions that fire
``host.launch_runner`` and launch their OWN fresh runner — so the first
message races the runner's boot, reproducing the true UI cold path.
:param database_uri: SQLAlchemy URI the server boots against. ``None``
(default) uses a fresh throwaway SQLite file in the temp dir — the
empty-DB path. Pass a pre-seeded URI (e.g. a seeded SQLite file, or a
@@ -130,34 +106,23 @@ class BenchEnvironment:
self,
*,
with_runner: bool = False,
with_host: bool = False,
database_uri: str | None = None,
harness: str = _DEFAULT_HARNESS,
model: str = _DEFAULT_MODEL,
) -> None:
# with_host is additive over with_runner: the boot runner still serves
# the warm journeys, and the host daemon additionally lets the cold-start
# journey create host-bound sessions that launch their own runners.
self.with_host = with_host
self.with_runner = with_runner or with_host
self.with_runner = with_runner
self.database_uri = database_uri
self.harness = harness
self.model = model
self.base_url = ""
self.mock_url = ""
self.runner_id = ""
self.host_id = ""
self.host_workspace = ""
self.client: httpx.AsyncClient | None = None
self._tmp = Path("/tmp") / f"omni-bench-{uuid.uuid4().hex[:8]}"
self._mock_proc: subprocess.Popen[bytes] | None = None
self._server_proc: subprocess.Popen[bytes] | None = None
self._runner_proc: subprocess.Popen[bytes] | None = None
self._host_proc: subprocess.Popen[bytes] | None = None
# Base env retained so the host daemon is built identically to the boot
# runner's server-facing env (worktree source, mock LLM routing).
self._runner_base_env: dict[str, str] = {}
self._log_handles: list[IO[bytes]] = []
self._agent_cache: dict[str, str] = {}
@@ -207,33 +172,15 @@ class BenchEnvironment:
base_env["OPENAI_BASE_URL"] = f"{self.mock_url}/v1"
# Prepend the worktree so subprocesses import this branch's source.
apply_server_env(base_env, _REPO_ROOT)
# Retained so the host daemon (with_host) is built with the same
# server-facing env as the boot runner.
self._runner_base_env = base_env
self._server_proc = self._spawn_server(port, base_env, binding_token, artifact_dir)
if self.with_runner:
self._runner_proc = self._spawn_runner(base_env, binding_token)
self._wait_ready()
# The host daemon is ADDITIVE — the boot runner above still serves the
# warm journeys; the daemon exists so the cold-start journey can create
# host-bound sessions that launch their OWN fresh runners on demand
# (the race the cold path measures). The two never share a runner id.
if self.with_host:
self._host_proc = self._spawn_host(base_env)
self._wait_host_online()
def _stop(self) -> None:
"""Terminate host, runner, server, and mock; remove the temp dir."""
# Host first: SIGTERM-ing the daemon reaps the runners IT spawned (they
# are daemon-owned children), so it must go before the server so those
# runners' tunnels close cleanly.
for proc in (
self._host_proc,
self._runner_proc,
self._server_proc,
self._mock_proc,
):
"""Terminate runner, server, and mock; remove the temp dir."""
for proc in (self._runner_proc, self._server_proc, self._mock_proc):
if proc is not None and proc.poll() is None:
proc.send_signal(signal.SIGTERM)
try:
@@ -274,7 +221,9 @@ class BenchEnvironment:
# four slashes; the temp path is absolute.
db_uri = self.database_uri or f"sqlite:///{self._tmp / 'bench.db'}"
args = [
_omni_executable(),
server_executable(),
"-m",
"omnigent.cli",
"server",
"--port",
str(port),
@@ -321,36 +270,10 @@ class BenchEnvironment:
# on teardown, rather than in the launch cwd (its default).
workspace = self._tmp / "workspace"
workspace.mkdir(exist_ok=True)
return self._spawn_runner_process(
base_env,
binding_token,
runner_id=self.runner_id,
workspace=workspace,
log_name="runner.log",
)
def _spawn_runner_process(
self,
base_env: dict[str, str],
binding_token: str,
*,
runner_id: str,
workspace: Path,
log_name: str,
) -> subprocess.Popen[bytes]:
"""Spawn one runner subprocess under *runner_id* + *binding_token*.
Factored out of :meth:`_spawn_runner` so the ``session_cold_start``
journey can spawn additional runners on demand, each under its own id,
binding token, and workspace. The caller must pair *runner_id* with the token
it derives from (``token_bound_runner_id(binding_token)``): the runner
derives its managed-mint URL from the token internally, so a mismatch
would register the tunnel under one id but mint under another (→ 401).
"""
runner_env = apply_runner_env(
{
**base_env,
"OMNIGENT_RUNNER_ID": runner_id,
"OMNIGENT_RUNNER_ID": self.runner_id,
"OMNIGENT_RUNNER_TUNNEL_BINDING_TOKEN": binding_token,
"OMNIGENT_RUNNER_PARENT_PID": str(os.getpid()),
"RUNNER_SERVER_URL": self.base_url,
@@ -361,40 +284,7 @@ class BenchEnvironment:
[runner_executable(), "-m", "omnigent.runner._entry"],
env=runner_env,
cwd=compat_runner_cwd(),
stdout=self._log(log_name),
stderr=subprocess.STDOUT,
)
def _spawn_host(self, base_env: dict[str, str]) -> subprocess.Popen[bytes]:
"""Spawn a real ``omni host`` daemon against the bench server.
Runs the user-facing ``omni host --server`` command — the same daemon a
developer starts by hand. Identity comes from :data:`HOST_ID_ENV_VAR` /
:data:`HOST_NAME_ENV_VAR`: with both set, ``load_or_create_host_identity``
returns that identity WITHOUT reading or writing any ``config.yaml``, so
the daemon never touches the developer's real ``~/.omnigent`` (nor
collides with a sibling bench leg). ``--non-interactive`` keeps it from
ever launching a browser login (moot for the loopback server, which is
not Databricks-fronted, but explicit for CI). The daemon self-registers
over loopback (single-user ``RESERVED_USER_LOCAL`` owner, no token) and
launches runners on demand when the server sends ``host.launch_runner``.
"""
# Bare 32-char hex uuid — host_id is a Uuid16 (binary) column, so it
# must be a valid uuid (a synthetic "host_bench_…" string no longer fits).
self.host_id = uuid.uuid4().hex
workspace = self._tmp / "host-workspace"
workspace.mkdir(exist_ok=True)
self.host_workspace = str(workspace)
host_env = {
**base_env,
HOST_ID_ENV_VAR: self.host_id,
HOST_NAME_ENV_VAR: f"bench-host-{self.host_id[-8:]}",
}
return subprocess.Popen(
[_omni_executable(), "host", "--server", self.base_url, "--non-interactive"],
env=host_env,
cwd=str(workspace),
stdout=self._log("host-daemon.log"),
stdout=self._log("runner.log"),
stderr=subprocess.STDOUT,
)
@@ -425,39 +315,12 @@ class BenchEnvironment:
raise RuntimeError(f"server not ready within {_HEALTH_TIMEOUT_S}s; logs in {self._tmp}")
def _runner_ready(self) -> bool:
"""Whether the boot runner reports online (always ``True`` server-only)."""
"""Whether the runner reports online (always ``True`` server-only)."""
if not self.with_runner:
return True
status = httpx.get(f"{self.base_url}/v1/runners/{self.runner_id}/status", timeout=2)
return status.status_code == 200 and status.json().get("online") is True
def _wait_host_online(self) -> None:
"""Block until the host daemon's row reads ``status=online``.
Polls ``GET /v1/hosts`` (the single-user owner is ``local``) until the
daemon we spawned has connected its tunnel and been upserted online, so
a host-bound session-create has a live launch target.
"""
deadline = time.monotonic() + _HOST_ONLINE_TIMEOUT_S
while time.monotonic() < deadline:
if self._host_proc is not None and self._host_proc.poll() is not None:
raise RuntimeError(
f"host daemon exited (code {self._host_proc.returncode}) before "
f"coming online; logs in {self._tmp}"
)
try:
resp = httpx.get(f"{self.base_url}/v1/hosts", timeout=2)
if resp.status_code == 200:
for host in resp.json().get("hosts", []):
if host.get("host_id") == self.host_id and host.get("status") == "online":
return
except httpx.HTTPError:
# Server not yet accepting requests, or a transient read error:
# keep polling until the deadline rather than failing the boot.
pass
time.sleep(_POLL_INTERVAL_S)
raise RuntimeError(f"host {self.host_id} not online within {_HOST_ONLINE_TIMEOUT_S}s")
# ── mock control (runner mode only) ──────────────────────
async def _mock_post(self, path: str, body: dict[str, object]) -> None:
@@ -564,33 +427,6 @@ class BenchEnvironment:
created.raise_for_status()
return str(created.json()["id"])
async def create_hosted_session(self, agent_id: str) -> str:
"""Create a host-bound session that fires ``host.launch_runner``.
The inline-launch ``POST /v1/sessions`` shape the Web UI's New Chat
wizard sends: passing ``host_id`` + ``workspace`` makes the server bind a
runner id and dispatch a launch frame to the host daemon, then return
immediately (~tens of ms) WITHOUT waiting for the runner to connect.
Returned without any readiness poll on purpose — the caller's first
message then races the runner's boot, which is the cold path we measure.
:raises RuntimeError: If the env was not built with ``with_host=True``.
"""
assert self.client is not None
if not self.with_host:
raise RuntimeError("create_hosted_session requires with_host=True")
created = await self.client.post(
"/v1/sessions",
json={
"agent_id": agent_id,
"host_id": self.host_id,
"host_type": "external",
"workspace": self.host_workspace,
},
)
created.raise_for_status()
return str(created.json()["id"])
async def seed_items(self, session_id: str, count: int) -> None:
"""Append *count* history items over HTTP, with no runner or LLM.
@@ -620,22 +456,13 @@ class BenchEnvironment:
# ── runner-mode session driving (phase 2) ────────────────
async def create_bound_session(self, agent_id: str) -> str:
"""Create a session for *agent_id* and bind it to the boot runner."""
return await self.create_session_bound_to(agent_id, self.runner_id)
async def create_session_bound_to(self, agent_id: str, runner_id: str) -> str:
"""Create a session for *agent_id* and bind it to *runner_id*.
Binds a session to an already-online runner by patching its
``runner_id`` — used by the warm journeys via :meth:`create_bound_session`
to pin the boot runner.
"""
"""Create a session for *agent_id* and bind it to the runner."""
assert self.client is not None
if not self.with_runner:
raise RuntimeError("create_session_bound_to requires with_runner=True")
raise RuntimeError("create_bound_session requires with_runner=True")
session_id = await self.create_session(agent_id)
bound = await self.client.patch(
f"/v1/sessions/{session_id}", json={"runner_id": runner_id}
f"/v1/sessions/{session_id}", json={"runner_id": self.runner_id}
)
bound.raise_for_status()
return session_id
@@ -719,42 +546,26 @@ class BenchEnvironment:
await asyncio.sleep(_POLL_INTERVAL_S)
raise RuntimeError(f"session did not reach idle within {timeout}s ({session_id})")
async def _post_and_await_first_delta(
self,
session_id: str,
text: str,
*,
wait_idle_first: bool,
timeout: float = _TURN_TIMEOUT_S,
async def time_to_first_delta(
self, session_id: str, text: str, *, timeout: float = _TURN_TIMEOUT_S
) -> None:
"""Imitate the UI first-token path: attach SSE, then post, then await.
"""Post a turn and return once the first output-text delta streams back.
The exact sequence the web client follows for a one-shot turn:
subscribe to ``GET …/stream``, wait for the stream's ready heartbeat (the
first SSE line — the server yields it right after registering the
live-tail slot, so no event can be missed), POST the message, and return
on the first response from the model — either a
``response.output_text.delta`` (streamed text) or a
``response.output_item.done`` (a completed output item, e.g. a tool call
for harnesses that don't stream text deltas). This measures time to *any*
first response, not just text. A terminal event before either arrives
means the turn produced no response at all (a failure).
The session SSE stream (``GET …/stream``) is separate from the message
POST, so we subscribe first (as a concurrent task), post the turn, then
return when the first ``response.output_text.delta`` event arrives. This
times omnigent's streaming-pipeline overhead to first token — with the
zero-latency mock there is no model latency in the number.
:param wait_idle_first: When ``True``, wait for the session to be ``idle``
before subscribing so a prior turn's terminal event can't race this
turn's response (warm-session TTFT). ``False`` for a fresh session whose
first turn is the only one — the cold path, where the timed span must
include runner launch + connect, so we must NOT poll it warm first.
:raises RuntimeError: If not in runner mode, or no response / a terminal
:raises RuntimeError: If not in runner mode, or no delta / a terminal
event arrives within *timeout*.
"""
assert self.client is not None
if not self.with_runner:
raise RuntimeError("first-delta timing requires with_runner=True")
raise RuntimeError("time_to_first_delta requires with_runner=True")
connected = asyncio.Event()
first_delta = asyncio.Event()
first_response = asyncio.Event()
outcome: dict[str, str] = {}
async def _read_stream() -> None:
@@ -763,9 +574,9 @@ class BenchEnvironment:
"GET", f"/v1/sessions/{session_id}/stream", timeout=timeout
) as resp:
# Any first line means the SSE connection is live (the server
# emits a ready heartbeat on connect). Signalling here lets us
# post the turn only once subscribed — without a blind sleep
# that would otherwise inflate the measured time-to-first-delta.
# emits a heartbeat on connect). Signalling here lets us post
# the turn only once subscribed — without a blind sleep that
# would otherwise inflate the measured time-to-first-delta.
connected.set()
async for line in resp.aiter_lines():
if not line.startswith("event:"):
@@ -774,9 +585,6 @@ class BenchEnvironment:
if etype == "response.output_text.delta":
first_delta.set()
return
if etype == "response.output_item.done":
first_response.set()
return
if etype in _STREAM_TERMINAL_EVENTS:
outcome["terminal"] = etype
first_delta.set()
@@ -786,16 +594,15 @@ class BenchEnvironment:
connected.set()
first_delta.set()
if wait_idle_first:
# Warm path: ensure any prior turn has settled so the fresh
# subscription's first terminal event can't be the previous turn
# completing (which would otherwise race ahead of this turn's delta).
await self._wait_idle(session_id, timeout=timeout)
# Ensure any prior turn has settled so the fresh subscription's first
# terminal event can't be the previous turn completing (which would
# otherwise race ahead of this turn's delta).
await self._wait_idle(session_id, timeout=timeout)
reader = asyncio.create_task(_read_stream())
try:
# Wait until the stream is actually connected (not a fixed sleep) so
# the measured window is post → first response, not subscription setup.
# the measured window is post → first delta, not subscription setup.
await asyncio.wait_for(connected.wait(), timeout=timeout)
posted = await self.client.post(
f"/v1/sessions/{session_id}/events",
@@ -805,71 +612,21 @@ class BenchEnvironment:
},
)
posted.raise_for_status()
# Return on the first response, whichever comes first: a streamed text
# delta or a completed output item (e.g. a tool call for harnesses that
# don't stream text).
waiters = [
asyncio.create_task(first_delta.wait()),
asyncio.create_task(first_response.wait()),
]
done, pending = await asyncio.wait(
waiters, timeout=timeout, return_when=asyncio.FIRST_COMPLETED
)
for task in pending:
task.cancel()
if not done:
try:
await asyncio.wait_for(first_delta.wait(), timeout=timeout)
except TimeoutError as exc:
raise RuntimeError(
"no output_text.delta or output_item.done within "
f"{timeout}s (session {session_id})"
)
f"no output_text.delta within {timeout}s (session {session_id})"
) from exc
if "error" in outcome:
raise RuntimeError(f"stream error: {outcome['error']}")
if "terminal" in outcome:
raise RuntimeError(
f"turn reached {outcome['terminal']} before any response "
f"(session {session_id})"
f"turn reached {outcome['terminal']} before any delta (session {session_id})"
)
finally:
reader.cancel()
async def time_to_first_delta(
self, session_id: str, text: str, *, timeout: float = _TURN_TIMEOUT_S
) -> None:
"""Post a turn on a WARM session and return on the first output delta.
Times omnigent's streaming-pipeline overhead to first token against an
already-connected runner — with the zero-latency mock there is no model
latency in the number. See :meth:`_post_and_await_first_delta`.
"""
await self._post_and_await_first_delta(
session_id, text, wait_idle_first=True, timeout=timeout
)
async def cold_start_first_delta(
self, agent_id: str, text: str, *, timeout: float = _TURN_TIMEOUT_S
) -> None:
"""Time the full UI cold path: create → attach SSE → send → first token.
Reproduces exactly what the Web UI does for a brand-new host-bound
session: create the session (which fires ``host.launch_runner`` and
returns before the runner connects), then run the standard first-token
sequence (attach the SSE stream, wait for its ready heartbeat, POST the
first message, await the first ``response.output_text.delta``). Because
the runner is still booting when the message posts, the server's
connect-grace wait is on the timed path — so the measured span captures
the real cold-start cost the ``session_cold_start`` journey exists for:
host launch + runner boot + reverse-tunnel connect + first-token
pipeline. No pre-warm and no ``GET /session`` status polling — the SSE
first-delta signal is the same one the UI renders on.
:raises RuntimeError: If not host-backed, or no delta / a terminal event
arrives within *timeout*.
"""
session_id = await self.create_hosted_session(agent_id)
await self._post_and_await_first_delta(
session_id, text, wait_idle_first=False, timeout=timeout
)
async def drive_and_interrupt(
self, session_id: str, *, timeout: float = _TURN_TIMEOUT_S
) -> None:
+5 -53
View File
@@ -19,14 +19,6 @@ v1 journeys are pure HTTP/API (server + DB, no runner, no LLM):
``read_runner_file`` needs a runner but no LLM turn: it plants a file in the
runner environment (setup) and times the server → runner filesystem read proxy.
Full-turn journeys (``needs_runner=True``) drive a real turn through the runner
+ mock LLM. ``session_cold_start`` (``needs_host=True``) measures the real UI
new-conversation cold path: it spawns a host daemon once, then per iteration
creates a host-bound session (which fires ``host.launch_runner``), attaches the
SSE stream, sends the first message, and times to the first output-text delta —
so the span includes the on-demand runner launch + reverse-tunnel handshake the
UI's first message races, exactly as a real new chat pays it.
The framework (``Journey`` + the two runners) is harness-agnostic and reused
verbatim by phase-2 full-turn journeys.
"""
@@ -77,10 +69,6 @@ class Journey:
:param needs_runner: Whether this journey drives a full agent turn and so
requires ``BenchEnvironment(with_runner=True)`` (mock LLM + runner).
HTTP/DB journeys leave this ``False``.
:param needs_host: Whether this journey needs a real host daemon
(``BenchEnvironment(with_host=True)``) so a host-bound session-create
fires ``host.launch_runner`` and the first message races the runner's
boot. Implies ``needs_runner``. Only ``session_cold_start`` sets this.
:param max_iterations: Upper bound on latency iterations for this journey,
clamping ``--iterations`` down (never up). Full-turn journeys cost ~1s+
per op, so 100+ iterations would blow the CI time budget; they cap at a
@@ -96,7 +84,6 @@ class Journey:
teardown: Callable[[BenchEnvironment, JourneyContext], Awaitable[None]] | None = None
concurrency_safe: bool = False
needs_runner: bool = False
needs_host: bool = False
max_iterations: int | None = None
description: str = ""
@@ -374,17 +361,6 @@ async def _setup_turn_agent(env: BenchEnvironment, *, stream: bool = False) -> s
return await env.agent_id(name)
async def _setup_cold_start_agent(env: BenchEnvironment) -> str:
"""Register a streaming-reply agent for the cold-start journey; return its id.
No session and no warm-up turn — the cold-start measure creates a fresh
host-bound session each iteration. The reply streams deltas so the measured
op can return on the first ``response.output_text.delta`` (the UI's
first-token signal).
"""
return await _setup_turn_agent(env, stream=True)
async def _setup_warm_session(env: BenchEnvironment) -> str:
"""Create+bind a session and drive one warm-up turn; return the session id.
@@ -420,31 +396,9 @@ async def _setup_interrupt_session(env: BenchEnvironment) -> str:
async def _measure_session_cold_start(env: BenchEnvironment, ctx: JourneyContext) -> None:
"""Time the real UI cold path: create host-bound session → first token.
Faithfully imitates the Web UI's New Chat flow on a fresh session (see
``BenchEnvironment.cold_start_first_delta``): create a host-bound session
(which fires ``host.launch_runner`` at the host daemon and returns before
the runner connects), attach the SSE stream, wait for its ready heartbeat,
POST the first message, and return on the first response.
Because the message posts while the runner is still booting, the server's
connect-grace wait is on the timed path — so the measured span captures the
true new-conversation cost: host launch + runner boot + reverse-tunnel
connect + first-token pipeline.
Each iteration is its own fresh session with its own host-launched runner.
The server never stops an external-host runner on idle (only on an explicit
stop/delete, neither of which the UI first-message path does), so each
iteration's runner stays connected until the daemon is SIGTERM'd at env
teardown, which reaps them together. That is bounded — ``_RUNNER_MAX_ITERATIONS``
(+ warmups) runners at most, all cleaned up at the end — so we deliberately
skip per-iteration teardown: stopping the runner would add a
stop-round-trip to a journey whose whole point is to time the fresh-launch
cost, and would not reflect what a real first message does.
"""
agent_id = cast(str, ctx) # _setup_turn_agent (stream=True)
await env.cold_start_first_delta(agent_id, _TURN_PROMPT)
agent_id = cast(str, ctx) # _setup_turn_agent
session_id = await env.create_bound_session(agent_id)
await env.drive_turn(session_id, _TURN_PROMPT)
async def _measure_warm_turn(env: BenchEnvironment, ctx: JourneyContext) -> None:
@@ -545,12 +499,10 @@ ALL_JOURNEYS: dict[str, Journey] = {
name="session_cold_start",
kind="latency",
measure=_measure_session_cold_start,
setup=_setup_cold_start_agent,
setup=_setup_turn_agent,
needs_runner=True,
needs_host=True,
max_iterations=_RUNNER_MAX_ITERATIONS,
description="Create a host-bound session (fires host.launch_runner) then "
"time create → attach SSE → send → first token — the real UI cold path.",
description="Create+bind a fresh session and drive its first turn to idle.",
),
Journey(
name="warm_turn",
+5 -18
View File
@@ -64,8 +64,6 @@ def _backend_of(database_uri: str | None) -> str:
return "sqlite"
if database_uri.startswith("postgres"):
return "postgres"
if database_uri.startswith("mysql"):
return "mysql"
return "other"
@@ -124,16 +122,10 @@ async def run_benchmark(args: argparse.Namespace) -> tuple[dict[str, object], bo
# Any full-turn journey needs the runner + mock LLM. A full env is a
# superset — HTTP journeys still run against it — so a mixed selection just
# boots with_runner=True. The harness label reflects what drove the turns.
# A host-backed journey (session_cold_start) additionally needs a host
# daemon; with_host is a further superset (it implies with_runner) so a
# mixed selection that includes it boots the host too.
with_runner = any(j.needs_runner for j in journeys)
with_host = any(j.needs_host for j in journeys)
harness = _RUNNER_HARNESS if with_runner else _HTTP_HARNESS
async with BenchEnvironment(
with_runner=with_runner, with_host=with_host, database_uri=args.database_uri
) as env:
async with BenchEnvironment(with_runner=with_runner, database_uri=args.database_uri) as env:
for journey in journeys:
console.print(f"\n[bold]Benchmarking[/bold] {journey.name} [dim]({backend})[/dim]")
kind, results = await _run_journey(journey, env, args)
@@ -141,11 +133,6 @@ async def run_benchmark(args: argparse.Namespace) -> tuple[dict[str, object], bo
block = aggregate(results)
block["kind"] = kind
block["backend"] = backend
# Hardcoded per-journey mapping: HTTP journeys are False, full-turn
# journeys True. Sourced from the journey itself, not the run-level
# env, so it stays correct in a mixed selection (where with_runner
# is True for the whole run because *some* journey needs it).
block["needs_runner"] = journey.needs_runner
journey_results[journey.name] = block
if not check_thresholds(
results,
@@ -191,10 +178,10 @@ def _parse_args(argv: list[str]) -> argparse.Namespace:
"--database-uri",
default=None,
metavar="URI",
help="DB the server boots against — a pre-seeded SQLite file, a "
"postgresql+psycopg://… instance, or a mysql+mysqldb://… instance "
"(see seed.py). Default: a fresh throwaway SQLite DB (empty — "
"best-case numbers). The report's `backend` field is derived from this.",
help="DB the server boots against — a pre-seeded SQLite file or a "
"postgresql+psycopg://… instance (see seed.py). Default: a fresh "
"throwaway SQLite DB (empty — best-case numbers). The report's "
"`backend` field is derived from this.",
)
parser.add_argument(
"--iterations",
+6 -11
View File
@@ -1,5 +1,5 @@
{
"schema_version": 2,
"schema_version": 1,
"generated_at": "2026-07-08T18:30:00+00:00",
"git_sha": "0000000000000000000000000000000000000000",
"git_branch": "main",
@@ -66,8 +66,7 @@
"avg_rps": 162.7669557884957
},
"kind": "latency",
"backend": "sqlite",
"needs_runner": false
"backend": "sqlite"
},
"create_session": {
"runs": [
@@ -116,8 +115,7 @@
"avg_rps": 40.548671853654
},
"kind": "latency",
"backend": "sqlite",
"needs_runner": false
"backend": "sqlite"
},
"get_session": {
"runs": [
@@ -166,8 +164,7 @@
"avg_rps": 199.59858100569238
},
"kind": "latency",
"backend": "sqlite",
"needs_runner": false
"backend": "sqlite"
},
"load_conversation_history": {
"runs": [
@@ -216,8 +213,7 @@
"avg_rps": 503.8957501013986
},
"kind": "latency",
"backend": "sqlite",
"needs_runner": false
"backend": "sqlite"
},
"search_sessions": {
"runs": [
@@ -266,8 +262,7 @@
"avg_rps": 12.317410535217997
},
"kind": "latency",
"backend": "sqlite",
"needs_runner": false
"backend": "sqlite"
}
}
}
+1 -1
View File
@@ -13,7 +13,7 @@ import platform
import subprocess
# Incremented on any breaking change to the report document shape below.
SCHEMA_VERSION = 2
SCHEMA_VERSION = 1
def _git(*args: str) -> str:
-92
View File
@@ -2,15 +2,6 @@
# It is not intended for manual editing.
version = 4
[[package]]
name = "aho-corasick"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301"
dependencies = [
"memchr",
]
[[package]]
name = "allocator-api2"
version = "0.2.21"
@@ -92,16 +83,6 @@ version = "2.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4388bee8683e3d04af747c73422af53102d2bd24d9eadb6cbc100baef4b43f8"
[[package]]
name = "bstr"
version = "1.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5cee35f73844aa3014bb606320a6c1f010249dbdf43342fe54b5a4f6a8ed4b79"
dependencies = [
"memchr",
"serde_core",
]
[[package]]
name = "bytes"
version = "1.12.0"
@@ -189,31 +170,6 @@ dependencies = [
"static_assertions",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crossterm"
version = "0.28.1"
@@ -319,19 +275,6 @@ dependencies = [
"libc",
]
[[package]]
name = "globset"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52dfc19153a48bde0cbd630453615c8151bce3a5adfac7a0aebfbf0a1e1f57e3"
dependencies = [
"aho-corasick",
"bstr",
"log",
"regex-automata",
"regex-syntax",
]
[[package]]
name = "hashbrown"
version = "0.15.5"
@@ -371,22 +314,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "ignore"
version = "0.4.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fe112b004901c62c2faa11f4f75e9864e0cc5af8da71c9115d184a3aa888749f"
dependencies = [
"crossbeam-deque",
"globset",
"log",
"memchr",
"regex-automata",
"same-file",
"walkdir",
"winapi-util",
]
[[package]]
name = "indexmap"
version = "2.14.0"
@@ -599,7 +526,6 @@ dependencies = [
"clap",
"crossterm",
"if-addrs",
"ignore",
"libc",
"notify",
"notify-debouncer-full",
@@ -608,7 +534,6 @@ dependencies = [
"serde_json",
"tokio",
"toml",
"unicode-width 0.2.0",
]
[[package]]
@@ -700,23 +625,6 @@ dependencies = [
"bitflags",
]
[[package]]
name = "regex-automata"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e1dd4122fc1595e8162618945476892eefca7b88c52820e74af6262213cae8f"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "rustix"
version = "0.38.44"
-2
View File
@@ -15,10 +15,8 @@ clap = { version = "4", features = ["derive"] }
crossterm = "0.28"
ratatui = "0.29"
ansi-to-tui = "7"
unicode-width = "0.2"
notify = "8"
notify-debouncer-full = "0.5"
ignore = "0.4"
libc = "0.2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
+12 -32
View File
@@ -20,11 +20,8 @@ replaces the three-terminal local dev flow (`omnigent server`, `omnigent host`,
- **supervises** the backend server, the host daemon, and the Vite frontend,
restarting any that crash (with backoff);
- **reloads the backend** (server → host) when you edit `omnigent/**/*.py`;
gitignored files under `omnigent/` (e.g. the build-time `_build_info.py`) are
skipped so generated churn doesn't reload; the frontend self-reloads through
Vite HMR;
- gives you **per-process log panes** plus a combined view, each a `less`-style
pager with wrap and search (see [Keys](#keys)).
the frontend self-reloads through Vite HMR;
- gives you **scrollable per-process log panes** plus a combined view.
## Build & run
@@ -45,8 +42,8 @@ Run it from anywhere inside the checkout — it walks up to the repo root
| Process | Command | Notes |
|---|---|---|
| server | `uv run omnigent --log-to-stderr server --host 127.0.0.1 --port <p> --database-uri … --artifact-location …` | Waited on via `GET /health`. |
| host | `uv run omnigent --log-to-stderr host --server http://127.0.0.1:<p>` | Started once the server is healthy. |
| server | `uv run omnigent server --host 127.0.0.1 --port <p> --database-uri … --artifact-location …` | Waited on via `GET /health`. |
| host | `uv run omnigent host --server http://127.0.0.1:<p>` | Started once the server is healthy. |
| vite | `npm run dev -- --host <host> --port <p> --strictPort` (cwd `web/`) | `OMNIGENT_URL` points its proxy at the pod's server. |
Before Vite starts (and on a manual Vite restart), omnidev runs `npm install`
@@ -59,19 +56,12 @@ Open the UI at the `ui` URL shown in the header (the Vite dev server).
## Isolation
Only Omnigent's own state is isolated per pod — enough that concurrent pods
never share a database, server pidfile, or `config.yaml` — via
`OMNIGENT_DATA_DIR`, `OMNIGENT_DATABASE_URI`, `OMNIGENT_URL`, and
`OMNIGENT_CONFIG_HOME`. Everything else (your real `HOME`, credentials, and
uv/npm caches) is inherited, because the agents Omnigent runs need it. This is
deliberately lighter than the hermetic `scripts/backend-smoke.sh` sandbox,
which repoints `HOME`/`XDG_*` to touch nothing real.
Each pod gets its own `config.yaml` under `<pod>/config/`, pointed to by
`OMNIGENT_CONFIG_HOME`. On first create it's **seeded** from your real
`~/.omnigent/config.yaml` (if present) so the pod works out of the box — it
keeps your providers — after which the two are independent: server-config edits
inside a pod (via the UI or `omnigent config`) don't touch your real config.
`--clean` wipes the pod dir, so the next run re-seeds from your real config.
never share a database or server pidfile — via `OMNIGENT_DATA_DIR`,
`OMNIGENT_DATABASE_URI`, and `OMNIGENT_URL`. Everything else (your real
`HOME`, credentials, config, and uv/npm caches) is inherited, because the
agents Omnigent runs need it. This is deliberately lighter than the hermetic
`scripts/backend-smoke.sh` sandbox, which repoints `HOME`/`XDG_*` to touch
nothing real.
The pod dir defaults to
`${XDG_CACHE_HOME:-~/.cache}/omnidev/<repo-name>-<hash>/`, keyed to the
@@ -88,7 +78,6 @@ canonical checkout path. Per-process logs are written through to
--pod-dir <PATH> Use a specific pod dir instead of the per-repo default
--no-vite Backend + host only (no frontend)
--clean Wipe the pod dir before starting
--debug Log each watched file change and whether it reloads
```
`--vite-host 0.0.0.0` exposes the Vite dev server on all interfaces for device
@@ -118,21 +107,12 @@ not auto-trusted (add those to `OMNIGENT_WS_ALLOWED_ORIGINS` yourself).
## Keys
The log pane is a `less`-style pager, so the movement and search keys should
feel familiar.
| Key | Action |
|---|---|
| `1` / `2` / `3` / `0` | Focus server / host / vite / combined pane |
| `Tab` | Cycle panes |
| `j` / `k` (or `↓` / `↑`) | Scroll one line |
| `f` / `Space` / `PgDn` (or `b` / `PgUp`) | Page forward / back one window |
| `d` / `u` | Half-page forward / back |
| `g` / `G` | Jump to top / bottom (bottom re-follows the tail) |
| `F` | Toggle follow-tail (like `less +F`) |
| `w` | Toggle line wrap (on by default) |
| `/` `?` | Search forward / back — type, `Enter` to jump, `Esc` to cancel |
| `n` / `N` | Next / previous match |
| `` `` `PgUp` `PgDn` | Scroll (detaches from tail) |
| `f` | Toggle follow-tail |
| `r` | Restart the focused process (server/host restart as a pair) |
| `R` | Restart the backend (server then host) |
| `c` | Clear the focused pane |
+1 -12
View File
@@ -77,11 +77,6 @@ struct RunArgs {
/// Wipe the pod directory before starting.
#[arg(long)]
clean: bool,
/// Log every observed file change and whether it triggers a backend reload
/// (with the skip reason otherwise).
#[arg(long)]
debug: bool,
}
#[derive(Subcommand, Debug)]
@@ -193,13 +188,7 @@ async fn run_supervisor(args: RunArgs) -> Result<()> {
// File watcher: Python changes -> Reload commands. Keep the debouncer alive
// for the whole session.
let _watcher = watcher::spawn(
&pod.repo_root,
&pod.omnigent_dir(),
shared.clone(),
args.debug,
cmd_tx.clone(),
)?;
let _watcher = watcher::spawn(&pod.omnigent_dir(), cmd_tx.clone())?;
// Supervisor runs on the tokio runtime; the TUI drives it via cmd_tx.
let supervisor = Supervisor::new(
+10 -200
View File
@@ -19,8 +19,8 @@ pub struct Pod {
impl Pod {
/// Create the pod directory tree (idempotent) and return the pod handle.
/// Only omnigent's own state is isolated (DB, artifacts, logs, config); the
/// pod inherits your real home, credentials, and caches.
/// Only omnigent's own state is isolated (DB, artifacts, logs); the pod
/// otherwise inherits your real home, credentials, config, and caches.
pub fn create(
repo_root: PathBuf,
dir: PathBuf,
@@ -28,28 +28,18 @@ impl Pod {
vite_host: String,
trusted_origins: Vec<String>,
) -> Result<Pod> {
for sub in ["data/omnigent", "artifacts", "logs", "config"] {
for sub in ["data/omnigent", "artifacts", "logs"] {
let p = dir.join(sub);
std::fs::create_dir_all(&p)
.with_context(|| format!("creating pod dir {}", p.display()))?;
}
let pod = Pod {
Ok(Pod {
repo_root,
dir,
ports,
vite_host,
trusted_origins,
};
// Seed the pod's config from the developer's real one so it works out
// of the box (keeps their providers). Best-effort: a copy failure just
// starts the pod with an empty config, so warn rather than abort.
if let Some(src) = real_config_path() {
let dest = pod.config_dir().join("config.yaml");
if let Err(e) = seed_config_file(&src, &dest) {
eprintln!("omnidev: could not seed pod config: {e:#}");
}
}
Ok(pod)
})
}
pub fn db_uri(&self) -> String {
@@ -63,13 +53,6 @@ impl Pod {
self.dir.join("artifacts")
}
/// The pod's isolated config home, exposed to children as
/// `OMNIGENT_CONFIG_HOME` so its `config.yaml` is separate from the
/// developer's real `~/.omnigent/config.yaml`.
pub fn config_dir(&self) -> PathBuf {
self.dir.join("config")
}
pub fn server_url(&self) -> String {
format!("http://127.0.0.1:{}", self.ports.server)
}
@@ -121,22 +104,17 @@ impl Pod {
}
/// The env overrides applied on top of the inherited parent env for every
/// child. We isolate omnigent's own state — the DB, data dir, and config
/// home — so concurrent pods don't share a database, pidfile, or
/// `config.yaml`. The rest (real `HOME`, credentials, uv/npm caches) is
/// inherited, since the agents omnigent runs need it. `OMNIGENT_URL` is the
/// seam `web/vite.config.ts` reads to point its proxy at this pod's backend;
/// `OMNIGENT_CONFIG_HOME` is where the server/host/runner read `config.yaml`.
/// child. We isolate only omnigent's own state — the DB and data dir — so
/// concurrent pods don't share a database or pidfile. Everything else
/// (real `HOME`, credentials, config, uv/npm caches) is inherited, since
/// the agents omnigent runs need it. `OMNIGENT_URL` is the seam
/// `web/vite.config.ts` reads to point its proxy at this pod's backend.
pub fn env(&self) -> Vec<(String, String)> {
let d = |p: &str| self.dir.join(p).display().to_string();
let mut env = vec![
("OMNIGENT_DATA_DIR".into(), d("data/omnigent")),
("OMNIGENT_DATABASE_URI".into(), self.db_uri()),
("OMNIGENT_URL".into(), self.server_url()),
(
"OMNIGENT_CONFIG_HOME".into(),
self.config_dir().display().to_string(),
),
];
if let Some(allowed) = self.allowed_origins_env() {
env.push(("OMNIGENT_WS_ALLOWED_ORIGINS".into(), allowed));
@@ -178,171 +156,3 @@ pub fn clean(dir: &Path) -> Result<()> {
}
Ok(())
}
/// The developer's real omnigent `config.yaml` to seed a fresh pod from.
///
/// Honors `OMNIGENT_CONFIG_HOME` if the parent env sets it (nested/test
/// setups), else `~/.omnigent/config.yaml` via `HOME`. Returns `None` when the
/// file does not exist — a fresh pod then starts with an empty config, just
/// like a first-run user.
fn real_config_path() -> Option<PathBuf> {
let home = match std::env::var_os("OMNIGENT_CONFIG_HOME") {
Some(h) if !h.is_empty() => PathBuf::from(h),
_ => PathBuf::from(std::env::var_os("HOME")?).join(".omnigent"),
};
let path = home.join("config.yaml");
path.exists().then_some(path)
}
/// Copy `src` to `dest`, but only when `dest` does not already exist — a normal
/// pod restart must not clobber config the developer edited inside the pod.
/// After `--clean` the whole pod dir is gone, so `dest` is absent and this
/// re-seeds.
fn seed_config_file(src: &Path, dest: &Path) -> Result<()> {
if dest.exists() {
return Ok(());
}
std::fs::copy(src, dest)
.with_context(|| format!("seeding {} from {}", dest.display(), src.display()))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Mutex;
// `real_config_path` reads process-global env; serialize the tests that
// set it so parallel runs don't observe each other's overrides.
static ENV_LOCK: Mutex<()> = Mutex::new(());
fn tempdir() -> PathBuf {
let unique = format!(
"omnidev-pod-test-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
);
let dir = std::env::temp_dir().join(unique);
std::fs::create_dir_all(&dir).unwrap();
dir
}
fn make_pod(pod_dir: PathBuf) -> Pod {
Pod::create(
tempdir(),
pod_dir,
Ports {
server: 19191,
vite: 19292,
},
"127.0.0.1".into(),
Vec::new(),
)
.unwrap()
}
/// Point `OMNIGENT_CONFIG_HOME` at `home` for the duration of `f`, restoring
/// the previous value afterwards. Serialized against other env-touching
/// tests via `ENV_LOCK`.
fn with_config_home<T>(home: &Path, f: impl FnOnce() -> T) -> T {
let _guard = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev = std::env::var_os("OMNIGENT_CONFIG_HOME");
std::env::set_var("OMNIGENT_CONFIG_HOME", home);
let out = f();
match prev {
Some(v) => std::env::set_var("OMNIGENT_CONFIG_HOME", v),
None => std::env::remove_var("OMNIGENT_CONFIG_HOME"),
}
out
}
#[test]
fn create_makes_config_dir() {
let real = tempdir(); // empty config home -> nothing to seed
let pod = with_config_home(&real, || make_pod(tempdir()));
assert!(pod.config_dir().is_dir());
}
#[test]
fn env_includes_config_home() {
let real = tempdir();
let pod = with_config_home(&real, || make_pod(tempdir()));
let env = pod.env();
let got = env
.iter()
.find(|(k, _)| k == "OMNIGENT_CONFIG_HOME")
.map(|(_, v)| v.clone());
assert_eq!(got, Some(pod.config_dir().display().to_string()));
}
#[test]
fn create_seeds_pod_config_from_real() {
let real = tempdir();
std::fs::write(real.join("config.yaml"), "providers:\n seeded: true\n").unwrap();
let pod = with_config_home(&real, || make_pod(tempdir()));
let seeded = std::fs::read_to_string(pod.config_dir().join("config.yaml")).unwrap();
assert_eq!(seeded, "providers:\n seeded: true\n");
}
#[test]
fn create_skips_seed_when_real_config_absent() {
let real = tempdir(); // no config.yaml inside
let pod = with_config_home(&real, || make_pod(tempdir()));
assert!(!pod.config_dir().join("config.yaml").exists());
}
#[test]
fn seed_does_not_overwrite_existing() {
let dir = tempdir();
let src = dir.join("src.yaml");
let dest = dir.join("dest.yaml");
std::fs::write(&src, "from: real\n").unwrap();
std::fs::write(&dest, "edited: in-pod\n").unwrap();
seed_config_file(&src, &dest).unwrap();
// Existing pod-local edits survive; the real config does not clobber them.
assert_eq!(std::fs::read_to_string(&dest).unwrap(), "edited: in-pod\n");
}
#[test]
fn real_config_path_honors_config_home() {
let real = tempdir();
std::fs::write(real.join("config.yaml"), "x: 1\n").unwrap();
let got = with_config_home(&real, real_config_path);
assert_eq!(got, Some(real.join("config.yaml")));
}
#[test]
fn real_config_path_falls_back_to_home_dot_omnigent() {
// With no OMNIGENT_CONFIG_HOME, the real config resolves under
// `$HOME/.omnigent/` — the path a normal pod run seeds from.
let _guard = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
let prev_cfg = std::env::var_os("OMNIGENT_CONFIG_HOME");
let prev_home = std::env::var_os("HOME");
let home = tempdir();
std::fs::create_dir_all(home.join(".omnigent")).unwrap();
std::fs::write(home.join(".omnigent/config.yaml"), "y: 2\n").unwrap();
std::env::remove_var("OMNIGENT_CONFIG_HOME");
std::env::set_var("HOME", &home);
let got = real_config_path();
match prev_cfg {
Some(v) => std::env::set_var("OMNIGENT_CONFIG_HOME", v),
None => std::env::remove_var("OMNIGENT_CONFIG_HOME"),
}
match prev_home {
Some(v) => std::env::set_var("HOME", v),
None => std::env::remove_var("HOME"),
}
assert_eq!(got, Some(home.join(".omnigent/config.yaml")));
}
}
+4 -61
View File
@@ -5,34 +5,22 @@ use std::path::PathBuf;
use crate::pod::Pod;
/// A resolved command line + working dir for one process. Env is applied by the
/// supervisor from `Pod::env()`, with per-process additions from `extra_env`.
/// supervisor from `Pod::env()`, so it is not duplicated here.
pub struct ProcSpec {
pub program: String,
pub args: Vec<String>,
pub cwd: PathBuf,
pub extra_env: Vec<(String, String)>,
}
impl ProcSpec {
fn omnigent_log_env() -> Vec<(String, String)> {
// Child stderr is a pipe that omnidev reads into its process panes.
// Let Omnigent's process logger mirror to that pipe despite it not
// being a terminal, and force ANSI colors because omnidev parses them.
vec![
("OMNIGENT_LOG_TTY_FD".into(), "2".into()),
("OMNIGENT_LOG_FORCE_COLOR".into(), "1".into()),
]
}
/// `uv run omnigent --log-to-stderr server --host 127.0.0.1 --port <p>
/// --database-uri <db> --artifact-location <dir>`, from the repo root.
/// `uv run omnigent server --host 127.0.0.1 --port <p> --database-uri <db>
/// --artifact-location <dir>`, from the repo root.
pub fn server(pod: &Pod) -> ProcSpec {
ProcSpec {
program: "uv".into(),
args: vec![
"run".into(),
"omnigent".into(),
"--log-to-stderr".into(),
"server".into(),
"--host".into(),
"127.0.0.1".into(),
@@ -44,25 +32,21 @@ impl ProcSpec {
pod.artifacts_dir().display().to_string(),
],
cwd: pod.repo_root.clone(),
extra_env: Self::omnigent_log_env(),
}
}
/// `uv run omnigent --log-to-stderr host --server http://127.0.0.1:<p>`,
/// from the repo root.
/// `uv run omnigent host --server http://127.0.0.1:<p>`, from the repo root.
pub fn host(pod: &Pod) -> ProcSpec {
ProcSpec {
program: "uv".into(),
args: vec![
"run".into(),
"omnigent".into(),
"--log-to-stderr".into(),
"host".into(),
"--server".into(),
pod.server_url(),
],
cwd: pod.repo_root.clone(),
extra_env: Self::omnigent_log_env(),
}
}
@@ -83,7 +67,6 @@ impl ProcSpec {
"http".into(),
],
cwd: pod.web_dir(),
extra_env: Vec::new(),
}
}
@@ -103,7 +86,6 @@ impl ProcSpec {
"--strictPort".into(),
],
cwd: pod.web_dir(),
extra_env: Vec::new(),
}
}
}
@@ -135,45 +117,6 @@ mod tests {
assert_eq!(pod.server_url(), "http://127.0.0.1:19191");
}
#[test]
fn omnigent_processes_mirror_logs_to_omnidev_pipe() {
let repo = tempdir();
let pod_dir = tempdir();
let pod = Pod::create(
repo,
pod_dir,
Ports {
server: 19191,
vite: 19292,
},
"127.0.0.1".into(),
Vec::new(),
)
.unwrap();
for spec in [ProcSpec::server(&pod), ProcSpec::host(&pod)] {
assert!(
spec.args.iter().any(|arg| arg == "--log-to-stderr"),
"omnigent command should request stderr logging: {:?}",
spec.args
);
assert_eq!(
spec.extra_env
.iter()
.find(|(key, _)| key == "OMNIGENT_LOG_TTY_FD")
.map(|(_, value)| value.as_str()),
Some("2")
);
assert_eq!(
spec.extra_env
.iter()
.find(|(key, _)| key == "OMNIGENT_LOG_FORCE_COLOR")
.map(|(_, value)| value.as_str()),
Some("1")
);
}
}
fn tempdir() -> std::path::PathBuf {
let unique = format!(
"omnidev-process-test-{}-{}",
-2
View File
@@ -211,7 +211,6 @@ impl Supervisor {
cmd.args(&spec.args)
.current_dir(&spec.cwd)
.envs(self.env.iter().cloned())
.envs(spec.extra_env.iter().cloned())
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
@@ -293,7 +292,6 @@ impl Supervisor {
cmd.args(&spec.args)
.current_dir(&spec.cwd)
.envs(self.env.iter().cloned())
.envs(spec.extra_env.iter().cloned())
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped());
+10 -481
View File
@@ -3,7 +3,6 @@
mod render;
use std::cell::Cell;
use std::io::{self, Stdout};
use std::sync::{Arc, Mutex};
use std::time::Duration;
@@ -31,35 +30,6 @@ pub enum View {
All,
}
/// Search direction. `Fwd` scans toward the tail (newer lines), `Back` toward
/// the head — matching `less`'s `/` and `?`.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub enum Dir {
Fwd,
Back,
}
impl Dir {
fn flip(self) -> Dir {
match self {
Dir::Fwd => Dir::Back,
Dir::Back => Dir::Fwd,
}
}
}
/// A committed search: the query and the direction it was entered with.
pub struct Search {
pub query: String,
pub dir: Dir,
}
/// The line-editor state while the user is typing a `/` or `?` query.
pub struct InputMode {
pub dir: Dir,
pub query: String,
}
impl View {
fn proc(self) -> Option<ProcId> {
match self {
@@ -76,23 +46,9 @@ pub struct App {
shared: Arc<Mutex<Shared>>,
cmds: mpsc::UnboundedSender<Cmd>,
view: View,
/// Display rows scrolled up from the bottom; 0 == pinned to tail. Counted in
/// *rendered rows*, so it stays correct whether or not lines wrap.
/// Lines scrolled up from the bottom; 0 == pinned to tail.
scroll_back: usize,
follow: bool,
/// Wrap long lines to the next row (default) vs. clip them at the edge.
wrap: bool,
/// Body size in rows/cols, refreshed by the renderer each frame so key
/// handling can page by a full/half window and lay out wraps for search.
/// Seeded so keys pressed before the first draw still behave.
viewport_h: Cell<usize>,
viewport_w: Cell<usize>,
/// The last committed search, if any (drives `n`/`N` and highlighting).
search: Option<Search>,
/// Logical line index of the match `n`/`N` last jumped to, for anchoring.
current_match: Option<usize>,
/// Set while the user is typing a query; steals keys from command mode.
input: Option<InputMode>,
should_quit: bool,
}
@@ -105,12 +61,6 @@ impl App {
view: View::All,
scroll_back: 0,
follow: true,
wrap: true,
viewport_h: Cell::new(20),
viewport_w: Cell::new(80),
search: None,
current_match: None,
input: None,
should_quit: false,
}
}
@@ -148,21 +98,9 @@ impl App {
if key.kind != KeyEventKind::Press {
return;
}
// Ctrl-C always quits, even mid-search.
if key.code == KeyCode::Char('c') && key.modifiers.contains(KeyModifiers::CONTROL) {
self.should_quit = true;
return;
}
// While typing a query, keys build/commit/cancel it instead of running
// commands.
if self.input.is_some() {
self.on_key_input(key);
return;
}
let window = self.viewport_h.get().max(1);
let half = (window / 2).max(1);
let page = 20;
match (key.code, key.modifiers) {
(KeyCode::Char('c'), KeyModifiers::CONTROL) => self.should_quit = true,
(KeyCode::Char('q'), _) => self.should_quit = true,
(KeyCode::Char('1'), _) => self.set_view(View::Server),
@@ -171,33 +109,17 @@ impl App {
(KeyCode::Char('0'), _) => self.set_view(View::All),
(KeyCode::Tab, _) => self.cycle_view(),
// Pager movement — full `less` semantics.
(KeyCode::Char('j'), _) | (KeyCode::Down, _) => self.scroll_down(1),
(KeyCode::Char('k'), _) | (KeyCode::Up, _) => self.scroll_up(1),
(KeyCode::Char('f'), _) | (KeyCode::Char(' '), _) | (KeyCode::PageDown, _) => {
self.scroll_down(window)
}
(KeyCode::Char('b'), _) | (KeyCode::PageUp, _) => self.scroll_up(window),
(KeyCode::Char('d'), _) => self.scroll_down(half),
(KeyCode::Char('u'), _) => self.scroll_up(half),
(KeyCode::Char('g'), _) | (KeyCode::Home, _) => self.scroll_to_top(),
(KeyCode::Char('G'), _) | (KeyCode::End, _) => self.scroll_to_bottom(),
(KeyCode::Up, _) => self.scroll(1),
(KeyCode::Down, _) => self.scroll_down(1),
(KeyCode::PageUp, _) => self.scroll(page),
(KeyCode::PageDown, _) => self.scroll_down(page),
// `less +F`: capital F toggles tail-follow.
(KeyCode::Char('F'), _) => {
(KeyCode::Char('f'), _) => {
self.follow = !self.follow;
if self.follow {
self.scroll_back = 0;
}
}
(KeyCode::Char('w'), _) => self.toggle_wrap(),
// Search.
(KeyCode::Char('/'), _) => self.begin_search(Dir::Fwd),
(KeyCode::Char('?'), _) => self.begin_search(Dir::Back),
(KeyCode::Char('n'), _) => self.repeat_search(false),
(KeyCode::Char('N'), _) => self.repeat_search(true),
(KeyCode::Char('r'), _) => {
if let Some(id) = self.view.proc() {
let _ = self.cmds.send(Cmd::Restart(id));
@@ -213,43 +135,9 @@ impl App {
}
}
/// Handle a key while a `/` or `?` query is being typed.
fn on_key_input(&mut self, key: KeyEvent) {
match key.code {
KeyCode::Enter => {
let input = self.input.take().unwrap();
if !input.query.is_empty() {
self.search = Some(Search {
query: input.query,
dir: input.dir,
});
self.current_match = None;
self.run_search(input.dir, true);
}
}
KeyCode::Esc => self.input = None,
KeyCode::Backspace => {
let done = {
let input = self.input.as_mut().unwrap();
input.query.pop();
input.query.is_empty()
};
if done {
self.input = None;
}
}
KeyCode::Char(c) if !key.modifiers.contains(KeyModifiers::CONTROL) => {
self.input.as_mut().unwrap().query.push(c);
}
_ => {}
}
}
fn set_view(&mut self, v: View) {
self.view = v;
self.scroll_back = 0;
// Match indices are per-view; drop the anchor on switch.
self.current_match = None;
}
fn cycle_view(&mut self) {
@@ -260,10 +148,9 @@ impl App {
View::Vite => View::All,
};
self.scroll_back = 0;
self.current_match = None;
}
fn scroll_up(&mut self, n: usize) {
fn scroll(&mut self, n: usize) {
// Scrolling up detaches from the tail.
self.follow = false;
self.scroll_back = self.scroll_back.saturating_add(n);
@@ -276,147 +163,6 @@ impl App {
}
}
fn scroll_to_top(&mut self) {
self.follow = false;
let lines = self.display_lines();
let counts = self.row_counts(&lines);
let total: usize = counts.iter().sum();
let height = self.viewport_h.get().max(1);
self.scroll_back = total.saturating_sub(height);
}
fn scroll_to_bottom(&mut self) {
self.scroll_back = 0;
self.follow = true;
}
fn toggle_wrap(&mut self) {
self.wrap = !self.wrap;
// Row counts change with wrap; re-anchor on the matched line if any,
// otherwise drop to the tail so we land somewhere sane.
match self.current_match {
Some(idx) => {
let lines = self.display_lines();
self.jump_to_logical(idx, &lines);
}
None => self.scroll_to_bottom(),
}
}
fn begin_search(&mut self, dir: Dir) {
self.input = Some(InputMode {
dir,
query: String::new(),
});
}
/// `n` repeats the committed search in its direction; `N` (opposite=true)
/// reverses it.
fn repeat_search(&mut self, opposite: bool) {
let Some(search) = self.search.as_ref() else {
return;
};
let dir = if opposite {
search.dir.flip()
} else {
search.dir
};
self.run_search(dir, false);
}
/// Scan for the next match and jump to it. `fresh` anchors from the current
/// viewport; otherwise it steps off the last matched line.
fn run_search(&mut self, dir: Dir, fresh: bool) {
let Some(query) = self.search.as_ref().map(|s| s.query.to_ascii_lowercase()) else {
return;
};
let lines = self.display_lines();
let n = lines.len();
if n == 0 || query.is_empty() {
return;
}
let start = if fresh {
self.anchor(&lines, dir)
} else {
match self.current_match {
Some(m) => match dir {
Dir::Fwd => (m + 1) % n,
Dir::Back => (m + n - 1) % n,
},
None => self.anchor(&lines, dir),
}
};
// Scan every line once, wrapping around the ends.
for k in 0..n {
let i = match dir {
Dir::Fwd => (start + k) % n,
Dir::Back => (start + n - (k % n)) % n,
};
if lines[i].to_ascii_lowercase().contains(&query) {
self.current_match = Some(i);
self.jump_to_logical(i, &lines);
return;
}
}
}
/// Displayed text (ANSI stripped, `[label]` prefix included in the combined
/// view) for every logical line of the focused channel — the exact text the
/// renderer shows, so search offsets and wrap counts line up.
fn display_lines(&self) -> Vec<String> {
let all_view = self.view == View::All;
let s = self.shared.lock().unwrap();
let iter: Box<dyn Iterator<Item = &String>> = match self.view {
View::Server => Box::new(s.buf(ProcId::Server).iter()),
View::Host => Box::new(s.buf(ProcId::Host).iter()),
View::Vite => Box::new(s.buf(ProcId::Vite).iter()),
View::All => Box::new(s.all.iter()),
};
iter.map(|l| render::display_text(l, all_view)).collect()
}
/// Per-line display-row counts at the current width/wrap.
fn row_counts(&self, lines: &[String]) -> Vec<usize> {
let width = self.viewport_w.get();
lines
.iter()
.map(|t| render::row_count(t, width, self.wrap))
.collect()
}
/// The logical line a fresh search should scan from: the top visible line
/// going forward, the bottom visible line going back.
fn anchor(&self, lines: &[String], dir: Dir) -> usize {
let counts = self.row_counts(lines);
let total: usize = counts.iter().sum();
let height = self.viewport_h.get().max(1);
let back = self.scroll_back.min(total.saturating_sub(height));
let end = total.saturating_sub(back); // one past the bottom visible row
let top_row = end.saturating_sub(height);
match dir {
Dir::Fwd => line_at_row(&counts, top_row),
Dir::Back => line_at_row(&counts, end.saturating_sub(1)),
}
}
/// Scroll so logical line `idx`'s first display row sits at the top of the
/// viewport (clamped so we never scroll past the tail).
fn jump_to_logical(&mut self, idx: usize, lines: &[String]) {
let counts = self.row_counts(lines);
if idx >= counts.len() {
return;
}
let height = self.viewport_h.get().max(1);
let below: usize = counts[idx + 1..].iter().sum();
let own = counts[idx];
let total: usize = counts.iter().sum();
let max_back = total.saturating_sub(height);
self.scroll_back = (own + below).saturating_sub(height).min(max_back);
self.follow = false;
}
fn clear_current(&mut self) {
let mut s = self.shared.lock().unwrap();
match self.view {
@@ -426,10 +172,9 @@ impl App {
View::All => s.all.clear(),
}
self.scroll_back = 0;
self.current_match = None;
}
/// Total logical line count of the focused channel, for the status readout.
/// Total line count of the focused channel, for the status readout.
pub fn line_count(&self) -> usize {
let s = self.shared.lock().unwrap();
match self.view {
@@ -439,53 +184,6 @@ impl App {
View::All => s.all.iter().count(),
}
}
/// The committed query, ASCII-lowercased, for the renderer's highlight
/// pass. `None` when no search is active.
pub fn search_query_lower(&self) -> Option<String> {
self.search
.as_ref()
.filter(|s| !s.query.is_empty())
.map(|s| s.query.to_ascii_lowercase())
}
/// The in-progress query prompt (`dir`, text) while the user is typing.
pub fn input_prompt(&self) -> Option<(Dir, &str)> {
self.input.as_ref().map(|i| (i.dir, i.query.as_str()))
}
/// Number of logical lines matching the committed search, for the status
/// readout, plus the 1-based rank of the current match within them.
pub fn match_stats(&self) -> Option<(usize, usize)> {
let query = self.search.as_ref()?.query.to_ascii_lowercase();
if query.is_empty() {
return None;
}
let lines = self.display_lines();
let mut total = 0;
let mut rank = 0;
for (i, l) in lines.iter().enumerate() {
if l.to_ascii_lowercase().contains(&query) {
total += 1;
if Some(i) == self.current_match {
rank = total;
}
}
}
Some((rank, total))
}
}
/// Map a display-row index to the logical line that contains it.
fn line_at_row(counts: &[usize], target_row: usize) -> usize {
let mut acc = 0;
for (i, &rc) in counts.iter().enumerate() {
if target_row < acc + rc {
return i;
}
acc += rc;
}
counts.len().saturating_sub(1)
}
fn setup_terminal() -> Result<Terminal<CrosstermBackend<Stdout>>> {
@@ -516,172 +214,3 @@ fn spawn_input() -> mpsc::UnboundedReceiver<KeyEvent> {
});
rx
}
#[cfg(test)]
mod tests {
//! Headless end-to-end: drive the real `on_key` and render through
//! ratatui's `TestBackend`, so the full key → state → draw path is
//! exercised without a TTY or a live pod.
use super::*;
use crate::ports::Ports;
use ratatui::backend::TestBackend;
use ratatui::Terminal;
/// Build an `App` over a throwaway pod and a channel whose receiver we keep
/// so `cmds.send` never fails.
fn app() -> (App, mpsc::UnboundedReceiver<Cmd>) {
let root = std::env::temp_dir().join(format!("omnidev-tui-{}", std::process::id()));
let dir = root.join("pod");
let pod = Arc::new(
Pod::create(
root.clone(),
dir,
Ports {
server: 6767,
vite: 5173,
},
"127.0.0.1".into(),
Vec::new(),
)
.unwrap(),
);
let shared = Shared::new(&pod);
let (tx, rx) = mpsc::unbounded_channel();
(App::new(pod, shared, tx), rx)
}
fn press(app: &mut App, code: KeyCode) {
app.on_key(KeyEvent::new(code, KeyModifiers::NONE));
}
fn type_str(app: &mut App, s: &str) {
for c in s.chars() {
press(app, KeyCode::Char(c));
}
}
/// Render one frame at the given size and return the body rows (everything
/// between the 4 header rows and the footer) as trimmed strings.
fn body(app: &App, w: u16, h: u16) -> Vec<String> {
let mut term = Terminal::new(TestBackend::new(w, h)).unwrap();
term.draw(|f| render::draw(f, app)).unwrap();
let buf = term.backend().buffer().clone();
let mut rows = Vec::new();
// Layout: 4 header rows, body fills the middle, 1 footer row.
for y in 4..h - 1 {
let mut s = String::new();
for x in 0..w {
s.push_str(buf.cell((x, y)).unwrap().symbol());
}
rows.push(s.trim_end().to_string());
}
rows
}
fn seed(app: &App, n: usize) {
let mut s = app.shared.lock().unwrap();
for i in 0..n {
s.all.push(format!("line{i:03}"));
}
}
#[test]
fn renders_tail_by_default() {
let (app, _rx) = app();
seed(&app, 100);
let rows = body(&app, 40, 12); // 4 header + 7 body + 1 footer
assert_eq!(rows.last().unwrap(), "line099");
assert!(rows.iter().any(|r| r == "line093"));
}
#[test]
fn paging_and_ends_move_the_window() {
let (mut app, _rx) = app();
seed(&app, 100);
// Establish viewport height via a first render (7 body rows).
let _ = body(&app, 40, 12);
press(&mut app, KeyCode::Char('b')); // page back one window
assert!(!app.follow);
let rows = body(&app, 40, 12);
assert_eq!(rows.last().unwrap(), "line092");
press(&mut app, KeyCode::Char('g')); // top
let rows = body(&app, 40, 12);
assert_eq!(rows.first().unwrap(), "line000");
press(&mut app, KeyCode::Char('G')); // bottom + follow
assert!(app.follow);
let rows = body(&app, 40, 12);
assert_eq!(rows.last().unwrap(), "line099");
}
#[test]
fn wrap_toggle_changes_row_shape() {
let (mut app, _rx) = app();
{
let mut s = app.shared.lock().unwrap();
s.all.push("X".repeat(30)); // wider than a 10-col body
}
// Default wrap ON: the 30-char line occupies multiple body rows.
let wrapped = body(&app, 10, 8);
let nonblank = wrapped.iter().filter(|r| !r.is_empty()).count();
assert!(nonblank >= 3, "expected wrap across rows, got {wrapped:?}");
press(&mut app, KeyCode::Char('w')); // wrap OFF → clipped to one row
let clipped = body(&app, 10, 8);
let nonblank = clipped.iter().filter(|r| !r.is_empty()).count();
assert_eq!(nonblank, 1);
}
#[test]
fn search_jumps_and_highlights() {
let (mut app, _rx) = app();
{
let mut s = app.shared.lock().unwrap();
for i in 0..100 {
let tag = if i == 5 { " ERROR here" } else { "" };
s.all.push(format!("line{i:03}{tag}"));
}
}
let _ = body(&app, 40, 12);
// `/error` + Enter jumps up to the match near the top of the body.
press(&mut app, KeyCode::Char('/'));
type_str(&mut app, "error");
press(&mut app, KeyCode::Enter);
assert_eq!(app.current_match, Some(5));
assert_eq!(app.match_stats(), Some((1, 1)));
// The matched line is visible and its "ERROR" is highlighted.
let mut term = Terminal::new(TestBackend::new(40, 12)).unwrap();
term.draw(|f| render::draw(f, &app)).unwrap();
let buf = term.backend().buffer().clone();
let mut highlit = 0;
for y in 4..11 {
for x in 0..40 {
let cell = buf.cell((x, y)).unwrap();
let is_match_char = matches!(cell.symbol(), "E" | "R" | "O");
if is_match_char && cell.bg == render::match_bg() {
highlit += 1;
}
}
}
assert!(
highlit >= 5,
"expected the match highlighted, got {highlit}"
);
}
#[test]
fn typing_query_does_not_run_commands() {
let (mut app, _rx) = app();
seed(&app, 100);
let _ = body(&app, 40, 12);
press(&mut app, KeyCode::Char('/'));
// 'q' would quit in command mode; here it's just query text.
type_str(&mut app, "q");
assert!(!app.should_quit);
assert_eq!(app.input_prompt(), Some((Dir::Fwd, "q")));
press(&mut app, KeyCode::Esc);
assert!(app.input_prompt().is_none());
}
}
+38 -387
View File
@@ -9,9 +9,8 @@ use ratatui::style::{Color, Modifier, Style};
use ratatui::text::{Line, Span};
use ratatui::widgets::{Paragraph, Tabs};
use ratatui::Frame;
use unicode_width::UnicodeWidthChar;
use super::{App, Dir, View};
use super::{App, View};
use crate::state::{ProcId, ProcStatus};
// Palette calibrated (Solarized accents) to stay legible on both light and
@@ -27,28 +26,16 @@ const SERVER: Color = Color::Rgb(38, 139, 210); // blue
const HOST: Color = Color::Rgb(42, 161, 152); // cyan
const VITE: Color = Color::Rgb(211, 54, 130); // magenta
const EVENT: Color = Color::Rgb(181, 137, 0); // amber (omnidev channel)
const LABEL_WIDTH: usize = 7;
const OK: Color = Color::Rgb(133, 153, 0); // green (running)
const WARN: Color = Color::Rgb(203, 75, 22); // orange (starting/restarting)
const ERR: Color = Color::Rgb(220, 50, 47); // red (crashed)
// Search-match highlight: amber background with near-black text, legible on
// either theme and distinct from the ANSI log colors underneath.
const MATCH_BG: Color = Color::Rgb(181, 137, 0);
const MATCH_FG: Color = Color::Rgb(20, 20, 20);
/// Style for the header/footer chrome bars.
fn chrome() -> Style {
Style::default().bg(CHROME_BG).fg(CHROME_FG)
}
/// The search-match background, exposed for tests that assert highlighting.
#[cfg(test)]
pub fn match_bg() -> Color {
MATCH_BG
}
pub fn draw(f: &mut Frame, app: &App) {
let chunks = Layout::default()
.direction(Direction::Vertical)
@@ -67,7 +54,7 @@ pub fn draw(f: &mut Frame, app: &App) {
draw_chips(f, app, chunks[2]);
draw_tabs_row(f, app, chunks[3]);
draw_body(f, app, chunks[4]);
draw_footer(f, app, chunks[5]);
draw_footer(f, chunks[5]);
}
fn draw_pod(f: &mut Frame, app: &App, area: Rect) {
@@ -119,7 +106,7 @@ fn draw_tabs_row(f: &mut Frame, app: &App, area: Rect) {
// Split the row: tabs on the left, scroll/follow status right-aligned.
let cols = Layout::default()
.direction(Direction::Horizontal)
.constraints([Constraint::Min(0), Constraint::Length(36)])
.constraints([Constraint::Min(0), Constraint::Length(24)])
.split(area);
let entries = [
@@ -147,18 +134,11 @@ fn draw_tabs_row(f: &mut Frame, app: &App, area: Rect) {
f.render_widget(tabs, cols[0]);
let total = app.line_count();
let mut status = format!("{total} ln");
if !app.wrap {
status.push_str(" · nowrap");
}
if let Some((rank, count)) = app.match_stats() {
status.push_str(&format!(" · {rank}/{count}"));
}
if app.follow {
status.push_str(" · follow ");
let status = if app.follow {
format!("{total} ln · follow ")
} else {
status.push_str(&format!(" · ↑{} ", app.scroll_back));
}
format!("{total} ln · ↑{} ", app.scroll_back)
};
f.render_widget(
Paragraph::new(Line::from(Span::styled(status, Style::default().fg(MUTED))))
.alignment(Alignment::Right)
@@ -169,12 +149,6 @@ fn draw_tabs_row(f: &mut Frame, app: &App, area: Rect) {
fn draw_body(f: &mut Frame, app: &App, area: Rect) {
let all_view = app.view == View::All;
let width = area.width as usize;
let height = area.height as usize;
// Publish the body geometry so key handling can page and search can wrap.
app.viewport_h.set(height);
app.viewport_w.set(width);
let shared = app.shared.lock().unwrap();
let lines: Vec<String> = match app.view {
View::Server => shared.buf(ProcId::Server).iter().cloned().collect(),
@@ -184,230 +158,54 @@ fn draw_body(f: &mut Frame, app: &App, area: Rect) {
};
drop(shared);
let query = app.search_query_lower();
let visible = visible_rows(
&lines,
all_view,
width,
height,
app.wrap,
app.scroll_back,
query.as_deref(),
let height = area.height as usize;
let total = lines.len();
let max_back = total.saturating_sub(height);
let back = app.scroll_back.min(max_back);
let end = total.saturating_sub(back);
let start = end.saturating_sub(height);
let rendered: Vec<Line> = lines[start..end]
.iter()
.map(|l| render_line(l, all_view))
.collect();
f.render_widget(Paragraph::new(rendered), area);
}
fn draw_footer(f: &mut Frame, area: Rect) {
let hint = " 1/2/3/0 view · Tab cycle · ↑↓/PgUp/PgDn scroll · f follow · r restart · R backend · c clear · q quit ";
f.render_widget(
Paragraph::new(Line::from(Span::styled(
hint,
Style::default().fg(CHROME_FG),
)))
.style(chrome()),
area,
);
f.render_widget(Paragraph::new(visible), area);
}
/// The window of display rows to show: the `height` rows sitting `scroll_back`
/// rows above the tail. Rows are built from the bottom up, wrapping only enough
/// logical lines to cover `scroll_back + height` so a full buffer isn't
/// re-parsed every frame. Equivalent to wrapping every line and slicing the
/// flat list, but without the wasted work.
fn visible_rows(
lines: &[String],
all_view: bool,
width: usize,
height: usize,
wrap: bool,
scroll_back: usize,
query: Option<&str>,
) -> Vec<Line<'static>> {
// `acc` holds rows bottom-to-top; each logical line yields one row (wrap
// off) or several (wrap on), so `scroll_back` counts rendered rows.
let needed = scroll_back.saturating_add(height);
let mut acc: Vec<Line> = Vec::with_capacity(needed + 8);
let mut exhausted = true;
for raw in lines.iter().rev() {
let spans = render_line(raw, all_view);
let ranges = query.map(|q| match_ranges(raw, all_view, q));
let mut line_rows: Vec<Line> = Vec::new();
wrap_spans(spans, width, wrap, ranges.as_deref(), &mut line_rows);
acc.extend(line_rows.into_iter().rev());
if acc.len() >= needed {
exhausted = false;
break;
}
}
// If we ran out of lines the buffer is shorter than the scroll offset, so
// clamp to the top; otherwise `scroll_back` is within range as-is.
let back = if exhausted {
scroll_back.min(acc.len().saturating_sub(height))
} else {
scroll_back
};
let end = (back + height).min(acc.len());
let mut visible: Vec<Line> = acc.drain(back..end).collect();
visible.reverse();
visible
}
fn draw_footer(f: &mut Frame, app: &App, area: Rect) {
// While typing a query the footer becomes the search prompt with a cursor
// block; otherwise it lists the key hints.
let line = if let Some((dir, query)) = app.input_prompt() {
let sigil = match dir {
Dir::Fwd => '/',
Dir::Back => '?',
};
Line::from(vec![
Span::styled(
format!(" {sigil}{query}"),
Style::default().fg(CHROME_FG).add_modifier(Modifier::BOLD),
),
Span::styled("", Style::default().fg(CHROME_FG)),
])
} else {
let hint = " f/b page · d/u half · j/k line · g/G ends · F follow · w wrap · / ? search · n/N next · 1230/Tab view · r/R restart · c clear · q quit ";
Line::from(Span::styled(hint, Style::default().fg(CHROME_FG)))
};
f.render_widget(Paragraph::new(line).style(chrome()), area);
}
/// Turn one stored log line into styled spans. In the combined view the leading
/// `[service]` tag is colored per service and the rest keeps its ANSI colors;
/// per-service panes just pass their ANSI through.
fn render_line(raw: &str, all_view: bool) -> Vec<Span<'static>> {
/// Turn one stored log line into a styled `Line`. In the combined view the
/// leading `[service]` tag is colored per service and the rest keeps its ANSI
/// colors; per-service panes just pass their ANSI through.
fn render_line(raw: &str, all_view: bool) -> Line<'static> {
if all_view {
if let Some(rest) = raw.strip_prefix('[') {
if let Some(end) = rest.find(']') {
let label = &rest[..end];
let body = &rest[end + 1..];
let mut spans = vec![Span::styled(
format!("[{label:<LABEL_WIDTH$}]"),
format!("[{label}]"),
Style::default()
.fg(label_color(label))
.add_modifier(Modifier::BOLD),
)];
spans.extend(ansi_spans(body));
return spans;
return Line::from(spans);
}
}
}
ansi_spans(raw)
}
/// The exact text `render_line` will display (ANSI stripped, `[label]` prefix
/// included), so search offsets and wrap-row counts line up with what's drawn.
pub fn display_text(raw: &str, all_view: bool) -> String {
render_line(raw, all_view)
.iter()
.map(|s| s.content.as_ref())
.collect()
}
/// Column width of a char for layout. Control and zero-width chars (including
/// tabs) count as 0 — good enough for log lines.
fn char_cols(c: char) -> usize {
UnicodeWidthChar::width(c).unwrap_or(0)
}
/// How many display rows `text` occupies at `width` columns. Must stay in step
/// with `wrap_spans`' row splitting so scroll math and search jumps agree.
pub fn row_count(text: &str, width: usize, wrap: bool) -> usize {
if !wrap || width == 0 {
return 1;
}
let mut rows = 1;
let mut col = 0;
for c in text.chars() {
let w = char_cols(c);
if col + w > width && col > 0 {
rows += 1;
col = 0;
}
col += w;
}
rows
}
/// Char-offset ranges of every case-insensitive occurrence of `query` (already
/// ASCII-lowercased) in the line's displayed text. Offsets are in chars so they
/// align with `wrap_spans`' per-char highlight test.
fn match_ranges(raw: &str, all_view: bool, query: &str) -> Vec<(usize, usize)> {
let mut ranges = Vec::new();
if query.is_empty() {
return ranges;
}
let hay: Vec<char> = display_text(raw, all_view)
.chars()
.map(|c| c.to_ascii_lowercase())
.collect();
let q: Vec<char> = query.chars().collect();
if hay.len() < q.len() {
return ranges;
}
let mut i = 0;
while i + q.len() <= hay.len() {
if hay[i..i + q.len()] == q[..] {
ranges.push((i, i + q.len()));
i += q.len();
} else {
i += 1;
}
}
ranges
}
/// Split one logical line's spans into display rows, pushing each row onto
/// `out`. When `wrap` is off (or width 0) the line stays a single row — clipped
/// at the edge by the renderer, as before. Contiguous same-style chars coalesce
/// into one span. Chars whose char-offset falls in a `matches` range get the
/// search-highlight style overlaid, so a match spanning a wrap boundary lights
/// up on both rows.
fn wrap_spans(
spans: Vec<Span<'static>>,
width: usize,
wrap: bool,
matches: Option<&[(usize, usize)]>,
out: &mut Vec<Line<'static>>,
) {
let matches = matches.unwrap_or(&[]);
// Nothing to reflow or highlight: emit the spans as one row untouched.
if (!wrap || width == 0) && matches.is_empty() {
out.push(Line::from(spans));
return;
}
let in_match = |off: usize| matches.iter().any(|&(s, e)| off >= s && off < e);
let mut row: Vec<Span<'static>> = Vec::new();
let mut run = String::new();
let mut run_style: Option<Style> = None;
let mut col = 0usize;
let mut offset = 0usize;
for span in &spans {
let base = span.style;
for c in span.content.chars() {
let w = char_cols(c);
if wrap && width > 0 && col + w > width && col > 0 {
flush_run(&mut run, run_style.unwrap_or_default(), &mut row);
out.push(Line::from(std::mem::take(&mut row)));
col = 0;
}
let style = if in_match(offset) {
base.bg(MATCH_BG).fg(MATCH_FG).add_modifier(Modifier::BOLD)
} else {
base
};
if run_style != Some(style) {
flush_run(&mut run, run_style.unwrap_or_default(), &mut row);
run_style = Some(style);
}
run.push(c);
col += w;
offset += 1;
}
}
flush_run(&mut run, run_style.unwrap_or_default(), &mut row);
out.push(Line::from(row));
}
/// Emit the buffered same-style run as a span, clearing the buffer.
fn flush_run(run: &mut String, style: Style, row: &mut Vec<Span<'static>>) {
if !run.is_empty() {
row.push(Span::styled(std::mem::take(run), style));
}
Line::from(ansi_spans(raw))
}
/// Parse a single line of possibly-ANSI text into owned spans, falling back to
@@ -453,150 +251,3 @@ fn status_color(st: &ProcStatus) -> Color {
ProcStatus::Idle => MUTED,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn rows(text: &str, width: usize, wrap: bool) -> Vec<String> {
let mut out = Vec::new();
wrap_spans(
vec![Span::raw(text.to_string())],
width,
wrap,
None,
&mut out,
);
out.iter()
.map(|l| l.spans.iter().map(|s| s.content.as_ref()).collect())
.collect()
}
#[test]
fn wrap_off_is_one_row() {
assert_eq!(rows("hello world", 4, false), vec!["hello world"]);
assert_eq!(row_count("hello world", 4, false), 1);
}
#[test]
fn wrap_splits_at_width_and_row_count_agrees() {
let text = "abcdefgh";
assert_eq!(rows(text, 3, true), vec!["abc", "def", "gh"]);
assert_eq!(row_count(text, 3, true), 3);
}
#[test]
fn wide_char_that_does_not_fit_wraps_first() {
// "a" then a 2-wide char into width 2: the wide char can't share the
// row with "a", so it starts the next one.
let rows = rows("a世", 2, true);
assert_eq!(rows, vec!["a", ""]);
assert_eq!(row_count("a世", 2, true), 2);
}
#[test]
fn zero_width_join_does_not_add_a_row() {
// A trailing combining mark rides the last column, not a new row.
assert_eq!(row_count("abc\u{0301}", 3, true), 1);
}
#[test]
fn width_zero_never_panics() {
assert_eq!(rows("abc", 0, true), vec!["abc"]);
assert_eq!(row_count("abc", 0, true), 1);
}
#[test]
fn match_ranges_are_case_insensitive_char_offsets() {
assert_eq!(
match_ranges("Error: ERROR", false, "error"),
vec![(0, 5), (7, 12)]
);
assert_eq!(match_ranges("nope", false, "error"), vec![]);
}
/// Reference: wrap every line into one flat list, then slice the window —
/// the obvious-but-wasteful version `visible_rows` optimizes.
fn naive_visible(
lines: &[String],
width: usize,
height: usize,
wrap: bool,
scroll_back: usize,
) -> Vec<String> {
let mut all: Vec<Line> = Vec::new();
for raw in lines {
wrap_spans(vec![Span::raw(raw.clone())], width, wrap, None, &mut all);
}
let total = all.len();
let back = scroll_back.min(total.saturating_sub(height));
let end = total.saturating_sub(back);
let start = end.saturating_sub(height);
all[start..end].iter().map(row_text).collect()
}
fn row_text(l: &Line) -> String {
l.spans.iter().map(|s| s.content.as_ref()).collect()
}
fn lazy_visible(
lines: &[String],
width: usize,
height: usize,
wrap: bool,
scroll_back: usize,
) -> Vec<String> {
visible_rows(lines, false, width, height, wrap, scroll_back, None)
.iter()
.map(row_text)
.collect()
}
#[test]
fn lazy_slice_matches_naive_across_offsets() {
let lines: Vec<String> = (0..30).map(|i| format!("line{i:02}=abcdefghij")).collect();
for &wrap in &[false, true] {
for width in [6usize, 8, 40] {
for height in [1usize, 5, 12] {
for back in [0usize, 3, 10, 25, 999] {
assert_eq!(
lazy_visible(&lines, width, height, wrap, back),
naive_visible(&lines, width, height, wrap, back),
"wrap={wrap} width={width} height={height} back={back}",
);
}
}
}
}
}
#[test]
fn empty_and_short_buffers_do_not_panic() {
assert!(lazy_visible(&[], 10, 5, true, 0).is_empty());
let one = vec!["hi".to_string()];
assert_eq!(lazy_visible(&one, 10, 5, true, 0), vec!["hi"]);
assert_eq!(lazy_visible(&one, 10, 5, true, 99), vec!["hi"]);
}
#[test]
fn highlight_survives_a_wrap_boundary() {
// "error" at chars 2..7 straddles the width-4 wrap between rows.
let ranges = match_ranges("--error--", false, "error");
let mut out = Vec::new();
wrap_spans(
vec![Span::raw("--error--".to_string())],
4,
true,
Some(&ranges),
&mut out,
);
// Every row that overlaps the match must carry a highlighted span.
let highlighted: usize = out
.iter()
.flat_map(|l| &l.spans)
.filter(|s| s.style.bg == Some(MATCH_BG))
.map(|s| s.content.chars().count())
.sum();
assert_eq!(highlighted, 5); // all five chars of "error"
}
}
+7 -117
View File
@@ -3,38 +3,24 @@
//! handles those.
use std::path::Path;
use std::sync::{Arc, Mutex};
use std::time::Duration;
use anyhow::{Context, Result};
use ignore::gitignore::{Gitignore, GitignoreBuilder};
use notify::RecursiveMode;
use notify_debouncer_full::new_debouncer;
use tokio::sync::mpsc;
use crate::state::Shared;
use crate::supervisor::Cmd;
/// Start watching `omnigent_dir` for `*.py` changes. Coalesced bursts become a
/// single `Cmd::Reload(n)` on `cmd_tx`. The returned debouncer must be kept
/// alive for the watch to persist.
///
/// Gitignored files (e.g. the build-time `omnigent/_build_info.py`) are skipped
/// so churn from generated files doesn't trigger reloads. With `debug` on, every
/// observed change is logged with whether it triggered a reload or why it was
/// skipped.
pub fn spawn(
repo_root: &Path,
omnigent_dir: &Path,
shared: Arc<Mutex<Shared>>,
debug: bool,
cmd_tx: mpsc::UnboundedSender<Cmd>,
) -> Result<impl Send + 'static> {
let ignore = build_ignore(repo_root);
let repo_root = repo_root.to_path_buf();
// The debouncer coalesces rapid saves; we still filter to *.py, skip caches
// and gitignored files so editor churn and generated writes don't reload.
// The debouncer coalesces rapid saves; we still filter to *.py and skip
// caches so editor churn and __pycache__ writes don't trigger reloads.
let mut debouncer = new_debouncer(
Duration::from_millis(500),
None,
@@ -43,18 +29,8 @@ pub fn spawn(
let mut changed = 0usize;
for event in &events {
for path in &event.paths {
match classify(path, &ignore) {
Ok(()) => {
changed += 1;
if debug {
log_watch(&shared, &repo_root, path, "reload trigger");
}
}
Err(reason) => {
if debug {
log_watch(&shared, &repo_root, path, &format!("skip ({reason})"));
}
}
if is_relevant(path) {
changed += 1;
}
}
}
@@ -72,95 +48,9 @@ pub fn spawn(
Ok(debouncer)
}
/// Build a gitignore matcher from the repo's root `.gitignore` and
/// `.git/info/exclude`. Both are best-effort — a missing or malformed file just
/// contributes no rules. Nested `.gitignore` files under `omnigent/` are not
/// consulted (the repo has none today); add them here if that changes.
fn build_ignore(repo_root: &Path) -> Gitignore {
let mut b = GitignoreBuilder::new(repo_root);
b.add(repo_root.join(".gitignore"));
b.add(repo_root.join(".git").join("info").join("exclude"));
b.build().unwrap_or_else(|_| Gitignore::empty())
}
/// Decide whether a changed path should trigger a reload, or why not. The `Err`
/// carries a short reason for the `--debug` log.
fn classify(path: &Path, ignore: &Gitignore) -> Result<(), &'static str> {
fn is_relevant(path: &Path) -> bool {
if path.extension().and_then(|e| e.to_str()) != Some("py") {
return Err("non-.py");
}
if path.components().any(|c| c.as_os_str() == "__pycache__") {
return Err("__pycache__");
}
// `_or_any_parents` so files inside a gitignored directory (build/, dist/,
// *.egg-info/, …) are skipped too, matching git's own behavior — plain
// `matched` only catches paths named by a rule directly.
if ignore.matched_path_or_any_parents(path, false).is_ignore() {
return Err("gitignored");
}
Ok(())
}
/// Emit a `--debug` watch line into the combined pane, path shown relative to
/// the repo root when possible.
fn log_watch(shared: &Arc<Mutex<Shared>>, repo_root: &Path, path: &Path, what: &str) {
let rel = path.strip_prefix(repo_root).unwrap_or(path);
shared
.lock()
.unwrap()
.event(format!("watch: {what} {}", rel.display()));
}
#[cfg(test)]
mod tests {
use super::*;
fn ignore_with(line: &str) -> Gitignore {
let mut b = GitignoreBuilder::new("/repo");
b.add_line(None, line).unwrap();
b.build().unwrap()
}
#[test]
fn plain_python_file_triggers_reload() {
let ig = ignore_with("omnigent/_build_info.py");
assert_eq!(classify(Path::new("/repo/omnigent/cli.py"), &ig), Ok(()));
}
#[test]
fn gitignored_python_file_is_skipped() {
let ig = ignore_with("omnigent/_build_info.py");
assert_eq!(
classify(Path::new("/repo/omnigent/_build_info.py"), &ig),
Err("gitignored")
);
}
#[test]
fn file_inside_gitignored_dir_is_skipped() {
// A directory rule must ignore everything beneath it, like git does.
let ig = ignore_with("build/");
assert_eq!(
classify(Path::new("/repo/omnigent/build/foo.py"), &ig),
Err("gitignored")
);
}
#[test]
fn non_python_file_is_skipped() {
let ig = ignore_with("omnigent/_build_info.py");
assert_eq!(
classify(Path::new("/repo/omnigent/notes.txt"), &ig),
Err("non-.py")
);
}
#[test]
fn pycache_file_is_skipped() {
let ig = ignore_with("omnigent/_build_info.py");
assert_eq!(
classify(Path::new("/repo/omnigent/__pycache__/cli.py"), &ig),
Err("__pycache__")
);
return false;
}
!path.components().any(|c| c.as_os_str() == "__pycache__")
}
-591
View File
@@ -1,591 +0,0 @@
# Omnigent Uninstaller Design
Status: Implemented in PR #2550
Owner: Pat Sukprasert (@PattaraS)
Related discussion: brainstormed and debated via Debby (claude + gpt partners)
Implementation note: PR #2550 ships the OSS CLI/script implementation as one
combined PR rather than the staged PR breakdown below. Checkboxes marked here
reflect the current implementation and focused test coverage in that PR.
This document specifies how Omnigent should be uninstalled. It is written to be
handed to an implementer without further design decisions. Track delivery with
the checklists in each section.
## 1. Overview and scope
Ship four coupled pieces around one shared removal codepath:
1. `scripts/uninstall_oss.sh` - pure POSIX `sh`, the actual removal logic. Works
even when the wheel is wedged or PATH is broken; usable via curl-pipe.
2. `omnigent uninstall` - the discoverable CLI entry. It performs graceful
process shutdown and state/JSON handling in Python, then execs
`uninstall_oss.sh` for the final self-removal steps. One implementation, two
entry points.
3. Install-side ledger writer - records what the installer did to
`~/.omnigent/install_ledger.json`.
4. Back-fill routine - reconstructs a ledger as observed evidence (never
invented memory) for the pre-ledger install base.
Out of scope: any cross-domain "reaper" spanning the wheel, the signed `.app`,
and mobile sandboxes. App-store surfaces (iOS/Android/Electron) use OS-native
uninstall and only point the user back at `omnigent uninstall --purge` for
`~/.omnigent`. Shared runtimes (uv/Node/tmux/bwrap) are report-only in this
version - never removed, even with `--yes`.
Design principles that recur below:
- Remove only what we own; report everything else.
- Preserve user data by default; destruction is a separate, explicit intent.
- Risk is a property of the artifact, not of how we learned about it.
- Stop before you delete.
- Idempotent by state-check, not error-swallowing.
## 2. install_ledger.json schema
- Path: `~/.omnigent/install_ledger.json`
- Mode: `0600` (local paths; treat as sensitive)
- Write: atomic - write `install_ledger.json.tmp` in the same dir, `fsync`,
`rename()` over target.
- `schema_version`: `1` for first ship. Bump on any breaking change.
### Top level
| Field | Type | Allowed / notes |
|---|---|---|
| `schema_version` | int | `1`. |
| `ledger_source` | enum | `installer` \| `backfill`. A backfill ledger never overwrites an installer one. |
| `generator` | object | `{name, version, strategy, os, wrote_at}`; `strategy` = `install` \| `fast-backfill` \| `deep-backfill`; `os` = `macos` \| `linux`. |
| `installation_id` | string \| null | Copied from `~/.omnigent/installation_id`; the anchor proving an install exists. |
| `created_at` / `updated_at` / `last_validated_at` | string | RFC3339 UTC. |
| `entries` | object | The reversible-action records (below). |
### Per-entry provenance (every entry carries both)
- `source`: `recorded` (installer saw itself act) \| `observed` (backfill saw
the artifact directly) \| `inferred` (backfill deduced it).
- `confidence`: `certain` \| `high` \| `medium` \| `low` \| `none`.
### entries sub-objects
`profiles` (array) - shell profiles that received the delimited PATH block:
`path`, `marker_begin` (`# >>> Omnigent installer >>>`), `marker_end`
(`# <<< Omnigent installer <<<`), `line_range` [int,int] (1-indexed inclusive,
advisory - removal re-locates by marker), `block_sha256` (of block text incl.
markers, for tamper detection), `content_matches_current` (bool), `source`,
`confidence`.
`injected_external_config` (array) - entries Omnigent wrote into third-party
files: `path`, `marker` (logical key, e.g. `mcp_servers.omnigent`), `format`
(`json` \| `toml` \| `delimited_block`), `allowlist` (array of exact key paths /
block markers we may remove - removal touches ONLY these), `block_sha256`
(\| null), `source`, `confidence`.
`deps` (object keyed by `uv`/`node`/`npm`/`tmux`/`bwrap`): `present` (bool),
`path` (\| null), `version` (\| null), `installed_by` (`omnigent` - only ever set
by a real installer that did the install; \| `preexisting` \| `unknown` -
backfill may only write `unknown`), `confidence` (`none` whenever
`installed_by=="unknown"`), optional `notes` (weak human hint, never actioned).
`wheel` (object): `installed` (bool), `uv_tool_dir` (\| null), `bin_dir`
(\| null, e.g. `~/.local/bin`), `console_scripts` (array, e.g.
`["omnigent","omni"]`), `source`, `confidence`.
`launch_agents` (array): `kind` (`launchd` \| `systemd_user`), `path`, `label`,
`source`, `confidence`.
`state_paths` (object, informational, only removed under `--purge`):
`omnigent_home` (`~/.omnigent`), `workspace` (`~/omnigent`), `desktop_data`
(array of observed Electron dirs).
### Annotated example
```json
{
"schema_version": 1,
"ledger_source": "installer",
"installation_id": "b1f3c9a2-7e40-4c11-9d2a-3f6e8c0a1b22",
"created_at": "2026-07-14T18:03:22Z",
"updated_at": "2026-07-14T18:03:22Z",
"last_validated_at": "2026-07-14T18:03:22Z",
"generator": { "name": "omnigent", "version": "1.42.0", "strategy": "install", "os": "macos", "wrote_at": "2026-07-14T18:03:22Z" },
"entries": {
"profiles": [
{ "path": "~/.zshrc", "marker_begin": "# >>> Omnigent installer >>>", "marker_end": "# <<< Omnigent installer <<<",
"line_range": [212, 215], "block_sha256": "9f2c...e1", "content_matches_current": true,
"source": "recorded", "confidence": "certain" }
],
"injected_external_config": [
{ "path": "~/.config/harness/hermes.json", "marker": "mcp_servers.omnigent", "format": "json",
"allowlist": ["mcp_servers.omnigent"], "block_sha256": null, "source": "recorded", "confidence": "certain" }
],
"deps": {
"uv": { "present": true, "path": "~/.local/bin/uv", "version": "0.5.11", "installed_by": "omnigent", "confidence": "high" },
"node": { "present": true, "path": "/usr/bin/node", "version": "22.3.0", "installed_by": "preexisting", "confidence": "high" }
},
"wheel": { "installed": true, "uv_tool_dir": "~/.local/share/uv/tools/omnigent", "bin_dir": "~/.local/bin",
"console_scripts": ["omnigent","omni"], "source": "recorded", "confidence": "certain" },
"launch_agents": [
{ "kind": "launchd", "path": "~/Library/LaunchAgents/dev.omnigent.daemon.plist", "label": "dev.omnigent.daemon",
"source": "recorded", "confidence": "certain" }
],
"state_paths": { "omnigent_home": "~/.omnigent", "workspace": "~/omnigent", "desktop_data": [] }
}
}
```
Checklist:
- [x] Schema documented and versioned (`schema_version = 1`)
- [x] Atomic writer (tmp + fsync + rename) with `0600` mode
- [x] Serializer / dataclass with round-trip unit tests
- [x] `omnigent _internal write-ledger --from-env` hidden subcommand
## 3. Install-side ledger writer
Hook point: in `scripts/install_oss.sh`, after all side effects succeed and
before `print_next_steps`. Since the installer is the source of truth, prefer
having it call the hidden serializer subcommand
`omnigent _internal write-ledger --from-env` (reuses the schema serializer, gets
atomic-write + `0600` for free) rather than hand-building JSON in `sh`. Provide a
`write_install_ledger` shell wrapper.
Records (all `source: recorded`): each profile actually edited (path, markers,
current `line_range`, `block_sha256`); each external-config injection (path,
marker, format, allowlist); the wheel install (`uv tool dir`, bin dir, console
scripts); deps the installer itself installed this run get
`installed_by: omnigent` + version, deps found already present get
`preexisting`; any LaunchAgent/systemd unit registered; `installation_id`;
`state_paths`. Do not shell out to package managers for versions - cheap
`--version` only.
Upgrade / repair sync:
1. If existing ledger is `backfill`, discard and write a fresh `installer`
ledger (a real record supersedes inference).
2. If `installer`, merge: refresh `block_sha256`/`line_range` for re-touched
profiles, refresh wheel/dep versions, add newly-injected external config,
bump `generator.version` + `updated_at`.
3. Never downgrade `installed_by` (`uv: omnigent` stays even if uv is now found
pre-present).
4. Atomic write.
Checklist:
- [x] `write_install_ledger` hooked into `scripts/install_oss.sh` (post
side-effects, pre next-steps)
- [x] Records profiles, external config, wheel, deps, launch agents, state paths
- [x] Upgrade/repair merge logic (backfill superseded by installer; never
downgrade `installed_by`)
- [x] Tests: fresh install, upgrade, backfill-superseded-by-installer
## 4. Back-fill routine
Reconstruction = observe current state, record with per-field confidence, never
invent provenance.
Anchor guard (refuse to fabricate): before writing anything, require at least
one genuine install signal: `~/.omnigent/installation_id` exists, OR the wheel
is installed (`uv tool list` shows `omnigent`), OR a known profile contains the
exact marker pair. If none, write nothing and report "no Omnigent install
detected."
Fast vs deep:
- Fast (startup, target <100ms, no package-manager subprocesses): stat the
ledger; if valid, return. Else cheap checks only - stat `installation_id`,
read + in-process scan of candidate profiles for markers (no shelling out to
`grep`), stat known `~/.omnigent` subdirs, existence checks for Electron
dirs. Mark wheel/deps `confidence: low` or omit; `generator.strategy =
fast-backfill`. Never spawn `uv`/`command -v` on the hot path.
- Deep (uninstall / doctor, no budget): fast steps plus `uv tool list`/
`uv tool dir`, `command -v omnigent omni uv node tmux bwrap`, version
resolution, allowlisted external-config marker scans, LaunchAgent/systemd
enumeration. `generator.strategy = deep-backfill`.
Per-field confidence assignment:
| Signal | source | confidence |
|---|---|---|
| PATH block present (marker match) | observed | certain |
| PATH block present, content != current | observed | certain (flag `content_matches_current:false`) |
| Wheel / bin dir / console scripts | observed | high |
| `~/.omnigent`, `installation_id` | observed | high |
| LaunchAgent by known label | observed | high |
| Injected external config (marker block) | observed | certain |
| Injected external config (header fingerprint, no marker) | inferred | medium |
| Any dep `installed_by` | inferred | unknown / none |
Dependency `installed_by` is unrecoverable by design: backfill may write
`present`/`path`/`version` but MUST write `installed_by: unknown`,
`confidence: none`. A `notes` hint is allowed for `--dry-run` readers but never
changes behavior.
Never-overwrite-real + double-ledger:
- If existing ledger is `installer`, backfill does nothing, ever.
- Backfill writes to `~/.omnigent/install_ledger.backfill.json`, not directly
over `install_ledger.json`.
- Uninstaller ledger resolution: use `install_ledger.json` if `installer`; else
use `install_ledger.backfill.json` if present; else run deep backfill on the
fly.
- Re-run replaces the backfill file only if content differs; else bump
`last_validated_at`.
Read-only-except-the-ledger: backfill never edits profiles, removes deps, or
stops processes. It only reads and writes the (backfill) ledger.
Triggers: eager fast-backfill on first CLI run when missing; lazy deep-backfill
at uninstall when missing; explicit
`omnigent doctor --migrate-ledger [--deep]` which prints a JSON diff and writes
only with `--apply`.
Checklist:
- [x] Fast reconstruction (<100ms, no package-manager subprocesses, in-process
marker scan) on startup when missing
- [x] Deep reconstruction at uninstall / doctor
- [x] Anchor guard (refuse to fabricate without an install signal)
- [x] Per-field confidence assignment per table
- [x] Never-overwrite-real + `install_ledger.backfill.json` double-ledger handling
- [x] `omnigent doctor --migrate-ledger [--deep] [--apply]`
- [x] Read-only-except-the-ledger guarantee (tested)
## 5. omnigent uninstall CLI
`omnigent uninstall [targets...] [flags...]` (execs `scripts/uninstall_oss.sh`
with the same args). Fallback: `scripts/uninstall_oss.sh [targets...]
[flags...]`.
Targets (default `cli` if none given):
- `cli` - remove the uv tool entry + PATH/profile block(s).
- `state` - remove user data under `~/.omnigent` and `~/omnigent` (backup by
default).
- `desktop-data` - remove Electron caches/support/logs (NOT the app bundle).
- `all` - alias for `cli state desktop-data`.
Flags:
- `--purge` - implies `state`; deletes state/caches; backs up first unless
`--no-backup`.
- `--dry-run` - print exact planned actions (paths, sizes, line ranges); make no
changes.
- With no destructive flag (`--yes`, `--purge`, `--force`,
`--modify-external-config`, `--no-backup`, `--assume-inferred`, or
`--purge-workspace`), uninstall defaults to dry-run preview mode.
- `--yes` - non-interactive; suppresses prompts for auto-removable artifacts
only. Does NOT imply `--purge`.
- `--json` - machine-readable output.
- `--force` - allow SIGKILL after the SIGTERM grace window; proceed if daemons
resist; override tamper-refusal.
- `--modify-external-config` - primary gate to touch third-party config files.
- `--no-backup` - with `state`/`--purge`, skip archive creation.
- `--assume-inferred` - secondary gate to act on `inferred` entries.
- `--purge-workspace` - the only way to clear `~/omnigent` (your working files)
non-interactively. Without it, `--purge --yes` still removes `~/.omnigent`
(credentials/history) but leaves `~/omnigent` untouched and prints a notice.
This keeps a stray `--yes` in automation from wiping user work.
Gate decision table. Two orthogonal gates. Intrinsic-risk (primary): own
reversible artifacts auto-remove under `--yes`; third-party edits and data
destruction need their explicit flag on both real and backfilled ledgers.
Confidence (secondary, tighten-only): an `inferred`/low-confidence entry
escalates one notch and won't auto-act under bare `--yes` - it can only add
friction, never grant it.
| Artifact | No destructive flags | `--yes` | Required gate |
|---|---|---|---|
| Wheel (`uv tool uninstall omnigent`) | dry-run preview | auto-remove | none |
| Delimited PATH block (marker match) | dry-run preview | auto-remove | none; refuse if `block_sha256` mismatch (tampered) unless `--force` |
| Injected external config, marker/observed | reported, skipped | reported, skipped | `--modify-external-config` |
| Injected external config, inferred (no marker) | reported, skipped | reported, skipped | `--modify-external-config` AND `--assume-inferred` |
| `~/.omnigent` state root | reported, skipped | removed only with `--purge` | `--purge` |
| `~/omnigent` workspace | reported, skipped | kept unless `--purge-workspace` | `--purge` AND (`--purge-workspace` or interactive confirm) |
| Desktop data | via `desktop-data`/`all` | same | none beyond target |
| Shared deps (uv/node/tmux/bwrap) | report-only | report-only | none - never removed this version |
Checklist:
- [x] Python `omnigent uninstall` subcommand that execs the shell script
- [x] Targets: `cli`, `state`, `desktop-data`, `all`
- [x] Flags: `--purge`, `--purge-workspace`, `--dry-run`, `--yes`, `--json`,
`--force`, `--modify-external-config`, `--no-backup`, `--assume-inferred`
- [x] Two-gate decision table implemented (intrinsic-risk + confidence
tighten-only)
- [x] External-config stripping (marker/allowlist scoped only)
## 6. Order of operations
`omnigent uninstall` performs graceful shutdown + state/JSON in Python, then
execs the shell script for removal. Sequence:
1. Resolve ledger (section 4 resolution order).
2. Stop processes first. Read pidfiles under `~/.omnigent/run/` (+ `daemons/`,
`runners/`, `local_server/`): SIGTERM -> wait 5s -> under `--force` SIGKILL.
Kill only `omnigent:*` tmux sessions. Unload ledger-recorded LaunchAgents/
systemd units. If a process won't stop, abort destructive steps (report and
exit nonzero) unless `--force`.
3. `--dry-run`? Print exact paths + sizes + line ranges, then exit 0.
4. Profile cleanup. Remove ONLY the delimited marker block, all shells incl.
fish (`config.fish` + `conf.d/`). Back up the profile file first. Refuse a
block whose `block_sha256` doesn't match the ledger (tampered) unless
`--force`.
5. Strip injected external config (gated per table; marker-scoped /
allowlist-scoped only).
6. Optional state / desktop-data (only with `--purge` / target). For `--purge`:
archive to a backup tarball OUTSIDE the target under `~/.omnigent-backups/`
(or `$XDG_STATE_HOME`). Prefer `<ts>.tar.zst` when `zstd` is present; fall
back to `<ts>.tar.gz` (gzip is POSIX-baseline) otherwise. Never silently skip
the backup because a compressor is missing - a purge that can't write its
backup must fail closed (exit 1) unless `--no-backup` was given. Print the
restore command, then delete. Never back up into `~/.omnigent`. Clearing
`~/omnigent` non-interactively requires `--purge-workspace` (see section 5);
otherwise it prompts for a separate confirm. Note that purging
`installation_id` makes a reinstall look like a new device (telemetry).
7. `uv tool uninstall omnigent` - LAST (so earlier Python-driven steps still
have the wheel available).
Checklist:
- [x] Process-shutdown protocol (pidfiles, SIGTERM->5s->`--force` SIGKILL,
`omnigent:*` tmux, ledger LaunchAgents, abort-if-won't-stop)
- [x] Profile block removal across all shells incl. fish; profile backed up
first; tamper-refusal
- [x] `--purge` archives OUTSIDE the target (`.tar.zst`, gzip fallback; fail
closed if it can't write the backup), prints restore command, then
deletes; `~/omnigent` gated behind `--purge-workspace` (or confirm)
- [x] `uv tool uninstall omnigent` runs last
## 7. Idempotency and exit codes
State-check semantics: already-absent = success (exit 0); tried-and-failed =
report, continue with remaining steps, exit nonzero, summarize at end. Never
swallow a real failure as success; distinguish "already gone" from "tried and
failed."
Exit codes:
- `0` - all planned actions done or already-absent
- `1` - one or more actions failed (details in summary)
- `2` - aborted before destructive steps (e.g. process would not stop without
`--force`)
- `3` - refused (tampered block / anchor guard / ambiguous, no `--force`)
`--json` output shape:
```json
{
"schema_version": 1,
"dry_run": false,
"ledger_source": "installer",
"actions": [
{ "artifact": "profile_block", "path": "~/.zshrc", "planned": "remove",
"status": "done", "gate": null, "detail": "block removed, backup at ~/.zshrc.omnigent.bak" },
{ "artifact": "external_config", "path": "~/.config/harness/hermes.json", "marker": "mcp_servers.omnigent",
"planned": "remove", "status": "skipped", "gate": "--modify-external-config", "detail": "gate not provided" },
{ "artifact": "shared_dep", "name": "uv", "planned": "report", "status": "reported",
"gate": null, "detail": "installed_by=unknown; not removed" }
],
"backups": ["~/.omnigent-backups/2026-07-14T18-40-02Z.tar.zst"],
"summary": { "done": 1, "skipped": 1, "failed": 0, "reported": 1 },
"exit_code": 0
}
```
Checklist:
- [x] State-check idempotency (already-absent = 0; tried-and-failed = nonzero +
continue + summarize)
- [x] Exit codes 0/1/2/3 as specified
- [x] `--json` output shape stable and tested
## 8. Test matrix
| # | Scenario | Expect |
|---|---|---|
| 1 | fish profiles (`config.fish` + `conf.d/omnigent.fish`) | block removed from both; other lines intact |
| 2 | Tampered / corrupted marker block (sha mismatch) | refuse without `--force`; exit 3 |
| 3 | No ledger, valid install signal | deep-backfill runs, uninstall proceeds |
| 4 | No ledger, no install signal | anchor guard: nothing written; "no install detected" |
| 5 | Backfilled ledger present | inferred entries need `--assume-inferred`; deps report-only |
| 6 | Live daemon running | stopped (SIGTERM->5s->`--force`); won't-stop aborts destructive steps |
| 7 | `--dry-run` | prints exact paths/sizes/ranges; zero mutations; exit 0 |
| 8 | `--purge` with backup | archive written OUTSIDE `~/.omnigent`; restore command printed; then delete |
| 9 | `--purge --no-backup` | delete without archive; `~/omnigent` kept unless `--purge-workspace` |
| 10 | Shared dep present (`installed_by:unknown`) | report-only, never removed, even with `--yes` |
| 11 | Double ledger (real + backfill both present) | keep real; backfill copy left as `.backfill.json` for inspection |
| 12 | Re-run after full uninstall (idempotency) | all already-absent; exit 0 |
| 13 | Injected external config, marker vs inferred | marker gated by `--modify-external-config`; inferred also needs `--assume-inferred` |
| 14 | uv tool uninstall runs last | earlier Python steps had the wheel available |
| 15 | `--purge` on a box without `zstd` | backup written as `.tar.gz`; not skipped |
| 16 | `--purge --yes` without `--purge-workspace` | `~/.omnigent` removed; `~/omnigent` kept + notice |
Checklist:
- [x] Rows 1-2, 6-7, 12, 14 covered by `uninstall_oss.sh` tests
- [x] Rows 3-5, 8-11, 13 covered by focused CLI, ledger, and
`uninstall_oss.sh` tests
## 9. Delivery plan (PR breakdown)
- [x] PR 1 - Ledger schema + serializer. Schema, atomic-write + `0600` writer,
`omnigent _internal write-ledger` hidden subcommand, round-trip unit
tests. No behavior change.
- [x] PR 2 - Install-side writer. Hook `write_install_ledger` into
`scripts/install_oss.sh` + upgrade/repair merge logic.
- [x] PR 3 - Back-fill routine. Fast + deep reconstruction, anchor guard,
confidence assignment, never-overwrite-real + double-ledger,
`doctor --migrate-ledger`.
- [x] PR 4 - `uninstall_oss.sh` core. Process shutdown, profile block removal
(all shells), `uv tool uninstall`, idempotency + exit codes,
`--dry-run`/`--json`.
- [x] PR 5 - `omnigent uninstall` subcommand + gates. Python front, targets/
flags, two-gate decision table, `--purge` backup-outside-target,
external-config stripping.
- [x] PR 6 - Docs + discovery. Installer next-steps + `--help` mention
uninstall; README documents the standalone fallback and purge behavior.
App-store and brew/apt-specific surfaces remain out of scope for this OSS
CLI/script PR.
## Appendix A: ELI5
Omnigent is a houseguest.
- Installing = the guest moves in: hangs a coat by the door (the PATH line in
your shell profile), keeps a box of their stuff in a closet (`~/.omnigent` -
settings, logins, chat history) and a desk they work at (`~/omnigent`).
Sometimes they borrow shared tools from your garage that may already have been
there (uv, Node, tmux). Occasionally they leave a sticky note inside a
roommate's notebook (config injected into other tools).
- Uninstalling = the guest moves out politely:
1. Finish what you're doing first. Stop working before packing (kill running
daemons/runners) - don't yank the desk out while they're typing.
2. Take only your own stuff. Grab your coat (remove only the marked PATH line,
not random lines), take your box, erase your sticky note from the
roommate's notebook.
3. Don't take the shared tools. The garage drill might belong to the house.
Just leave a note: "I think I brought this - you decide." Never haul it off
on your own.
4. Your box stays unless you say "throw it out." Moving out is not shredding
your photos. Only if you explicitly say `--purge` does the box go - and
even then it is boxed up in the garage first (a backup tarball OUTSIDE the
room) so you can get it back.
- The ledger = a move-in checklist the guest writes on arrival: "hung a coat
here, borrowed this drill, left a note in that notebook." On move-out they
read the checklist and undo exactly those things - no guessing.
- Back-fill = for guests who moved in before checklists existed, walk the house
and reconstruct the checklist from what you can see, writing down how sure you
are ("coat on hook - definitely mine" vs "this drill - no idea who brought it,
don't touch"). A reconstructed checklist never lets you auto-toss the risky
stuff.
- Bare uninstall = "show me what would happen first." Nothing changes until you
add a destructive flag such as `--yes` or `--purge`.
- `--yes` = "apply the previewed safe moves." It grabs the coat, but it still
leaves the box unless you add `--purge`, and still will not erase a roommate's
notebook unless you add `--modify-external-config`. Risky actions are gated by
what you are touching, not by which checklist you have.
## Appendix B: Flowchart
```
+-----------------------------+
| omnigent uninstall [...] |
| targets: cli | state | |
| desktop-data | all |
| flags: --purge --dry-run |
| --yes --json --force |
| --modify-external-config |
+--------------+--------------+
|
+--------------v--------------+
| Load install_ledger.json |
+--------------+--------------+
|
+--------------------+--------------------+
| | |
ledger source=installer source=backfill NO ledger
(real, trust) (evidence + per- |
| field confidence) |
| | v
| | +----------------------+
| | | Genuine install |
| | | signal present? |
| | | (installation_id / |
| | | wheel / marker) |
| | +-------+----------+----+
| | no | yes |
| | v v
| | +------------+ +--------------+
| | | Refuse: | | Back-fill |
| | | nothing to | | from markers |
| | | uninstall | | (read-only) |
| | +------------+ +------+-------+
+---------+----------+------------------------------+
|
v
=====================================
|| 1. PLAN/STOP PROCESSES FIRST ||
|| dry-run reports planned stops; ||
|| apply unloads LaunchAgents, then ||
|| pidfiles/tmux -> SIGTERM/force ||
=================+===================
| won't stop? --> ABORT destructive steps (exit 2)
v
=====================================
|| 2. --dry-run? -- yes -> print ||
|| planned stops, paths, sizes, ||
|| EXIT 0 ||
=================+===================
| no
v
+--------------------------------------------------+
| For each planned action, apply the GATES: |
| |
| INTRINSIC-RISK gate (primary): |
| - own + reversible (wheel, marked PATH block) |
| -> auto under --yes |
| - third-party file edit (injected config) |
| -> needs --modify-external-config |
| - data destruction (~/.omnigent, ~/omnigent) |
| -> needs --purge (defaults to No) |
| - shared deps (uv/Node/tmux, installed_by |
| =unknown) -> REPORT ONLY, never remove |
| |
| CONFIDENCE gate (secondary, tighten-only): |
| - inferred / low-confidence entry |
| -> +1 notch friction, no auto under |
| bare --yes (never loosens) |
+----------------------+---------------------------+
|
v
ORDER OF OPERATIONS (each gated above):
+-------------------------------------------+
| (processes already stopped) |
| 3. Profile cleanup - remove ONLY delimited |
| marker block, all shells incl. fish; |
| back up profile; refuse if tampered |
| 4. Strip injected external config (marker- |
| scoped, ledger-recorded) |
| 5. --purge? archive to backup tarball |
| OUTSIDE target (~/.omnigent-backups/), |
| then delete state; keep ~/omnigent |
| unless --purge-workspace or confirm |
| 6. uv tool uninstall omnigent (LAST) |
+--------------------+----------------------+
|
v
+--------------------------------------+
| Idempotency by STATE-CHECK: |
| already-absent = success (exit 0) |
| tried & failed = report, non-zero, |
| continue, summarize|
| --json summary of what was done/kept |
+--------------------------------------+
Other package surfaces:
OS/package-manager uninstall owns package files. The Omnigent
uninstaller handles local profile/state cleanup and uses
uv tool uninstall for uv-installed wheels; it does not remove
shared dependencies or act as a cross-domain reaper.
```
Binary file not shown.

Before

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 759 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 129 KiB

+203 -144
View File
@@ -6,10 +6,9 @@ available", "is steering possible", "does policy DENY actually block a call" —
instead of a human hand-maintaining a spreadsheet and hoping it still reflects
reality.
> **Status:** shipped and in use. The bench on `main` has three transport
> drivers, six P0 probes, five report-only P1 probes, automatic live/offline
> selection, and a capability-derived matrix that has already caught and
> corrected real declaration drift. See
> **Status:** shipped and in use. The MVP plus most of phase-2 is on `main` —
> three transport drivers, the six P0 probes, and a capability-derived matrix
> that has already caught and corrected real declaration drift. See
> [Current state](#current-state-shipped) for what is live vs. still open. The
> sections before it describe the design and the decisions behind it.
@@ -126,42 +125,48 @@ list" to "discover"; probes, profiles, and reports are untouched.
## Architecture
The implementation has three layers plus reporting:
Three layers plus a report step.
```
tests/harness_bench/
profile.py # BenchProfile and profile-name resolution
manifest.py # official profiles derived from capabilities + e2e metadata
verdict.py # verdict vocabulary, priority, and drift reconciliation
transport.py # semantic Driver protocol and transport resolution
driver.py # sdk-inproc driver + shared TurnResult/usage helpers
full_server.py # shared server/runner lifecycle and registration
full_server_driver.py # full-server driver and session polling
native_tui_driver.py # native vendor CLI + host-daemon/tmux driver
session_items.py # shared session-item envelope parsing
runtime_env.py # config/credential resolution matching `omni run`
probes/ # one module per capability dimension
events.py # structured progress events and plain sink
richreport.py # optional live Rich matrix
bench.py # orchestration, concurrency, and shared-server wiring
report.py # terminal, Markdown, and JSON rendering
profile.py # BenchProfile: per-harness self-declared facts
manifest.py # registry of official BenchProfiles (the spreadsheet as data)
verdict.py # Verdict enum, ProbeResult, priority (P0/P1)
transports/ # transport drivers keyed by class
_base.py # TransportDriver: launch/session/turn against a harness
sdk_inproc.py # in-proc HTTP (reuses existing e2e server helpers)
tmux_tui.py # (phase 2)
app_server.py # (phase 2)
http_sse.py # (phase 2)
probes/ # one module per dimension
_base.py # CapabilityProbe: name, priority, applies_to, declared(), run()
basic_turn.py
streaming.py
tool_calling.py # incl. "connects to Omnigent MCP"
interrupt.py
policy_deny.py
model_override.py
... # (phase 2: steering, live_queue, resume_fork, elicitation,
# reasoning, images, cost, compaction)
bench.py # driver: iterate probes x harnesses -> matrix
report.py # render Markdown + JSON, with a DRIFT column
test_bench.py # pytest wrapper (parametrized) for CI
```
Reusable configuration and runtime primitives live in production modules such
as `omnigent.config`, `find_free_port`, and the harness registry rather than
being reimplemented under tests.
- **Layer 0 — Profile / manifest.** Static facts and declared verdicts are
derived from `harness_capabilities()` plus the existing e2e harness metadata.
- **Layer 1Offline conformance.** No network or credentials. It validates
registration, profile shape, capability derivation, transport resolution,
rendering, and orchestration behavior in normal CI.
- **Layer 2 — Live probes.** Drivers execute behavioral probes through the
wrap boundary or the real server/runner session API. Missing credentials,
vendor binaries, or vendor login produce capability-neutral skips.
- **Report.** The CLI renders the declared matrix offline or reconciles live
observations into terminal, Markdown, and JSON reports. `DRIFT` produces a
non-zero exit status.
- **Layer 0 — Profile / manifest.** The spreadsheet, as data. Source of truth
for the static columns and the *expected* verdicts for behavioral ones.
- **Layer 1 — Offline conformance** (no network, always in CI). Harness
registers, `create_app()` builds, required routes exist, `Executor` flags are
internally consistent, a `BenchProfile` exists. Fast, catches structural
regressions.
- **Layer 2Live probes** (gated on CLI + creds; reuses
`skip_if_harness_cli_missing`). Runs the behavioral table against a live
server, exactly like the existing e2e tests
(`/v1/sessions` + `send_user_message_to_session` +
`poll_session_until_terminal` + `final_assistant_text`).
- **Report.** `python -m tests.harness_bench --harness codex` prints one
harness's matrix; no filter regenerates the whole sheet with a `DRIFT` column
diffing declared vs observed.
### Build on `HarnessProbe`, don't reinvent it
@@ -201,38 +206,24 @@ Validated for presence and shape only: `Owner`, `Transport`, `Implementation`,
| Dimension | How the probe proves it |
|---|---|
| Basic turn (P0 prerequisite) | complete a marker-echo turn and require assistant text |
| Fork replay (P1) | clone the session after Basic turn, require copied marker history, and require the clone to recall it |
| Streaming (P0) | count output-text deltas; repeated single-delta output is `PARTIAL` |
| Reasoning (P1) | request high effort and require a forwarded reasoning delta or persisted reasoning item; no observation is inconclusive because the model may emit none |
| Tool calling (P0) | provoke the transport's tool mechanism and require a surfaced call |
| Omnigent MCP (P1, native only) | call read-only `sys_session_list` through the generated `omnigent` MCP relay and require a matching function-call item |
| Policy DENY (P0) | apply a tool-call deny and require a blocked-call signal |
| Policy ALLOW (P1) | attach an explicit allow and require a non-blocked tool output; native hooks expose no positive ALLOW event |
| Policy ASK (P1) | apply ask and require an elicitation/approval request |
| Model override (P0) | validate the requested harness/model pair and complete a turn |
| Cost tracking (P1) | read priced cost or token usage from the turn/session |
| Interrupt (P0) | interrupt a long turn and require cancellation or early termination |
Planned dimensions are steering, live queue, resume, images, and compaction.
Their declarations already have a place in `HarnessCapabilities`: resume uses
the `Resume` mechanism enum, while steering, live queue, images, and compaction
are optional booleans. An unset optional value makes no claim and therefore
stays `UNKNOWN` until the corresponding probe work establishes the harness's
expected behavior.
| Basic turn (prereq) | ask model to reply with `<marker>`, assert marker in final text |
| Connects to Omnigent MCP | expose an Omnigent tool, ask model to call it, assert `ToolCallRequest` dispatched through the relay |
| Streaming | count `TextChunk` events: >1 delta = `deltas`, single blob = `complete-only` |
| Model override | launch with a chosen model, assert routing (gateway request / `TurnComplete` usage model); cross-family reject verified via `model_family_mismatch` |
| Policy: DENY | set DENY on a tool, ask model to call it, assert the call is blocked + surfaced |
| Policy: ASK -> Elicitation | set ASK, assert an elicitation event is emitted upstream (web-surfaceable) |
| Interrupt | start a long turn, call `interrupt_session`, assert it stops promptly |
| Live queue (concurrent) | `enqueue_session_message` mid-turn, assert accepted (not rejected) |
| Tool-boundary steer | inject steering text at a tool boundary, assert the next turn reflects it |
| Resume/fork from transcript | run a convo, resume in a fresh session, assert prior context present; fork = branch diverges |
| Compaction | assert `CompactionComplete` surfaced when triggered |
| Reasoning | reasoning-heavy prompt, assert `ReasoningChunk` emitted |
| Images | send an image, assert the model describes it |
| Cost tracking | assert `TurnComplete` carries usage / cost |
Every behavioral probe also reads the corresponding declared flag and returns
`DRIFT` when observed disagrees with declared.
The CLI can slice this catalog with repeatable or comma-separated
`--dimension` values. A slice always includes `basic_turn` because it proves
the harness is exercisable before interpreting another probe's result. Reports
and the live Rich grid contain only the selected columns. Each repeated
`--harness NAME[=MODEL]` binds an optional model override directly to that
harness, avoiding both test model-pool environment variables and positional
cross-family assignment. Omitting `=MODEL` keeps that profile's default.
### Illustrative probe shape
```python
@@ -256,56 +247,80 @@ class StreamingProbe(CapabilityProbe):
## Transport drivers: the real ceiling on "all dimensions"
Behavioral probes call semantic driver methods such as `run_basic_turn`,
`run_tool_turn`, `run_policy_turn`, and `run_interrupt_turn`. Drivers own the
transport-specific mechanism; probes interpret a common `TurnResult`.
Behavioral probes run through a **transport driver** resolved from the
harness *family* plus flags: SDK harnesses default to `full-server` (`--fast`
picks `sdk-inproc`), natives use `native-tui`, and `--transport NAME` overrides
the family for any harness. A probe calls
*semantic* methods on the driver (`run_basic_turn`, `run_streaming_turn`,
`run_tool_turn(deny=...)`, `run_interrupt_turn`); the driver owns the
*mechanism* and the probe owns the *interpretation*, so one probe runs across
transports that reach the same capability by different means.
Three drivers exist:
Three drivers exist today (see "Current state" above): `sdk-inproc`,
`full-server`, `native-tui`. Two consequences fall out of this design:
- `full-server` is the SDK-family default. It drives a real server and runner,
uses a server-dispatched builtin for tool probes, and observes fixed
ALLOW/ASK/DENY policies.
- `native-tui` drives a resident vendor CLI in a runner-owned tmux pane through
the server session API. It observes vendor tool calls and tool-call DENY via
the native policy hook. ALLOW/ASK are not yet implemented.
- `sdk-inproc` drives the harness wrap directly. It is selected by `--fast` and
provides cheaper wrap-level coverage, but no server-side policy surface.
- A dimension is only observable where a driver exercises it. Tool calling and
Policy DENY need `full-server`; on `sdk-inproc`/`native-tui` they report `·`.
A `·` therefore often means "this transport can't exercise it here," not "the
harness lacks it" (see "Which transport exercises which dimension").
- A harness that invents a *novel* transport (neither wrap-subprocess, full
server, nor native tmux) would degrade its transport-dependent probes to
`SKIPPED`/`UNKNOWN` until a driver for that class exists.
A `SKIPPED` verdict therefore means the behavior was not measurable in that
transport or environment, not that the harness lacks the capability. A novel
transport class still requires a driver, but harnesses reusing one of these
families flow through the existing probes without per-harness probe code.
So "run the bench, see all verdicts, zero code" is true *for any harness
reusing a known transport class*, and honest about the cases where a dimension
or a transport is not yet wired.
## Current state (shipped)
The bench on `main` includes:
The MVP and most of phase-2 are landed. What exists on `main` today:
- **Six P0 probes:** Basic turn, Streaming, Tool calling, Policy DENY, Model
override, and Interrupt.
- **Six P1 probes:** Fork replay, Reasoning, Omnigent MCP, Policy ALLOW, Policy ASK, and Cost tracking. P1 verdicts
are report-only and do not gate the same way as P0 declarations.
- **Three transport drivers:** `full-server`, `native-tui`, and `sdk-inproc`,
selected by harness family with `--transport` and `--fast` overrides.
- **Automatic live selection:** without an explicit mode, the CLI runs live
when credentials are resolvable and otherwise renders the declared matrix.
`--live` and `--no-live` force either mode. Credentials are derived like
`omni run`; `--profile` is only an override.
- **Concurrent execution and shared infrastructure:** `--jobs` runs harnesses
concurrently while preserving report order, and full-server harnesses share
one server/runner pair within a run.
- **Structured progress and reports:** plain or Rich live progress plus terminal,
Markdown, JSON, and optional report-file output.
- **Capability-derived registration:** official SDK and native profiles derive
from `harness_capabilities()` and existing e2e metadata. Session-item parsing,
config loading, free-port selection, and polling helpers are shared rather
than duplicated.
- **Layer 0/1/2** — profile/manifest, offline conformance (runs in CI via the
`misc` pytest group), and the six P0 live probes (basic turn, streaming,
tool calling, policy DENY, model override, interrupt) with the `DRIFT`
column.
- **Three transport drivers**, selected by harness *family* with flag overrides:
- `sdk-inproc` — drives a harness wrap subprocess directly (the four P0 SDK
harnesses: claude-sdk, codex, pi, openai-agents).
- `full-server` — a real server + runner; the only transport that exercises
**Tool calling** and **Policy DENY** as server-dispatched, policy-gated
calls (SDK harnesses only — it registers via an agent bundle).
- `native-tui` — a resident vendor CLI in a runner-owned tmux pane, driven
over the session HTTP surface via a host daemon.
SDK harnesses default to **`full-server`** — the fullest coverage, and a
strict superset of what `sdk-inproc` observes (everything sdk-inproc does,
*plus* Tool calling + Policy DENY). `--fast` opts the SDK family down to
`sdk-inproc` when you want to skip the server boot (those two dimensions then
report `·`). Native harnesses have a single transport `--fast` does not touch.
An explicit `--transport NAME` overrides the family default for any harness
and is mutually exclusive with `--fast`.
- **Capability-derived matrix** — descriptive columns and declared verdicts
come from `harness_capabilities()` (the seam; see
`designs/harness-capabilities-bench-seam.md`), so a harness added to the
registry — in-repo *or* a community plugin — flows into the bench with no
bench edit.
- **Native harnesses auto-derived** — every `NATIVE_TUI` harness is registered
and drivable by name; `native_vendor()` derives what the driver needs from
capabilities, with no per-vendor table.
### Not yet wired
- Registry-driven server seeding for community native UI agents.
- Steering, live queue, resume, images, and compaction probes.
- Automatic provisioning of vendor login/provider configuration for native
harnesses; unavailable environments skip cleanly.
- **Bench observation of Tool calling / Policy DENY on `native-tui`** — a
*driver gap, not a native-harness limitation*. Native harnesses do call tools
and enforce permissions; the bench cannot yet observe it on this transport.
A native tool call is the vendor's own tool (Bash/Read/...), not a
server-dispatched `function_call_output` the bench can force, and a native
deny is a vendor permission decision, not a server-side policy evaluation the
probe can assert against. So both cells show `·` (not measured), never `✗`.
Wiring the observation needs new driver work. (SDK harnesses get these via
`full-server`.)
- **P1 dimensions** — steering, live-queue, resume/fork, elicitation ASK,
reasoning, images, cost, compaction. Probes not written yet (report
`UNKNOWN`).
- **Server-side native-agent seeding is a hardcoded list** — see the
plugin-seamlessness note below; this is the main gap between "the bench is
plugin-ready" and "a plugged-in native harness just works end to end".
## CI integration
@@ -317,29 +332,37 @@ The bench on `main` includes:
## Running the bench and reading the result
```
# Declared matrix only, with no credentials.
python -m tests.harness_bench --no-live
# Offline: the declared matrix, no creds, every harness. Fast.
python -m tests.harness_bench
# Auto-live when configured or ambient credentials are available.
python -m tests.harness_bench --harness codex
# Live: probe one harness against a gateway profile.
python -m tests.harness_bench --harness codex-native --profile oss
# Force a named profile and probe several harnesses concurrently.
python -m tests.harness_bench --profile oss --jobs 4 --rich
# Live: probe every official harness (SDK + native) sequentially.
python -m tests.harness_bench --profile oss
# A community harness that ships its own BenchProfile.
python -m tests.harness_bench --harness mypkg.harness:PROFILE --live
python -m tests.harness_bench --harness mypkg.harness:PROFILE --profile oss
```
Without `--live` or `--no-live`, resolvable credentials select live mode and
missing credentials select the offline declared matrix. Native harnesses also
need their vendor CLI installed and logged in; the bench cannot provision those
accounts, so unavailable harnesses skip without aborting the run.
**You do not need to live-probe every harness on every host — and you cannot.**
Each native harness needs its own vendor CLI logged in (a login the bench
cannot provision), so no single host has them all. The two layers split the
work:
Offline conformance covers every registered harness in CI. Live runs are
spot-checks of observed behavior and can vary with model behavior and timing;
re-run an isolated timeout or skip before treating it as a regression. The
signals that matter most are `DRIFT` and repeatable unexpected
`UNSUPPORTED`/`PARTIAL` verdicts on a runnable harness.
- **Offline conformance** already covers every harness in CI — registration,
the declared matrix, capability derivation. No host access needed.
- **Live probes** only answer "does observed behavior match the declaration?"
You get value from live-probing a harness where the declaration is unverified
or might be wrong — not from chasing 100% coverage on one box.
Run the full set on whatever host you have (`--profile oss`); harnesses whose
vendor CLI is absent or logged out **skip cleanly** (they do not fail or abort
the run). Read two signals only: any `!!` DRIFT, and any harness you *can* run
that shows an unexpected `✗` / `·`. A single live run is a spot-check, not a
gate — live probes are non-deterministic (model behavior, timing), so re-run
before treating one `·`/timeout as a regression. Drift coverage is cumulative:
each host that has harness X logged in contributes a live check for X.
## Streaming is a binary declared capability
@@ -363,20 +386,46 @@ stream, the bench flags a real drift on the next run, rather than a false
## Which transport exercises which dimension
| Dimension | `sdk-inproc` (`--fast`) | `full-server` (SDK default) | `native-tui` |
|---|---|---|---|
| Basic turn, Streaming, Reasoning, Model override, Interrupt | Wrap-level observation; reasoning effort is set per request | End-to-end server/runner observation; reasoning effort is set on the session | End-to-end server/runner/vendor observation; reasoning effort is set on the session |
| Fork replay | Not observable | Clone + copied-history replay through server/runner | Clone + copied-history replay through server/runner/vendor |
| Tool calling | Request-level wrap tool | Server-dispatched builtin | Vendor tool mirrored into session items |
| Omnigent MCP | Not applicable | Not applicable | Generated `omnigent` MCP relay when supported by the vendor |
| Policy DENY | Not observable | Fixed policy blocks the builtin | Session CEL policy triggers the native policy hook |
| Policy ALLOW / ASK | Not observable | Fixed policy; ASK observes and resolves an elicitation | Temporary session CEL policy; ASK observes and resolves an elicitation |
| Cost tracking | Completed-response usage when forwarded | Session snapshot usage/cost | Session snapshot when the vendor forwards usage |
Not every dimension is observable on every transport, so a `·` (SKIPPED) in a
run always means "the bench did not measure this here," never "the harness
lacks it." Two dimensions in particular only get a real verdict on the
`full-server` transport:
`full-server` remains the SDK default because it covers the deployed server
path and all three policy actions. `--fast` trades that policy coverage for
lower startup cost. `native-tui` now has real Tool calling and all three policy
action probes through the native hook path.
| Dimension | sdk-inproc (`--fast`) | full-server (default) | native-tui |
|---|---|---|---|
| Basic turn, Streaming, Model override, Interrupt | ✓ | ✓ | ✓ |
| **Tool calling** | · (harness dispatches tools internally) | ✓ (server-dispatched builtin) | · (bench can't observe vendor tools yet) |
| **Policy DENY** | · (wrap-direct: no tool-call policy hook) | ✓ (spec-baked deny, enforced) | · (bench can't observe vendor deny yet) |
The `native-tui` `·` is a *bench observation gap, not a native-harness
limitation*: native harnesses do call tools and enforce permissions, but a
native tool call is the vendor's own (Bash/Read/...) and a native deny is a
vendor permission decision, neither of which is the server-dispatched,
policy-gated call the probe watches for. Giving those cells a real verdict
needs new driver work, not a change to the harnesses.
Because `full-server` sees everything `sdk-inproc` does *plus* these two, it is
the **default** for SDK harnesses — a plain live run proves Tool calling and
Policy DENY out of the box:
```
python -m tests.harness_bench --harness claude-sdk --profile oss
```
Live-verified: `claude-sdk` completes the full matrix on `full-server`
Tool calling `✓` and Policy DENY `✓` (the deny is delivered and the blocked
call does not stall the turn). Add `--fast` to trade that coverage for a quicker
run on `sdk-inproc`; those two columns then show `·`, since neither `sdk-inproc`
nor `native-tui` (for natives) routes a tool call through a server policy
evaluation.
`full-server` covers **SDK harnesses only** — it registers the harness via an
agent bundle, which is the SDK-wrap path; native harnesses need the host-daemon
provisioning the `native-tui` driver owns. So Tool calling / Policy DENY on
native harnesses are not observed by *any* transport yet — a bench follow-up,
not a native-harness gap — distinct from the `--fast` (sdk-inproc) `·`, which
is a transport limitation the default `full-server` run already answers for SDK
harnesses.
## Plugin seamlessness: where it is and isn't
@@ -429,15 +478,25 @@ agree with it.
## Open items
- **Declarative native tool-relay mechanism** — extend the harness capability
model to distinguish generated MCP, native registration, and no relay. Derive
the Omnigent MCP probe's applicability from that declaration instead of the
bench's temporary `_NATIVE_OMNIGENT_MCP_HARNESSES` list.
- **Registry-driven native-agent seeding** — replace the hardcoded server
seeding list with registry iteration so community native harnesses work end
to end after plugin installation.
- **Per-harness native provisioning** — some vendors require login or provider
configuration that the bench deliberately cannot create. Improve diagnostics
where possible while retaining clean skips.
- **Additional dimensions** — steering, live queue, resume, images, and
compaction.
- **Registry-driven native-agent seeding** (highest leverage) — replace the
hardcoded `_ensure_default_*_agent()` list in `server/app.py` with a loop over
`native_agents()`, so any native harness (in-repo or plugin) registers
automatically. This is the fix for the plugin-seamlessness seam above.
- **Bench observation of Tool calling / Policy DENY on `native-tui`** — a
driver gap, not a native-harness limitation: native harnesses call tools and
enforce permissions, but a native tool call is the vendor's own and a native
deny is a vendor permission decision, not the server-dispatched
`function_call_output` the probe watches for. The cells show `·` (not
measured), never `✗`. Needs new driver work. (SDK harnesses get these via
`full-server`.)
- **Per-harness native provisioning gaps** the bench has surfaced but not yet
resolved: goose-native returns a 500 on the terminal-ensure endpoint;
hermes-native's forwarder does not wire up (a lazy-chat / first-turn gate to
confirm); kimi-native and own-auth natives need a vendor provider setup the
bench cannot provision (kimi in particular has no gateway path — it routes
via `kimi provider add`, out of band).
- **P1 dimensions + their probes** — steering, live-queue, resume/fork,
elicitation ASK, reasoning, images, cost, compaction.
- Exact `BenchProfile` field set and whether it subsumes `HarnessProbe` or wraps
it; whether the manifest fully retires the spreadsheet or diffs against an
exported CSV during transition.
-35
View File
@@ -1,35 +0,0 @@
# AWS Analyst
An example Omnigent agent that answers questions over **governed AWS data** through
the official [AWS Labs MCP servers](https://github.com/awslabs/mcp) — no custom
connector code required. It shows how any AWS Labs MCP server plugs into Omnigent as
a `type: mcp` tool.
Wired connectors (both **read-only** by default):
| Connector | AWS Labs server | Tools surfaced |
|---|---|---|
| `redshift` | `awslabs.redshift-mcp-server` | `list_clusters`, `list_databases`, `list_schemas`, `list_tables`, `list_columns`, `execute_query` |
| `s3-tables` | `awslabs.s3-tables-mcp-server` | metadata discovery + read-only SQL |
## Prerequisites
- [`uv`/`uvx`](https://docs.astral.sh/uv/) on `PATH` — the AWS Labs servers are
published to PyPI as `awslabs.*` and launched via `uvx ...@latest`.
- AWS credentials the servers can resolve: an `AWS_PROFILE` + `AWS_REGION`, or an
IAM role on the host.
## Run
```bash
AWS_PROFILE=my-profile AWS_REGION=us-east-1 omnigent run examples/aws_analyst
```
## Notes
- The S3 Tables server defaults to read-only; this recipe intentionally does **not**
pass `--allow-write`.
- The `tools:` allow-list on the Redshift connector limits what the model can call —
a good default for a governed analytics agent.
- Pairs naturally with a Databricks Genie connector for a Databricks-on-AWS
"better together" analyst that reasons across both platforms.
-70
View File
@@ -1,70 +0,0 @@
# AWS Analyst — query governed AWS data through official awslabs MCP servers.
#
# This example agent wires two AWS Labs MCP servers as Omnigent connectors:
# - Amazon Redshift (awslabs.redshift-mcp-server)
# - Amazon S3 Tables (awslabs.s3-tables-mcp-server)
# Both run read-only by default. The agent uses them to answer analytical
# questions over data governed in AWS — a natural companion to Databricks Genie
# in a Databricks-on-AWS "better together" setup.
#
# Prerequisites:
# - `uvx` on PATH (the awslabs servers are published to PyPI as awslabs.*).
# - AWS credentials resolvable by the servers (AWS_PROFILE + AWS_REGION, or a role).
#
# Usage:
# AWS_PROFILE=my-profile AWS_REGION=us-east-1 omnigent run examples/aws_analyst
spec_version: 1
name: aws_analyst
description: >-
An analyst agent that answers questions over governed AWS data — Amazon Redshift
and Amazon S3 Tables — through the official awslabs MCP servers (read-only).
executor:
type: omnigent
config:
harness: claude-sdk
tools:
# Amazon Redshift — discovery + read-only SQL over your clusters.
redshift:
type: mcp
command: uvx
args: [awslabs.redshift-mcp-server@latest]
env:
AWS_PROFILE: ${AWS_PROFILE}
AWS_REGION: ${AWS_REGION}
FASTMCP_LOG_LEVEL: INFO
# Allow-list: only these tools are surfaced to the model. execute_query is
# read-only on the server side; the discovery tools let the agent map the
# environment before querying.
tools: [list_clusters, list_databases, list_schemas, list_tables, list_columns, execute_query]
# Amazon S3 Tables — read-only metadata discovery + SQL over table buckets.
# (Server defaults to read-only; --allow-write is intentionally NOT set.)
s3-tables:
type: mcp
command: uvx
args: [awslabs.s3-tables-mcp-server@latest]
env:
AWS_PROFILE: ${AWS_PROFILE}
AWS_REGION: ${AWS_REGION}
prompt: |
You are an AWS data analyst. You answer questions over governed AWS data using
two toolsets:
- `redshift` — Amazon Redshift. Start by discovering the environment
(list_clusters → list_databases → list_schemas → list_tables → list_columns)
before writing SQL, then use execute_query for read-only analytical queries.
- `s3-tables` — Amazon S3 Tables. Use it for metadata discovery and read-only
SQL over table buckets.
Rules:
- Prefer discovery before querying; never assume a table or column exists —
confirm it with the list_* tools first.
- These tools are read-only. Do not attempt inserts, updates, or deletes.
- Always state which source (Redshift or S3 Tables) and which table an answer
came from, so results are auditable.
- When a question spans multiple tables, explain your join logic before running
the query.
@@ -22,8 +22,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
-2
View File
@@ -22,8 +22,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
+4 -15
View File
@@ -3,23 +3,14 @@ name: cursor
description: Cursor coding sub-agent — implements, cross-vendor reviews, or explores a scoped task in its own worktree.
# Native Cursor TUI harness (`cursor-agent`): runs in its own terminal the
# human can open in the UI's Subagents panel and TAKE OVER. Headless workers
# can't answer ApprovalCards, so YOLO skips cursor-agent's in-terminal
# prompts (and the mirrored web cards). Omnigent ``blast_radius`` still
# DENYs the catastrophic set. Opt out with ``yolo: false``; or set
# ``permission_mode: auto`` for Smart Auto (``--auto-review``) instead.
# human can open in the UI's Subagents panel and TAKE OVER. cursor-agent owns
# its own tool-approval gating (omnigent does not intercept it), so dangerous
# actions surface in the cursor TUI / mirrored web cards rather than being
# auto-bypassed.
executor:
type: omnigent
# Faster default for Polly Cursor workers; override per-session with ``/model``.
# Use the base id from Cursor's model list / SDK (``grok-4.5``). The compound
# ``cursor-grok-4.5-high`` also works on cursor-agent, but the SDK catalog
# exposes ``grok-4.5``.
model: grok-4.5
config:
harness: cursor-native
# YOLO: headless workers can't answer approval prompts, so run
# cursor-agent with full bypass (``--yolo``).
yolo: true
prompt: |
You are Cursor, a coding sub-agent dispatched by the polly
@@ -32,8 +23,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
-2
View File
@@ -22,8 +22,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
@@ -23,8 +23,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
-2
View File
@@ -21,8 +21,6 @@ prompt: |
contract.
- Make the change, then drive it to green: run the relevant tests, lint, and
typecheck for the code you touched.
- When you report test results, include the exact command and file set. If
you mention counts, distinguish collected test cases from test functions.
- Co-sign every commit you author: end each commit message with a blank line
followed by this exact trailer as its final line —
`Co-authored-by: omnigent <noreply@omnigent.ai>`
+1 -11
View File
@@ -24,7 +24,7 @@ spawn: true
# (`omnigent setup --no-internal-beta`) — an Anthropic API key, a Claude
# subscription, an OpenAI-compatible gateway, or a Databricks workspace. With
# no model named the claude-sdk harness resolves the configured provider's
# default Claude model.
# default Claude model (the bundled catalog default is claude-opus-4-8).
executor:
type: omnigent
context_window: 1000000
@@ -178,16 +178,6 @@ prompt: |
write or edit source code or tests, run a deep code investigation for your own
answer, or merge a PR — those go to sub-agents.
Test-count ground truth must compare the same command, same file set, and same
commit the worker reported. For pytest, collected CASES are the count: use
`python -m pytest --collect-only -q <same files>` when reconciling a reported
total, and never use `grep -c 'def test_'` as a correctness oracle. A single
test function can expand into many collected cases via parametrized tests, and
a one-file function count cannot be compared to a multi-file gate. Do not record
`miscount`, `over-report`, or `fabrication` in `.polly/registry.json` or a
handoff unless you have re-collected the same gate at the same commit and the
numbers still disagree.
For long-running processes that can't block a single tool call (a local dev
server on localhost:PORT, file watchers, tailing logs) or ad-hoc shell where
`sys_os_shell`'s one-shot blocking model doesn't fit, launch the `shell`
@@ -15,11 +15,6 @@ anyone needs to read through.
2. Run the deterministic gates first — tests / lint / typecheck via
`sys_os_shell`. If red, re-dispatch the implementer to drive it green first;
don't involve the reviewer yet.
If a pytest result's count must be recorded or reconciled, collect ground
truth with `python -m pytest --collect-only -q <same files>` against the
exact file set/command/commit the implementer reported. Never use
`grep -c 'def test_'` as a pytest count: it counts functions, not collected
cases, and misses parametrized case expansion.
3. Dispatch a DIFFERENT-vendor sub-agent as reviewer: pick any AVAILABLE worker
whose vendor differs from the implementer's — `claude_code`, `codex`,
`opencode`, `cursor`, `hermes`, or `pi` (e.g. Claude built it → any of
+1 -1
View File
@@ -8,7 +8,7 @@
# Memory is keyed by the agent id, so all of Remy's runs share one memory bank.
#
# Setup:
# pip install 'omnigent[hindsight]'
# pip install 'omnigent[memory]'
# export HINDSIGHT_API_KEY=hsk_... # https://ui.hindsight.vectorize.io
#
# Usage:
-22
View File
@@ -1,22 +0,0 @@
SLACK_BOT_TOKEN=xoxb-your-bot-token
SLACK_APP_TOKEN=xapp-your-app-level-token
OMNIGENT_AGENT_NAME=your_agent_name
# Optional. Defaults to the local Omnigent server from docs/api-1.yaml.
OMNIGENT_BASE_URL=http://127.0.0.1:6767
# Optional Omnigent auth modes.
# OMNIGENT_AUTH_EMAIL=slack-bot@example.com
# OMNIGENT_AUTH_HEADER_NAME=X-Forwarded-Email
# OMNIGENT_SESSION_COOKIE=ap_session=...
# Optional runner fallback. If no online runner exists, launch one on a host.
# Defaults to the bot process current working directory.
# OMNIGENT_RUNNER_WORKSPACE=/absolute/path/to/workspace
# OMNIGENT_RUNNER_HOST_ID=host_optional_specific_host
# OMNIGENT_RUNNER_LAUNCH_TIMEOUT_SECONDS=60
# Optional runtime tuning.
# LOG_LEVEL=INFO
# OMNIGENT_SLACK_DATABASE_PATH=data/omnigent_slack.sqlite3
# SLACK_UPDATE_INTERVAL_SECONDS=1.0
-10
View File
@@ -1,10 +0,0 @@
.env
.venv/
.uv-cache/
__pycache__/
*.py[cod]
.pytest_cache/
.ruff_cache/
.mypy_cache/
data/*.sqlite3
data/*.sqlite3-*
-1
View File
@@ -1 +0,0 @@
3.12
-39
View File
@@ -1,39 +0,0 @@
# Omnigent Slack Bot
Slack Socket Mode bot that maps one Slack thread to one Omnigent session.
## Setup
1. Create a Slack app with Socket Mode enabled.
2. Add bot scopes for `app_mentions:read`, `chat:write`, and the history scopes needed for the channel types where the bot will run.
3. Install the app into the workspace.
4. Copy `.env.example` to `.env` and fill in Slack and Omnigent values.
5. Run the bot:
```bash
UV_CACHE_DIR=.uv-cache uv run omnigent-slack
```
Set `LOG_LEVEL=DEBUG` in `.env` when diagnosing why Slack events are not producing replies.
If Omnigent has no online runners, the bot launches one on an online host using
the current working directory as the workspace. Set `OMNIGENT_RUNNER_WORKSPACE`
when the host needs a different absolute path.
Mention the bot with a message to start a session:
```text
@your-bot help me inspect this failure
```
Replies in that Slack thread continue the same Omnigent session.
## Development
```bash
UV_CACHE_DIR=.uv-cache uv run pytest
UV_CACHE_DIR=.uv-cache uv run ruff check
UV_CACHE_DIR=.uv-cache uv run mypy src
```
The Omnigent API reference used for implementation is stored at `docs/api-1.yaml`.
-56
View File
@@ -1,56 +0,0 @@
[project]
name = "omnigent-slack"
version = "0.1.0"
description = "Slack Socket Mode bot that drives Omnigent sessions."
readme = "README.md"
requires-python = ">=3.11"
dependencies = [
"aiosqlite>=0.21.0",
"aiohttp>=3.12.0",
"httpx>=0.28.0",
"pydantic-settings>=2.10.0",
"python-dotenv>=1.1.0",
"slack-bolt>=1.23.0",
"markdown-to-mrkdwn>=0.3.3",
]
[project.scripts]
omnigent-slack = "omnigent_slack.__main__:main"
[dependency-groups]
dev = [
"mypy>=1.16.0",
"pytest>=8.4.0",
"pytest-asyncio>=1.0.0",
"respx>=0.22.0",
"ruff>=0.12.0",
]
[build-system]
requires = ["uv_build>=0.8.0,<0.9.0"]
build-backend = "uv_build"
[tool.ruff]
line-length = 100
target-version = "py311"
exclude = [".uv-cache", ".venv", "docs"]
[tool.ruff.lint]
select = ["E", "F", "I", "UP", "B", "ASYNC"]
[tool.mypy]
python_version = "3.11"
strict = true
warn_unreachable = true
[[tool.mypy.overrides]]
module = [
"slack_bolt.*",
"slack_sdk.*",
"markdown_to_mrkdwn.*",
]
ignore_missing_imports = true
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
@@ -1,5 +0,0 @@
"""Slack bot for Omnigent sessions."""
__all__ = ["__version__"]
__version__ = "0.1.0"
@@ -1,13 +0,0 @@
from __future__ import annotations
import asyncio
from omnigent_slack.app import run
def main() -> None:
asyncio.run(run())
if __name__ == "__main__":
main()
@@ -1,114 +0,0 @@
from __future__ import annotations
import logging
from typing import Any
from dotenv import load_dotenv
from slack_bolt.adapter.socket_mode.aiohttp import AsyncSocketModeHandler
from slack_bolt.async_app import AsyncApp
from omnigent_slack.config import load_settings
from omnigent_slack.omnigent import OmnigentAuth, OmnigentClient
from omnigent_slack.service import SlackOmnigentService
from omnigent_slack.store import SQLiteStore
async def run() -> None:
load_dotenv()
settings = load_settings()
logging.basicConfig(
level=getattr(logging, settings.log_level.upper(), logging.INFO),
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
logger.info(
"Starting Omnigent Slack bot base_url=%s database=%s runner_workspace=%s",
settings.omnigent_base_url,
settings.database_path,
settings.omnigent_runner_workspace,
)
store = SQLiteStore(settings.database_path)
await store.initialize()
omnigent = OmnigentClient(
base_url=str(settings.omnigent_base_url),
auth=OmnigentAuth(
email=settings.omnigent_auth_email,
header_name=settings.omnigent_auth_header_name,
session_cookie=settings.omnigent_session_cookie,
),
runner_workspace=settings.omnigent_runner_workspace,
runner_host_id=settings.omnigent_runner_host_id,
runner_launch_timeout_seconds=settings.omnigent_runner_launch_timeout_seconds,
)
logger.info("Checking Omnigent server availability base_url=%s", settings.omnigent_base_url)
try:
agents = await omnigent.list_agents()
except Exception:
logger.exception(
"Omnigent server is not reachable at %s; aborting startup", settings.omnigent_base_url
)
await omnigent.aclose()
raise
logger.info("Omnigent server is up; found %s built-in agents", len(agents))
agent_id = _resolve_agent_id(agents, settings.omnigent_agent_name)
if agent_id is None:
available = ", ".join(sorted(str(a.get("name")) for a in agents if a.get("name"))) or "none"
await omnigent.aclose()
raise RuntimeError(
f"No Omnigent agent named {settings.omnigent_agent_name!r} was found. "
f"Available agents: {available}"
)
logger.info("Resolved Omnigent agent name=%s to id=%s", settings.omnigent_agent_name, agent_id)
service = SlackOmnigentService(
store=store,
omnigent=omnigent,
omnigent_agent_id=agent_id,
update_interval_seconds=settings.slack_update_interval_seconds,
)
app = AsyncApp(token=settings.slack_bot_token)
register_handlers(app, service)
handler = AsyncSocketModeHandler(app, settings.slack_app_token)
try:
logger.info("Connecting to Slack Socket Mode")
await handler.start_async() # type: ignore[no-untyped-call]
finally:
logger.info("Shutting down Omnigent Slack bot")
await service.shutdown()
await omnigent.aclose()
def _resolve_agent_id(agents: list[dict[str, Any]], agent_name: str) -> str | None:
for agent in agents:
if agent.get("name") == agent_name:
agent_id = agent.get("id")
if isinstance(agent_id, str):
return agent_id
return None
def register_handlers(app: AsyncApp, service: SlackOmnigentService) -> None:
@app.event("app_mention")
async def handle_app_mention(
body: dict[str, Any],
event: dict[str, Any],
client: Any,
context: dict[str, Any],
) -> None:
await service.handle_app_mention(body=body, event=event, client=client, context=context)
@app.event("message")
async def handle_message(
body: dict[str, Any],
event: dict[str, Any],
client: Any,
context: dict[str, Any],
) -> None:
if not body.get("team_id") and not event.get("team"):
return
await service.handle_message(body=body, event=event, client=client, context=context)
@@ -1,61 +0,0 @@
from __future__ import annotations
from pathlib import Path
from pydantic import Field
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
extra="ignore",
case_sensitive=False,
)
slack_bot_token: str = Field(validation_alias="SLACK_BOT_TOKEN")
slack_app_token: str = Field(validation_alias="SLACK_APP_TOKEN")
omnigent_agent_name: str = Field(validation_alias="OMNIGENT_AGENT_NAME")
omnigent_base_url: str = Field(
default="http://127.0.0.1:6767",
validation_alias="OMNIGENT_BASE_URL",
)
omnigent_auth_email: str | None = Field(default=None, validation_alias="OMNIGENT_AUTH_EMAIL")
omnigent_auth_header_name: str = Field(
default="X-Forwarded-Email",
validation_alias="OMNIGENT_AUTH_HEADER_NAME",
)
omnigent_session_cookie: str | None = Field(
default=None,
validation_alias="OMNIGENT_SESSION_COOKIE",
)
omnigent_runner_workspace: str = Field(
default_factory=lambda: str(Path.cwd()),
validation_alias="OMNIGENT_RUNNER_WORKSPACE",
)
omnigent_runner_host_id: str | None = Field(
default=None,
validation_alias="OMNIGENT_RUNNER_HOST_ID",
)
omnigent_runner_launch_timeout_seconds: float = Field(
default=60.0,
ge=1.0,
validation_alias="OMNIGENT_RUNNER_LAUNCH_TIMEOUT_SECONDS",
)
database_path: Path = Field(
default=Path("data/omnigent_slack.sqlite3"),
validation_alias="OMNIGENT_SLACK_DATABASE_PATH",
)
log_level: str = Field(default="INFO", validation_alias="LOG_LEVEL")
slack_update_interval_seconds: float = Field(
default=1.0,
ge=0.0,
validation_alias="SLACK_UPDATE_INTERVAL_SECONDS",
)
def load_settings() -> Settings:
return Settings() # type: ignore[call-arg]
@@ -1,70 +0,0 @@
from __future__ import annotations
import asyncio
import logging
from collections.abc import Awaitable, Callable
from omnigent_slack.models import SlackTurn, ThreadKey
TurnWorker = Callable[[SlackTurn], Awaitable[None]]
class ThreadTurnDispatcher:
def __init__(self, worker: TurnWorker, idle_timeout_seconds: float = 60.0) -> None:
self._worker = worker
self._idle_timeout_seconds = idle_timeout_seconds
self._queues: dict[ThreadKey, asyncio.Queue[SlackTurn]] = {}
self._tasks: dict[ThreadKey, asyncio.Task[None]] = {}
self._lock = asyncio.Lock()
self._logger = logging.getLogger(__name__)
async def enqueue(self, turn: SlackTurn) -> None:
async with self._lock:
queue = self._queues.get(turn.key)
if queue is None:
queue = asyncio.Queue()
self._queues[turn.key] = queue
self._tasks[turn.key] = asyncio.create_task(self._run_queue(turn.key, queue))
self._logger.debug("Created turn queue for %s", turn.key.display())
await queue.put(turn)
self._logger.info(
"Queued Slack turn thread=%s queue_size=%s create_if_missing=%s",
turn.key.display(),
queue.qsize(),
turn.create_if_missing,
)
async def shutdown(self) -> None:
async with self._lock:
tasks = list(self._tasks.values())
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
async def _run_queue(self, key: ThreadKey, queue: asyncio.Queue[SlackTurn]) -> None:
try:
while True:
try:
turn = await asyncio.wait_for(queue.get(), timeout=self._idle_timeout_seconds)
except TimeoutError:
self._logger.debug("Closing idle turn queue for %s", key.display())
return
try:
self._logger.info("Running queued Slack turn thread=%s", key.display())
await self._worker(turn)
except Exception:
self._logger.exception("Slack turn failed for %s", key.display())
finally:
queue.task_done()
finally:
async with self._lock:
if self._queues.get(key) is queue:
if queue.empty():
self._queues.pop(key, None)
self._tasks.pop(key, None)
else:
# A turn slipped in after the idle timeout fired but
# before this teardown reacquired the lock. The queue
# stays registered, so no future enqueue would spawn a
# worker — re-arm one here to keep draining it.
self._tasks[key] = asyncio.create_task(self._run_queue(key, queue))
@@ -1,30 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Any
@dataclass(frozen=True, slots=True)
class ThreadKey:
team_id: str
channel_id: str
thread_ts: str
@classmethod
def from_event(cls, team_id: str, event: dict[str, object]) -> ThreadKey:
channel_id = str(event["channel"])
thread_ts = str(event.get("thread_ts") or event["ts"])
return cls(team_id=team_id, channel_id=channel_id, thread_ts=thread_ts)
def display(self) -> str:
return f"{self.team_id}:{self.channel_id}:{self.thread_ts}"
@dataclass(frozen=True, slots=True)
class SlackTurn:
key: ThreadKey
text: str
user_id: str
create_if_missing: bool
title: str
slack_client: Any
@@ -1,570 +0,0 @@
from __future__ import annotations
import asyncio
import json
import logging
import random
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import Any
import httpx
class OmnigentError(RuntimeError):
pass
class RunnerUnavailableError(OmnigentError):
pass
@dataclass(frozen=True, slots=True)
class OmnigentAuth:
email: str | None = None
header_name: str = "X-Forwarded-Email"
session_cookie: str | None = None
def headers(self) -> dict[str, str]:
headers: dict[str, str] = {}
if self.email:
headers[self.header_name] = self.email
if self.session_cookie:
headers["Cookie"] = (
self.session_cookie
if "=" in self.session_cookie
else f"ap_session={self.session_cookie}"
)
return headers
class OmnigentClient:
def __init__(
self,
base_url: str,
auth: OmnigentAuth | None = None,
timeout: float = 30.0,
runner_workspace: str | None = None,
runner_host_id: str | None = None,
runner_launch_timeout_seconds: float = 60.0,
) -> None:
self._client = httpx.AsyncClient(
base_url=base_url.rstrip("/"),
timeout=httpx.Timeout(timeout, read=None),
headers=(auth or OmnigentAuth()).headers(),
)
self._runner_workspace = runner_workspace
self._runner_host_id = runner_host_id
self._runner_launch_timeout_seconds = runner_launch_timeout_seconds
self._logger = logging.getLogger(__name__)
async def aclose(self) -> None:
await self._client.aclose()
async def create_session(self, agent_id: str, title: str) -> str:
self._logger.info("Creating Omnigent session agent_id=%s title=%r", agent_id, title)
response = await self._client.post(
"/v1/sessions",
json={"agent_id": agent_id, "title": title},
)
await _raise_for_status(response)
payload = response.json()
session_id = _extract_session_id(payload)
if session_id is None:
raise OmnigentError(f"Create session response did not include an id: {payload!r}")
self._logger.info("Created Omnigent session session_id=%s", session_id)
return session_id
async def submit_message(self, session_id: str, text: str) -> None:
self._logger.info(
"Submitting Slack message to Omnigent session_id=%s chars=%s",
session_id,
len(text),
)
payload = {
"type": "message",
"data": {
"role": "user",
"content": [{"type": "input_text", "text": text}],
},
}
response = await self._client.post(f"/v1/sessions/{session_id}/events", json=payload)
await _raise_for_status(response)
self._logger.debug("Submitted Omnigent message session_id=%s", session_id)
async def bind_random_runner(self, session_id: str) -> str:
runner_ids = await self.list_runner_ids()
if not runner_ids:
return await self.launch_random_runner(session_id)
runner_id = random.choice(runner_ids)
self._logger.info(
"Binding random Omnigent runner session_id=%s runner_id=%s candidates=%s",
session_id,
runner_id,
len(runner_ids),
)
response = await self._client.patch(
f"/v1/sessions/{session_id}",
json={"runner_id": runner_id},
)
await _raise_for_status(response)
return runner_id
async def list_runner_ids(self) -> list[str]:
runner_ids = await self.list_runner_ids_from_hosts()
self._logger.info("Loaded Omnigent runner ids from hosts count=%s", len(runner_ids))
return runner_ids
async def launch_random_runner(self, session_id: str) -> str:
if not self._runner_workspace:
raise OmnigentError(
"No online Omnigent runners are available, and runner auto-launch is not "
"configured. Set OMNIGENT_RUNNER_WORKSPACE to an absolute workspace path; "
"optionally set OMNIGENT_RUNNER_HOST_ID to use a specific host."
)
host_id = self._runner_host_id or await self._select_random_online_host()
self._logger.info(
"Launching Omnigent runner session_id=%s host_id=%s workspace=%s",
session_id,
host_id,
self._runner_workspace,
)
response = await self._client.post(
f"/v1/hosts/{host_id}/runners",
json={"session_id": session_id, "workspace": self._runner_workspace},
)
await _raise_for_status(response)
payload = response.json()
runner_id = _extract_runner_id(payload)
if runner_id is None:
raise OmnigentError(f"Launch runner response did not include a runner id: {payload!r}")
await self.wait_for_runner_online(runner_id)
self._logger.info(
"Launched Omnigent runner session_id=%s runner_id=%s host_id=%s",
session_id,
runner_id,
host_id,
)
return runner_id
async def list_agents(self) -> list[dict[str, Any]]:
self._logger.debug("Listing built-in Omnigent agents")
response = await self._client.get("/v1/agents")
await _raise_for_status(response)
payload = response.json()
data = _extract_list(payload, "data") or _extract_list(payload, "agents")
if data is None:
data = payload if isinstance(payload, list) else []
agents = [item for item in data if isinstance(item, dict)]
self._logger.info("Found built-in Omnigent agents count=%s", len(agents))
return agents
async def list_runners(self) -> list[dict[str, Any]]:
self._logger.debug("Listing online Omnigent runners")
response = await self._client.get("/v1/runners")
await _raise_for_status(response)
payload = response.json()
data = _extract_list(payload, "data") or _extract_list(payload, "runners")
if data is None:
data = payload if isinstance(payload, list) else []
runners = [item for item in data if isinstance(item, dict)]
self._logger.info("Found online Omnigent runners count=%s", len(runners))
return runners
async def list_hosts(self) -> list[dict[str, Any]]:
self._logger.debug("Listing Omnigent hosts")
response = await self._client.get("/v1/hosts")
await _raise_for_status(response)
payload = response.json()
data = _extract_list(payload, "hosts") or _extract_list(payload, "data")
if data is None:
data = payload if isinstance(payload, list) else []
hosts = [item for item in data if isinstance(item, dict)]
self._logger.info("Found Omnigent hosts count=%s", len(hosts))
return hosts
async def list_runner_ids_from_hosts(self) -> list[str]:
hosts = await self.list_hosts()
runner_ids: list[str] = []
for host in hosts:
if not _is_host_online(host):
continue
runner_ids.extend(_runner_ids_from_host(host))
return sorted(set(runner_ids))
async def wait_for_runner_online(self, runner_id: str) -> None:
deadline = asyncio.get_running_loop().time() + self._runner_launch_timeout_seconds
while True:
response = await self._client.get(f"/v1/runners/{runner_id}/status")
await _raise_for_status(response)
payload = response.json()
if isinstance(payload, dict) and payload.get("online") is True:
return
if asyncio.get_running_loop().time() >= deadline:
raise OmnigentError(
f"Timed out waiting for launched Omnigent runner to come online: {runner_id}"
)
await asyncio.sleep(1)
async def _select_random_online_host(self) -> str:
hosts = await self.list_hosts()
host_ids = [
host_id
for host in hosts
if _is_host_online(host) and (host_id := _host_id(host)) is not None
]
if not host_ids:
raise OmnigentError(
"No online Omnigent hosts are available to launch a runner. "
"Set OMNIGENT_RUNNER_HOST_ID to a specific online host, or start a host."
)
host_id = random.choice(host_ids)
self._logger.info(
"Selected random Omnigent host host_id=%s candidates=%s",
host_id,
len(host_ids),
)
return host_id
@asynccontextmanager
async def stream_session_events(
self,
session_id: str,
) -> AsyncIterator[AsyncIterator[dict[str, Any]]]:
async with self._client.stream(
"GET",
f"/v1/sessions/{session_id}/stream",
params={"idle": "false"},
) as response:
await _raise_for_status(response)
self._logger.debug("Connected to Omnigent SSE stream session_id=%s", session_id)
yield iter_sse_events(response.aiter_lines())
async def run_turn(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
try:
async for event in self._run_turn_once(session_id, text):
yield event
return
except RunnerUnavailableError:
self._logger.info(
"Session has no available runner; "
"binding a random runner and retrying session_id=%s",
session_id,
)
await self.bind_random_runner(session_id)
async for event in self._run_turn_once(session_id, text):
yield event
async def _run_turn_once(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
async with self.stream_session_events(session_id) as events:
await self.submit_message(session_id, text)
async for event in events:
self._logger.debug(
"Received Omnigent event session_id=%s type=%s",
session_id,
event.get("type"),
)
yield event
if is_terminal_event(event):
self._logger.info(
"Omnigent turn reached terminal event session_id=%s type=%s",
session_id,
event.get("type"),
)
break
async def latest_assistant_text(self, session_id: str) -> str | None:
self._logger.debug("Fetching latest Omnigent assistant item session_id=%s", session_id)
response = await self._client.get(
f"/v1/sessions/{session_id}/items",
params={"limit": 100, "order": "desc"},
)
await _raise_for_status(response)
payload = response.json()
items = payload.get("data", [])
if not isinstance(items, list):
return None
for item in items:
if isinstance(item, dict):
text = extract_assistant_text(item)
if text:
return text
return None
async def iter_sse_events(lines: AsyncIterator[str]) -> AsyncIterator[dict[str, Any]]:
event_name: str | None = None
data_lines: list[str] = []
async for raw_line in lines:
line = raw_line.rstrip("\r")
if line == "":
event = _decode_sse_event(event_name, data_lines)
event_name = None
data_lines = []
if event is None:
continue
if event == "[DONE]":
break
if isinstance(event, str):
continue
yield event
continue
if line.startswith(":"):
continue
field, separator, value = line.partition(":")
if separator and value.startswith(" "):
value = value[1:]
if field == "event":
event_name = value
elif field == "data":
data_lines.append(value)
event = _decode_sse_event(event_name, data_lines)
if isinstance(event, dict):
yield event
def is_terminal_event(event: dict[str, Any]) -> bool:
# A turn ends at the SESSION level, not the response level. Orchestrator
# agents emit a `response.completed`/`turn.completed` every time they end a
# turn to wait on a background sub-agent, then resume with more responses in
# the same turn — so treating those as terminal cuts the stream off at the
# first sub-agent dispatch. `session.status` is the authoritative signal:
# `running` -> `waiting` (parked on async work) -> `running` -> `idle`, and
# only `idle`/`failed` mean the turn is truly over.
event_type = str(event.get("type"))
if event_type == "session.status":
return str(event.get("status")) in {"idle", "failed"}
# Explicit turn/response failure and cancellation still end the turn; keep
# them as a fallback in case the session settles without an `idle` edge.
return event_type in {
"response.failed",
"response.cancelled",
"turn.failed",
"turn.cancelled",
}
def extract_delta(event: dict[str, Any]) -> str | None:
if event.get("type") != "response.output_text.delta":
return None
delta = event.get("delta")
return delta if isinstance(delta, str) else None
def extract_error_text(event: dict[str, Any]) -> str | None:
event_type = str(event.get("type"))
if event_type == "response.error":
error = event.get("error")
if isinstance(error, dict):
message = error.get("message")
if isinstance(message, str):
return message
message = event.get("message")
if isinstance(message, str):
return message
if event_type in {"response.failed", "turn.failed"}:
response = event.get("response")
if isinstance(response, dict):
last_error = response.get("error") or response.get("last_error")
if isinstance(last_error, dict):
message = last_error.get("message")
if isinstance(message, str):
return message
error = event.get("error")
if isinstance(error, dict):
message = error.get("message")
if isinstance(message, str):
return message
if isinstance(error, str):
return error
return None
def extract_assistant_text(event_or_item: dict[str, Any]) -> str | None:
if event_or_item.get("type") == "response.output_item.done":
item = event_or_item.get("item")
return extract_assistant_text(item) if isinstance(item, dict) else None
item_type = event_or_item.get("type")
if item_type != "message":
return None
data = event_or_item.get("data")
message = data if isinstance(data, dict) else event_or_item
if message.get("role") != "assistant":
return None
content = message.get("content")
if not isinstance(content, list):
return None
parts: list[str] = []
for block in content:
if not isinstance(block, dict):
continue
text = block.get("text")
if isinstance(text, str):
parts.append(text)
return "".join(parts).strip() or None
def _decode_sse_event(event_name: str | None, data_lines: list[str]) -> dict[str, Any] | str | None:
if not data_lines:
return None
data = "\n".join(data_lines)
if data == "[DONE]":
return data
try:
payload = json.loads(data)
except json.JSONDecodeError as exc:
raise OmnigentError(f"Invalid SSE JSON payload: {data}") from exc
if not isinstance(payload, dict):
return None
if event_name and "type" not in payload:
payload["type"] = event_name
return payload
def _extract_session_id(payload: Any) -> str | None:
if isinstance(payload, dict):
for key in ("id", "session_id", "conversation_id"):
value = payload.get(key)
if isinstance(value, str):
return value
for key in ("session", "data"):
value = _extract_session_id(payload.get(key))
if value:
return value
return None
def _extract_list(payload: Any, key: str) -> list[Any] | None:
if not isinstance(payload, dict):
return None
value = payload.get(key)
return value if isinstance(value, list) else None
def _runner_id(runner: dict[str, Any]) -> str | None:
for key in ("id", "runner_id"):
value = runner.get(key)
if isinstance(value, str):
return value
return None
def _runner_ids_from_host(host: dict[str, Any]) -> list[str]:
runner_ids: list[str] = []
for key in (
"runner_id",
"active_runner_id",
"current_runner_id",
):
value = host.get(key)
if isinstance(value, str):
runner_ids.append(value)
for key in (
"runner_ids",
"active_runner_ids",
"current_runner_ids",
"live_runner_ids",
"online_runner_ids",
):
value = host.get(key)
if isinstance(value, list):
runner_ids.extend(item for item in value if isinstance(item, str))
for key in ("runner", "active_runner", "current_runner"):
value = host.get(key)
if isinstance(value, str):
runner_ids.append(value)
elif isinstance(value, dict):
runner_id = _runner_id(value)
if runner_id:
runner_ids.append(runner_id)
for key in (
"runners",
"active_runners",
"current_runners",
"live_runners",
"online_runners",
):
value = host.get(key)
if not isinstance(value, list):
continue
for item in value:
if isinstance(item, str):
runner_ids.append(item)
elif isinstance(item, dict):
runner_id = _runner_id(item)
if runner_id:
runner_ids.append(runner_id)
return runner_ids
def _extract_runner_id(payload: Any) -> str | None:
if isinstance(payload, dict):
value = _runner_id(payload)
if value:
return value
for key in ("runner", "data"):
value = _extract_runner_id(payload.get(key))
if value:
return value
return None
def _host_id(host: dict[str, Any]) -> str | None:
for key in ("id", "host_id"):
value = host.get(key)
if isinstance(value, str):
return value
return None
def _is_host_online(host: dict[str, Any]) -> bool:
if host.get("online") is True or host.get("host_online") is True:
return True
status = host.get("status")
return isinstance(status, str) and status.lower() == "online"
async def _raise_for_status(response: httpx.Response) -> None:
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
error_code = _extract_error_code(response)
if response.status_code == 503 and error_code == "runner_unavailable":
raise RunnerUnavailableError(
f"Omnigent runner unavailable for {response.request.url}: {response.text}"
) from exc
raise OmnigentError(
f"Omnigent request failed with {response.status_code}: {response.text}"
) from exc
def _extract_error_code(response: httpx.Response) -> str | None:
try:
payload = response.json()
except json.JSONDecodeError:
return None
if not isinstance(payload, dict):
return None
error = payload.get("error")
if not isinstance(error, dict):
return None
code = error.get("code")
return code if isinstance(code, str) else None
@@ -1,391 +0,0 @@
from __future__ import annotations
import logging
import time
from typing import Any, Protocol
from omnigent_slack.dispatcher import ThreadTurnDispatcher
from omnigent_slack.models import SlackTurn, ThreadKey
from omnigent_slack.omnigent import (
OmnigentClient,
extract_assistant_text,
extract_delta,
extract_error_text,
)
from omnigent_slack.store import SQLiteStore
from omnigent_slack.text import (
normalize_whitespace,
split_for_slack,
strip_bot_mention,
to_mrkdwn,
truncate_for_slack,
)
class SlackClientProtocol(Protocol):
async def chat_postMessage(self, **kwargs: Any) -> dict[str, Any]: ...
async def chat_update(self, **kwargs: Any) -> dict[str, Any]: ...
class SlackOmnigentService:
def __init__(
self,
*,
store: SQLiteStore,
omnigent: OmnigentClient,
omnigent_agent_id: str,
update_interval_seconds: float = 1.0,
bot_user_id: str | None = None,
) -> None:
self._store = store
self._omnigent = omnigent
self._omnigent_agent_id = omnigent_agent_id
self._update_interval_seconds = update_interval_seconds
self._bot_user_id = bot_user_id
self._dispatcher = ThreadTurnDispatcher(self._run_turn)
self._logger = logging.getLogger(__name__)
async def shutdown(self) -> None:
await self._dispatcher.shutdown()
async def handle_app_mention(
self,
*,
body: dict[str, Any],
event: dict[str, Any],
client: SlackClientProtocol,
context: dict[str, Any] | None = None,
) -> None:
self._logger.info(
"Received Slack app_mention team=%s channel=%s ts=%s user=%s event_id=%s",
body.get("team_id") or event.get("team"),
event.get("channel"),
event.get("ts"),
event.get("user"),
body.get("event_id") or event.get("client_msg_id"),
)
if not await self._claim_event(body, event):
self._logger.info(
"Ignoring duplicate Slack app_mention event_id=%s",
body.get("event_id") or event.get("client_msg_id"),
)
return
bot_user_id = self._resolve_bot_user_id(context)
if self._should_ignore_message(event, bot_user_id):
self._logger.info(
"Ignoring Slack app_mention subtype=%s bot_id=%s user=%s bot_user_id=%s",
event.get("subtype"),
event.get("bot_id"),
event.get("user"),
bot_user_id,
)
return
team_id = _team_id(body, event)
key = ThreadKey.from_event(team_id, event)
text = strip_bot_mention(str(event.get("text") or ""), bot_user_id)
if not text:
self._logger.info(
"Slack app_mention had no text after mention thread=%s",
key.display(),
)
await client.chat_postMessage(
channel=key.channel_id,
thread_ts=key.thread_ts,
text="Send a message after mentioning me to start a session.",
)
return
self._logger.info("Accepted Slack app_mention thread=%s chars=%s", key.display(), len(text))
await self._dispatcher.enqueue(
SlackTurn(
key=key,
text=text,
user_id=str(event.get("user") or ""),
create_if_missing=True,
title=_session_title(event, text),
slack_client=client,
)
)
async def handle_message(
self,
*,
body: dict[str, Any],
event: dict[str, Any],
client: SlackClientProtocol,
context: dict[str, Any] | None = None,
) -> None:
self._logger.info(
"Received Slack message team=%s channel=%s ts=%s thread_ts=%s user=%s event_id=%s",
body.get("team_id") or event.get("team"),
event.get("channel"),
event.get("ts"),
event.get("thread_ts"),
event.get("user"),
body.get("event_id") or event.get("client_msg_id"),
)
if not await self._claim_event(body, event):
self._logger.info(
"Ignoring duplicate Slack message event_id=%s",
body.get("event_id") or event.get("client_msg_id"),
)
return
bot_user_id = self._resolve_bot_user_id(context)
if self._should_ignore_message(event, bot_user_id):
self._logger.info(
"Ignoring Slack message subtype=%s bot_id=%s user=%s bot_user_id=%s",
event.get("subtype"),
event.get("bot_id"),
event.get("user"),
bot_user_id,
)
return
raw_text = str(event.get("text") or "")
if bot_user_id and f"<@{bot_user_id}" in raw_text:
self._logger.info("Ignoring generic message containing bot mention")
return
team_id = _team_id(body, event)
key = ThreadKey.from_event(team_id, event)
if await self._store.get_session_id(key) is None:
self._logger.info(
"Ignoring Slack message with no Omnigent session thread=%s",
key.display(),
)
return
text = normalize_whitespace(raw_text)
if not text:
self._logger.info("Ignoring empty Slack message thread=%s", key.display())
return
self._logger.info(
"Accepted Slack thread reply thread=%s chars=%s",
key.display(),
len(text),
)
await self._dispatcher.enqueue(
SlackTurn(
key=key,
text=text,
user_id=str(event.get("user") or ""),
create_if_missing=False,
title=_session_title(event, text),
slack_client=client,
)
)
async def _run_turn(self, turn: SlackTurn) -> None:
self._logger.info("Starting turn thread=%s chars=%s", turn.key.display(), len(turn.text))
session_id = await self._store.get_session_id(turn.key)
if session_id is None:
if not turn.create_if_missing:
self._logger.info(
"No session found and creation disabled thread=%s",
turn.key.display(),
)
return
session_id = await self._omnigent.create_session(self._omnigent_agent_id, turn.title)
runner_id = await self._omnigent.bind_random_runner(session_id)
await self._store.upsert_session(turn.key, session_id, turn.title)
self._logger.info(
"Mapped Slack thread to new Omnigent session thread=%s session_id=%s runner_id=%s",
turn.key.display(),
session_id,
runner_id,
)
else:
self._logger.info(
"Using existing Omnigent session thread=%s session_id=%s",
turn.key.display(),
session_id,
)
slack_client = turn.slack_client
placeholder = await slack_client.chat_postMessage(
channel=turn.key.channel_id,
thread_ts=turn.key.thread_ts,
text="Working...",
)
message_ts = str(placeholder.get("ts") or "")
if not message_ts:
self._logger.error("Slack placeholder response missing ts: %r", placeholder)
return
self._logger.info(
"Posted Slack placeholder thread=%s message_ts=%s",
turn.key.display(),
message_ts,
)
streamed_text = ""
final_text: str | None = None
error_text: str | None = None
last_update = 0.0
try:
async for omnigent_event in self._omnigent.run_turn(session_id, turn.text):
delta = extract_delta(omnigent_event)
if delta:
streamed_text += delta
self._logger.debug(
"Accumulated Omnigent delta thread=%s total_chars=%s",
turn.key.display(),
len(streamed_text),
)
now = time.monotonic()
if now - last_update >= self._update_interval_seconds:
# A progress edit is best-effort: a failure here (e.g. a
# transient Slack error) must not abort the turn or
# clobber the real answer delivered below.
try:
await self._update_slack(
slack_client, turn.key, message_ts, streamed_text
)
except Exception:
self._logger.warning(
"Slack progress update failed thread=%s; continuing",
turn.key.display(),
exc_info=True,
)
last_update = now
item_text = extract_assistant_text(omnigent_event)
if item_text:
final_text = item_text
event_error = extract_error_text(omnigent_event)
if event_error:
error_text = event_error
except Exception as exc:
self._logger.exception("Omnigent turn failed for %s", turn.key.display())
error_text = str(exc)
# Resolve the answer independently of any error so a failure never
# erases what the user already saw stream in.
if not final_text:
final_text = streamed_text.strip() or await self._omnigent.latest_assistant_text(
session_id
)
if final_text:
# Deliver the real answer, then, if the turn also errored, report
# the failure as a separate reply instead of overwriting it.
await self._deliver_final(slack_client, turn.key, message_ts, final_text)
if error_text:
await self._post_failure_reply(slack_client, turn.key, error_text)
else:
# Nothing to preserve — surface the error (or a fallback) in the
# placeholder itself.
fallback = (
f"Omnigent request failed: {error_text}"
if error_text
else "Omnigent completed without returning response text."
)
await self._deliver_final(slack_client, turn.key, message_ts, fallback)
self._logger.info(
"Completed Slack turn thread=%s session_id=%s final_chars=%s errored=%s",
turn.key.display(),
session_id,
len(final_text or ""),
bool(error_text),
)
async def _post_failure_reply(
self,
client: SlackClientProtocol,
key: ThreadKey,
error_text: str,
) -> None:
# Post the failure as its own thread reply so the already-delivered
# answer stays intact. Keep it to a single message.
await client.chat_postMessage(
channel=key.channel_id,
thread_ts=key.thread_ts,
text=truncate_for_slack(f":warning: Omnigent request failed: {error_text}"),
)
async def _deliver_final(
self,
client: SlackClientProtocol,
key: ThreadKey,
message_ts: str,
text: str,
) -> None:
# The server returns standard Markdown; convert it to Slack's mrkdwn
# dialect before display. A single Slack message can't hold a long
# answer, so split the converted text and edit the placeholder to the
# first chunk, posting the rest as thread replies — delivering the full
# answer instead of truncating it.
chunks = split_for_slack(to_mrkdwn(text))
await self._update_slack(client, key, message_ts, chunks[0])
for chunk in chunks[1:]:
await client.chat_postMessage(
channel=key.channel_id,
thread_ts=key.thread_ts,
text=chunk,
)
if len(chunks) > 1:
self._logger.info(
"Delivered long Slack answer across parts thread=%s parts=%s",
key.display(),
len(chunks),
)
async def _update_slack(
self,
client: SlackClientProtocol,
key: ThreadKey,
message_ts: str,
text: str,
) -> None:
self._logger.debug(
"Updating Slack message thread=%s message_ts=%s chars=%s",
key.display(),
message_ts,
len(text),
)
await client.chat_update(
channel=key.channel_id,
ts=message_ts,
text=truncate_for_slack(text),
)
async def _claim_event(self, body: dict[str, Any], event: dict[str, Any]) -> bool:
event_id = body.get("event_id") or event.get("client_msg_id")
return await self._store.claim_event(str(event_id) if event_id else None)
def _resolve_bot_user_id(self, context: dict[str, Any] | None) -> str | None:
bot_user_id = None if context is None else context.get("bot_user_id")
if isinstance(bot_user_id, str):
self._bot_user_id = bot_user_id
return bot_user_id
return self._bot_user_id
@staticmethod
def _should_ignore_message(event: dict[str, Any], bot_user_id: str | None) -> bool:
subtype = event.get("subtype")
if subtype in {"bot_message", "message_changed", "message_deleted"}:
return True
if event.get("bot_id"):
return True
user_id = event.get("user")
return bool(bot_user_id and user_id == bot_user_id)
def _team_id(body: dict[str, Any], event: dict[str, Any]) -> str:
team_id = body.get("team_id") or event.get("team")
if not team_id:
raise ValueError("Slack event is missing team_id")
return str(team_id)
def _session_title(event: dict[str, Any], text: str) -> str:
channel = str(event.get("channel") or "channel")
thread_ts = str(event.get("thread_ts") or event.get("ts") or "thread")
summary = truncate_for_slack(text, limit=80).replace("\n", " ")
return f"Slack {channel}/{thread_ts}: {summary}"
@@ -1,92 +0,0 @@
from __future__ import annotations
import time
from pathlib import Path
import aiosqlite
from omnigent_slack.models import ThreadKey
class SQLiteStore:
def __init__(self, path: Path) -> None:
self._path = path
async def initialize(self) -> None:
self._path.parent.mkdir(parents=True, exist_ok=True)
async with aiosqlite.connect(self._path) as db:
await db.execute("PRAGMA journal_mode=WAL")
await db.execute(
"""
CREATE TABLE IF NOT EXISTS thread_sessions (
team_id TEXT NOT NULL,
channel_id TEXT NOT NULL,
thread_ts TEXT NOT NULL,
omnigent_session_id TEXT NOT NULL,
title TEXT NOT NULL,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
PRIMARY KEY (team_id, channel_id, thread_ts)
)
"""
)
await db.execute(
"""
CREATE TABLE IF NOT EXISTS slack_events (
event_id TEXT PRIMARY KEY,
created_at INTEGER NOT NULL
)
"""
)
await db.commit()
async def get_session_id(self, key: ThreadKey) -> str | None:
async with aiosqlite.connect(self._path) as db:
cursor = await db.execute(
"""
SELECT omnigent_session_id
FROM thread_sessions
WHERE team_id = ? AND channel_id = ? AND thread_ts = ?
""",
(key.team_id, key.channel_id, key.thread_ts),
)
row = await cursor.fetchone()
await cursor.close()
if row is None:
return None
return str(row[0])
async def upsert_session(self, key: ThreadKey, session_id: str, title: str) -> None:
now = int(time.time())
async with aiosqlite.connect(self._path) as db:
await db.execute(
"""
INSERT INTO thread_sessions (
team_id, channel_id, thread_ts, omnigent_session_id,
title, created_at, updated_at
)
VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(team_id, channel_id, thread_ts) DO UPDATE SET
omnigent_session_id = excluded.omnigent_session_id,
title = excluded.title,
updated_at = excluded.updated_at
""",
(key.team_id, key.channel_id, key.thread_ts, session_id, title, now, now),
)
await db.commit()
async def claim_event(self, event_id: str | None, ttl_seconds: int = 7 * 24 * 60 * 60) -> bool:
if not event_id:
return True
now = int(time.time())
async with aiosqlite.connect(self._path) as db:
cursor = await db.execute(
"INSERT OR IGNORE INTO slack_events (event_id, created_at) VALUES (?, ?)",
(event_id, now),
)
claimed = cursor.rowcount == 1
await cursor.close()
await db.execute("DELETE FROM slack_events WHERE created_at < ?", (now - ttl_seconds,))
await db.commit()
return claimed
@@ -1,79 +0,0 @@
from __future__ import annotations
import re
from markdown_to_mrkdwn import SlackMarkdownConverter
MENTION_RE = re.compile(r"<@([A-Z0-9]+)(?:\|[^>]+)?>")
WHITESPACE_RE = re.compile(r"\s+")
# Slack renders its own `mrkdwn` dialect, not standard Markdown (e.g. *bold* is
# single-asterisk, links are <url|text>). Reuse one converter instance — it
# compiles regex patterns on init.
_MRKDWN_CONVERTER = SlackMarkdownConverter()
def to_mrkdwn(text: str) -> str:
"""Convert standard Markdown to Slack's mrkdwn dialect for display."""
return str(_MRKDWN_CONVERTER.convert(text))
def strip_bot_mention(text: str, bot_user_id: str | None) -> str:
if bot_user_id:
text = re.sub(rf"<@{re.escape(bot_user_id)}(?:\|[^>]+)?>", " ", text)
else:
text = MENTION_RE.sub(" ", text, count=1)
return normalize_whitespace(text)
def normalize_whitespace(text: str) -> str:
return WHITESPACE_RE.sub(" ", text).strip()
# Slack accepts up to 40,000 characters in a message `text`, but its own
# guidance is to keep messages under 4,000 so they render without a "Show more"
# fold. Stay at that best-practice ceiling and split longer answers across
# replies (see `split_for_slack`).
SLACK_MESSAGE_CHAR_LIMIT = 4000
def truncate_for_slack(text: str, limit: int = SLACK_MESSAGE_CHAR_LIMIT) -> str:
if len(text) <= limit:
return text
suffix = "\n\n[truncated]"
if limit <= len(suffix):
return text[:limit]
return text[: limit - len(suffix)].rstrip() + suffix
def split_for_slack(text: str, limit: int = SLACK_MESSAGE_CHAR_LIMIT) -> list[str]:
"""Split ``text`` into chunks no longer than ``limit`` characters.
Preserves every character so a long assistant answer (code blocks,
reports) is delivered in full across multiple messages instead of being
truncated. Prefers to break after a newline, then a space, and only
hard-cuts a run with no whitespace (e.g. a long URL). A blank string
yields ``[""]`` so the caller always has a message to post.
"""
if limit <= 0:
raise ValueError("limit must be positive")
if not text:
return [""]
chunks: list[str] = []
start = 0
length = len(text)
while start < length:
end = start + limit
if end >= length:
chunks.append(text[start:])
break
window = text[start:end]
boundary = window.rfind("\n")
if boundary == -1:
boundary = window.rfind(" ")
# Include the delimiter in the current chunk; hard-cut if none found.
cut = end if boundary <= 0 else start + boundary + 1
chunks.append(text[start:cut])
start = cut
return chunks
@@ -1,97 +0,0 @@
import asyncio
from omnigent_slack.dispatcher import ThreadTurnDispatcher
from omnigent_slack.models import SlackTurn, ThreadKey
def _turn(key: ThreadKey, text: str) -> SlackTurn:
return SlackTurn(
key=key,
text=text,
user_id="U",
create_if_missing=False,
title="title",
slack_client=object(),
)
async def test_dispatcher_runs_turns_in_thread_order() -> None:
seen: list[str] = []
done = asyncio.Event()
async def worker(turn: SlackTurn) -> None:
await asyncio.sleep(0)
seen.append(turn.text)
if len(seen) == 3:
done.set()
dispatcher = ThreadTurnDispatcher(worker, idle_timeout_seconds=0.1)
key = ThreadKey(team_id="T", channel_id="C", thread_ts="1")
for text in ["one", "two", "three"]:
await dispatcher.enqueue(_turn(key, text))
await asyncio.wait_for(done.wait(), timeout=1)
await dispatcher.shutdown()
assert seen == ["one", "two", "three"]
async def test_enqueue_during_idle_teardown_is_not_wedged() -> None:
"""A turn that arrives while an idle worker is tearing down must still run.
Reproduces the race where ``_run_queue`` times out on an empty queue and
decides to exit, then ``enqueue`` slips a turn in before the teardown
``finally`` reacquires the lock. The queue stays registered, so no new
worker is ever spawned and the turn is stranded.
"""
seen: list[str] = []
processed = asyncio.Event()
async def worker(turn: SlackTurn) -> None:
seen.append(turn.text)
processed.set()
dispatcher = ThreadTurnDispatcher(worker, idle_timeout_seconds=0.05)
key = ThreadKey(team_id="T", channel_id="C", thread_ts="1")
# Gate the teardown's lock acquisition so a concurrent enqueue wins the
# race: the worker has decided to exit but has not yet run its finally.
original_lock = dispatcher._lock
teardown_reached = asyncio.Event()
release_teardown = asyncio.Event()
class _GatedLock:
def __init__(self) -> None:
self._enter_count = 0
async def __aenter__(self) -> None:
self._enter_count += 1
# The first acquisition after startup is enqueue's; the teardown
# acquisition is the one we stall so enqueue can slip ahead.
if self._enter_count == 2:
teardown_reached.set()
await release_teardown.wait()
await original_lock.acquire()
async def __aexit__(self, *exc: object) -> None:
original_lock.release()
dispatcher._lock = _GatedLock() # type: ignore[assignment]
# Let the worker spawn and hit its idle timeout → enter teardown.
await dispatcher.enqueue(_turn(key, "first"))
await asyncio.wait_for(processed.wait(), timeout=1)
processed.clear()
await asyncio.wait_for(teardown_reached.wait(), timeout=1)
# Enqueue arrives before teardown finishes. Restore the real lock so the
# new enqueue path (and any re-armed worker) runs unhindered.
dispatcher._lock = original_lock # type: ignore[assignment]
await dispatcher.enqueue(_turn(key, "second"))
release_teardown.set()
await asyncio.wait_for(processed.wait(), timeout=1)
await dispatcher.shutdown()
assert seen == ["first", "second"]
-295
View File
@@ -1,295 +0,0 @@
from collections.abc import AsyncIterator
import httpx
import respx
from omnigent_slack.omnigent import (
OmnigentAuth,
OmnigentClient,
OmnigentError,
RunnerUnavailableError,
extract_assistant_text,
is_terminal_event,
iter_sse_events,
)
def test_is_terminal_event_only_ends_on_session_idle_or_failed() -> None:
# Per-response completions are NOT terminal: an orchestrator emits one each
# time it ends a turn to wait on a sub-agent, then resumes the same turn.
assert not is_terminal_event({"type": "response.completed"})
assert not is_terminal_event({"type": "turn.completed"})
assert not is_terminal_event({"type": "response.output_text.delta", "delta": "x"})
assert not is_terminal_event({"type": "session.status", "status": "running"})
assert not is_terminal_event({"type": "session.status", "status": "waiting"})
# The session settling is the authoritative turn boundary.
assert is_terminal_event({"type": "session.status", "status": "idle"})
assert is_terminal_event({"type": "session.status", "status": "failed"})
# Explicit failure/cancel still ends the turn as a fallback.
assert is_terminal_event({"type": "response.failed"})
assert is_terminal_event({"type": "turn.cancelled"})
async def _lines(values: list[str]) -> AsyncIterator[str]:
for value in values:
yield value
async def test_iter_sse_events_parses_json_and_done() -> None:
events = [
event
async for event in iter_sse_events(
_lines(
[
"event: response.output_text.delta",
'data: {"delta":"hel"}',
"",
'data: {"type":"response.output_text.delta","delta":"lo"}',
"",
"data: [DONE]",
"",
]
)
)
]
assert events == [
{"type": "response.output_text.delta", "delta": "hel"},
{"type": "response.output_text.delta", "delta": "lo"},
]
def test_extract_assistant_text_from_stream_item() -> None:
assert (
extract_assistant_text(
{
"type": "response.output_item.done",
"item": {
"type": "message",
"role": "assistant",
"content": [{"type": "output_text", "text": "done"}],
},
}
)
== "done"
)
@respx.mock
async def test_client_create_and_submit_request_shapes() -> None:
create = respx.post("http://omnigent.test/v1/sessions").mock(
return_value=httpx.Response(201, json={"id": "conv_1"})
)
submit = respx.post("http://omnigent.test/v1/sessions/conv_1/events").mock(
return_value=httpx.Response(200, json={})
)
client = OmnigentClient(
"http://omnigent.test",
auth=OmnigentAuth(email="bot@example.com", session_cookie="cookie-value"),
)
try:
session_id = await client.create_session("ag_1", "Slack C/1")
await client.submit_message(session_id, "hello")
finally:
await client.aclose()
assert session_id == "conv_1"
assert create.calls.last.request.headers["X-Forwarded-Email"] == "bot@example.com"
assert create.calls.last.request.headers["Cookie"] == "ap_session=cookie-value"
assert create.calls.last.request.read() == b'{"agent_id":"ag_1","title":"Slack C/1"}'
assert submit.calls.last.request.read() == (
b'{"type":"message","data":{"role":"user","content":[{"type":"input_text",'
b'"text":"hello"}]}}'
)
@respx.mock
async def test_client_binds_random_runner() -> None:
respx.get("http://omnigent.test/v1/hosts").mock(
return_value=httpx.Response(
200,
json={"hosts": [{"id": "host_1", "online": True, "runners": [{"id": "runner_a"}]}]},
)
)
bind = respx.patch("http://omnigent.test/v1/sessions/conv_1").mock(
return_value=httpx.Response(200, json={"id": "conv_1", "runner_id": "runner_a"})
)
client = OmnigentClient("http://omnigent.test")
try:
runner_id = await client.bind_random_runner("conv_1")
finally:
await client.aclose()
assert runner_id == "runner_a"
assert bind.calls.last.request.read() == b'{"runner_id":"runner_a"}'
@respx.mock
async def test_client_launches_runner_when_no_online_runner_exists() -> None:
respx.get("http://omnigent.test/v1/hosts").mock(
return_value=httpx.Response(200, json={"hosts": []})
)
launch = respx.post("http://omnigent.test/v1/hosts/host_1/runners").mock(
return_value=httpx.Response(200, json={"runner_id": "runner_launched"})
)
respx.get("http://omnigent.test/v1/runners/runner_launched/status").mock(
return_value=httpx.Response(200, json={"runner_id": "runner_launched", "online": True})
)
client = OmnigentClient(
"http://omnigent.test",
runner_workspace="/tmp/workspace",
runner_host_id="host_1",
)
try:
runner_id = await client.bind_random_runner("conv_1")
finally:
await client.aclose()
assert runner_id == "runner_launched"
assert launch.calls.last.request.read() == (
b'{"session_id":"conv_1","workspace":"/tmp/workspace"}'
)
@respx.mock
async def test_client_launches_runner_on_random_online_host() -> None:
respx.get("http://omnigent.test/v1/hosts").mock(
return_value=httpx.Response(
200,
json={
"hosts": [
{"id": "host_offline", "status": "offline"},
{"id": "host_online", "online": True},
]
},
)
)
launch = respx.post("http://omnigent.test/v1/hosts/host_online/runners").mock(
return_value=httpx.Response(200, json={"runner_id": "runner_launched"})
)
respx.get("http://omnigent.test/v1/runners/runner_launched/status").mock(
return_value=httpx.Response(200, json={"runner_id": "runner_launched", "online": True})
)
client = OmnigentClient("http://omnigent.test", runner_workspace="/tmp/workspace")
try:
runner_id = await client.bind_random_runner("conv_1")
finally:
await client.aclose()
assert runner_id == "runner_launched"
assert launch.called
@respx.mock
async def test_client_binds_runner_loaded_from_hosts() -> None:
respx.get("http://omnigent.test/v1/hosts").mock(
return_value=httpx.Response(
200,
json={
"hosts": [
{"id": "host_offline", "online": False, "runners": [{"id": "runner_no"}]},
{
"id": "host_online",
"online": True,
"runners": [{"runner_id": "runner_from_host"}],
},
]
},
)
)
bind = respx.patch("http://omnigent.test/v1/sessions/conv_1").mock(
return_value=httpx.Response(200, json={"id": "conv_1"})
)
client = OmnigentClient("http://omnigent.test")
try:
runner_id = await client.bind_random_runner("conv_1")
finally:
await client.aclose()
assert runner_id == "runner_from_host"
assert bind.calls.last.request.read() == b'{"runner_id":"runner_from_host"}'
@respx.mock
async def test_client_errors_when_no_runner_and_no_launch_workspace() -> None:
respx.get("http://omnigent.test/v1/hosts").mock(
return_value=httpx.Response(200, json={"hosts": []})
)
client = OmnigentClient("http://omnigent.test")
try:
try:
await client.bind_random_runner("conv_1")
except OmnigentError as exc:
message = str(exc)
else:
message = ""
finally:
await client.aclose()
assert "OMNIGENT_RUNNER_WORKSPACE" in message
@respx.mock
async def test_run_turn_streams_across_multiple_responses_until_session_idle() -> None:
# An orchestrator ends its first response to wait on a sub-agent, then
# resumes with the real answer in a second response. The turn is only over
# once the session settles to idle — `response.completed` alone must not
# cut the stream off after the "dispatched, waiting" message.
sse_body = (
'data: {"type":"response.output_text.delta","delta":"Explorer dispatched."}\n\n'
'data: {"type":"response.completed","response":{"status":"completed"}}\n\n'
'data: {"type":"response.output_text.delta","delta":"Here is the report."}\n\n'
'data: {"type":"response.completed","response":{"status":"completed"}}\n\n'
'data: {"type":"session.status","conversation_id":"conv_1","status":"idle"}\n\n'
"data: [DONE]\n\n"
)
respx.get("http://omnigent.test/v1/sessions/conv_1/stream").mock(
return_value=httpx.Response(200, text=sse_body)
)
respx.post("http://omnigent.test/v1/sessions/conv_1/events").mock(
return_value=httpx.Response(200, json={})
)
client = OmnigentClient("http://omnigent.test")
try:
deltas = [
event.get("delta")
async for event in client.run_turn("conv_1", "hello")
if event.get("type") == "response.output_text.delta"
]
finally:
await client.aclose()
# Both responses stream; the second (the real answer) is not dropped.
assert deltas == ["Explorer dispatched.", "Here is the report."]
@respx.mock
async def test_client_raises_runner_unavailable() -> None:
respx.post("http://omnigent.test/v1/sessions/conv_1/events").mock(
return_value=httpx.Response(
503,
json={"error": {"code": "runner_unavailable", "message": "No runner bound"}},
)
)
client = OmnigentClient("http://omnigent.test")
try:
try:
await client.submit_message("conv_1", "hello")
except RunnerUnavailableError:
raised = True
else:
raised = False
finally:
await client.aclose()
assert raised is True
-479
View File
@@ -1,479 +0,0 @@
import asyncio
from collections.abc import AsyncIterator
from pathlib import Path
from typing import Any
from omnigent_slack.models import ThreadKey
from omnigent_slack.service import SlackOmnigentService
from omnigent_slack.store import SQLiteStore
class FakeSlackClient:
def __init__(self) -> None:
self.posts: list[dict[str, Any]] = []
self.updates: list[dict[str, Any]] = []
async def chat_postMessage(self, **kwargs: Any) -> dict[str, Any]:
self.posts.append(kwargs)
return {"ok": True, "ts": f"bot-{len(self.posts)}"}
async def chat_update(self, **kwargs: Any) -> dict[str, Any]:
self.updates.append(kwargs)
return {"ok": True}
# Slack accepts up to 40,000 characters in a message ``text`` and rejects
# anything larger with ``msg_too_long`` (docs.slack.dev chat.postMessage).
SLACK_HARD_LIMIT = 40000
class LimitEnforcingSlackClient(FakeSlackClient):
"""Fake that mimics Slack rejecting oversized ``text`` with msg_too_long."""
async def chat_postMessage(self, **kwargs: Any) -> dict[str, Any]:
self._enforce_limit(kwargs)
return await super().chat_postMessage(**kwargs)
async def chat_update(self, **kwargs: Any) -> dict[str, Any]:
self._enforce_limit(kwargs)
return await super().chat_update(**kwargs)
@staticmethod
def _enforce_limit(kwargs: dict[str, Any]) -> None:
if len(str(kwargs.get("text") or "")) > SLACK_HARD_LIMIT:
raise RuntimeError("msg_too_long")
class FlakyUpdateSlackClient(FakeSlackClient):
"""Fake whose streaming ``chat_update`` calls fail before the final one.
The final delivery re-edits the placeholder to the first chunk; only the
interim progress updates raise, so we can prove a progress-update failure
never aborts the turn or clobbers the real answer.
"""
def __init__(self, fail_first: int = 1) -> None:
super().__init__()
self._remaining_failures = fail_first
async def chat_update(self, **kwargs: Any) -> dict[str, Any]:
if self._remaining_failures > 0:
self._remaining_failures -= 1
raise RuntimeError("msg_too_long")
return await super().chat_update(**kwargs)
class FakeOmnigentClient:
def __init__(self, final_text: str = "hello final") -> None:
self.created: list[tuple[str, str]] = []
self.bound: list[str] = []
self.turns: list[tuple[str, str]] = []
self.next_session_id = "conv_1"
self.final_text = final_text
async def create_session(self, agent_id: str, title: str) -> str:
self.created.append((agent_id, title))
return self.next_session_id
async def bind_random_runner(self, session_id: str) -> str:
self.bound.append(session_id)
return "runner_1"
async def run_turn(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
self.turns.append((session_id, text))
yield {"type": "response.output_text.delta", "delta": "hel"}
yield {"type": "response.output_text.delta", "delta": "lo"}
yield {
"type": "response.output_item.done",
"item": {
"type": "message",
"role": "assistant",
"content": [{"type": "output_text", "text": self.final_text}],
},
}
yield {"type": "response.completed", "response": {"status": "completed"}}
async def latest_assistant_text(self, session_id: str) -> str | None:
return None
async def _store(tmp_path: Path) -> SQLiteStore:
store = SQLiteStore(tmp_path / "store.sqlite3")
await store.initialize()
return store
async def _wait_for_updates(client: FakeSlackClient, count: int) -> None:
for _ in range(50):
if len(client.updates) >= count:
return
await asyncio.sleep(0.02)
raise AssertionError(f"Timed out waiting for {count} updates")
async def test_app_mention_creates_session_and_posts_response(tmp_path: Path) -> None:
store = await _store(tmp_path)
slack = FakeSlackClient()
omnigent = FakeOmnigentClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 2)
await service.shutdown()
key = ThreadKey(team_id="T1", channel_id="C1", thread_ts="100.1")
assert await store.get_session_id(key) == "conv_1"
assert omnigent.created[0][0] == "ag_1"
assert omnigent.bound == ["conv_1"]
assert omnigent.turns == [("conv_1", "hello")]
assert slack.posts[0]["thread_ts"] == "100.1"
assert slack.updates[-1]["text"] == "hello final"
async def test_long_answer_is_split_across_thread_replies(tmp_path: Path) -> None:
from omnigent_slack.text import SLACK_MESSAGE_CHAR_LIMIT
store = await _store(tmp_path)
slack = FakeSlackClient()
long_answer = "x" * (SLACK_MESSAGE_CHAR_LIMIT * 2 + 100)
omnigent = FakeOmnigentClient(final_text=long_answer)
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 1)
# Placeholder update + two overflow replies = original placeholder post + 2.
for _ in range(50):
if len(slack.posts) >= 3:
break
await asyncio.sleep(0.02)
await service.shutdown()
# Every message stays within Slack's limit and the full answer is preserved.
parts = [slack.updates[-1]["text"]]
parts.extend(post["text"] for post in slack.posts[1:])
assert all(len(part) <= SLACK_MESSAGE_CHAR_LIMIT for part in parts)
assert "".join(parts) == long_answer
# Overflow replies land in the same thread.
assert all(post["thread_ts"] == "100.1" for post in slack.posts[1:])
class StreamingLongAnswerClient(FakeOmnigentClient):
"""Streams a long answer as deltas, then reports it as the final item.
Mirrors the real session where an orchestrator streams a multi-thousand
character synthesis: the interim progress updates carry the whole
accumulated text, which overflows Slack's ``text`` ceiling.
"""
async def run_turn(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
self.turns.append((session_id, text))
# Two deltas so the accumulated progress text is multi-thousand chars.
half = "a" * 3000
yield {"type": "response.output_text.delta", "delta": half}
yield {"type": "response.output_text.delta", "delta": half}
yield {
"type": "response.output_item.done",
"item": {
"type": "message",
"role": "assistant",
"content": [{"type": "output_text", "text": self.final_text}],
},
}
yield {"type": "response.completed", "response": {"status": "completed"}}
async def test_streaming_progress_update_failure_does_not_clobber_answer(
tmp_path: Path,
) -> None:
store = await _store(tmp_path)
slack = FlakyUpdateSlackClient(fail_first=1)
omnigent = StreamingLongAnswerClient(final_text="the real answer")
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 1)
await service.shutdown()
# A failed progress update must not surface as an error answer.
assert slack.updates[-1]["text"] == "the real answer"
async def test_long_streamed_answer_never_exceeds_slack_limit(tmp_path: Path) -> None:
from omnigent_slack.text import SLACK_MESSAGE_CHAR_LIMIT
# Every delivered chunk stays within Slack's hard ceiling.
assert SLACK_MESSAGE_CHAR_LIMIT <= SLACK_HARD_LIMIT
store = await _store(tmp_path)
slack = LimitEnforcingSlackClient()
long_answer = "y" * (SLACK_MESSAGE_CHAR_LIMIT * 2 + 100)
omnigent = StreamingLongAnswerClient(final_text=long_answer)
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 1)
for _ in range(50):
if len(slack.posts) >= 3:
break
await asyncio.sleep(0.02)
await service.shutdown()
# No update or post exceeded the hard limit (else the fake would have
# raised), every chunk honors the best-practice ceiling, and the full
# answer was preserved across parts.
parts = [slack.updates[-1]["text"]]
parts.extend(post["text"] for post in slack.posts[1:])
assert all(len(part) <= SLACK_MESSAGE_CHAR_LIMIT for part in parts)
assert "".join(parts) == long_answer
async def test_turn_error_posts_separate_reply_and_keeps_answer(tmp_path: Path) -> None:
"""An error after content streamed must not erase the delivered answer.
The failure is reported as its own thread reply so the user keeps both the
real answer and the failure notice.
"""
store = await _store(tmp_path)
slack = FakeSlackClient()
class ErroringAfterAnswerClient(FakeOmnigentClient):
async def run_turn(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
self.turns.append((session_id, text))
yield {
"type": "response.output_item.done",
"item": {
"type": "message",
"role": "assistant",
"content": [{"type": "output_text", "text": self.final_text}],
},
}
yield {
"type": "response.failed",
"response": {"error": {"message": "boom"}},
}
omnigent = ErroringAfterAnswerClient(final_text="the real answer")
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 1)
for _ in range(50):
if len(slack.posts) >= 2:
break
await asyncio.sleep(0.02)
await service.shutdown()
# The placeholder holds the real answer, not the error.
assert slack.updates[-1]["text"] == "the real answer"
# The failure is a separate reply in the same thread.
failure_posts = [p for p in slack.posts if "failed" in str(p.get("text", ""))]
assert len(failure_posts) == 1
assert "boom" in failure_posts[0]["text"]
assert failure_posts[0]["thread_ts"] == "100.1"
async def test_turn_error_without_answer_uses_placeholder(tmp_path: Path) -> None:
"""When nothing streamed, the error surfaces in the placeholder itself."""
store = await _store(tmp_path)
slack = FakeSlackClient()
class ErroringNoAnswerClient(FakeOmnigentClient):
async def run_turn(self, session_id: str, text: str) -> AsyncIterator[dict[str, Any]]:
self.turns.append((session_id, text))
yield {
"type": "response.failed",
"response": {"error": {"message": "boom"}},
}
async def latest_assistant_text(self, session_id: str) -> str | None:
return None
omnigent = ErroringNoAnswerClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 1)
await service.shutdown()
assert "boom" in slack.updates[-1]["text"]
# No extra failure reply when there was no answer to preserve.
assert len(slack.posts) == 1
async def test_empty_app_mention_prompts_without_creating_session(tmp_path: Path) -> None:
store = await _store(tmp_path)
slack = FakeSlackClient()
omnigent = FakeOmnigentClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
)
await service.handle_app_mention(
body={"team_id": "T1", "event_id": "Ev1"},
event={"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1>"},
client=slack,
context={"bot_user_id": "B1"},
)
await service.shutdown()
assert omnigent.created == []
assert omnigent.bound == []
assert "Send a message" in slack.posts[0]["text"]
async def test_thread_reply_reuses_existing_session(tmp_path: Path) -> None:
store = await _store(tmp_path)
key = ThreadKey(team_id="T1", channel_id="C1", thread_ts="100.1")
await store.upsert_session(key, "conv_existing", "title")
slack = FakeSlackClient()
omnigent = FakeOmnigentClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_message(
body={"team_id": "T1", "event_id": "Ev2"},
event={
"channel": "C1",
"thread_ts": "100.1",
"ts": "101.1",
"user": "U1",
"text": "next",
},
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 2)
await service.shutdown()
assert omnigent.created == []
assert omnigent.bound == []
assert omnigent.turns == [("conv_existing", "next")]
assert slack.updates[-1]["text"] == "hello final"
async def test_duplicate_event_is_ignored(tmp_path: Path) -> None:
store = await _store(tmp_path)
slack = FakeSlackClient()
omnigent = FakeOmnigentClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
body = {"team_id": "T1", "event_id": "Ev1"}
event = {"channel": "C1", "ts": "100.1", "user": "U1", "text": "<@B1> hello"}
await service.handle_app_mention(
body=body,
event=event,
client=slack,
context={"bot_user_id": "B1"},
)
await service.handle_app_mention(
body=body,
event=event,
client=slack,
context={"bot_user_id": "B1"},
)
await _wait_for_updates(slack, 2)
await service.shutdown()
assert len(omnigent.turns) == 1
async def test_generic_message_with_bot_mention_is_ignored(tmp_path: Path) -> None:
store = await _store(tmp_path)
key = ThreadKey(team_id="T1", channel_id="C1", thread_ts="100.1")
await store.upsert_session(key, "conv_existing", "title")
slack = FakeSlackClient()
omnigent = FakeOmnigentClient()
service = SlackOmnigentService(
store=store,
omnigent=omnigent, # type: ignore[arg-type]
omnigent_agent_id="ag_1",
update_interval_seconds=0,
)
await service.handle_message(
body={"team_id": "T1", "event_id": "Ev2"},
event={
"channel": "C1",
"thread_ts": "100.1",
"ts": "101.1",
"user": "U1",
"text": "<@B1> next",
},
client=slack,
context={"bot_user_id": "B1"},
)
await service.shutdown()
assert omnigent.turns == []
assert slack.posts == []
-27
View File
@@ -1,27 +0,0 @@
from pathlib import Path
from omnigent_slack.models import ThreadKey
from omnigent_slack.store import SQLiteStore
async def test_store_persists_thread_session(tmp_path: Path) -> None:
store = SQLiteStore(tmp_path / "store.sqlite3")
await store.initialize()
key = ThreadKey(team_id="T1", channel_id="C1", thread_ts="100.1")
assert await store.get_session_id(key) is None
await store.upsert_session(key, "conv_1", "title")
assert await store.get_session_id(key) == "conv_1"
await store.upsert_session(key, "conv_2", "title")
assert await store.get_session_id(key) == "conv_2"
async def test_store_claim_event_dedupes(tmp_path: Path) -> None:
store = SQLiteStore(tmp_path / "store.sqlite3")
await store.initialize()
assert await store.claim_event("Ev1") is True
assert await store.claim_event("Ev1") is False
assert await store.claim_event(None) is True
-63
View File
@@ -1,63 +0,0 @@
from omnigent_slack.text import (
split_for_slack,
strip_bot_mention,
to_mrkdwn,
truncate_for_slack,
)
def test_to_mrkdwn_converts_markdown_to_slack_dialect() -> None:
result = to_mrkdwn("# Title\n\n**bold** and [link](https://example.com)")
# Bold collapses to single asterisks, headings lose '#', links become <url|text>.
assert "**" not in result
assert "*bold*" in result
assert "<https://example.com|link>" in result
assert "#" not in result
def test_to_mrkdwn_preserves_code_blocks() -> None:
result = to_mrkdwn("```python\nprint('hi')\n```")
assert "```" in result
assert "print('hi')" in result
def test_strip_bot_mention_removes_target_mention() -> None:
assert strip_bot_mention("<@B123> hello world", "B123") == "hello world"
def test_strip_bot_mention_falls_back_to_first_mention() -> None:
assert strip_bot_mention("<@B123> hello <@U456>", None) == "hello <@U456>"
def test_truncate_for_slack() -> None:
result = truncate_for_slack("a" * 20, limit=15)
assert result.endswith("[truncated]")
assert len(result) <= 15
def test_split_for_slack_returns_single_chunk_when_within_limit() -> None:
assert split_for_slack("hello", limit=10) == ["hello"]
def test_split_for_slack_empty_yields_one_empty_chunk() -> None:
assert split_for_slack("", limit=10) == [""]
def test_split_for_slack_preserves_all_content_and_respects_limit() -> None:
text = "\n".join(f"line {i}" for i in range(200))
chunks = split_for_slack(text, limit=40)
assert all(len(chunk) <= 40 for chunk in chunks)
assert "".join(chunks) == text
def test_split_for_slack_breaks_on_whitespace_when_possible() -> None:
chunks = split_for_slack("aaaa bbbb cccc", limit=6)
# Breaks after a space rather than mid-word.
assert chunks[0] == "aaaa "
assert "".join(chunks) == "aaaa bbbb cccc"
def test_split_for_slack_hard_cuts_runs_without_whitespace() -> None:
text = "a" * 25
chunks = split_for_slack(text, limit=10)
assert chunks == ["a" * 10, "a" * 10, "a" * 5]
-1214
View File
File diff suppressed because it is too large Load Diff
-6
View File
@@ -1,6 +0,0 @@
"""API schemas (protobuf) for omnigent services.
Each API lives in its own subpackage, versioned to mirror its proto ``package``
declaration — e.g. the routing API's ``package omnigent.api.routing.v1`` maps to
``omnigent/api/routing/v1/``. A new major version is a sibling ``vN`` package.
"""
-2
View File
@@ -1,2 +0,0 @@
"""AI-gateway routing API schema. Versioned subpackages (``v1``, ...) hold the
proto and its generated bindings."""
-2
View File
@@ -1,2 +0,0 @@
"""Routing API v1 (``package omnigent.api.routing.v1``): the ``routing.proto``
schema and its generated ``routing_pb2`` bindings."""
-110
View File
@@ -1,110 +0,0 @@
syntax = "proto3";
// Routing API for AI gateways.
//
// This schema is versioned independently of any particular gateway
// implementation so that the request and response contracts can evolve
// (v1, v2, ...) without being coupled to a gateway's release cycle, while
// still being served under a gateway's existing API scope.
package omnigent.api.routing.v1;
import "google/protobuf/struct.proto";
// A candidate destination a request may be routed to.
message RouteOption {
// Identifier of the model to serve the request, e.g. "gpt-5-5".
optional string model = 1;
// Harness that drives the model. May be omitted for a native harness;
// required when the model is served through a meta-harness.
optional string harness = 2;
}
// Selects the routing strategy to apply and its configuration.
//
// A gateway resolves `router_name` to a routing implementation and passes
// `config` to it. Both the set of available routers and their configuration
// are gateway-defined, so `config` is an opaque structure rather than a fixed
// message. Examples:
//
// Server-side judge model (implementation is a gateway-side black box):
// {
// "router_name": "judge_v20260708",
// "config": {
// "user_guidance": "prefer the most capable model available",
// "org_guidance": "keep cost reasonable"
// }
// }
//
// Fixed model and harness:
// {
// "router_name": "fixed",
// "config": { "model": "gpt-5-5", "harness": "codex" }
// }
//
// User-supplied lambda:
// {
// "router_name": "python_lambda",
// "config": { "lambda": "gpt-5-5 if len(prompt) > 10 else kimi" }
// }
message RouteSelector {
// Name of the routing strategy to invoke.
optional string router_name = 1;
// Router-specific configuration, interpreted by the selected router.
optional google.protobuf.Struct config = 2;
}
// The routing decision produced for a single request.
message RouteSelection {
// The chosen destination.
optional RouteOption route_option = 1;
// Router-specific parameters emitted alongside the decision, interpreted
// by the caller or the serving path.
optional google.protobuf.Struct params = 2;
}
// A single unit of work submitted for routing.
message Task {
// The prompt to be served.
optional string prompt = 1;
}
// One prior turn in a session: the task that was submitted and the routing
// decision made for it.
message SessionTurn {
optional Task task = 1;
optional RouteSelection route_selection = 2;
}
// The routing history for a session, ordered oldest to newest. Routers may
// use it to keep successive turns consistent.
message SessionHistory {
repeated SessionTurn session_turns = 1;
}
// Request to select a route for a task.
message SelectRouteRequest {
// Candidate destinations the router may choose from.
repeated RouteOption route_options = 1;
// The task to route.
optional Task task = 2;
// The routing strategy to apply. Required in practice; a gateway rejects a
// request that omits it.
optional RouteSelector route_selector = 3;
// Prior turns in the session, when available.
optional SessionHistory session_history = 4;
}
// Response containing the selected route.
message SelectRouteResponse {
// The routing decision.
repeated RouteSelection route_selection = 1;
// Human-readable explanation of why this route was selected.
optional string rationale = 2;
}
-51
View File
@@ -1,51 +0,0 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# NO CHECKED-IN PROTOBUF GENCODE
# source: omnigent/api/routing/v1/routing.proto
# Protobuf Python Version: 6.33.5
"""Generated protocol buffer code."""
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import runtime_version as _runtime_version
from google.protobuf import symbol_database as _symbol_database
from google.protobuf.internal import builder as _builder
_runtime_version.ValidateProtobufRuntimeVersion(
_runtime_version.Domain.PUBLIC,
6,
33,
5,
'',
'omnigent/api/routing/v1/routing.proto'
)
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
from google.protobuf import struct_pb2 as google_dot_protobuf_dot_struct__pb2
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n%omnigent/api/routing/v1/routing.proto\x12\x17omnigent.api.routing.v1\x1a\x1cgoogle/protobuf/struct.proto\"M\n\x0bRouteOption\x12\x12\n\x05model\x18\x01 \x01(\tH\x00\x88\x01\x01\x12\x14\n\x07harness\x18\x02 \x01(\tH\x01\x88\x01\x01\x42\x08\n\x06_modelB\n\n\x08_harness\"r\n\rRouteSelector\x12\x18\n\x0brouter_name\x18\x01 \x01(\tH\x00\x88\x01\x01\x12,\n\x06\x63onfig\x18\x02 \x01(\x0b\x32\x17.google.protobuf.StructH\x01\x88\x01\x01\x42\x0e\n\x0c_router_nameB\t\n\x07_config\"\x9b\x01\n\x0eRouteSelection\x12?\n\x0croute_option\x18\x01 \x01(\x0b\x32$.omnigent.api.routing.v1.RouteOptionH\x00\x88\x01\x01\x12,\n\x06params\x18\x02 \x01(\x0b\x32\x17.google.protobuf.StructH\x01\x88\x01\x01\x42\x0f\n\r_route_optionB\t\n\x07_params\"&\n\x04Task\x12\x13\n\x06prompt\x18\x01 \x01(\tH\x00\x88\x01\x01\x42\t\n\x07_prompt\"\xa3\x01\n\x0bSessionTurn\x12\x30\n\x04task\x18\x01 \x01(\x0b\x32\x1d.omnigent.api.routing.v1.TaskH\x00\x88\x01\x01\x12\x45\n\x0froute_selection\x18\x02 \x01(\x0b\x32\'.omnigent.api.routing.v1.RouteSelectionH\x01\x88\x01\x01\x42\x07\n\x05_taskB\x12\n\x10_route_selection\"M\n\x0eSessionHistory\x12;\n\rsession_turns\x18\x01 \x03(\x0b\x32$.omnigent.api.routing.v1.SessionTurn\"\xbf\x02\n\x12SelectRouteRequest\x12;\n\rroute_options\x18\x01 \x03(\x0b\x32$.omnigent.api.routing.v1.RouteOption\x12\x30\n\x04task\x18\x02 \x01(\x0b\x32\x1d.omnigent.api.routing.v1.TaskH\x00\x88\x01\x01\x12\x43\n\x0eroute_selector\x18\x03 \x01(\x0b\x32&.omnigent.api.routing.v1.RouteSelectorH\x01\x88\x01\x01\x12\x45\n\x0fsession_history\x18\x04 \x01(\x0b\x32\'.omnigent.api.routing.v1.SessionHistoryH\x02\x88\x01\x01\x42\x07\n\x05_taskB\x11\n\x0f_route_selectorB\x12\n\x10_session_history\"}\n\x13SelectRouteResponse\x12@\n\x0froute_selection\x18\x01 \x03(\x0b\x32\'.omnigent.api.routing.v1.RouteSelection\x12\x16\n\trationale\x18\x02 \x01(\tH\x00\x88\x01\x01\x42\x0c\n\n_rationaleb\x06proto3')
_globals = globals()
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'omnigent.api.routing.v1.routing_pb2', _globals)
if not _descriptor._USE_C_DESCRIPTORS:
DESCRIPTOR._loaded_options = None
_globals['_ROUTEOPTION']._serialized_start=96
_globals['_ROUTEOPTION']._serialized_end=173
_globals['_ROUTESELECTOR']._serialized_start=175
_globals['_ROUTESELECTOR']._serialized_end=289
_globals['_ROUTESELECTION']._serialized_start=292
_globals['_ROUTESELECTION']._serialized_end=447
_globals['_TASK']._serialized_start=449
_globals['_TASK']._serialized_end=487
_globals['_SESSIONTURN']._serialized_start=490
_globals['_SESSIONTURN']._serialized_end=653
_globals['_SESSIONHISTORY']._serialized_start=655
_globals['_SESSIONHISTORY']._serialized_end=732
_globals['_SELECTROUTEREQUEST']._serialized_start=735
_globals['_SELECTROUTEREQUEST']._serialized_end=1054
_globals['_SELECTROUTERESPONSE']._serialized_start=1056
_globals['_SELECTROUTERESPONSE']._serialized_end=1181
# @@protoc_insertion_point(module_scope)
-72
View File
@@ -1,72 +0,0 @@
from google.protobuf import struct_pb2 as _struct_pb2
from google.protobuf.internal import containers as _containers
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from collections.abc import Iterable as _Iterable, Mapping as _Mapping
from typing import ClassVar as _ClassVar, Optional as _Optional, Union as _Union
DESCRIPTOR: _descriptor.FileDescriptor
class RouteOption(_message.Message):
__slots__ = ("model", "harness")
MODEL_FIELD_NUMBER: _ClassVar[int]
HARNESS_FIELD_NUMBER: _ClassVar[int]
model: str
harness: str
def __init__(self, model: _Optional[str] = ..., harness: _Optional[str] = ...) -> None: ...
class RouteSelector(_message.Message):
__slots__ = ("router_name", "config")
ROUTER_NAME_FIELD_NUMBER: _ClassVar[int]
CONFIG_FIELD_NUMBER: _ClassVar[int]
router_name: str
config: _struct_pb2.Struct
def __init__(self, router_name: _Optional[str] = ..., config: _Optional[_Union[_struct_pb2.Struct, _Mapping]] = ...) -> None: ...
class RouteSelection(_message.Message):
__slots__ = ("route_option", "params")
ROUTE_OPTION_FIELD_NUMBER: _ClassVar[int]
PARAMS_FIELD_NUMBER: _ClassVar[int]
route_option: RouteOption
params: _struct_pb2.Struct
def __init__(self, route_option: _Optional[_Union[RouteOption, _Mapping]] = ..., params: _Optional[_Union[_struct_pb2.Struct, _Mapping]] = ...) -> None: ...
class Task(_message.Message):
__slots__ = ("prompt",)
PROMPT_FIELD_NUMBER: _ClassVar[int]
prompt: str
def __init__(self, prompt: _Optional[str] = ...) -> None: ...
class SessionTurn(_message.Message):
__slots__ = ("task", "route_selection")
TASK_FIELD_NUMBER: _ClassVar[int]
ROUTE_SELECTION_FIELD_NUMBER: _ClassVar[int]
task: Task
route_selection: RouteSelection
def __init__(self, task: _Optional[_Union[Task, _Mapping]] = ..., route_selection: _Optional[_Union[RouteSelection, _Mapping]] = ...) -> None: ...
class SessionHistory(_message.Message):
__slots__ = ("session_turns",)
SESSION_TURNS_FIELD_NUMBER: _ClassVar[int]
session_turns: _containers.RepeatedCompositeFieldContainer[SessionTurn]
def __init__(self, session_turns: _Optional[_Iterable[_Union[SessionTurn, _Mapping]]] = ...) -> None: ...
class SelectRouteRequest(_message.Message):
__slots__ = ("route_options", "task", "route_selector", "session_history")
ROUTE_OPTIONS_FIELD_NUMBER: _ClassVar[int]
TASK_FIELD_NUMBER: _ClassVar[int]
ROUTE_SELECTOR_FIELD_NUMBER: _ClassVar[int]
SESSION_HISTORY_FIELD_NUMBER: _ClassVar[int]
route_options: _containers.RepeatedCompositeFieldContainer[RouteOption]
task: Task
route_selector: RouteSelector
session_history: SessionHistory
def __init__(self, route_options: _Optional[_Iterable[_Union[RouteOption, _Mapping]]] = ..., task: _Optional[_Union[Task, _Mapping]] = ..., route_selector: _Optional[_Union[RouteSelector, _Mapping]] = ..., session_history: _Optional[_Union[SessionHistory, _Mapping]] = ...) -> None: ...
class SelectRouteResponse(_message.Message):
__slots__ = ("route_selection", "rationale")
ROUTE_SELECTION_FIELD_NUMBER: _ClassVar[int]
RATIONALE_FIELD_NUMBER: _ClassVar[int]
route_selection: _containers.RepeatedCompositeFieldContainer[RouteSelection]
rationale: str
def __init__(self, route_selection: _Optional[_Iterable[_Union[RouteSelection, _Mapping]]] = ..., rationale: _Optional[str] = ...) -> None: ...
+28 -43
View File
@@ -58,12 +58,6 @@ from omnigent.harness_aliases import canonicalize_harness
from omnigent.inner import _proc
from omnigent.inner.databricks_executor import _DatabricksBearerAuth, _read_databrickscfg
from omnigent.native_coding_agents import native_coding_agent_for_wrapper_label
from omnigent.process_logging import (
PROCESS_LOG_FILE_ENV_VAR,
child_logging_popen_kwargs,
logs_root,
open_process_log_file,
)
from omnigent.spec import load as load_spec
from omnigent.spec._omnigent_compat import OMNIGENT_EXECUTOR_TYPE
from omnigent.spec.parser import discover_host_skills
@@ -3229,16 +3223,6 @@ def _apply_overrides_to_raw(raw: _YamlMapping, overrides: ChatOverrides) -> None
executor_block["model"] = overrides.model
if overrides.harness is not None:
_apply_harness_override_to_executor(raw, executor_block, overrides.harness)
# A harness-only override drops any prior model pin so the new
# harness resolves its provider default — e.g. ``omnigent run
# examples/polly --harness pi`` must not keep Polly's Claude-only
# a Claude-only ``executor.model``. An explicit ``--model``
# (applied above) wins and is left alone.
if overrides.model is None:
executor_block.pop("model", None)
llm_block = raw.get("llm")
if isinstance(llm_block, dict):
llm_block.pop("model", None)
# When neither harness nor model is declared — after overrides —
# inject the ad-hoc default. Gated on harness absence so a YAML
# like ``claude_code_agent.yaml`` (declares harness, no model)
@@ -3436,7 +3420,7 @@ def _omnigent_log_dir() -> Path:
:returns: ``~/.omnigent/logs``, created if needed.
"""
log_dir = logs_root()
log_dir = Path.home() / ".omnigent" / "logs"
log_dir.mkdir(parents=True, exist_ok=True)
return log_dir
@@ -3505,7 +3489,11 @@ def _start_local_server(
:returns: The server handle bundling the subprocess and
the path to its captured stdout/stderr log file.
"""
log_path, log_fh = open_process_log_file("server", root=_omnigent_log_dir())
log_dir = _omnigent_log_dir() / "server"
log_dir.mkdir(parents=True, exist_ok=True)
log_fd, log_name = tempfile.mkstemp(prefix="server-", suffix=".log", dir=log_dir)
log_path = Path(log_name)
log_fh = os.fdopen(log_fd, "wb")
if ephemeral:
data_tmpdir = tempfile.mkdtemp(prefix="ap-chat-data-")
db_path = Path(data_tmpdir) / "chat.db"
@@ -3546,7 +3534,6 @@ def _start_local_server(
child_env = {
**os.environ,
"OMNIGENT_RUNNER_TUNNEL_TOKEN": binding_token,
PROCESS_LOG_FILE_ENV_VAR: str(log_path),
# Single-user loopback runtime — see ensure_local_omnigent_server for why
# this lets the host tunnel re-own this machine's host_id across an
# auth-mode flip without weakening the deployed multi-user boundary.
@@ -3579,30 +3566,28 @@ def _start_local_server(
child_env["DATABRICKS_CONFIG_PROFILE"] = _spec.executor.profile
try:
with child_logging_popen_kwargs(child_env) as logging_kwargs:
server_proc = subprocess.Popen(
[
sys.executable,
"-m",
"omnigent.cli",
"server",
"--host",
"127.0.0.1",
"--port",
str(port),
"--database-uri",
f"sqlite:///{db_path}",
"--artifact-location",
str(artifact_path),
"--agent",
str(agent_path),
],
env=child_env,
stdout=log_fh,
stderr=log_fh,
**_proc.spawn_kwargs(),
**logging_kwargs,
)
server_proc = subprocess.Popen(
[
sys.executable,
"-m",
"omnigent.cli",
"server",
"--host",
"127.0.0.1",
"--port",
str(port),
"--database-uri",
f"sqlite:///{db_path}",
"--artifact-location",
str(artifact_path),
"--agent",
str(agent_path),
],
env=child_env,
stdout=log_fh,
stderr=log_fh,
**_proc.spawn_kwargs(),
)
finally:
log_fh.close()
+2 -86
View File
@@ -322,16 +322,6 @@ def build_native_claude_terminal_env(
terminal_env.update(claude_config.env)
terminal_env[_CLAUDE_CODE_ENABLE_TOOL_SEARCH_ENV] = "true"
terminal_env[_CLAUDE_CODE_DISABLE_AGENT_VIEW_ENV] = "1"
# On the apiKeyHelper path the credential reaches Claude Code via the
# helper; a raw ANTHROPIC_API_KEY here re-triggers Claude Code's "Detected a
# custom API key" menu, which hangs tmux delivery. Fail loud if one leaks.
if claude_config is not None and claude_config.api_key_helper:
if _ANTHROPIC_API_KEY_ENV in terminal_env:
raise RuntimeError(
"native-claude: apiKeyHelper is configured but the terminal env "
f"carries a raw {_ANTHROPIC_API_KEY_ENV}; the credential must reach "
"Claude Code via the helper, not the environment."
)
return terminal_env
@@ -3671,25 +3661,17 @@ def _claude_transcript_record_from_session_item(
if not isinstance(output, str):
output = "" if output is None else json.dumps(output, separators=(",", ":"))
record_type = "user"
# Image (and other structured) tool results are persisted as a
# stringified content-block array. Rehydrate them into real blocks
# so ``claude --resume`` sends screenshots as images — not as ~250K
# tokens of base64 text — and the model actually sees them again.
content_blocks = _claude_tool_result_content_blocks(output)
content: str | list[dict[str, Any]] = (
content_blocks if content_blocks is not None else output
)
message = {
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": call_id,
"content": content,
"content": output,
}
],
}
extra["toolUseResult"] = _json_safe_tool_use_result(output)
extra["toolUseResult"] = output
else:
return None
return {
@@ -3809,72 +3791,6 @@ def _json_object_from_string(value: object) -> dict[str, Any]:
return parsed if isinstance(parsed, dict) else {}
def _json_safe_tool_use_result(output: str) -> str:
"""
Return a ``toolUseResult`` value Claude Code can ``JSON.parse``.
Some built-in result renderers (notably ``TaskOutput``) call
``JSON.parse`` on ``toolUseResult`` when the transcript is resumed.
A raw display string such as ``"<retrieval_status>timeout</...>"``
throws ``JSON Parse error: Unrecognized token '<'`` at TUI boot,
before the input prompt renders — so the whole resume fails and the
first web-UI message is never delivered.
Outputs that are already JSON (e.g. an image content-block array)
pass through verbatim; anything else is wrapped as a JSON string
literal so the parse always succeeds. The plain-text output still
lives verbatim in the ``tool_result`` content block, so this does
not change what the model or the web UI sees.
:param output: The tool result string synthesized for the
transcript, e.g. ``"<retrieval_status>timeout</...>"`` or
``'[{"type":"image",...}]'``.
:returns: A JSON-parseable string for the record's
``toolUseResult`` field.
"""
try:
json.loads(output)
except (json.JSONDecodeError, ValueError):
return json.dumps(output)
return output
def _claude_tool_result_content_blocks(output: str) -> list[dict[str, Any]] | None:
"""
Rehydrate a stringified content-block array into real blocks.
Tool results that return image content are persisted as a JSON *string*
like ``'[{"type":"image","source":{...}}]'``. Passing that string
straight into a ``tool_result`` content block makes ``claude --resume``
send the base64 to the API as plain text — a single screenshot balloons
to ~250K text tokens instead of the ~1.5K an image block costs, which is
what pushes a resumed conversation over the context limit.
Only ``text`` and ``image`` blocks are rehydrated: those are the block
types the API accepts inside a ``tool_result``. Anything else (plain
text, or a JSON array of some other shape) stays a raw string so the
resume request keeps sending exactly what it did before.
:param output: The persisted tool-result string, e.g.
``'[{"type":"image","source":{"type":"base64","data":"..."}}]'``
or plain text like ``"file written"``.
:returns: A list of content blocks when *output* parses to a non-empty
list of ``text``/``image`` block dicts; ``None`` otherwise, so the
caller keeps the raw string as the block content.
"""
try:
parsed = json.loads(output)
except (json.JSONDecodeError, ValueError):
return None
if not isinstance(parsed, list) or not parsed:
return None
if not all(
isinstance(block, dict) and block.get("type") in ("text", "image") for block in parsed
):
return None
return parsed
def _preflight_local_tools(command: str) -> None:
"""
Verify local executables required by the native Claude wrapper.
+2 -82
View File
@@ -124,10 +124,6 @@ _TMUX_SEND_TIMEOUT_S = 5.0
# The glyph persists while Claude is busy responding, so its presence
# means "input box mounted" (not "idle"), which is what injection needs.
_CLAUDE_PROMPT_GLYPH = ""
# Matches a selected numbered menu row (`` 2. No (recommended)``): the glyph
# followed by a numbered choice, which the chat input never renders. Used to
# exclude startup menus from the readiness scan (see ``_is_selected_menu_row``).
_SELECTED_MENU_ROW_RE = re.compile(rf"{_CLAUDE_PROMPT_GLYPH}\s*\d+\.\s")
# Box-drawing glyphs Claude Code's input-box frame is made of. A line of
# these below ```` marks the live input box (see ``_is_box_rule``),
# distinguishing it from a bare prompt echoed into scrollback.
@@ -1090,9 +1086,6 @@ def build_hook_settings(
ap_server_url: str | None = None,
ap_auth_headers: dict[str, str] | None = None,
api_key_helper: str | None = None,
launch_model: str | None = None,
launch_permission_mode: str | None = None,
launch_effort: str | None = None,
) -> dict[str, Any]:
"""
Build invocation-local Claude Code hook settings.
@@ -1112,15 +1105,6 @@ def build_hook_settings(
:param api_key_helper: Optional Claude Code ``apiKeyHelper``
command from ucode state, e.g. ``"databricks auth token
--host https://example.databricks.com ..."``.
:param launch_model: Effective launch model from ``--model``. Mirrored
into the invocation-local settings sidecar so a wrapped Claude Code
re-exec that preserves ``--settings`` but rebuilds argv cannot fall
back to the user's global default model.
:param launch_permission_mode: Effective launch permission mode from
``--permission-mode``. Mirrored into ``permissions.defaultMode``
for the same re-exec hardening.
:param launch_effort: Effective launch effort from ``--effort``.
Mirrored into ``effortLevel`` for restart/re-exec parity.
:returns: JSON-serializable Claude settings fragment.
"""
python = python_executable or sys.executable
@@ -1306,12 +1290,6 @@ def build_hook_settings(
# prompts, since both fire UserPromptSubmit.
hooks["UserPromptSubmit"].append({"hooks": [evaluate_policy_hook]})
settings: dict[str, Any] = {"hooks": hooks}
if launch_model:
settings["model"] = launch_model
if launch_permission_mode:
settings["permissions"] = {"defaultMode": launch_permission_mode}
if launch_effort and launch_effort in CLAUDE_EFFORTS:
settings["effortLevel"] = launch_effort
if api_key_helper:
settings["apiKeyHelper"] = api_key_helper
# Override Claude Code's statusLine so we receive its stdin (the
@@ -1408,9 +1386,6 @@ def augment_claude_args(
ap_server_url=ap_server_url,
ap_auth_headers=ap_auth_headers,
api_key_helper=api_key_helper,
launch_model=_arg_value(claude_args, "--model"),
launch_permission_mode=_arg_value(claude_args, "--permission-mode"),
launch_effort=_arg_value(claude_args, "--effort"),
)
args = _merge_disallowed_tools(list(claude_args), _OMNIGENT_DISALLOWED_TOOLS)
args.extend(
@@ -1431,32 +1406,6 @@ def augment_claude_args(
return args
def _arg_value(args: tuple[str, ...], flag: str) -> str | None:
"""Return the effective CLI flag value from ``args``.
Supports both ``--flag value`` and ``--flag=value`` spellings. When a
flag appears more than once, the last valid occurrence wins, matching the
usual CLI precedence for repeated long options.
:param args: Claude CLI args, e.g. ``("--model", "sonnet")``.
:param flag: Long flag to read, e.g. ``"--model"``.
:returns: The flag value, or ``None`` when absent/empty.
"""
joined_prefix = f"{flag}="
value: str | None = None
for idx, arg in enumerate(args):
if arg.startswith(joined_prefix):
candidate = arg[len(joined_prefix) :]
if candidate:
value = candidate
continue
if arg == flag and idx + 1 < len(args):
candidate = args[idx + 1]
if candidate and not candidate.startswith("--"):
value = candidate
return value
def _merge_disallowed_tools(args: list[str], extra: tuple[str, ...]) -> list[str]:
"""
Add ``extra`` tool names to a ``--disallowedTools`` flag in ``args``.
@@ -2933,24 +2882,12 @@ def _claude_prompt_rendered(pane: str) -> bool:
it's framed by a box rule — the ``────`` closing line the live input
box always renders below ```` but a bare echoed prompt never has.
A bare ```` on a selected numbered menu row is not the chat input. A
numbered line with an input-box rule below it still counts, however: the
readiness gate runs before every injection, so a restored composer draft
may legitimately begin with text such as ``2. buy milk``.
:param pane: Captured pane text from :func:`_capture_pane`.
:returns: ``True`` when the input box appears mounted.
"""
non_empty = [line for line in pane.splitlines() if line.strip()]
tail_start = max(0, len(non_empty) - _PROMPT_SCAN_TAIL_LINES)
for idx in range(tail_start, len(non_empty)):
line = non_empty[idx]
if _CLAUDE_PROMPT_GLYPH not in line:
continue
if not _is_selected_menu_row(line) or any(
_is_box_rule(rule) for rule in non_empty[idx + 1 :]
):
return True
if any(_CLAUDE_PROMPT_GLYPH in line for line in non_empty[-_PROMPT_SCAN_TAIL_LINES:]):
return True
# Above that window, trust the glyph only when a box rule sits below
# it — the live input box's closing frame, absent from scrollback.
# The footer height scales with concurrent subagents (a fan-out of
@@ -2965,23 +2902,6 @@ def _claude_prompt_rendered(pane: str) -> bool:
return False
def _is_selected_menu_row(line: str) -> bool:
"""
Return whether a ```` line is a selected numbered menu row.
Claude Code's startup menus (e.g. the "Detected a custom API key"
confirmation) mark the highlighted choice with the same ```` glyph the
chat input uses (`` 2. No (recommended)``). The readiness scan must not
treat such a row as the chat composer, or the first message gets typed
into the menu. A chat prompt never renders a numbered choice after the
glyph, so the ``<glyph> <digit>.`` shape distinguishes them.
:param line: A single pane line, e.g. ``" 2. No (recommended)"``.
:returns: ``True`` when the line is a selected numbered menu choice.
"""
return bool(_SELECTED_MENU_ROW_RE.match(line.strip()))
def _is_box_rule(line: str) -> bool:
"""
Return whether a line is a TUI box-drawing horizontal rule.
+4 -15
View File
@@ -119,23 +119,12 @@ def _state_dir_for_conversation_id(conversation_id: str) -> Path:
every byte that lands in the path is hex, so the result is
always a single child of the state root.
Sessions created before ids dropped the ``conv_`` prefix hashed the
prefixed string, so their directories live under the legacy digest; when
the bare-digest directory is absent, the legacy one is returned (never
renamed files inside may embed their own absolute path).
:param conversation_id: Omnigent conversation id, bare 32-char hex
(a legacy ``conv_``-prefixed form is accepted and normalised).
:param conversation_id: Omnigent conversation id, e.g.
``"conv_abc123"``.
:returns: Absolute directory path; not guaranteed to exist.
"""
bare = conversation_id.removeprefix("conv_")
root = _claude_native_state_root()
state_dir = root / hashlib.sha256(bare.encode("utf-8")).hexdigest()[:_ID_HASH_CHARS]
if not state_dir.exists():
legacy = root / hashlib.sha256(f"conv_{bare}".encode()).hexdigest()[:_ID_HASH_CHARS]
if legacy.exists():
return legacy
return state_dir
digest = hashlib.sha256(conversation_id.encode("utf-8")).hexdigest()[:_ID_HASH_CHARS]
return _claude_native_state_root() / digest
def write_launch_state(conversation_id: str, working_directory: str) -> None:
+150 -659
View File
File diff suppressed because it is too large Load Diff
+2 -41
View File
@@ -226,37 +226,6 @@ def load_databricks_org_id(server_url: str) -> str | None:
DATABRICKS_ORG_ID_HEADER = "X-Databricks-Org-Id"
# Opaque extra request headers for dev/test: a JSON object of header name→value
# in :data:`DATABRICKS_EXTRA_HEADERS_ENV_VAR`. Databricks deployments use it to
# carry request-routing selector headers so a request pins to a specific server
# instance/replica instead of the default one. Folded into
# :func:`databricks_request_headers` below so it travels with every
# client→server connection built through that one helper — a per-call-site
# bearer that skips this helper misses the selectors. Unset in prod.
DATABRICKS_EXTRA_HEADERS_ENV_VAR = "OMNIGENT_DATABRICKS_EXTRA_HEADERS"
def _databricks_extra_headers() -> dict[str, str]:
"""Return the opaque extra request headers when configured, else ``{}``.
Reads :data:`DATABRICKS_EXTRA_HEADERS_ENV_VAR`, a JSON object of header
namevalue. Missing or malformed (unset, not JSON, or not an object)
``{}``, so production and local runs are unaffected.
:returns: A header dict parsed from the env var, or an empty dict.
"""
raw = os.environ.get(DATABRICKS_EXTRA_HEADERS_ENV_VAR, "").strip()
if not raw:
return {}
try:
parsed = json.loads(raw)
except (json.JSONDecodeError, ValueError):
return {}
if not isinstance(parsed, dict):
return {}
return {str(key): str(value) for key, value in parsed.items()}
def databricks_request_headers(
server_url: str, *, bearer_token: str | None = None
) -> dict[str, str]:
@@ -274,17 +243,12 @@ def databricks_request_headers(
Both values are omitted when absent, so single-workspace and
local-unauthenticated callers get ``{}`` and are unaffected.
Also folds in any opaque dev/test headers from
:data:`DATABRICKS_EXTRA_HEADERS_ENV_VAR` (request-routing selectors set by
some Databricks deployments) so every chokepoint that builds headers through
this one helper carries them when set.
:param server_url: The server URL, e.g.
``"https://example.databricks.com/api/2.0/omnigent"``.
:param bearer_token: The workspace bearer token, or ``None`` when the
credential is supplied by a separate mechanism (or there is none).
:returns: A header dict carrying ``Authorization``, ``X-Databricks-Org-Id``,
and/or the configured extra headers as available, possibly empty.
:returns: A header dict carrying ``Authorization`` and/or
``X-Databricks-Org-Id`` as available, possibly empty.
"""
headers: dict[str, str] = {}
if bearer_token:
@@ -292,9 +256,6 @@ def databricks_request_headers(
org_id = load_databricks_org_id(server_url)
if org_id:
headers[DATABRICKS_ORG_ID_HEADER] = org_id
# Opaque dev/test extra headers (request-routing selectors); no-op in prod
# (env unset).
headers.update(_databricks_extra_headers())
return headers
+13 -28
View File
@@ -2,7 +2,7 @@
Always-on CLI diagnostics log.
Captures exceptions, warnings, and diagnostic info to a per-invocation
log file under ``<data-dir>/logs/cli/cli-*.log``. Separate from the
log file under ``~/.omnigent/logs/cli-*.log``. Separate from the
``--log`` conversation JSON transcript and the ``--debug-events`` SSE
tape this layer is always on so crash context is available even when
the user didn't know to enable debugging ahead of time.
@@ -36,20 +36,14 @@ from logging.handlers import RotatingFileHandler
from pathlib import Path
from typing import cast
from omnigent.process_logging import (
TerminalLogFormatter,
effective_log_level,
env_truthy,
process_log_dir,
terminal_supports_color,
)
from omnigent_ui_sdk import state_dir
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
#: Destination subdirectory under ``<data-dir>/logs`` for CLI diagnostics.
_LOG_DESTINATION = "cli"
#: Subdirectory under :func:`state_dir` for CLI diagnostic logs.
_LOGS_SUBDIR = "logs"
#: Maximum number of ``cli-*.log`` files kept before pruning.
MAX_LOG_FILES = 20
@@ -138,7 +132,7 @@ def _redact(text: str) -> str:
return text
class _RedactingFormatter(TerminalLogFormatter):
class _RedactingFormatter(logging.Formatter):
"""
Formatter that scrubs obvious secrets from the *final* formatted
output after ``%``-interpolation of ``record.args`` and after
@@ -230,12 +224,12 @@ def _log_dir() -> Path:
"""
Return the CLI diagnostics log directory.
Uses the shared Omnigent runtime data dir so ``OMNIGENT_DATA_DIR``
isolates diagnostics with the DB, artifacts, and process logs.
Uses :func:`omnigent_ui_sdk.state_dir` as the shared
``~/.omnigent`` root so the path is defined in one place.
:returns: ``<data-dir>/logs/cli``.
:returns: ``~/.omnigent/logs``.
"""
return process_log_dir(_LOG_DESTINATION)
return Path(state_dir()) / _LOGS_SUBDIR
def setup_cli_logging(argv: list[str]) -> CliLogContext:
@@ -267,38 +261,29 @@ def setup_cli_logging(argv: list[str]) -> CliLogContext:
log_path = log_dir / filename
# Rotating handler — caps a single invocation at MAX_LOG_BYTES.
log_level = effective_log_level()
handler = RotatingFileHandler(
log_path,
maxBytes=MAX_LOG_BYTES,
backupCount=_BACKUP_COUNT,
encoding="utf-8",
)
handler.setLevel(log_level)
# Best-effort 0600 permissions on the log file.
with contextlib.suppress(OSError):
os.chmod(log_path, 0o600)
handler.setFormatter(
_RedactingFormatter(
use_colors=False,
fmt="%(asctime)s %(levelname)-5s [%(name)s] %(message)s",
datefmt="%H:%M:%S",
)
)
stream_handler: logging.Handler | None = None
if env_truthy(os.environ.get("OMNIGENT_LOG_TO_STDERR")) and sys.stderr.isatty():
stream_handler = logging.StreamHandler(sys.stderr)
stream_handler.setLevel(log_level)
stream_handler.setFormatter(_RedactingFormatter(use_colors=terminal_supports_color()))
# Wire our two package hierarchies at the effective level so their records reach
# Wire our two package hierarchies at INFO so their records reach
# the file handler.
for name in ("omnigent", "omnigent_ui_sdk"):
logger = logging.getLogger(name)
logger.setLevel(log_level)
logger.setLevel(logging.INFO)
logger.addHandler(handler)
if stream_handler is not None:
logger.addHandler(stream_handler)
logger.propagate = False
# Suppress noisy third-party loggers that are commonly present.
+1 -176
View File
@@ -19,21 +19,6 @@ CODEX_NATIVE_REQUEST_SESSION_ID_ENV_VAR = "HARNESS_CODEX_NATIVE_REQUEST_SESSION_
_STATE_FILE = "state.json"
_STARTUP_ERROR_FILE = "startup_error.json"
# Per-MCP-server startup state mirrored from Codex's
# ``mcpServer/startupStatus/updated`` notifications. Written by the
# forwarder (and by ``wait_for_thread_started`` while it drains startup
# events), read by the executor's first-turn gate and the runner's
# Stop handler.
_MCP_STARTUP_FILE = "mcp_startup.json"
# Startup states mirrored from Codex's ``McpServerStartupState`` enum.
MCP_STARTUP_STARTING = "starting"
MCP_STARTUP_READY = "ready"
MCP_STARTUP_FAILED = "failed"
MCP_STARTUP_CANCELLED = "cancelled"
MCP_STARTUP_STATES = frozenset(
{MCP_STARTUP_STARTING, MCP_STARTUP_READY, MCP_STARTUP_FAILED, MCP_STARTUP_CANCELLED}
)
# Must match ``_CONFIG_FILE`` in ``claude_native_bridge.py`` because
# ``serve-mcp`` reads this filename for the token.
_MCP_CONFIG_FILE = "bridge.json"
@@ -356,7 +341,7 @@ def clear_bridge_state(bridge_dir: Path) -> None:
:param bridge_dir: Native Codex bridge directory.
:returns: None.
"""
for name in (_STATE_FILE, _STARTUP_ERROR_FILE, _MCP_STARTUP_FILE):
for name in (_STATE_FILE, _STARTUP_ERROR_FILE):
try:
(bridge_dir / name).unlink()
except FileNotFoundError:
@@ -407,166 +392,6 @@ def read_bridge_startup_error(bridge_dir: Path) -> str | None:
return message if isinstance(message, str) and message else None
def read_mcp_startup(bridge_dir: Path) -> dict[str, dict[str, str | None]]:
"""
Read the recorded per-MCP-server startup state.
:param bridge_dir: Native Codex bridge directory.
:returns: Mapping of server name to its latest startup record, e.g.
``{"safe": {"status": "starting", "error": None}}``. Empty when
no state has been recorded or the file is unreadable.
"""
path = bridge_dir / _MCP_STARTUP_FILE
if not path.is_file():
return {}
try:
raw = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return {}
servers = raw.get("servers") if isinstance(raw, dict) else None
if not isinstance(servers, dict):
return {}
parsed: dict[str, dict[str, str | None]] = {}
for name, record in servers.items():
if not (isinstance(name, str) and name and isinstance(record, dict)):
continue
status = record.get("status")
if status not in MCP_STARTUP_STATES:
continue
error = record.get("error")
parsed[name] = {
"status": status,
"error": error if isinstance(error, str) and error else None,
}
return parsed
def _write_mcp_startup(bridge_dir: Path, servers: dict[str, dict[str, str | None]]) -> None:
"""
Persist the per-MCP-server startup map atomically (best-effort).
:param bridge_dir: Native Codex bridge directory.
:param servers: Full startup map, e.g.
``{"safe": {"status": "ready", "error": None}}``.
:returns: None.
"""
try:
bridge_dir.mkdir(mode=0o700, parents=True, exist_ok=True)
path = bridge_dir / _MCP_STARTUP_FILE
fd, tmp_name = tempfile.mkstemp(prefix=f"{_MCP_STARTUP_FILE}.", dir=str(bridge_dir))
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump({"servers": servers}, handle, sort_keys=True)
handle.write("\n")
os.replace(tmp_name, path)
finally:
if os.path.exists(tmp_name):
os.unlink(tmp_name)
except OSError:
return # best-effort; surfacing MCP state must never sink startup
def update_mcp_server_startup(
bridge_dir: Path,
name: str,
status: str,
error: str | None = None,
) -> dict[str, dict[str, str | None]]:
"""
Record one Codex MCP-server startup update.
:param bridge_dir: Native Codex bridge directory.
:param name: MCP server name, e.g. ``"storage-console"``.
:param status: One of :data:`MCP_STARTUP_STATES`.
:param error: Failure detail when ``status == "failed"``, e.g.
``"handshaking with MCP server failed"``. ``None`` otherwise.
:returns: The full startup map after the update.
"""
servers = read_mcp_startup(bridge_dir)
servers[name] = {"status": status, "error": error}
_write_mcp_startup(bridge_dir, servers)
return servers
def pending_mcp_servers(servers: dict[str, dict[str, str | None]]) -> list[str]:
"""
Return the MCP servers still reported as ``starting``.
:param servers: Startup map from :func:`read_mcp_startup`.
:returns: Sorted server names whose latest status is ``starting``.
"""
return sorted(
name for name, record in servers.items() if record.get("status") == MCP_STARTUP_STARTING
)
def cancel_pending_mcp_startup(bridge_dir: Path) -> list[str]:
"""
Mark every still-``starting`` MCP server as ``cancelled``.
Used by the Stop path so the executor's first-turn gate unblocks
immediately, even when Codex's own ``cancelled`` notifications are
delayed or lost.
:param bridge_dir: Native Codex bridge directory.
:returns: Sorted names of the servers that were flipped, e.g.
``["storage-console"]``. Empty when nothing was pending.
"""
servers = read_mcp_startup(bridge_dir)
pending = pending_mcp_servers(servers)
if not pending:
return []
for name in pending:
servers[name] = {"status": MCP_STARTUP_CANCELLED, "error": servers[name].get("error")}
_write_mcp_startup(bridge_dir, servers)
return pending
def settle_pending_mcp_startup(bridge_dir: Path) -> tuple[dict[str, dict[str, str | None]], bool]:
"""
Drop every still-``starting`` MCP server from the recorded map.
Codex delivers per-server terminal states (ready/failed) only to the
connection that owns the thread never to Omnigent's observer
connection so when a settle signal arrives (the thread went idle
after a turn, or the startup window elapsed) the round is known to be
over but the per-server outcomes are not. Unresolved entries are
removed rather than guessed; locally-known terminal states
(``cancelled`` from a Stop) are preserved.
:param bridge_dir: Native Codex bridge directory.
:returns: ``(map_after, changed)`` the settled map and whether any
entry was dropped.
"""
# The read→write below is not locked across processes: a runner Stop
# can flip an entry to ``cancelled`` in between, and this write drops
# it. Cosmetic only — both outcomes end the round, and the Stop path
# publishes its cancelled map independently.
servers = read_mcp_startup(bridge_dir)
pending = pending_mcp_servers(servers)
if not pending:
return servers, False
for name in pending:
servers.pop(name, None)
_write_mcp_startup(bridge_dir, servers)
return servers, True
def mcp_startup_waiting_detail(servers: dict[str, dict[str, str | None]]) -> str | None:
"""
Describe the MCP servers a startup wait is still blocked on.
:param servers: Startup map from :func:`read_mcp_startup`.
:returns: Text naming the pending servers, e.g.
``"MCP startup still waiting on storage-console"``, or ``None``
when nothing is pending.
"""
pending = pending_mcp_servers(servers)
if not pending:
return None
return f"MCP startup still waiting on {', '.join(pending)}"
def read_bridge_state(bridge_dir: Path) -> CodexNativeBridgeState | None:
"""
Read shared native Codex bridge state.
-326
View File
@@ -34,18 +34,12 @@ from omnigent.codex_native_app_server import (
)
from omnigent.codex_native_bridge import (
CODEX_NATIVE_BRIDGE_ID_LABEL_KEY,
MCP_STARTUP_STARTING,
MCP_STARTUP_STATES,
CodexNativeBridgeState,
clear_active_turn_id_if_matches,
codex_home_for_bridge_dir,
pending_mcp_servers,
read_bridge_state,
read_codex_config_model,
read_mcp_startup,
settle_pending_mcp_startup,
update_active_turn_id,
update_mcp_server_startup,
update_thread_id,
write_bridge_state,
)
@@ -118,31 +112,6 @@ _CODEX_ELICITATION_CONNECT_TIMEOUT_SECONDS = 30.0
_CODEX_ELICITATION_RETRY_INITIAL_BACKOFF_SECONDS = 1.0
_CODEX_ELICITATION_RETRY_MAX_BACKOFF_SECONDS = 30.0
_CODEX_MCP_ELICITATION_REQUEST_METHOD = "mcpServer/elicitation/request"
# Per-server MCP startup progress (issue #2058). Codex runs an MCP
# startup round when a thread starts, but delivers the per-server
# ``mcpServer/startupStatus/updated`` edges ONLY to the connection that
# owns the thread (the TUI) — verified against codex 0.142.5 — so this
# observer connection cannot passively mirror them. Instead the round is
# SYNTHESIZED: at forwarder start the config-declared servers are
# recorded as ``starting`` (true — codex boots them all at thread start)
# in the bridge dir and posted to Omnigent as ``external_mcp_startup``; the
# round is settled (unresolved entries dropped) when the thread goes
# idle after a turn — codex defers turn execution until startup ends, so
# an idle edge proves the round is over — or when the config-derived
# startup window elapses. ``cancelled`` states are recorded locally by
# the Stop path. The notification handler is kept as a zero-cost path
# for any delivery codex broadens later (it fully supersedes synthesis
# when edges do arrive).
_CODEX_MCP_STARTUP_STATUS_METHOD = "mcpServer/startupStatus/updated"
_CODEX_THREAD_STATUS_CHANGED_METHOD = "thread/status/changed"
_EXTERNAL_MCP_STARTUP_TYPE = "external_mcp_startup"
# Codex bounds each MCP server's spawn+handshake by its per-server
# ``startup_timeout_sec`` (codex default 10s); the round cannot outlive
# the slowest server's budget. The synthesis settle timer mirrors that
# bound, with floor/grace/cap keeping a misconfigured value sane.
_MCP_STARTUP_DEFAULT_TIMEOUT_SECONDS = 10.0
_MCP_STARTUP_SETTLE_GRACE_SECONDS = 15.0
_MCP_STARTUP_SETTLE_MAX_SECONDS = 240.0
_CODEX_TOOL_REQUEST_USER_INPUT_METHOD = "item/tool/requestUserInput"
_CODEX_COMMAND_EXECUTION_REQUEST_APPROVAL_METHOD = "item/commandExecution/requestApproval"
_CODEX_FILE_CHANGE_REQUEST_APPROVAL_METHOD = "item/fileChange/requestApproval"
@@ -1632,14 +1601,6 @@ async def supervise_forwarder(
# outage or restart). Runs before live forwarding begins, so no
# other writer races the dead-letter files (#1579).
await _replay_dead_letters_on_startup(ap_client, bridge_dir)
# Synthesize the thread's MCP startup round (see the comment on
# _CODEX_MCP_STARTUP_STATUS_METHOD): the fresh-launch forwarder
# starts right at thread creation, which is when codex boots its
# configured MCP servers. Skipped when the bridge already carries
# round state (forwarder reconnect mid-session).
mcp_settle_timer = await _seed_mcp_startup_round(
ap_client, session_id=session_id, bridge_dir=bridge_dir
)
target = _ForwarderTarget(
session_id=session_id,
thread_id=thread_id,
@@ -1725,10 +1686,6 @@ async def supervise_forwarder(
except Exception: # noqa: BLE001 - keep the long-lived mirror alive.
_logger.warning("Codex forwarder event handling failed", exc_info=True)
finally:
if mcp_settle_timer is not None:
mcp_settle_timer.cancel()
with contextlib.suppress(asyncio.CancelledError):
await mcp_settle_timer
await target.delta_coalescer.close()
await target.usage_coalescer.close()
await target.elicitation_tracker.close()
@@ -2297,39 +2254,6 @@ async def _handle_event(
_parent_thread_id_from_started_event(event),
)
return
if method == _CODEX_MCP_STARTUP_STATUS_METHOD:
# MCP startup is bridge-level state, surfaced on the parent
# session. The notification's ``threadId`` is nullable; a child
# thread's startup (different id) is not mirrored.
event_thread_id = _thread_id_from_params(params)
if (
event_thread_id is None
or expected_thread_id is None
or event_thread_id == expected_thread_id
):
parent_session_id = (
forwarder_state.parent_session_id
if forwarder_state is not None and forwarder_state.parent_session_id is not None
else session_id
)
await _handle_mcp_startup_status(
client,
session_id=parent_session_id,
bridge_dir=bridge_dir,
params=params,
)
return
if _is_thread_idle_status_event(method, params) and _thread_id_from_params(params) in {
None,
expected_thread_id,
}:
# A completed turn proves MCP startup settled (codex defers turn
# execution until the round ends) — resolve the synthesized round.
# Not an exclusive handler: idle status also feeds the subscribe
# release below, so fall through.
await _settle_mcp_startup(
client, session_id=session_id, bridge_dir=bridge_dir, reason="thread went idle"
)
# Resolve routing: parent thread, known child thread, or stale/ignored.
route_session_id, is_child = _resolve_event_session(
params, method, expected_thread_id, forwarder_state, fallback_session_id=session_id
@@ -3065,256 +2989,6 @@ def _handle_turn_diff_updated(
forwarder_state.note_turn_diff(turn_id, diff if isinstance(diff, str) else "")
async def _handle_mcp_startup_status(
client: httpx.AsyncClient,
*,
session_id: str,
bridge_dir: Path,
params: dict[str, Any],
) -> None:
"""
Mirror one Codex MCP-server startup update.
Records the update into the bridge dir (the Stop path and turn-error
text read it) and republishes the full per-server map to Omnigent so the
web session shows startup progress. In practice codex delivers these
edges only to the thread-owning connection (see the comment on
:data:`_CODEX_MCP_STARTUP_STATUS_METHOD`); when they do arrive they
carry real terminal states and supersede the synthesized round.
:param client: HTTP client for Omnigent event posts.
:param session_id: Omnigent conversation id, e.g. ``"conv_abc123"``.
:param bridge_dir: Native Codex bridge directory.
:param params: Codex ``mcpServer/startupStatus/updated`` params, e.g.
``{"name": "safe", "status": "failed", "error": "..."}``.
:returns: None.
"""
name = params.get("name")
status = params.get("status")
if not (isinstance(name, str) and name and status in MCP_STARTUP_STATES):
_logger.info("Codex forwarder ignored malformed MCP startup update: %r", params)
return
error = params.get("error")
servers = update_mcp_server_startup(
bridge_dir,
name,
status,
error=error if isinstance(error, str) and error else None,
)
await _post_mcp_startup(client, session_id, servers)
def _expected_mcp_servers_from_config(bridge_dir: Path) -> list[str]:
"""
Read the enabled MCP server names from the session's Codex config.
The per-session ``config.toml`` (private ``CODEX_HOME``) is what the
app-server loads, so its ``[mcp_servers.*]`` tables are exactly the
servers codex boots at thread start including the injected
``omnigent`` relay server. Codex-internal servers that are not
config-declared (e.g. ``codex_apps``) are not visible here and are
simply absent from the synthesized round.
:param bridge_dir: Native Codex bridge directory.
:returns: Sorted enabled server names, e.g. ``["omnigent", "safe"]``.
Empty when the config is missing or unparsable.
"""
import tomllib
config_path = codex_home_for_bridge_dir(bridge_dir) / "config.toml"
try:
config = tomllib.loads(config_path.read_text(encoding="utf-8"))
except (OSError, tomllib.TOMLDecodeError):
return []
servers = config.get("mcp_servers")
if not isinstance(servers, dict):
return []
return sorted(
name
for name, table in servers.items()
if isinstance(name, str)
and name
and isinstance(table, dict)
and table.get("enabled") is not False
)
def _mcp_startup_settle_timeout_seconds(bridge_dir: Path) -> float:
"""
Derive the synthesized round's settle window from the session config.
Codex bounds each server's spawn+handshake by its per-server
``startup_timeout_sec`` (default
:data:`_MCP_STARTUP_DEFAULT_TIMEOUT_SECONDS`), so the round cannot
outlive the slowest server's budget; a grace period absorbs spawn
overhead and the cap keeps a misconfigured budget from pinning the
band for many minutes.
:param bridge_dir: Native Codex bridge directory.
:returns: Settle timeout in seconds, e.g. ``135.0`` for a config whose
slowest server declares ``startup_timeout_sec = 120``.
"""
import tomllib
slowest = _MCP_STARTUP_DEFAULT_TIMEOUT_SECONDS
config_path = codex_home_for_bridge_dir(bridge_dir) / "config.toml"
try:
config = tomllib.loads(config_path.read_text(encoding="utf-8"))
except (OSError, tomllib.TOMLDecodeError):
config = {}
servers = config.get("mcp_servers")
if isinstance(servers, dict):
for table in servers.values():
# Same enabled filter as _expected_mcp_servers_from_config:
# codex never boots a disabled server, so its budget must not
# stretch the window for a round it is not part of.
if not isinstance(table, dict) or table.get("enabled") is False:
continue
timeout = table.get("startup_timeout_sec")
if isinstance(timeout, (int, float)) and timeout > slowest:
slowest = float(timeout)
return min(slowest + _MCP_STARTUP_SETTLE_GRACE_SECONDS, _MCP_STARTUP_SETTLE_MAX_SECONDS)
def _arm_mcp_settle_timer(
client: httpx.AsyncClient,
*,
session_id: str,
bridge_dir: Path,
) -> asyncio.Task[None]:
"""
Arm the bounded settle window for an in-flight MCP startup round.
:param client: HTTP client for Omnigent event posts.
:param session_id: Omnigent conversation id, e.g. ``"conv_abc123"``.
:param bridge_dir: Native Codex bridge directory.
:returns: The settle-timer task.
"""
timeout = _mcp_startup_settle_timeout_seconds(bridge_dir)
async def settle_after_window() -> None:
"""Settle the synthesized round once the startup window elapses."""
await _sleep(timeout)
await _settle_mcp_startup(
client, session_id=session_id, bridge_dir=bridge_dir, reason="startup window elapsed"
)
return asyncio.create_task(settle_after_window(), name="codex-native-mcp-settle")
async def _seed_mcp_startup_round(
client: httpx.AsyncClient,
*,
session_id: str,
bridge_dir: Path,
) -> asyncio.Task[None] | None:
"""
Record the config-declared MCP servers as ``starting`` and post them.
Seeds once per app-server launch: ``clear_bridge_state`` wipes the
recorded map before each launch, and an existing map means a
forwarder reconnect mid-session reseeding then would flash a false
"starting" band for servers that finished booting long ago. A
reconnect that finds the round still pending does re-arm the settle
window, though: the previous forwarder's timer died with it, and
without a replacement a missed idle edge would leave the band stuck
on "starting" for the rest of the session.
:param client: HTTP client for Omnigent event posts.
:param session_id: Omnigent conversation id, e.g. ``"conv_abc123"``.
:param bridge_dir: Native Codex bridge directory.
:returns: The armed settle-timer task, or ``None`` when the recorded
round has already settled.
"""
existing = read_mcp_startup(bridge_dir)
if existing:
if not pending_mcp_servers(existing):
return None
_logger.info("Codex MCP startup round still pending after reconnect; re-arming settle")
return _arm_mcp_settle_timer(client, session_id=session_id, bridge_dir=bridge_dir)
expected = _expected_mcp_servers_from_config(bridge_dir)
if not expected:
return None
servers: dict[str, dict[str, str | None]] = {}
for name in expected:
servers = update_mcp_server_startup(bridge_dir, name, MCP_STARTUP_STARTING)
_logger.info("Codex MCP startup round synthesized: %s", ", ".join(expected))
await _post_mcp_startup(client, session_id, servers)
return _arm_mcp_settle_timer(client, session_id=session_id, bridge_dir=bridge_dir)
async def _settle_mcp_startup(
client: httpx.AsyncClient,
*,
session_id: str,
bridge_dir: Path,
reason: str,
) -> None:
"""
Settle the synthesized MCP startup round, if any of it is unresolved.
Drops still-``starting`` entries from the bridge map (their real
terminal states are only ever delivered to the thread-owning
connection) and posts the settled map so the web band clears.
Locally-recorded terminal states ``cancelled`` from a Stop are
preserved. Idempotent: a fully settled map is left untouched.
:param client: HTTP client for Omnigent event posts.
:param session_id: Omnigent conversation id, e.g. ``"conv_abc123"``.
:param bridge_dir: Native Codex bridge directory.
:param reason: Settle trigger for logs, e.g. ``"thread went idle"``.
:returns: None.
"""
servers, changed = settle_pending_mcp_startup(bridge_dir)
if not changed:
return
_logger.info("Codex MCP startup round settled (%s)", reason)
await _post_mcp_startup(client, session_id, servers)
def _is_thread_idle_status_event(method: str, params: dict[str, Any]) -> bool:
"""
Return whether an event reports the thread going idle.
Codex defers turn execution until MCP startup settles, so a thread
reaching ``idle`` after a turn proves the startup round is over. This
is one of the few notifications codex broadcasts to non-owning
connections, making it the natural live settle signal for the
synthesized round.
:param method: Codex method value, e.g. ``"thread/status/changed"``.
:param params: Codex notification params.
:returns: ``True`` for an idle ``thread/status/changed``.
"""
if method != _CODEX_THREAD_STATUS_CHANGED_METHOD:
return False
status = params.get("status")
return isinstance(status, dict) and status.get("type") == "idle"
async def _post_mcp_startup(
client: httpx.AsyncClient,
session_id: str,
servers: dict[str, dict[str, str | None]],
) -> None:
"""
Post the current per-MCP-server startup map to Omnigent.
:param client: HTTP client for Omnigent event posts.
:param session_id: Omnigent conversation id, e.g. ``"conv_abc123"``.
:param servers: Full startup map, e.g.
``{"safe": {"status": "starting", "error": None}}``.
:returns: None.
"""
response = await _post_session_event(
client,
session_id,
event_type=_EXTERNAL_MCP_STARTUP_TYPE,
data={"servers": servers},
)
_log_failed_session_event_post(_EXTERNAL_MCP_STARTUP_TYPE, response)
def _is_codex_elicitation_request(event: CodexMessage) -> bool:
"""
Return whether an app-server frame asks this client for input.
+4 -15
View File
@@ -61,23 +61,12 @@ def _state_dir_for_conversation_id(conversation_id: str) -> Path:
Hashing the conversation id prevents path traversal if a server
ever returned an attacker-controlled id such as ``"../etc"``.
Sessions created before ids dropped the ``conv_`` prefix hashed the
prefixed string, so their directories live under the legacy digest; when
the bare-digest directory is absent, the legacy one is returned (never
renamed files inside may embed their own absolute path).
:param conversation_id: Omnigent conversation id, bare 32-char hex
(a legacy ``conv_``-prefixed form is accepted and normalised).
:param conversation_id: Omnigent conversation id, e.g.
``"conv_abc123"``.
:returns: Absolute directory path; not guaranteed to exist.
"""
bare = conversation_id.removeprefix("conv_")
root = _codex_native_state_root()
state_dir = root / hashlib.sha256(bare.encode("utf-8")).hexdigest()[:_ID_HASH_CHARS]
if not state_dir.exists():
legacy = root / hashlib.sha256(f"conv_{bare}".encode()).hexdigest()[:_ID_HASH_CHARS]
if legacy.exists():
return legacy
return state_dir
digest = hashlib.sha256(conversation_id.encode("utf-8")).hexdigest()[:_ID_HASH_CHARS]
return _codex_native_state_root() / digest
def write_launch_state(conversation_id: str, working_directory: str) -> None:

Some files were not shown because too many files have changed in this diff Show More