main
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zeyi (Rice) Fan
|
244ded1ed9 |
chore(deps): move development tooling to internal groups (#4626)
## Related issue N/A ## Summary The published `dev` extra mixed repository workflows with installable Omnigent capabilities. Move contributor-only dependencies to PEP 735 groups so package extras describe product functionality and CI installs only the workflow dependencies it executes. - Replace the `dev` extra with local-only `lint`, `test`, and aggregate `dev` groups; configure no default groups so plain `uv sync` matches the published base package. - Remove the retired mypy dependency/configuration, `types-PyYAML`, the orphaned `pathspec` declaration, and the duplicate `filelock` declaration. - Migrate workflows, actions, contributor commands, tests, and development skills from `--extra dev` to the smallest required group, or no group for application/benchmark jobs. - Compose Pyrefly's lint environment from the `lint` group plus the existing `hindsight`, `nimble`, `s3`, and `tracing` capability extras. Remove the OpenTelemetry missing-import configuration and Nimble's inline missing-import suppression so real package types remain checked. - Update OpenShell, e2e, browser-test, Slack, and implementation-plan commands to compose capability extras with repository groups explicitly. Document why read-only/tools-less agent workflows intentionally keep runtime-only environments. - Avoid `--all-extras`: it resolves but selects 240 product packages, including unrelated large/native integrations. Keep capability ownership explicit instead. ELI5: product features remain extras users can install; lint and test toolboxes become private repository groups that never appear in the wheel. ```text published wheel: base + capability extras repository: lint group | test group | dev = lint + test CI lint: lint + explicitly type-checked capability extras ``` ## Test Plan - `uv lock && just normalize-locks` - Built the wheel and verified its metadata contains no `dev` extra or lint/test dependencies. - Verified a fresh base environment imports Omnigent, excludes lint/test/pathspec packages, and imports each release benchmark script. - `uv run --isolated --frozen --group lint --extra hindsight --extra nimble --extra s3 --extra tracing pre-commit run pyrefly --all-files` - `uv run --isolated --frozen --group lint python scripts/gen_routing_pb2.py --check` - Verified isolated `test` and aggregate `dev` group membership independently. - `uv run --isolated --frozen --group test pytest tests/tools/builtins/test_hindsight.py tests/tools/builtins/test_nimble_research.py tests/stores/test_s3_artifact_store.py tests/db/test_d1_fts_dialect.py -q` (172 passed) - `uv run --isolated --frozen --group test --extra tracing pytest tests/runtime/test_telemetry.py tests/inner/test_tracing_genai_semconv.py -q` (69 passed) - `uv run --isolated --frozen --extra openshell --group test pytest tests/onboarding/sandboxes/test_openshell.py tests/server/test_managed_hosts.py -q` (259 passed) - Verified load-test modules import with only `loadtest` and `agents-sdk` extras. - Ran the exact locked lint sync against PyPI and Pyrefly passed. - Rebased onto current `origin/main`; migrated the newly added compatibility-smoke test actions and host benchmark workflow. - Surveyed all tracked uv install/run commands and removed every remaining published-`dev`/implicit-tooling command. Verified the documented e2e and Slack environments and collected the Kimi/live-DDG tests in fresh group-selected environments. - `uv run --frozen pre-commit run --all-files` ## Demo N/A — dependency metadata and CI configuration only. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] UI / frontend change - [x] Refactor / chore - [ ] Docs - [ ] Test / CI - [ ] Breaking change ## Test coverage - [ ] Unit tests added / updated - [ ] Integration tests added / updated - [ ] E2E tests added / updated - [x] Manual verification completed - [x] Existing tests cover this change - [ ] Not applicable ## Coverage notes Fresh isolated environments validated the base, lint, test, aggregate dev, tracing-test, and load-test dependency boundaries. Focused tests prove retained optional clients are genuine test runtimes, while wheel inspection proves repository groups are not published. Signed-off-by: Zeyi (Rice) Fan <zeyi.f@databricks.com> |
||
|
Dhruv Gupta
|
14eb2a515d |
refactor(harness): declarative catalog for builtin ACP CLI harnesses (#3988)
Promoting an ACP-speaking vendor CLI to a first-class harness has meant touching 6+ registration points (capabilities, valid set, module map, aliases, labels, install spec, readiness, setup steps, a per-harness spawn-env builder, the live e2e matrix exclusion) plus a near-identical thin inner module. Recent PRs each re-derived this by hand and one shipped without its spawn-env builder, silently dropping the session cwd and the spec sandbox. Add omnigent/acp_cli_harnesses.py: one AcpCliHarness row per vendor CLI (label, binary, ACP argv, aliases, install and login metadata). Every registration derives from the row: - harness_plugins: validity, module routing (all rows run the shared omnigent/inner/acp_harness.py wrap), aliases, labels, capabilities (the generic acp profile), install specs and install keys - onboarding: one-click install allowlist (npm rows) and vendor-login setup steps derive; readiness rides the existing install-key gate - runtime/workflow: one shared _build_acp_cli_spawn_env forwarding the session cwd and serialized os_env, shell-quoting the resolved binary - runner dispatch: one membership check covers every current and future row - tests: readiness spelling lists and the live-matrix exclusion extend from the catalog; tests/test_acp_cli_harnesses.py drives a fake row through the builder and dispatch and asserts full registration per real row The catalog ships empty; the first rows land with the Grok Build (#3075) and Qoder (#3560) PRs, each reduced to one dict entry plus docs. Co-authored-by: Isaac Signed-off-by: Dhruv Gupta <dhruv.gupta@databricks.com> |
||
|
Shivam Mittal
|
dc00417841 |
Add WebSocket load test (dev/loadtest/) + run-load-test skill (#3591)
* Add WebSocket load test (dev/loadtest/) + run-load-test skill Adds a Locust load test that opens N concurrent WebSocket connections to WS /v1/sessions/updates and holds them open, measuring the server's WebSocket fan-out (handshake, origin/auth gating, watch-set diffing, heartbeat) under concurrency — no runner, LLM, or agent turns. - dev/loadtest/ws_load_test.py: the locustfile (SessionUpdatesUser). - dev/loadtest/run.py: runner taking server + host + load params, runs locust headless, and writes a result set (summary.md, CSV, HTML, config). - loadtest extra (locust + websocket-client) in pyproject.toml + uv.lock. - .claude/skills/run-load-test: skill that gathers inputs, runs, and explains the latency results. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> * Launch locust via sys.executable -m locust in the load-test runner run.py launched locust as a bare `locust` command, which resolves through PATH and can pick up a stale/broken locust from a different Python (e.g. a ~/.local 3.10 install missing gevent's zope.event) even when run.py itself runs under a venv — crashing the run with ModuleNotFoundError before locust starts. Launch it as `sys.executable -m locust` so it always uses the same interpreter + site-packages that run.py runs under. Preflight now checks importlib.util.find_spec (the actual interpreter) instead of shutil.which (PATH), and --web execs sys.executable too. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> * Genericize --mount-prefix docs to reverse-proxy sub-paths Replace deployment-specific mount-prefix details with a provider-neutral "behind a reverse proxy at a sub-path" framing (neutral /omnigent example) across the README, the run-load-test skill, and the run.py / ws_load_test.py help + docstrings. The --mount-prefix flag itself is unchanged. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> * Add runner-level turn load test (real multi-turn conversations, mocked LLM) turn_load.py drives real agent turns through the runner — the full POST .../events → server → runner → executor → LLM → stream → idle loop — under concurrency, with the LLM mocked (zero latency) so the numbers isolate Omnigent's own per-turn / history-handling overhead. Runs N concurrent conversations of M sequential turns each on one durable session, so history grows across the turns (a real long conversation, not N one-shots). It boots the whole stack itself (server + zero-latency mock LLM + runner) by reusing the benchmark harness's BenchEnvironment, using the in-process openai-agents harness — no vendor CLI, no real API key — so it runs from a repo checkout with no server to point at. Concurrency is asyncio (the runner stack is async), not Locust. Writes the same summary.md / run_config.json result format as the WS runner. Documents both scenarios (WebSocket fan-out vs runner turns) in the README and the run-load-test skill. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> * Address review: fix socket leak, URL/timeout edge cases, double-count; add tests Copilot review follow-ups on the load-test harness: - ws_load_test: assign self.ws before the send/recv steps so a post-create failure closes the socket instead of leaking it. - ws_load_test: _ws_url now treats a schemeless host (localhost:8000, which Locust accepts) as ws:// rather than emitting an invalid URL. - ws_load_test: _read_until_snapshot caps each recv to the remaining deadline so a late frame can't overrun by a full read timeout. - ws_load_test: WS_READ_TIMEOUT falls back to the default on a non-numeric value instead of raising in on_start. - run.py: preflight websocket-client as well as locust; rename _fmt_ms -> _fmt_num (it also formats Requests/s). - run.py: _write_summary skips locust's Aggregated row when totaling, which was double-counting the headline request/failure counts. - docs: the scenario reads AUTH_TOKEN from the environment; drop the wrong `-e AUTH_TOKEN` locust-flag examples (AUTH_TOKEN=... locust ...). - tests: add tests/loadtest unit tests for the pure helpers (URL/env/argv wiring, summary formatting, timeout parsing) — deterministic, no server boot. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> * Redesign as one load test: each user is a real host driving real turns Collapse the two scenarios (ws_load_test.py + turn_load.py) into a single load test where each Locust user IS a real omnigent host. Each user spawns a real `omnigent host` subprocess (unique identity + per-host $HOME so the host-daemon singleton guard doesn't collide), registers it over the host tunnel, then creates host-bound sessions and drives real multi-turn conversations — every turn is a genuine post→idle loop through a runner the host spawns, with the LLM mocked (zero latency). `-u N` scales the number of hosts; Locust does the concurrency. run.py boots the whole stack (server + mock LLM via BenchEnvironment), registers one agent, sets the mock reply, then runs Locust against it — there is no --server to pass, since mocking the LLM requires a stack we control. It reuses the CSV→summary.md machinery (Aggregated-row dedupe kept). Capacity-limited by design: N hosts × M sessions = N×M real runner processes on the load box, so it drives genuine end-to-end turns rather than faking the runner, but does not scale to hundreds on one machine (documented). Removes the websocket-client dep (no longer used); needs [loadtest,dev,agents-sdk]. README, skill, and tests updated for the single scenario. Verified locally: 5 hosts × 3 turns → 133 turns, 0 failures. Co-authored-by: Isaac Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> --------- Signed-off-by: Shivam Mittal <shivam.mittal@databricks.com> Co-authored-by: Shivam Mittal <shivam.mittal@databricks.com> |
||
|
Elliot Sun
|
38cc498b75 |
fix(onboarding): detect agy settings.json as login fallback on macOS (#3289)
* fix(onboarding): detect agy settings.json as login fallback on macOS On macOS, agy 1.1.7+ stores OAuth credentials in Keychain and writes only ~/.gemini/antigravity-cli/settings.json (no oauth_creds.json). The existing gemini_auth_has_credential() missed this and falsely reported 'harness antigravity-native is not configured'. Accept the existence of settings.json as a fallback signal when no token files are found. This is safe because the caller (resolve_native_antigravity_launch) uses it only for an informational warning — agy always re-drives OAuth on first run regardless. - Update gemini_auth_has_credential() with settings.json fallback - Update docstrings to document the third detection path - Update warning message in antigravity_native_launch.py - Add unit test for settings.json-only detection - Fix _GEMINI_DIR isolation in existing test Signed-off-by: ElliotSun <elros1109@gmail.com> * fix(onboarding): prove agy login via CLI, not settings.json existence The macOS lockout this fixes is real: agy 1.1.7+ keeps OAuth in the Keychain and writes no token file, so the file-only check reported antigravity-native as unconfigured and connect.py refused to spawn a runner for a user who was in fact signed in. Accepting the bare existence of ~/.gemini/antigravity-cli/settings.json as the fallback signal does not work, because omnigent creates that file itself: the CLI launch path calls ensure_agy_feedback_survey_disabled under the real home before agy starts, and build_agy_launch emits no HOME override. One `omni antigravity` run therefore satisfied the credential gate forever, on every platform — turning a hard launch gate into a no-op and letting a runner spawn that dies on its first turn. That is worst on headless hosts, where agy's OAuth prompt has no TTY. Ask the CLI instead. `agy models` exits 0 only when signed in and reads the credential wherever agy stored it, Keychain included, so nothing omnigent writes can satisfy it. This mirrors ambient._claude_login_detected, which already solves the identical Keychain split for Claude Code, and reuses the probe harness_install already wires as the gemini family's status command. The fallback is gated on macOS: Linux writes a real token file, so its absence is a true negative there and the fallback would only add a subprocess while weakening a signal that works. Failures — missing binary, non-zero exit, timeout, unreadable home — all read as False, because readiness must never raise. Content inspection of settings.json was the alternative considered. It was rejected as unverifiable from here: no key in that file is known to mark a completed sign-in on 1.1.7, so keying on one risks reintroducing the very lockout being fixed. Signed-off-by: SabhyaC26 <sabhyachhabria@gmail.com> * docs(skills): note agy's macOS Keychain credential in the e2e pre-flight The pre-flight tells the reader agy's token lives under ~/.gemini, which leaves a Mac developer on agy 1.1.7+ hunting for a file that is never written. Name the Keychain case and the `agy models` fallback that gemini_login_detected() now uses there. Signed-off-by: SabhyaC26 <sabhyachhabria@gmail.com> --------- Signed-off-by: ElliotSun <elros1109@gmail.com> Signed-off-by: SabhyaC26 <sabhyachhabria@gmail.com> Co-authored-by: SabhyaC26 <sabhyachhabria@gmail.com> |
||
|
Zeyi (Rice) Fan
|
c7b24b8b05 |
refactor(cli): replace omni server start with omni server --background (#3105)
## Related issue N/A ## Summary - Removes the `omni server start` subcommand. `omni server` already starts the server (in the foreground), so `start` was a redundant way to launch it; the only thing it added was the detached/background mode. - Adds a `--background` flag to `omni server` that reproduces the former `start` behavior: spawn (or reuse) the managed detached local server instead of running uvicorn in the foreground. `omni server stop` / `omni server status` are unchanged. - Updates the desktop app's CLI shell-out, docs, skill files, and tests to the new invocation. ## Test Plan - `omni server start` now exits `2` with "No such command 'start'" (verified via `CliRunner`). - `omni server --background` routes to `ensure_local_omnigent_server()` and short-circuits before the foreground port-bind check; prints the URL and captured log path on spawn, "already running" on reuse, and omits the log line when `log_path` is unknown (3 renamed tests pass). - `omni server stop` / `omni server status` behave as before (verified via CliRunner with stubbed registry). - `server --help` lists `--background` and only the `stop`/`status` subcommands; bare `omni server` still reaches the foreground port-bind check. - `node --check web/electron/src/omnigent_cli.js` passes; the spawn primitive in `host/local_server.py` invokes the bare `omnigent.cli server` foreground command, so it is unaffected by the `start` removal. ## Demo N/A ## Type of change - [ ] Bug fix - [ ] Feature - [ ] UI / frontend change - [x] Refactor / chore - [ ] Docs - [ ] Test / CI - [x] Breaking change ## Test coverage - [ ] Unit tests added / updated - [ ] Integration tests added / updated - [ ] E2E tests added / updated - [x] Manual verification completed - [x] Existing tests cover this change - [ ] Not applicable ## Coverage notes Renamed the three `test_server_start_*` tests in `tests/cli/test_server_lifecycle.py` to `test_server_background_*` (invoking `server --background`); updated comments in `tests/host/test_local_server.py`. Manually verified routing, help output, and the desktop CLI arg via ad-hoc CliRunner/node checks. ## Changelog `omni server start` is removed; use `omni server --background` to launch the detached managed server instead. |
||
|
Sabhya Chhabria
|
5b4be623c2 |
feat(skills): add polly-e2e-dev skill for orchestrator CUJ testing (#1714)
* feat(skills): add polly-e2e-dev skill for orchestrator CUJ testing Add a polly-e2e-dev agent skill that end-to-end tests the polly multi-agent coding orchestrator's critical user journeys. Ships a deterministic mock-LLM driver (polly_cuj.py) that boots a throwaway local server + mock LLM, rewrites the examples/polly bundle to the openai-agents harness, and scripts the brain to assert the substrate: boot, bridged sys_* tool dispatch, the blast_radius and headless_subagent_purpose_guard guardrail DENYs, and fan-out delegation. SKILL.md adds the live real-CLI recipe (real claude/codex/pi, worktrees, PRs) for polly's judgment-level journeys (investigate/fanout/cross-review) and documents known sharp edges (e.g. the stateful spawn_bounds cap not tripping in the per-call server-side engine). The driver reaps the host-daemon/runner subprocesses an omni-run turn spawns, scoped to the invoking interpreter, so runs never leak processes. * style(skills): apply ruff format to polly_cuj.py Run the repo's ruff-format pre-commit hook so the driver's signatures match the formatter (it collapses wrapped defs that fit on one line), fixing the Pre-commit checks CI job. No behavior change; all five driver scenarios still pass. |
||
|
Sabhya Chhabria
|
30d0692d95 |
feat(skills): add antigravity-native-e2e-dev skill for live local harness testing (#1693)
Document how to exercise the native Antigravity (agy) TUI harness (antigravity-native) end-to-end against a real local Omnigent server + daemon-spawned runner: prerequisites (agy CLI on PATH + OAuth sign-in, tmux), launching `omnigent antigravity`, driving a turn over the web path (the executor types it into the agy TUI as a real USER_INPUT step, mirrored back by the connect-RPC read driver), inspecting the per-session bridge dir + isolated agy HOME + Omnigent MCP relay, targeted scenarios, gotchas, code/test pointers, and tmux/process-tree teardown. Mirrors the cursor/copilot/antigravity-sdk-e2e-dev, pi-native, and claude-native-e2e-test harness skills. Distinct from the in-process `antigravity` Gemini SDK harness. |
||
|
Sabhya Chhabria
|
06d756a1e9 |
feat(skills): add pi-native-e2e-dev skill for live local harness testing (#1675)
Document how to exercise the native Pi TUI harness (pi-native) end-to-end against a real local Omnigent server + daemon-spawned runner: prerequisites (pi CLI / tmux / node / provider auth), launching `omnigent pi`, driving turns through the web -> bridge inbox -> extension path that exercises PiNativeExecutor, inspecting the per-session bridge dir, targeted scenarios, gotchas, code/test pointers, and teardown. Mirrors the existing cursor/copilot/antigravity-sdk-e2e-dev and claude-native-e2e-test harness skills so others can run pi-native locally. |
||
|
Pat Sukprasert
|
db8c58ebe0 |
docs(harness-guide): tier native-harness capabilities (P0/P1/stretch) and add missing rows (#1270)
The native-harness checklist flatly marked all capabilities "required", but even codex-native (one of the most complete native harnesses) fails several. Reorganize the Part 2 checklist into P0 (core), P1 (parity), and Stretch (vendor-dependent) tiers, and add capability rows surfaced by a codex-native audit: tool-output streaming granularity, working-tree diff, generated/viewed media, and vendor-specific modes. Refs: #1254 #1255 #1256 #1257 #1258 Co-authored-by: Isaac |
||
|
Tomu Hirata
|
803cc7d73e |
docs: add harness-integration-guide skill (#1234)
* docs: add harness-integration-guide skill Reference skill describing the full harness feature matrix, implementation patterns, and a prioritized checklist for building new harness integrations. Co-authored-by: Isaac * docs: separate harness and native tracks, make all capabilities required Split the skill into Part 1 (SDK/subprocess) and Part 2 (native) with separate capability matrices, current status tables, and checklists. Removed priority tiers — all capabilities are now required. Co-authored-by: Isaac * docs: remove per-harness status tables and harness-specific examples The skill should describe requirements, not track progress. Removed both "Current harness status" tables and stripped harness names from the implementation pattern tables. Co-authored-by: Isaac * docs: split policies and elicitation into separate capabilities Omnigent policies (DENY, pre-gated, pre-tool hooks) and native elicitation (canUseTool ASK, request_permission, 2-stage cards) are distinct concerns — separate them in the capability matrix, strategy tables, and checklists. Co-authored-by: Isaac * docs: specify ALLOW/ASK/DENY verdicts for tool call and tool result Omnigent policies must support all three verdicts at both checkpoints (tool call and tool result), not just DENY. Co-authored-by: Isaac * docs: simplify native elicitation — it's the web UI for ASK verdicts Native elicitation is just surfacing ASK verdicts in the Omnigent web UI, not a separate strategy taxonomy. Co-authored-by: Isaac * docs: remove stdio serve-mcp implementation detail Co-authored-by: Isaac * docs: add cost tracking, remove transport types section Co-authored-by: Isaac * docs: clarify MCP connectivity — list all Omnigent builtin tools MCP connectivity means the harness bridges Omnigent's builtin MCP tools (session, agent, policy, async, skill, comments, web) to the model. Co-authored-by: Isaac * docs: remove E2E skill checklist item Co-authored-by: Isaac |
||
|
Sabhya Chhabria
|
c3554abee7 |
feat(skills): add cli-setup-verify skill for isolated CLI setup/UX verification (#1085)
* feat(skills): add cli-setup-verify skill for isolated CLI setup/UX verification Adds a skill that lets an agent drive the real `omnigent` CLI through a PTY inside a throwaway OMNIGENT_CONFIG_HOME / OMNIGENT_DATA_DIR sandbox to verify the setup/onboarding flow, terminal UI/UX, and critical user journeys — without a browser, without real credentials, and without touching the developer's real ~/.omnigent. The bundled `verify_cli.py` engine: - isolates every write via the CLI's own knobs and fingerprints the real ~/.omnigent (stat-only) before/after, reporting `real_config_untouched`; - simulates a fresh machine (`--isolate-home`, `--strip-path`) and captures ANSI-stripped frames at 80x24 for UX inspection; - ships 5 scenarios (check-isolation, cold-start, setup-snapshot, help-snapshot, repl-commands) whose checks/notes flip between a before→after baseline diff, so a fix is provable rather than asserted; unreachable surfaces report `skipped`, never a false pass. Builds on the existing pexpect/snapshot e2e infrastructure (tests/e2e/omnigent/_pexpect_harness.py, _snapshot.py). Co-authored-by: Isaac * fix(skills): make HOME isolation the default + detect diagnostics-log writes Addresses the Polly review's blocking issue: the "never touches the real ~/.omnigent" guarantee was false without --isolate-home, because the CLI's diagnostics logger writes cli-*.log under state_dir() = Path.home()/.omnigent, which ignores OMNIGENT_CONFIG_HOME / OMNIGENT_DATA_DIR. - Redirect HOME into the sandbox BY DEFAULT (the only knob that contains diagnostics); replace opt-in --isolate-home with opt-out --inherit-home for the credentialed-REPL case, documented as the less-safe mode. - Broaden fingerprint_real_config() to also tripwire new logs/cli-*.log basenames (stat-only, bounded by the log cap), so real_config_untouched can actually detect a real-home write. Verified: default run → untouched=True; --inherit-home running a non-help command → untouched=False (guard trips). - repl-commands: drop the misleading `/help or /quit` check; assert the /help command list rendered and keep /quit as the quit_advertised note. - _kill_tree: reap the full descendant tree (recursive pgrep -P walk), snapshot before close() so reparented grandchildren are still reachable — matching the "non-negotiable teardown" framing. - SKILL.md: correct the safety prose to reflect default HOME isolation, the --inherit-home tradeoff, and the broadened fingerprint. Co-authored-by: Isaac |
||
|
Sabhya Chhabria
|
ae774b8f79 |
feat(harness): add GitHub Copilot SDK harness (#330)
* feat(harness): add GitHub Copilot SDK harness
Add a first-party `harness: copilot` that drives the GitHub Copilot SDK
(`github-copilot-sdk`), mirroring how the cursor and antigravity SDK
harnesses are wired. The Python SDK bundles the Copilot CLI binary it
drives as a backing server, so the harness needs only the pip dependency
(optional `copilot` extra, lazy-imported) — no separate CLI install.
- `omnigent/inner/copilot_executor.py`: `CopilotExecutor` — one persistent
`CopilotClient` + `CopilotSession` per conversation, streaming
`SessionEvent`s into ExecutorEvents (text/reasoning deltas, tool
execution, usage). Omnigent `sys_*` tools bridge in-process via SDK
`Tool`s whose async handler routes to `_tool_executor` (awaited in the
SDK's own loop — no thread hop). PHASE_LLM_REQUEST/RESPONSE policy parity.
- `omnigent/inner/copilot_harness.py`: the `create_app()` wrap reading
`HARNESS_COPILOT_*` env vars.
- `omnigent/onboarding/copilot_auth.py`: a GitHub token store (dedicated
`copilot:` config block + secret store), resolved like the cursor key.
- Wiring: harness registry, spec allowlist + `github-copilot` alias,
spawn-env builder, runner dispatch + model-env map, model-override set,
readiness check, `omnigent setup` management, ap-web label, docs.
- Auth: a GitHub token with Copilot access (fine-grained PAT w/ "Copilot
Requests", or a gh/Copilot-CLI OAuth token). No Databricks gateway path.
- `pyproject.toml` / `uv.lock`: `copilot` extra (`github-copilot-sdk>=1,<2`).
- Tests: executor (fake-SDK), harness wrap, spawn-env, auth; readiness
test updated for the new spellings.
Verified end-to-end against a local server: a standalone copilot agent,
an agentic file create/read tool loop, and polly + debby running their
orchestrator brain on `--harness copilot`.
Co-authored-by: Isaac
* fix(copilot): reap CLI on start failure + don't mask mid-turn errors; add e2e skill
Fixes found by a live multi-agent bug-bash of the copilot harness:
- HIGH: `client.start()` ran outside the cleanup try/except, so a start
failure (bad token, version skew) dropped the only reference to the
client without stopping it — orphaning the bundled Copilot CLI subprocess
(the SDK only reaps it in `stop()`, never on a start error path). Moved
`start()` inside the try so `_safe_stop(client)` covers it.
- LOW: a `SESSION_ERROR` / `MODEL_CALL_FAILURE` arriving after partial text
streamed was masked — the turn was reported as a clean `TurnComplete`
with the partial text. Now surface it as an `ExecutorError` whenever the
SDK returned no successful final message, even if some text streamed.
- Document the known limitation (parity with cursor): Copilot's *native*
tools (create/view/edit/bash) run inside the SDK, so they bypass
`on:[tool_call]` policies and leave no transcript item; bridged `sys_*`
tools are gated + recorded. Gate built-ins at the LLM phase or sandbox.
- Add the `copilot-sdk-e2e-dev` skill (parity with cursor/antigravity),
capturing the test recipe and the bug-bash's known sharp edges.
- Tests: cover the start-failure teardown and the mid-turn-error-not-masked
paths.
The bug-bash also surfaced two pre-existing, harness-agnostic issues left
out of scope (native-tool transcript items in the shared executor adapter;
top-level `policies:` silently dropped in the shared spec parser).
Co-authored-by: Isaac
* test(copilot): address review findings + prove polly-on-copilot brain e2e
Adversarial swarm review + live polly e2e of the Copilot SDK harness
surfaced small correctness fixes and coverage gaps; this addresses them
and adds durable e2e coverage for copilot as polly's orchestrator brain.
Code fixes:
- copilot_executor: unwrap the SDK's structured TOOL_EXECUTION_COMPLETE
error ({"message","code"}) and result wrapper ({"content",...}) so the
tool error/result carry the payload, not a Python dict repr.
- cli: list `copilot` in the --harness help text (parity with peers).
Tests (executor): policy-deny gates (PHASE_LLM_REQUEST/RESPONSE), session
restart on tool/model change, mid-turn send_and_wait failure (retryable +
recreate), tool-result unwrap + BLOCKED/CANCELLED classification, interrupt,
empty-prompt, no-tool-executor branch, paragraph break, cache_read accumulation.
Tests (harness wrap): assert real adapter routes + os_env/bundle_dir/ambient
token. Tests (auth): inline github_token + dangling keychain ref.
E2E:
- add gated real-network tests/e2e/test_polly_copilot_e2e.py (polly brain on
--harness copilot; skipped without a Copilot token, like the CLI probes).
- document the polly-brain recipe in the copilot-sdk-e2e-dev skill.
- exclude copilot from the gateway-auth live-matrix coverage test (it auths
via a GitHub token, no Databricks gateway — same as cursor/antigravity).
Also fix model_override.py formatting (ruff).
Co-authored-by: Isaac
|
||
|
Sabhya Chhabria
|
fc8bcf9d8c |
docs(skill): add antigravity-sdk-e2e-dev skill for live antigravity harness dev/testing (#287)
A doc-based recipe (modeled on cursor-sdk-e2e-dev, #238) to exercise the Gemini-native Antigravity SDK harness end-to-end against a live local server. |
||
|
Sabhya Chhabria
|
439eb645fe |
docs(skill): add cursor-sdk-e2e-dev skill for live cursor harness dev/testing (#238)
* docs(skill): add cursor-sdk-e2e-dev skill for live harness dev/testing Captures the proven recipe for exercising the Cursor SDK harness end-to-end: start a local server, build a cursor agent bundle, run real turns via the local-runner topology, smoke-test, and bug-bash. Documents the gotchas that bite in practice — config `server:` defaults to a remote server so `--server` is required for local testing; a spec_version spec must be a dir + config.yaml, not a single yaml; the crsr_ key comes from `omni setup`; cursor has no Databricks gateway (databricks-* silently -> auto); turns take 30-90s — and points at the harness code + the unit / gated-e2e tests. Co-authored-by: Isaac * docs(skill): fold live bug-bash learnings into cursor-sdk-e2e-dev Add valid-model-id gotcha (bare gpt-5 is rejected; use the SDK's catalog) and a 'known sharp edges' section capturing live-observed cursor behaviors: swallowed start failures, built-in coding tools bypassing on:[tool_call] guardrails, run-on assistant text, and bridge orphaning on non-graceful exit. Co-authored-by: Isaac |