feat(sandbox): inject omnigent host config into managed sandboxes at launch (#2306)
Managed sandbox hosts boot in a fresh HOME with env-var credentials only,
so there was no way to give them config.yaml-level configuration — locking
provider-agnostic harnesses like pi out of self-hosted model gateways
(LiteLLM/vLLM) in managed sessions.
- New top-level `sandbox.host_config:` server config key — verbatim
in-sandbox ~/.omnigent/config.yaml content (e.g. a providers: block with
kind: gateway, default: [pi]), provider-agnostic across all managed
launch providers.
- Validated fail-loud at server startup: mapping shape, providers block
through the same provider_config parser omnigent itself uses (secrets
deliberately not resolved — api_key_ref: env:VAR names sandbox env),
inline api_key literals rejected at parse time, the block's own default
scopes checked for collisions, plus a JSON round-trip so YAML-native
values can't fail every launch at runtime.
- Materialized before `omnigent host` starts, from one shared rendering
primitive so merge semantics can't drift between providers: exec-model
providers run a self-contained python3 -c merge script (stdlib+yaml
only) via the shared SandboxLauncher.start_host; kubernetes appends the
same rendered command to its init-container prep script, landing the
file on the HOME emptyDir before the main container boots the host.
- Merge mirrors cli.py's deep_merge_keys=("providers",): providers entries
merge one level deep (injected wins), other top-level keys replace
wholesale. The payload rides base64, so arbitrary YAML content never
touches shell quoting.
- Server-managed replacement semantics: a marker file records what was
injected, and each launch/resume removes those entries by name before
merging the current payload — a renamed gateway or a removed host_config
block cleans up on the next wake instead of stranding stale providers.
User-created config in the sandbox survives; config and marker are
written atomically. A missing or corrupt marker degrades to additive
merging — never delete without evidence of what was injected.
Closes #2126
Signed-off-by: Bryan Li <bryan.li@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Bryan Li
@@ -59,6 +59,18 @@ sandbox:
|
||||
server_url: https://omnigent.example.com # the in-box host dials this back
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
`provider` + `server_url` is a complete config: the image defaults to
|
||||
the official prebaked host image and boxes run locally.
|
||||
|
||||
|
||||
@@ -136,6 +136,18 @@ sandbox:
|
||||
server_url: https://your-host # public URL sandboxes dial back to
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
`provider` + `server_url` is a complete config. `server_url` **must be reachable
|
||||
from CoreWeave** — the host inside the sandbox opens an outbound WebSocket to it,
|
||||
not `localhost`. For local testing, expose your server with a tunnel
|
||||
|
||||
@@ -151,6 +151,18 @@ sandbox:
|
||||
env: [OPENAI_API_KEY, ANTHROPIC_API_KEY, GIT_TOKEN]
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
## Credentials for the sandbox (LLM keys, git tokens)
|
||||
|
||||
Daytona has no provider-side named-secret store to attach at sandbox
|
||||
|
||||
@@ -138,6 +138,18 @@ sandbox:
|
||||
server_url: https://your-host # public URL sandboxes dial back to
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
`server_url` must be reachable *from E2B's cloud* — a public HTTPS URL,
|
||||
not `localhost`. Sessions created with `host_type: "managed"` (the API
|
||||
call or the Web UI's New Sandbox option) then run on a fresh E2B sandbox;
|
||||
|
||||
@@ -191,6 +191,18 @@ sandbox:
|
||||
server_url: https://your-host # public URL sandboxes dial back to
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
`server_url` must be reachable *from Islo's cloud* — a public HTTPS URL,
|
||||
not `localhost`. The server itself needs `ISLO_API_KEY` (and optional
|
||||
`ISLO_BASE_URL`) in its environment. Sessions created with
|
||||
|
||||
@@ -128,6 +128,7 @@ writing nothing to disk — use HTTPS repository URLs. Details by provider match
|
||||
| Key | Meaning |
|
||||
|---|---|
|
||||
| `server_url` | URL the runner Pod's host dials back to (in-cluster service DNS by default). |
|
||||
| `host_config` | Optional, top-level under `sandbox:` (provider-agnostic, not inside `kubernetes:`): verbatim in-sandbox `~/.omnigent/config.yaml` content installed before `omnigent host` starts — e.g. a `providers:` block routing the `pi` harness through a self-hosted gateway (LiteLLM/vLLM). Server-managed: entries injected by a previous launch are replaced or removed on the next launch/resume; config created inside the sandbox survives. Keep secrets out via `api_key_ref: env:VAR`, resolved inside the runner Pod against the `secret_name` Secret. Validated at server startup. |
|
||||
| `namespace` | Runner-Pod namespace (defaults to `omnigent-sandboxes`). |
|
||||
| `secret_name` | Harness-creds Secret projected into every Pod via `envFrom`. |
|
||||
| `service_account` | ServiceAccount the runner Pods run as (powerless). |
|
||||
@@ -138,6 +139,11 @@ writing nothing to disk — use HTTPS repository URLs. Details by provider match
|
||||
| `in_cluster` | Optional cluster-config source: `true` (in-cluster SA only), `false` (kubeconfig only), omit (try in-cluster, then kubeconfig). |
|
||||
| `kubeconfig` | Optional kubeconfig path for the out-of-cluster fallback (env: `OMNIGENT_KUBERNETES_KUBECONFIG`). |
|
||||
|
||||
To verify `host_config` end to end against a live cluster, run
|
||||
`python tests/e2e/integrations/deploy/kubernetes/e2e_managed_host_config.py
|
||||
--server <url>` — it creates a managed session and asserts the injected
|
||||
config inside the runner Pod.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- **Launch fails fast with a clear reason.** When a Pod can't schedule, pull its
|
||||
|
||||
@@ -19,6 +19,16 @@ data:
|
||||
# Service listens on port 80) is simplest; use your ingress URL if runner
|
||||
# Pods must reach the server through it.
|
||||
server_url: http://omnigent.omnigent.svc.cluster.local
|
||||
# ── optional, provider-agnostic ──
|
||||
# host_config: # verbatim in-sandbox ~/.omnigent/config.yaml content,
|
||||
# providers: # merged in before `omnigent host` starts — e.g. route
|
||||
# litellm: # the `pi` harness through a self-hosted gateway.
|
||||
# kind: gateway # Keep secrets out: api_key_ref: env: resolves inside
|
||||
# default: [pi] # the runner Pod against the secret_name Secret below.
|
||||
# openai:
|
||||
# base_url: http://litellm.litellm.svc.cluster.local/v1
|
||||
# api_key_ref: env:LITELLM_API_KEY
|
||||
# wire_api: chat
|
||||
kubernetes:
|
||||
# Runner-Pod namespace (secret_name / service_account resolve here).
|
||||
namespace: omnigent-sandboxes
|
||||
|
||||
@@ -302,6 +302,18 @@ sandbox:
|
||||
secrets: [omnigent-llm] # Modal secrets to inject
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
### LLM credentials for managed sandboxes
|
||||
|
||||
A fresh sandbox has no API keys. Park your provider credentials in a
|
||||
|
||||
@@ -188,6 +188,18 @@ sandbox:
|
||||
server_url: https://your-host # public URL sandboxes dial back to
|
||||
```
|
||||
|
||||
A top-level `sandbox.host_config:` (provider-agnostic) holds verbatim
|
||||
in-sandbox `~/.omnigent/config.yaml` content — e.g. a `providers:`
|
||||
block routing a harness through a self-hosted gateway — installed into
|
||||
the sandbox before `omnigent host` starts. The block is server-managed:
|
||||
entries injected by a previous launch are replaced or removed on the
|
||||
next launch/resume, while config created inside the sandbox survives.
|
||||
Keep secrets out via
|
||||
`api_key_ref: env:VAR` (resolved in the sandbox against the injected
|
||||
env). See the [sandbox-runners config
|
||||
table](../kubernetes/overlays/sandbox-runners/README.md#configuration-sandbox-configyaml)
|
||||
for the shape.
|
||||
|
||||
`provider` + `server_url` is a complete config. Sessions created with
|
||||
`host_type: "managed"` (the API call or the Web UI's New Sandbox option) then run
|
||||
on a fresh OpenShell sandbox; the create returns immediately and provisioning
|
||||
|
||||
@@ -10,10 +10,16 @@ foreground process open. Everything provider-specific (CLI bootstrap, SSH
|
||||
quirks, image contents, pip flags) lives behind a :class:`SandboxLauncher`
|
||||
implementation; everything provider-agnostic (wheel builds, the in-sandbox
|
||||
App OAuth dance, host registration) lives in ``bootstrap``.
|
||||
|
||||
Injected host config uses the loader's ``OMNIGENT_CONFIG_HOME`` resolution,
|
||||
atomically replaces its config and ownership-marker files, and removes a
|
||||
previously injected value only while it remains unchanged by the user.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import secrets
|
||||
import shlex
|
||||
from abc import ABC, abstractmethod
|
||||
@@ -152,6 +158,136 @@ def foreground_kill_command(pidfile: str) -> str:
|
||||
)
|
||||
|
||||
|
||||
# In-sandbox write of an injected host config, run via ``python3 -c``.
|
||||
# Self-contained on purpose (stdlib + yaml, both baked into any image that can
|
||||
# run ``omnigent host``): importing merge logic from the sandbox's installed
|
||||
# omnigent package would tie the feature to the IMAGE's package version, and
|
||||
# operator-supplied images may predate it. ``__PAYLOAD__`` is replaced with a
|
||||
# base64 Python literal — its alphabet has no quote or shell metacharacter, so
|
||||
# arbitrary YAML content can never break out of the script.
|
||||
#
|
||||
# A marker records the previous payload. Each run removes exactly the names it
|
||||
# injected last time — the server OWNS the names/keys it injects, so a renamed
|
||||
# gateway or a removed block never strands a stale entry that could collide on a
|
||||
# ``default`` scope. User-created entries under OTHER names are never in the
|
||||
# marker and so are never touched. A missing or corrupt marker skips removal
|
||||
# entirely — never delete without evidence of what was injected.
|
||||
_HOST_CONFIG_WRITE_SCRIPT: str = """\
|
||||
import base64, json, os, tempfile, yaml
|
||||
|
||||
config_home = os.environ.get("OMNIGENT_CONFIG_HOME")
|
||||
config_dir = config_home if config_home else os.path.join(os.path.expanduser("~"), ".omnigent")
|
||||
path = os.path.join(config_dir, "config.yaml")
|
||||
marker = os.path.join(config_dir, ".injected_host_config.json")
|
||||
existing = {}
|
||||
if os.path.exists(path):
|
||||
with open(path) as f:
|
||||
loaded = yaml.safe_load(f)
|
||||
if isinstance(loaded, dict):
|
||||
existing = loaded
|
||||
previous = {}
|
||||
try:
|
||||
with open(marker) as f:
|
||||
loaded = json.load(f)
|
||||
if isinstance(loaded, dict):
|
||||
previous = loaded
|
||||
except (OSError, ValueError):
|
||||
pass
|
||||
for key, value in previous.items():
|
||||
if key == "providers" and isinstance(value, dict):
|
||||
current = existing.get(key)
|
||||
if isinstance(current, dict):
|
||||
for name in value:
|
||||
current.pop(name, None)
|
||||
if not current:
|
||||
existing.pop(key, None)
|
||||
else:
|
||||
existing.pop(key, None)
|
||||
injected = json.loads(base64.b64decode(__PAYLOAD__).decode())
|
||||
for key, value in injected.items():
|
||||
current = existing.get(key)
|
||||
if key == "providers" and isinstance(current, dict) and isinstance(value, dict):
|
||||
existing[key] = {**current, **value}
|
||||
else:
|
||||
existing[key] = value
|
||||
|
||||
def atomic_write(path, dump):
|
||||
temp_path = None
|
||||
try:
|
||||
with tempfile.NamedTemporaryFile("w", dir=os.path.dirname(path), delete=False) as f:
|
||||
temp_path = f.name
|
||||
dump(f)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(temp_path, path)
|
||||
except BaseException:
|
||||
if temp_path is not None:
|
||||
try:
|
||||
os.remove(temp_path)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
if injected or previous:
|
||||
os.makedirs(config_dir, exist_ok=True)
|
||||
atomic_write(
|
||||
path,
|
||||
lambda f: yaml.safe_dump(existing, f, default_flow_style=False, sort_keys=True),
|
||||
)
|
||||
if injected:
|
||||
atomic_write(marker, lambda f: json.dump(injected, f))
|
||||
elif previous:
|
||||
os.remove(marker)
|
||||
"""
|
||||
|
||||
|
||||
def render_host_config_write_command(host_config: dict[str, object]) -> str:
|
||||
"""
|
||||
Build the remote command that installs *host_config* into the
|
||||
sandbox's config directory before ``omnigent host`` starts. The directory
|
||||
is ``$OMNIGENT_CONFIG_HOME`` when truthy, otherwise ``~/.omnigent``, exactly
|
||||
matching :func:`omnigent.onboarding.provider_config._config_path`.
|
||||
|
||||
Server-managed replacement semantics: the server OWNS the names/keys it
|
||||
injects. Entries recorded in the previous marker are removed first BY NAME,
|
||||
then the current payload merges in with
|
||||
``omnigent.cli._save_global_config``'s
|
||||
``deep_merge_keys=("providers",)`` semantics — ``providers`` one
|
||||
level deep and every other top-level key wholesale. Removing by name (rather
|
||||
than only when unchanged) is deliberate: a renamed gateway must not leave
|
||||
its old entry behind, since two entries claiming the same ``default`` scope
|
||||
is a sandbox load error. User-created config under names the server never
|
||||
injects is never in the marker and so always survives; a name the server
|
||||
injects is server-managed, and an in-sandbox edit to it does not persist
|
||||
across the next replacement. An empty *host_config* renders a pure cleanup
|
||||
command. A missing or corrupt marker skips removal rather than guessing
|
||||
ownership. Shared by both launch seams — the exec-model
|
||||
:meth:`SandboxLauncher.start_host` and the Kubernetes init container — so the
|
||||
behavior cannot drift between providers.
|
||||
|
||||
Both config and marker writes use a fully-written, fsynced temporary file
|
||||
in the destination directory followed by :func:`os.replace`, so an
|
||||
interrupted write cannot expose a truncated destination file. A pre-existing
|
||||
``config.yaml`` symlink is replaced by a real file — durability of the write
|
||||
is favored over following the link, which an internal sandbox config never
|
||||
relies on.
|
||||
|
||||
The payload travels as base64-encoded JSON substituted into a fixed
|
||||
Python script, and the whole script is ``shlex.quote``-wrapped —
|
||||
operator-supplied YAML content (quotes, ``$``, newlines) never
|
||||
reaches shell or Python quoting.
|
||||
|
||||
:param host_config: The validated ``sandbox.host_config`` mapping
|
||||
(see :func:`omnigent.server.managed_hosts.parse_sandbox_config`),
|
||||
or ``{}`` to only remove previously injected entries.
|
||||
:returns: A ``python3 -c '<script>'`` shell command, safe to pass to
|
||||
:meth:`SandboxLauncher.run` or embed in a larger shell script.
|
||||
"""
|
||||
payload = base64.b64encode(json.dumps(host_config).encode()).decode()
|
||||
script = _HOST_CONFIG_WRITE_SCRIPT.replace("__PAYLOAD__", repr(payload))
|
||||
return f"python3 -c {shlex.quote(script)}"
|
||||
|
||||
|
||||
class SandboxCapabilityError(click.ClickException):
|
||||
"""
|
||||
Raised when a launcher does not support an optional primitive.
|
||||
@@ -314,6 +450,7 @@ class SandboxLauncher(ABC):
|
||||
repo_url: str | None = None,
|
||||
repo_branch: str | None = None,
|
||||
repo_name: str | None = None,
|
||||
host_config: dict[str, object] | None = None,
|
||||
on_stage: Callable[[str], None] | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
@@ -321,8 +458,9 @@ class SandboxLauncher(ABC):
|
||||
|
||||
The default is the EXEC model: probe ``$HOME``, create
|
||||
``<HOME>/workspace``, optionally materialize the repository into it (via
|
||||
:meth:`materialize_workspace`, which clones by default), and start the
|
||||
host detached (``setsid``-backgrounded, identity + token in the process
|
||||
:meth:`materialize_workspace`, which clones by default), merge any
|
||||
*host_config* into ``~/.omnigent/config.yaml``, and start the host
|
||||
detached (``setsid``-backgrounded, identity + token in the process
|
||||
environment) — all driven through :meth:`run` / :meth:`run_background`.
|
||||
It is shared by every provider whose sandbox is a bare box the server
|
||||
execs into (Modal, Daytona, …); entrypoint-as-host providers (e.g.
|
||||
@@ -347,6 +485,12 @@ class SandboxLauncher(ABC):
|
||||
:param repo_branch: Branch to clone, or ``None`` for the default branch.
|
||||
:param repo_name: Directory the clone lands in under the workspace, or
|
||||
``None`` when *repo_url* is ``None``.
|
||||
:param host_config: Deployment-supplied ``~/.omnigent/config.yaml``
|
||||
content (the server's ``sandbox.host_config``) installed into the
|
||||
sandbox's config BEFORE the host starts, or ``None``. On resumable
|
||||
launchers ``None`` still runs the cleanup so entries injected by a
|
||||
since-removed block don't outlive it — see
|
||||
:func:`render_host_config_write_command`.
|
||||
:param on_stage: Progress observer invoked with ``"cloning"`` before the
|
||||
clone (when *repo_url* is set) and ``"starting"`` before the host
|
||||
launches. Runs on this (worker) thread, so it must be thread-safe.
|
||||
@@ -379,6 +523,12 @@ class SandboxLauncher(ABC):
|
||||
# online poll resolves it.
|
||||
if on_stage is not None:
|
||||
on_stage("starting")
|
||||
# Resumable sandboxes keep their filesystem, so even with no
|
||||
# host_config the cleanup must run: an operator who removed the block
|
||||
# expects previously injected entries gone on the next wake. Fresh
|
||||
# sandboxes can't carry a stale marker — skip the extra exec there.
|
||||
if host_config is not None or self.can_resume:
|
||||
self.run(sandbox_id, render_host_config_write_command(host_config or {}))
|
||||
env_prefix = " ".join(
|
||||
f"{key}={shlex.quote(value)}"
|
||||
for key, value in (
|
||||
|
||||
@@ -28,7 +28,9 @@ Platform notes that shape this launcher:
|
||||
the Pod runs as the image's non-root ``sandbox`` user (:data:`_RUN_AS_UID`)
|
||||
for least privilege, so ``$HOME`` would be unwritable. The Pod sets ``HOME``
|
||||
to :data:`_HOME_DIR`, mounts an ``emptyDir`` there shared by both containers,
|
||||
and ``fsGroup`` makes it group-writable.
|
||||
and ``fsGroup`` makes it group-writable. When the host receives a literal
|
||||
``OMNIGENT_CONFIG_HOME``, the init container receives the same value so its
|
||||
config injection lands where the host loader reads it.
|
||||
- **PID-1 reaper.** The in-sandbox host re-parents orphaned runner processes to
|
||||
PID 1, so the container command is a tiny supervisor that spawns
|
||||
``omnigent host``, reaps any children, and forwards SIGTERM for prompt,
|
||||
@@ -47,6 +49,7 @@ import contextlib
|
||||
import importlib
|
||||
import logging
|
||||
import os
|
||||
import posixpath
|
||||
import re
|
||||
import shlex
|
||||
import time
|
||||
@@ -65,6 +68,7 @@ from omnigent.onboarding.sandboxes.base import (
|
||||
DEFAULT_HOST_IMAGE,
|
||||
RemoteCommandResult,
|
||||
SandboxLauncher,
|
||||
render_host_config_write_command,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -361,21 +365,27 @@ def _render_workspace_prep_command(
|
||||
clone_dir: str | None,
|
||||
repo_url: str | None,
|
||||
repo_branch: str | None,
|
||||
host_config: dict[str, object] | None = None,
|
||||
) -> list[str]:
|
||||
"""
|
||||
Render the init container command that prepares the workspace.
|
||||
|
||||
Creates ``<workspace>`` and, when a repository is requested, clones it into
|
||||
``<clone_dir>`` BEFORE the host starts. Running in an init container means a
|
||||
clone failure terminates the init container non-zero — surfaced fast by the
|
||||
start wait with the git error as the container log tail — rather than
|
||||
silently leaving the host without its workspace.
|
||||
Creates ``<workspace>``, clones the repository into ``<clone_dir>`` when
|
||||
requested, and merges *host_config* into ``config.yaml`` under
|
||||
``$OMNIGENT_CONFIG_HOME`` or the default ``~/.omnigent`` when set — all
|
||||
BEFORE the host starts. Running in an init container means a failure
|
||||
terminates the init container non-zero — surfaced fast by the start wait
|
||||
with the error as the container log tail — rather than silently leaving the
|
||||
host without its workspace or provider config.
|
||||
|
||||
:param workspace: The workspace root to create, e.g. ``"/home/omnigent/workspace"``.
|
||||
:param clone_dir: Directory the clone lands in, or ``None`` for no clone.
|
||||
:param repo_url: Repository clone URL, or ``None`` for an empty workspace.
|
||||
:param repo_branch: Branch to clone (``--branch … --single-branch``), or
|
||||
``None`` for the default branch.
|
||||
:param host_config: Deployment-supplied config content to merge in (lands
|
||||
under the same config directory seen by the host container), or
|
||||
``None``.
|
||||
:returns: The ``["bash", "-lc", script]`` command.
|
||||
"""
|
||||
script = f"set -e\nmkdir -p {shlex.quote(workspace)}\n"
|
||||
@@ -390,6 +400,8 @@ def _render_workspace_prep_command(
|
||||
else ""
|
||||
)
|
||||
script += f"git clone {branch}-- {shlex.quote(repo_url)} {shlex.quote(clone_dir)}\n"
|
||||
if host_config is not None:
|
||||
script += render_host_config_write_command(host_config) + "\n"
|
||||
return ["bash", "-lc", script]
|
||||
|
||||
|
||||
@@ -461,6 +473,7 @@ def build_pod_manifest(
|
||||
clone_dir: str | None = None,
|
||||
repo_url: str | None = None,
|
||||
repo_branch: str | None = None,
|
||||
host_config: dict[str, object] | None = None,
|
||||
resources: dict[str, object] | None = None,
|
||||
) -> dict[str, object]:
|
||||
"""
|
||||
@@ -510,6 +523,12 @@ def build_pod_manifest(
|
||||
:param clone_dir: Directory the clone lands in, or ``None`` for no clone.
|
||||
:param repo_url: Repository clone URL, or ``None`` for an empty workspace.
|
||||
:param repo_branch: Branch to clone, or ``None`` for the default branch.
|
||||
:param host_config: Deployment-supplied config content merged in by the
|
||||
init container under the host's resolved config directory, or ``None``.
|
||||
Non-secret by design:
|
||||
credentials stay behind ``api_key_ref: env:`` indirection (resolved in
|
||||
the sandbox against the ``envFrom`` harness Secret), so embedding the
|
||||
content in the init container's command is as safe as the clone URL.
|
||||
:param resources: Configured resources block, or ``None`` for the defaults.
|
||||
:returns: The Pod manifest dict.
|
||||
"""
|
||||
@@ -520,12 +539,42 @@ def build_pod_manifest(
|
||||
}
|
||||
home_mount = [{"name": "home", "mountPath": _HOME_DIR}]
|
||||
|
||||
init_env = [{"name": "HOME", "value": _HOME_DIR}]
|
||||
config_home = env_literals.get("OMNIGENT_CONFIG_HOME")
|
||||
if config_home is not None:
|
||||
# Init and host containers share ONLY the HOME emptyDir, and both run
|
||||
# with workingDir=_HOME_DIR. The injected config the init container
|
||||
# writes is visible to the host only if its directory resolves under
|
||||
# HOME — otherwise the write lands in the init container's private
|
||||
# filesystem and the host silently boots without its providers. An empty
|
||||
# value is falsy: the writer (and host loader) treat it as unset
|
||||
# (~/.omnigent), so only a non-empty override is checked. Resolve
|
||||
# relative to HOME (the shared workingDir) and normalize so a ``..``
|
||||
# segment can't slip past the prefix check, then fail the launch loudly.
|
||||
# A runtime symlink under HOME pointing elsewhere can still defeat this
|
||||
# lexical check, so an operator must not aim OMNIGENT_CONFIG_HOME inside
|
||||
# the cloned workspace. Use posixpath: the target is always a POSIX Pod,
|
||||
# even when the server building this manifest runs on Windows.
|
||||
resolved_home = posixpath.normpath(posixpath.join(_HOME_DIR, config_home))
|
||||
if (
|
||||
config_home
|
||||
and host_config is not None
|
||||
and not (resolved_home == _HOME_DIR or resolved_home.startswith(_HOME_DIR + "/"))
|
||||
):
|
||||
raise ValueError(
|
||||
f"OMNIGENT_CONFIG_HOME ({config_home!r}) must resolve under {_HOME_DIR!r} "
|
||||
"when sandbox.host_config is set — the init container that writes the "
|
||||
"injected config shares only the HOME volume with the host"
|
||||
)
|
||||
init_env.append({"name": "OMNIGENT_CONFIG_HOME", "value": config_home})
|
||||
init_container: dict[str, object] = {
|
||||
"name": _INIT_CONTAINER_NAME,
|
||||
"image": image,
|
||||
"workingDir": _HOME_DIR,
|
||||
"command": _render_workspace_prep_command(workspace, clone_dir, repo_url, repo_branch),
|
||||
"env": [{"name": "HOME", "value": _HOME_DIR}],
|
||||
"command": _render_workspace_prep_command(
|
||||
workspace, clone_dir, repo_url, repo_branch, host_config
|
||||
),
|
||||
"env": init_env,
|
||||
"resources": pod_resources,
|
||||
"securityContext": container_security,
|
||||
"volumeMounts": home_mount,
|
||||
@@ -1021,6 +1070,7 @@ class KubernetesSandboxLauncher(SandboxLauncher):
|
||||
repo_url: str | None = None,
|
||||
repo_branch: str | None = None,
|
||||
repo_name: str | None = None,
|
||||
host_config: dict[str, object] | None = None,
|
||||
on_stage: Callable[[str], None] | None = None,
|
||||
) -> str:
|
||||
"""
|
||||
@@ -1045,6 +1095,9 @@ class KubernetesSandboxLauncher(SandboxLauncher):
|
||||
:param repo_url: Repository clone URL, or ``None`` for an empty workspace.
|
||||
:param repo_branch: Branch to clone, or ``None`` for the default branch.
|
||||
:param repo_name: Directory the clone lands in, or ``None``.
|
||||
:param host_config: Deployment-supplied ``~/.omnigent/config.yaml``
|
||||
content the init container merges in before the host starts, or
|
||||
``None``.
|
||||
:param on_stage: Progress observer; invoked with ``"starting"``.
|
||||
:returns: The absolute in-sandbox workspace path (the cloned repository
|
||||
directory when *repo_url* is set).
|
||||
@@ -1069,17 +1122,9 @@ class KubernetesSandboxLauncher(SandboxLauncher):
|
||||
)
|
||||
try:
|
||||
try:
|
||||
# Secret first so the Pod's secretKeyRef resolves immediately —
|
||||
# a Pod referencing a missing Secret would sit in
|
||||
# CreateContainerConfigError (which the start wait treats as
|
||||
# terminal).
|
||||
core.create_namespaced_secret(
|
||||
namespace,
|
||||
build_token_secret_manifest(
|
||||
secret_name=secret_name, namespace=namespace, token=token
|
||||
),
|
||||
_request_timeout=_POD_READY_REQUEST_TIMEOUT_S,
|
||||
)
|
||||
# Build the (side-effect-free) manifest first: it validates
|
||||
# host_config placement and can raise, so nothing should have
|
||||
# been created in the cluster yet when it does.
|
||||
manifest = build_pod_manifest(
|
||||
pod_name=sandbox_id,
|
||||
namespace=namespace,
|
||||
@@ -1096,8 +1141,20 @@ class KubernetesSandboxLauncher(SandboxLauncher):
|
||||
clone_dir=clone_dir,
|
||||
repo_url=repo_url,
|
||||
repo_branch=repo_branch,
|
||||
host_config=host_config,
|
||||
resources=self._resources,
|
||||
)
|
||||
# Secret before Pod so the Pod's secretKeyRef resolves
|
||||
# immediately — a Pod referencing a missing Secret would sit in
|
||||
# CreateContainerConfigError (which the start wait treats as
|
||||
# terminal).
|
||||
core.create_namespaced_secret(
|
||||
namespace,
|
||||
build_token_secret_manifest(
|
||||
secret_name=secret_name, namespace=namespace, token=token
|
||||
),
|
||||
_request_timeout=_POD_READY_REQUEST_TIMEOUT_S,
|
||||
)
|
||||
core.create_namespaced_pod(
|
||||
namespace, manifest, _request_timeout=_POD_READY_REQUEST_TIMEOUT_S
|
||||
)
|
||||
|
||||
@@ -34,6 +34,25 @@ stores into ``create_app``):
|
||||
sandbox:
|
||||
provider: modal # lakebox|modal|daytona|boxlite|cwsandbox|islo|e2b|openshell
|
||||
server_url: https://omnigent.example.com
|
||||
host_config: # optional; provider-agnostic. Verbatim
|
||||
# in-sandbox ~/.omnigent/config.yaml content,
|
||||
# installed before `omnigent host` starts
|
||||
# (e.g. route the `pi` harness through a
|
||||
# self-hosted gateway). Server-managed:
|
||||
# entries injected earlier are replaced or
|
||||
# removed on the next launch/resume; user
|
||||
# config in the sandbox survives. Keep
|
||||
# secrets out via api_key_ref: env: —
|
||||
# resolved in the SANDBOX env (harness
|
||||
# Secret / provider env lane).
|
||||
providers:
|
||||
litellm:
|
||||
kind: gateway
|
||||
default: [pi]
|
||||
openai:
|
||||
base_url: http://litellm.litellm.svc.cluster.local/v1
|
||||
api_key_ref: env:LITELLM_API_KEY
|
||||
wire_api: chat
|
||||
modal: # optional block
|
||||
image: docker.io/me/omnigent-host:latest # default: official image
|
||||
secrets: [omnigent-llm] # Modal secrets injected as sandbox env
|
||||
@@ -367,6 +386,17 @@ class ManagedSandboxConfig:
|
||||
falls back to the generic "New Sandbox" label. Exposed (when
|
||||
managed launch is supported) on the unauthenticated
|
||||
``GET /v1/info`` as ``sandbox_provider``.
|
||||
:param host_config: Verbatim in-sandbox ``~/.omnigent/config.yaml``
|
||||
content (e.g. a ``providers:`` block routing a harness through
|
||||
a self-hosted gateway) installed into the sandbox's config before
|
||||
``omnigent host`` starts, or ``None``. Server-managed: previously
|
||||
injected entries are replaced or removed on each launch/resume so
|
||||
the sandbox always reflects the current block. Provider-agnostic:
|
||||
forwarded to every launcher's ``start_host`` — see
|
||||
:func:`omnigent.onboarding.sandboxes.base.render_host_config_write_command`.
|
||||
Non-secret by design: credentials stay behind
|
||||
``api_key_ref: env:VAR`` indirection, resolved inside the
|
||||
sandbox against its own environment.
|
||||
"""
|
||||
|
||||
server_url: str
|
||||
@@ -374,6 +404,7 @@ class ManagedSandboxConfig:
|
||||
token_ttl_s: int
|
||||
managed_launch_supported: bool = True
|
||||
provider: str | None = None
|
||||
host_config: dict[str, object] | None = None
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -602,6 +633,95 @@ def _unsupported_launcher_factory(provider: str) -> Callable[[], SandboxLauncher
|
||||
return _reject
|
||||
|
||||
|
||||
def _parse_host_config(raw: dict[str, object]) -> dict[str, object] | None:
|
||||
"""
|
||||
Extract and validate the top-level ``sandbox.host_config`` block.
|
||||
|
||||
Verbatim in-sandbox ``~/.omnigent/config.yaml`` content forwarded at
|
||||
managed launch (see :class:`ManagedSandboxConfig`). When a
|
||||
``providers`` key is present, its SHAPE is validated through the same
|
||||
parser ``omnigent`` itself uses — structurally only: secret
|
||||
references (``api_key_ref: env:VAR``) name variables in the
|
||||
SANDBOX's environment, not the server's, so they are deliberately
|
||||
never resolved here. Validating at parse time matters doubly for
|
||||
this block: inside the sandbox a malformed ``providers`` entry
|
||||
degrades silently (the harness falls back to its own login), so
|
||||
server startup is the only place a typo can fail loud.
|
||||
|
||||
:param raw: The raw ``sandbox`` mapping.
|
||||
:returns: The validated ``host_config`` mapping, or ``None`` when
|
||||
the key is absent.
|
||||
:raises ValueError: When present but not a mapping, or when its
|
||||
``providers`` block fails shape validation.
|
||||
"""
|
||||
host_config = raw.get("host_config")
|
||||
if host_config is None:
|
||||
return None
|
||||
if not isinstance(host_config, dict):
|
||||
raise ValueError(
|
||||
"server config 'sandbox.host_config' must be a mapping — verbatim "
|
||||
"in-sandbox ~/.omnigent/config.yaml content merged in before "
|
||||
"'omnigent host' starts"
|
||||
)
|
||||
# Key presence, not get(): an explicit `providers: null` would skip
|
||||
# validation here yet still ride to the sandbox, where the merge writes
|
||||
# `providers: null` over any existing block — the silent degradation this
|
||||
# parse exists to prevent.
|
||||
if "providers" in host_config:
|
||||
providers = host_config["providers"]
|
||||
# load_providers silently ignores a non-mapping providers value, so
|
||||
# the mapping check must happen here to fail loud.
|
||||
if not isinstance(providers, dict):
|
||||
raise ValueError("server config 'sandbox.host_config.providers' must be a mapping")
|
||||
# Lazy imports, matching the provider branches below: the parse path
|
||||
# must not pull the onboarding layer in at module import time.
|
||||
from omnigent.errors import OmnigentError
|
||||
from omnigent.onboarding.provider_config import get_default_provider, load_providers
|
||||
|
||||
try:
|
||||
parsed_providers = load_providers(host_config)
|
||||
default_scopes = {
|
||||
scope
|
||||
for provider in parsed_providers.values()
|
||||
for scope in provider.default_families
|
||||
}
|
||||
for scope in sorted(default_scopes):
|
||||
get_default_provider(host_config, scope)
|
||||
except OmnigentError as exc:
|
||||
raise ValueError(
|
||||
f"server config 'sandbox.host_config.providers' is invalid: {exc}"
|
||||
) from exc
|
||||
for provider in parsed_providers.values():
|
||||
for family_name, family in provider.families.items():
|
||||
if family.api_key is not None:
|
||||
raise ValueError(
|
||||
"server config "
|
||||
f"'sandbox.host_config.providers.{provider.name}."
|
||||
f"{family_name}.api_key' must not contain an inline API key — "
|
||||
"use api_key_ref: env:VAR instead"
|
||||
)
|
||||
# The block rides json.dumps to the sandbox on every launch, and
|
||||
# yaml.safe_load produces values json can't take (an unquoted date
|
||||
# becomes datetime.date) — round-trip now so that fails startup, not
|
||||
# every launch.
|
||||
import json
|
||||
|
||||
try:
|
||||
serialized = json.dumps(host_config)
|
||||
round_tripped = json.loads(serialized)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise ValueError(
|
||||
f"server config 'sandbox.host_config' must be JSON-serializable "
|
||||
f"(quote YAML scalars like dates): {exc}"
|
||||
) from exc
|
||||
if round_tripped != host_config:
|
||||
raise ValueError(
|
||||
"server config 'sandbox.host_config' must be JSON-serializable without loss "
|
||||
"(mapping keys must be strings and values must preserve their JSON types)"
|
||||
)
|
||||
return host_config
|
||||
|
||||
|
||||
def parse_sandbox_config(raw: object) -> ManagedSandboxConfig | None:
|
||||
"""
|
||||
Parse and validate the server config's ``sandbox:`` section.
|
||||
@@ -633,6 +753,9 @@ def parse_sandbox_config(raw: object) -> ManagedSandboxConfig | None:
|
||||
"server config 'sandbox.server_url' is required — the public URL "
|
||||
"of this server that sandboxed hosts connect back to"
|
||||
)
|
||||
# Validated regardless of provider (like server_url): a malformed
|
||||
# host_config should stop startup even for staged/unsupported providers.
|
||||
host_config = _parse_host_config(raw)
|
||||
if provider == "modal":
|
||||
launcher_factory = _modal_launcher_factory(
|
||||
_parse_modal_image(raw), _parse_modal_secrets(raw)
|
||||
@@ -721,6 +844,7 @@ def parse_sandbox_config(raw: object) -> ManagedSandboxConfig | None:
|
||||
token_ttl_s=token_ttl_s,
|
||||
managed_launch_supported=provider in PROVIDERS_WITH_MANAGED_LAUNCH,
|
||||
provider=provider,
|
||||
host_config=host_config,
|
||||
)
|
||||
|
||||
|
||||
@@ -1943,6 +2067,9 @@ async def _arm_and_start_host(
|
||||
repo_branch=repo.branch if repo is not None else None,
|
||||
repo_name=repo.repo_name if repo is not None else None,
|
||||
on_stage=on_stage,
|
||||
# Omitted entirely when unset: a deployment-injected launcher
|
||||
# predating the host_config parameter must keep launching.
|
||||
**({"host_config": config.host_config} if config.host_config is not None else {}),
|
||||
)
|
||||
await _wait_for_host_online(host_store, host_id)
|
||||
except Exception as exc:
|
||||
@@ -2166,6 +2293,13 @@ async def resume_managed_host(
|
||||
host_name=host.name,
|
||||
server_url=config.server_url,
|
||||
repo_url=None, # the persistent volume already holds the workspace
|
||||
# Re-materialized on every wake, so an operator's host_config
|
||||
# change lands on the next resume without a new sandbox.
|
||||
# Omitted entirely when unset: a deployment-injected launcher
|
||||
# predating the host_config parameter must keep resuming.
|
||||
# (Base start_host still cleans up previously injected entries
|
||||
# on resumable launchers when the block is removed.)
|
||||
**({"host_config": config.host_config} if config.host_config is not None else {}),
|
||||
)
|
||||
await _wait_for_host_online(host_store, host.host_id)
|
||||
except Exception as exc:
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
End-to-end test: `sandbox.host_config` injection on the Kubernetes provider.
|
||||
|
||||
Runs against an EXISTING omnigent server already configured with
|
||||
``sandbox.provider: kubernetes`` and a ``sandbox.host_config:`` block (see
|
||||
``deploy/kubernetes/overlays/sandbox-runners/`` — apply the overlay, put a
|
||||
``host_config:`` block in ``sandbox-config.yaml``, and make the server URL
|
||||
reachable from where this script runs, e.g. via ``kubectl port-forward``).
|
||||
|
||||
The script creates a managed session, waits for the runner Pod's host to
|
||||
register, then ``kubectl exec``'s into the Pod and asserts the injected
|
||||
config landed at ``/home/omnigent/.omnigent/config.yaml`` before the host
|
||||
came up. It needs ``kubectl`` on PATH with access to the runner namespace.
|
||||
|
||||
python tests/e2e/integrations/deploy/kubernetes/e2e_managed_host_config.py \
|
||||
--server http://localhost:8080
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
import httpx
|
||||
|
||||
# Constants pinned by the kubernetes launcher (see
|
||||
# omnigent/onboarding/sandboxes/kubernetes.py): the Pod's fixed HOME, the
|
||||
# main container and host-id env names, and the labels stamped on every runner Pod.
|
||||
POD_HOME = "/home/omnigent"
|
||||
HOST_CONTAINER = "host"
|
||||
HOST_ID_ENV_VAR = "OMNIGENT_HOST_ID"
|
||||
POD_SELECTOR = "app.kubernetes.io/managed-by=omnigent,omnigent.ai/role=sandbox-host"
|
||||
DEFAULT_EXPECTED_CONFIG = (
|
||||
"litellm:",
|
||||
"base_url: http://litellm.litellm.svc.cluster.local/v1",
|
||||
)
|
||||
|
||||
|
||||
def log(msg: str) -> None:
|
||||
print(msg, flush=True)
|
||||
|
||||
|
||||
def check_server(base: str) -> None:
|
||||
log(f"[1/5] checking {base}/v1/info")
|
||||
info = httpx.get(f"{base}/v1/info", timeout=10.0).json()
|
||||
if not info.get("managed_sandboxes_enabled"):
|
||||
raise SystemExit("server does not advertise managed sandboxes — is sandbox: configured?")
|
||||
if info.get("sandbox_provider") != "kubernetes":
|
||||
raise SystemExit(
|
||||
f"server's sandbox provider is {info.get('sandbox_provider')!r}, not 'kubernetes'"
|
||||
)
|
||||
log(" ✓ managed sandboxes enabled (kubernetes)")
|
||||
|
||||
|
||||
def pick_agent(base: str, agent_id: str | None) -> str:
|
||||
resp = httpx.get(f"{base}/v1/agents", timeout=10.0)
|
||||
resp.raise_for_status()
|
||||
agents = resp.json()["data"]
|
||||
if not agents:
|
||||
raise SystemExit("no agents registered on the server to bind a session to")
|
||||
if agent_id:
|
||||
if not any(a.get("id") == agent_id for a in agents):
|
||||
raise SystemExit(f"agent_id {agent_id!r} not found on the server")
|
||||
return agent_id
|
||||
chosen = agents[0]
|
||||
log(f" agent_id={chosen['id']} ({chosen.get('name')})")
|
||||
return chosen["id"]
|
||||
|
||||
|
||||
def create_managed_session(base: str, agent_id: str) -> str:
|
||||
log("[2/5] creating managed session")
|
||||
r = httpx.post(
|
||||
f"{base}/v1/sessions",
|
||||
json={"agent_id": agent_id, "host_type": "managed"},
|
||||
timeout=180.0,
|
||||
)
|
||||
if r.status_code >= 300:
|
||||
raise SystemExit(f"create session failed: HTTP {r.status_code}: {r.text[:600]}")
|
||||
conv_id = r.json()["id"]
|
||||
log(f" session={conv_id}")
|
||||
return conv_id
|
||||
|
||||
|
||||
def wait_host_online(base: str, conv_id: str, timeout_s: float) -> str:
|
||||
log("[3/5] waiting for the runner Pod's host to register")
|
||||
deadline = time.monotonic() + timeout_s
|
||||
while time.monotonic() < deadline:
|
||||
d = httpx.get(f"{base}/v1/sessions/{conv_id}", timeout=10.0).json()
|
||||
if d.get("host_id"):
|
||||
log(f" ✓ host online: host_id={d['host_id']}")
|
||||
return d["host_id"]
|
||||
status = d.get("sandbox_status") or {}
|
||||
if status.get("stage") == "failed":
|
||||
raise SystemExit(f"managed launch failed: {status.get('error')}")
|
||||
time.sleep(5.0)
|
||||
raise SystemExit(f"host did not come online within {timeout_s:.0f}s")
|
||||
|
||||
|
||||
def runner_pod_for_host(kubectl: str, namespace: str, host_id: str) -> str:
|
||||
out = subprocess.run(
|
||||
[
|
||||
*shlex.split(kubectl),
|
||||
"get",
|
||||
"pods",
|
||||
"-n",
|
||||
namespace,
|
||||
"-l",
|
||||
POD_SELECTOR,
|
||||
"-o",
|
||||
"json",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
).stdout
|
||||
pods = json.loads(out).get("items", [])
|
||||
matches = []
|
||||
for pod in pods:
|
||||
containers = pod.get("spec", {}).get("containers", [])
|
||||
host = next((item for item in containers if item.get("name") == HOST_CONTAINER), None)
|
||||
env = host.get("env", []) if host else []
|
||||
if any(
|
||||
item.get("name") == HOST_ID_ENV_VAR and item.get("value") == host_id for item in env
|
||||
):
|
||||
matches.append(pod["metadata"]["name"])
|
||||
if not matches:
|
||||
raise SystemExit(f"no runner Pod matching host_id {host_id!r} in namespace {namespace!r}")
|
||||
if len(matches) > 1:
|
||||
raise SystemExit(f"multiple runner Pods match host_id {host_id!r}: {', '.join(matches)}")
|
||||
return f"pod/{matches[0]}"
|
||||
|
||||
|
||||
def assert_injected_config(
|
||||
kubectl: str, namespace: str, pod: str, expected: list[str] | tuple[str, ...]
|
||||
) -> str:
|
||||
log(f"[4/5] reading {POD_HOME}/.omnigent/config.yaml from {pod}")
|
||||
proc = subprocess.run(
|
||||
[
|
||||
*shlex.split(kubectl),
|
||||
"exec",
|
||||
"-n",
|
||||
namespace,
|
||||
pod,
|
||||
"-c",
|
||||
HOST_CONTAINER,
|
||||
"--",
|
||||
"cat",
|
||||
f"{POD_HOME}/.omnigent/config.yaml",
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
if proc.returncode != 0:
|
||||
raise SystemExit(
|
||||
f"config.yaml missing in the runner Pod — host_config was not injected?\n"
|
||||
f"{proc.stderr.strip()}"
|
||||
)
|
||||
content = proc.stdout
|
||||
log(" --- config.yaml ---")
|
||||
log(content.rstrip())
|
||||
missing = [fragment for fragment in expected if fragment not in content]
|
||||
if missing:
|
||||
raise SystemExit(f"config.yaml does not contain expected fragment(s): {missing!r}")
|
||||
log(f" ✓ contains expected fragments: {expected!r}")
|
||||
return content
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--server", required=True, help="Omnigent server base URL")
|
||||
parser.add_argument("--agent-id", default=None, help="Agent to bind (default: first)")
|
||||
parser.add_argument("--namespace", default="omnigent-sandboxes", help="Runner-Pod namespace")
|
||||
parser.add_argument(
|
||||
"--expect",
|
||||
action="append",
|
||||
default=None,
|
||||
help=(
|
||||
"Substring the injected config.yaml must contain; repeat for multiple fragments "
|
||||
"(default: the documented litellm provider and base_url)"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--kubectl",
|
||||
default="kubectl",
|
||||
help="kubectl command, split shell-style (e.g. 'kubectl --context my-cluster')",
|
||||
)
|
||||
parser.add_argument("--timeout", type=float, default=300.0, help="Host-online wait (s)")
|
||||
parser.add_argument("--keep", action="store_true", help="Skip session cleanup")
|
||||
args = parser.parse_args()
|
||||
base = args.server.rstrip("/")
|
||||
|
||||
check_server(base)
|
||||
agent_id = pick_agent(base, args.agent_id)
|
||||
conv_id = create_managed_session(base, agent_id)
|
||||
try:
|
||||
host_id = wait_host_online(base, conv_id, args.timeout)
|
||||
pod = runner_pod_for_host(args.kubectl, args.namespace, host_id)
|
||||
assert_injected_config(
|
||||
args.kubectl,
|
||||
args.namespace,
|
||||
pod,
|
||||
args.expect or DEFAULT_EXPECTED_CONFIG,
|
||||
)
|
||||
finally:
|
||||
if args.keep:
|
||||
log(f"[5/5] --keep: leaving session {conv_id} (and its Pod) running")
|
||||
else:
|
||||
log(f"[5/5] deleting session {conv_id} (terminates the runner Pod)")
|
||||
try:
|
||||
httpx.delete(f"{base}/v1/sessions/{conv_id}", timeout=60.0)
|
||||
except httpx.HTTPError as exc:
|
||||
log(f" cleanup failed (Pod may linger): {exc}")
|
||||
log("PASS")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -8,10 +8,20 @@ minimal recording launcher rather than per provider.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import ClassVar
|
||||
|
||||
from omnigent.onboarding.sandboxes.base import RemoteCommandResult, SandboxLauncher
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
from omnigent.onboarding.sandboxes.base import (
|
||||
RemoteCommandResult,
|
||||
SandboxLauncher,
|
||||
render_host_config_write_command,
|
||||
)
|
||||
|
||||
|
||||
class _RecordingLauncher(SandboxLauncher):
|
||||
@@ -175,3 +185,427 @@ def test_materialize_workspace_override_resolves_local_checkout_without_cloning(
|
||||
# The host still launched, in the resolved workspace.
|
||||
[raw] = launcher.backgrounded
|
||||
assert raw.endswith("omnigent host --server https://srv")
|
||||
|
||||
|
||||
# ── host_config materialization ────────────────────────────
|
||||
|
||||
_GATEWAY_HOST_CONFIG: dict[str, object] = {
|
||||
"providers": {
|
||||
"litellm": {
|
||||
"kind": "gateway",
|
||||
"default": ["pi"],
|
||||
"openai": {
|
||||
"base_url": "http://litellm.litellm.svc.cluster.local/v1",
|
||||
"api_key_ref": "env:LITELLM_API_KEY",
|
||||
"wire_api": "chat",
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def _run_write_command(
|
||||
command: str,
|
||||
home: Path,
|
||||
*,
|
||||
config_home: Path | None = None,
|
||||
extra_env: dict[str, str] | None = None,
|
||||
check: bool = True,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Run the rendered write command through a real shell + python3."""
|
||||
env = {**os.environ, "HOME": str(home)}
|
||||
env.pop("OMNIGENT_CONFIG_HOME", None)
|
||||
if config_home is not None:
|
||||
env["OMNIGENT_CONFIG_HOME"] = str(config_home)
|
||||
if extra_env is not None:
|
||||
env.update(extra_env)
|
||||
return subprocess.run(
|
||||
["sh", "-c", command],
|
||||
env=env,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=check,
|
||||
)
|
||||
|
||||
|
||||
def _materialize(
|
||||
command: str, home: Path, *, config_home: Path | None = None
|
||||
) -> dict[str, object]:
|
||||
"""Run the command and return the config from its resolved directory."""
|
||||
_run_write_command(command, home, config_home=config_home)
|
||||
config_dir = config_home if config_home is not None else home / ".omnigent"
|
||||
with open(config_dir / "config.yaml") as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def test_render_host_config_write_command_creates_config_from_scratch(tmp_path: Path) -> None:
|
||||
"""A fresh sandbox (no ~/.omnigent at all) gets the injected config verbatim."""
|
||||
written = _materialize(render_host_config_write_command(_GATEWAY_HOST_CONFIG), tmp_path)
|
||||
assert written == _GATEWAY_HOST_CONFIG
|
||||
|
||||
|
||||
def test_render_host_config_write_command_honors_omnigent_config_home(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""The writer uses OMNIGENT_CONFIG_HOME as the config directory itself."""
|
||||
home = tmp_path / "home"
|
||||
home.mkdir()
|
||||
config_home = tmp_path / "custom-config"
|
||||
|
||||
written = _materialize(
|
||||
render_host_config_write_command(_GATEWAY_HOST_CONFIG),
|
||||
home,
|
||||
config_home=config_home,
|
||||
)
|
||||
|
||||
assert written == _GATEWAY_HOST_CONFIG
|
||||
assert (config_home / ".injected_host_config.json").exists()
|
||||
assert not (home / ".omnigent").exists()
|
||||
|
||||
|
||||
def test_render_host_config_write_command_merges_providers_and_replaces_other_keys(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""
|
||||
The merge mirrors cli.py's ``deep_merge_keys=("providers",)``: sibling
|
||||
provider entries survive, an injected entry of the same name wins
|
||||
wholesale, other top-level keys replace, untouched keys persist.
|
||||
"""
|
||||
(tmp_path / ".omnigent").mkdir()
|
||||
(tmp_path / ".omnigent" / "config.yaml").write_text(
|
||||
yaml.safe_dump(
|
||||
{
|
||||
"providers": {
|
||||
"anthropic": {"kind": "key"},
|
||||
"litellm": {"kind": "gateway", "default": True},
|
||||
},
|
||||
"server": "https://old.example.com",
|
||||
"host": {"name": "keep-me"},
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
injected = {**_GATEWAY_HOST_CONFIG, "server": "https://new.example.com"}
|
||||
written = _materialize(render_host_config_write_command(injected), tmp_path)
|
||||
|
||||
providers = written["providers"]
|
||||
assert providers["anthropic"] == {"kind": "key"} # sibling survives
|
||||
# Same-name entry replaced wholesale (no per-entry merge), injected wins.
|
||||
assert providers["litellm"] == _GATEWAY_HOST_CONFIG["providers"]["litellm"]
|
||||
assert written["server"] == "https://new.example.com"
|
||||
assert written["host"] == {"name": "keep-me"}
|
||||
|
||||
|
||||
def test_render_host_config_write_command_survives_hostile_yaml_content(tmp_path: Path) -> None:
|
||||
"""
|
||||
Quotes, ``$VAR``-looking strings, backticks, newlines, and unicode round-trip
|
||||
byte-exact: the payload rides base64 through the shell/python layers, so no
|
||||
operator YAML can break out of the quoting.
|
||||
"""
|
||||
hostile: dict[str, object] = {
|
||||
"providers": {
|
||||
'we\'ird "name"': {
|
||||
"kind": "gateway",
|
||||
"note": "line1\nline2 `tick` $HOME 'single' — ünïcode ✓",
|
||||
}
|
||||
}
|
||||
}
|
||||
written = _materialize(render_host_config_write_command(hostile), tmp_path)
|
||||
assert written == hostile
|
||||
|
||||
|
||||
def test_materialized_config_routes_pi_to_the_gateway(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""
|
||||
The point of the injection: a host booted with the materialized config
|
||||
resolves the gateway as pi's provider through the REAL config loader and
|
||||
harness-routing chain — before any ambient env credential is consulted.
|
||||
"""
|
||||
_materialize(render_host_config_write_command(_GATEWAY_HOST_CONFIG), tmp_path)
|
||||
|
||||
from omnigent.onboarding.provider_config import default_provider_for_harness, load_config
|
||||
|
||||
monkeypatch.setenv("OMNIGENT_CONFIG_HOME", str(tmp_path / ".omnigent"))
|
||||
entry = default_provider_for_harness(load_config(), "pi")
|
||||
|
||||
assert entry is not None
|
||||
assert entry.name == "litellm"
|
||||
assert entry.kind == "gateway"
|
||||
|
||||
|
||||
def test_start_host_writes_host_config_before_launching_the_host() -> None:
|
||||
"""The config write runs via ``run`` strictly before the host is backgrounded."""
|
||||
launcher = _RecordingLauncher()
|
||||
|
||||
launcher.start_host(
|
||||
"sb-1",
|
||||
token="tok-123",
|
||||
host_id="host_abc",
|
||||
host_name="managed-abc",
|
||||
server_url="https://srv",
|
||||
host_config=_GATEWAY_HOST_CONFIG,
|
||||
)
|
||||
|
||||
write_index = launcher.commands.index(render_host_config_write_command(_GATEWAY_HOST_CONFIG))
|
||||
# run_background funnels through run(), so the wrapped host launch is
|
||||
# also in `commands` — the write must precede it.
|
||||
host_index = next(
|
||||
i for i, cmd in enumerate(launcher.commands) if "omnigent host --server" in cmd
|
||||
)
|
||||
assert write_index < host_index
|
||||
|
||||
|
||||
def test_start_host_without_host_config_writes_nothing() -> None:
|
||||
"""No host_config on a fresh-sandbox launcher → no config command at all.
|
||||
|
||||
Non-resumable sandboxes can't carry a stale injection marker, so the
|
||||
cleanup run would be dead weight (and a python3+yaml image requirement
|
||||
for operators who never use the feature).
|
||||
"""
|
||||
launcher = _RecordingLauncher()
|
||||
|
||||
launcher.start_host(
|
||||
"sb-1",
|
||||
token="tok-123",
|
||||
host_id="host_abc",
|
||||
host_name="managed-abc",
|
||||
server_url="https://srv",
|
||||
)
|
||||
|
||||
assert not any(cmd.startswith("python3 -c") for cmd in launcher.commands)
|
||||
|
||||
|
||||
# ── server-managed replacement semantics ────────────────────
|
||||
|
||||
|
||||
def _read_marker(home: Path) -> dict[str, object] | None:
|
||||
marker = home / ".omnigent" / ".injected_host_config.json"
|
||||
if not marker.exists():
|
||||
return None
|
||||
return json.loads(marker.read_text())
|
||||
|
||||
|
||||
def test_render_host_config_write_command_replaces_previously_injected_entries(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""
|
||||
Renaming a gateway in ``sandbox.host_config`` must not leave the old
|
||||
entry behind: two providers claiming the same ``default`` scope is a
|
||||
load error inside the sandbox. Previously injected providers and
|
||||
top-level keys are removed before the current payload merges in;
|
||||
user-created entries survive.
|
||||
"""
|
||||
(tmp_path / ".omnigent").mkdir()
|
||||
(tmp_path / ".omnigent" / "config.yaml").write_text(
|
||||
yaml.safe_dump({"providers": {"mine": {"kind": "key"}}})
|
||||
)
|
||||
first = {
|
||||
"providers": {"gateway_a": {"kind": "gateway", "default": ["pi"]}},
|
||||
"server": "https://old.example.com",
|
||||
}
|
||||
second = {"providers": {"gateway_b": {"kind": "gateway", "default": ["pi"]}}}
|
||||
|
||||
_materialize(render_host_config_write_command(first), tmp_path)
|
||||
written = _materialize(render_host_config_write_command(second), tmp_path)
|
||||
|
||||
assert written["providers"] == {
|
||||
"mine": {"kind": "key"},
|
||||
"gateway_b": {"kind": "gateway", "default": ["pi"]},
|
||||
}
|
||||
# The previously injected non-providers key is gone, not just replaced.
|
||||
assert "server" not in written
|
||||
assert _read_marker(tmp_path) == second
|
||||
|
||||
|
||||
def test_render_host_config_write_command_empty_payload_removes_injected_config(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""Removing ``host_config`` from server config cleans up on the next run."""
|
||||
(tmp_path / ".omnigent").mkdir()
|
||||
(tmp_path / ".omnigent" / "config.yaml").write_text(
|
||||
yaml.safe_dump({"host": {"name": "keep-me"}})
|
||||
)
|
||||
|
||||
_materialize(render_host_config_write_command(_GATEWAY_HOST_CONFIG), tmp_path)
|
||||
written = _materialize(render_host_config_write_command({}), tmp_path)
|
||||
|
||||
assert written == {"host": {"name": "keep-me"}}
|
||||
assert _read_marker(tmp_path) is None
|
||||
|
||||
|
||||
def test_render_host_config_write_command_preserves_user_created_entries(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""
|
||||
The server owns the names it injects; user-created config under OTHER names
|
||||
always survives. A provider the user adds themselves is never in the marker,
|
||||
so cleanup leaves it untouched while removing the injected entries by name.
|
||||
"""
|
||||
injected = {
|
||||
"providers": {"gateway": {"kind": "gateway", "default": ["pi"]}},
|
||||
"server": "https://injected.example.com",
|
||||
}
|
||||
_materialize(render_host_config_write_command(injected), tmp_path)
|
||||
config_path = tmp_path / ".omnigent" / "config.yaml"
|
||||
with open(config_path) as f:
|
||||
merged = yaml.safe_load(f)
|
||||
merged["providers"]["mine"] = {"kind": "key"} # user-created, never injected
|
||||
merged["default_agent"] = "/user/agent.yaml" # user-owned top-level key
|
||||
config_path.write_text(yaml.safe_dump(merged))
|
||||
|
||||
written = _materialize(render_host_config_write_command({}), tmp_path)
|
||||
|
||||
assert written["providers"] == {"mine": {"kind": "key"}} # user entry survives
|
||||
assert "gateway" not in written["providers"] # injected name removed
|
||||
assert "server" not in written # injected top-level key removed
|
||||
assert written["default_agent"] == "/user/agent.yaml" # user key untouched
|
||||
assert _read_marker(tmp_path) is None
|
||||
|
||||
|
||||
def test_render_host_config_write_command_rename_after_user_edit_leaves_no_stale_default(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""
|
||||
Renaming a gateway must not strand the old entry even when the user edited
|
||||
it in place — two providers claiming the same ``default`` scope is a sandbox
|
||||
load error. Removal is by name, so the old injected name goes regardless.
|
||||
"""
|
||||
_materialize(
|
||||
render_host_config_write_command(
|
||||
{"providers": {"gateway_a": {"kind": "gateway", "default": ["pi"]}}}
|
||||
),
|
||||
tmp_path,
|
||||
)
|
||||
config_path = tmp_path / ".omnigent" / "config.yaml"
|
||||
with open(config_path) as f:
|
||||
edited = yaml.safe_load(f)
|
||||
edited["providers"]["gateway_a"]["base_url"] = "http://user-edited" # user edit
|
||||
config_path.write_text(yaml.safe_dump(edited))
|
||||
|
||||
written = _materialize(
|
||||
render_host_config_write_command(
|
||||
{"providers": {"gateway_b": {"kind": "gateway", "default": ["pi"]}}}
|
||||
),
|
||||
tmp_path,
|
||||
)
|
||||
|
||||
assert sorted(written["providers"]) == ["gateway_b"]
|
||||
defaults = [n for n, v in written["providers"].items() if v.get("default") == ["pi"]]
|
||||
assert defaults == ["gateway_b"] # exactly one default, no collision
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("failure_mode", "target_name"),
|
||||
[("config", "config.yaml"), ("marker", ".injected_host_config.json")],
|
||||
)
|
||||
def test_render_host_config_write_command_interrupted_write_keeps_complete_file(
|
||||
tmp_path: Path,
|
||||
failure_mode: str,
|
||||
target_name: str,
|
||||
) -> None:
|
||||
"""A partial temp-file write never truncates either destination file."""
|
||||
first = {"providers": {"gateway_a": {"kind": "gateway"}}}
|
||||
second = {"providers": {"gateway_b": {"kind": "gateway"}}}
|
||||
_materialize(render_host_config_write_command(first), tmp_path)
|
||||
config_dir = tmp_path / ".omnigent"
|
||||
target = config_dir / target_name
|
||||
complete_contents = target.read_bytes()
|
||||
|
||||
hook_dir = tmp_path / "python-hooks"
|
||||
hook_dir.mkdir()
|
||||
(hook_dir / "sitecustomize.py").write_text(
|
||||
"""\
|
||||
import json
|
||||
import os
|
||||
import yaml
|
||||
|
||||
failure = os.environ.get("FAIL_ATOMIC_WRITE")
|
||||
if failure == "config":
|
||||
def fail_yaml(_data, stream, **_kwargs):
|
||||
stream.write("partial")
|
||||
raise OSError("simulated interrupted config write")
|
||||
yaml.safe_dump = fail_yaml
|
||||
elif failure == "marker":
|
||||
def fail_json(_data, stream, *_args, **_kwargs):
|
||||
stream.write("{")
|
||||
raise OSError("simulated interrupted marker write")
|
||||
json.dump = fail_json
|
||||
"""
|
||||
)
|
||||
|
||||
result = _run_write_command(
|
||||
render_host_config_write_command(second),
|
||||
tmp_path,
|
||||
extra_env={
|
||||
"FAIL_ATOMIC_WRITE": failure_mode,
|
||||
"PYTHONPATH": str(hook_dir),
|
||||
},
|
||||
check=False,
|
||||
)
|
||||
|
||||
assert result.returncode != 0
|
||||
assert target.read_bytes() == complete_contents
|
||||
assert set(config_dir.iterdir()) == {
|
||||
config_dir / "config.yaml",
|
||||
config_dir / ".injected_host_config.json",
|
||||
}
|
||||
|
||||
|
||||
def test_render_host_config_write_command_empty_payload_without_marker_is_noop(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""The cleanup run on a sandbox that never saw an injection touches nothing."""
|
||||
_run_write_command(render_host_config_write_command({}), tmp_path)
|
||||
|
||||
assert not (tmp_path / ".omnigent" / "config.yaml").exists()
|
||||
assert _read_marker(tmp_path) is None
|
||||
|
||||
|
||||
def test_render_host_config_write_command_corrupt_marker_degrades_to_additive(
|
||||
tmp_path: Path,
|
||||
) -> None:
|
||||
"""
|
||||
Never delete without evidence: an unreadable marker skips the removal
|
||||
(today's additive behavior) rather than guessing what the server owns,
|
||||
and the run repairs the marker for the next cycle.
|
||||
"""
|
||||
(tmp_path / ".omnigent").mkdir()
|
||||
(tmp_path / ".omnigent" / "config.yaml").write_text(
|
||||
yaml.safe_dump({"providers": {"gateway_a": {"kind": "gateway"}}})
|
||||
)
|
||||
(tmp_path / ".omnigent" / ".injected_host_config.json").write_text("{not json")
|
||||
|
||||
written = _materialize(render_host_config_write_command(_GATEWAY_HOST_CONFIG), tmp_path)
|
||||
|
||||
providers = written["providers"]
|
||||
assert providers["gateway_a"] == {"kind": "gateway"} # not removed
|
||||
assert "litellm" in providers
|
||||
assert _read_marker(tmp_path) == _GATEWAY_HOST_CONFIG
|
||||
|
||||
|
||||
def test_start_host_without_host_config_runs_cleanup_on_resumable_launcher() -> None:
|
||||
"""
|
||||
A resumable sandbox keeps its filesystem across wakes, so the cleanup
|
||||
must run even with no host_config — otherwise entries injected by a
|
||||
since-removed block outlive it forever.
|
||||
"""
|
||||
|
||||
class _ResumableLauncher(_RecordingLauncher):
|
||||
can_resume: ClassVar[bool] = True
|
||||
|
||||
launcher = _ResumableLauncher()
|
||||
|
||||
launcher.start_host(
|
||||
"sb-1",
|
||||
token="tok-123",
|
||||
host_id="host_abc",
|
||||
host_name="managed-abc",
|
||||
server_url="https://srv",
|
||||
)
|
||||
|
||||
cleanup_index = launcher.commands.index(render_host_config_write_command({}))
|
||||
host_index = next(
|
||||
i for i, cmd in enumerate(launcher.commands) if "omnigent host --server" in cmd
|
||||
)
|
||||
assert cleanup_index < host_index
|
||||
|
||||
@@ -23,7 +23,10 @@ from omnigent.host.identity import (
|
||||
HOST_NAME_ENV_VAR,
|
||||
HOST_TOKEN_ENV_VAR,
|
||||
)
|
||||
from omnigent.onboarding.sandboxes.base import SandboxCapabilityError
|
||||
from omnigent.onboarding.sandboxes.base import (
|
||||
SandboxCapabilityError,
|
||||
render_host_config_write_command,
|
||||
)
|
||||
from omnigent.onboarding.sandboxes.kubernetes import (
|
||||
KubernetesSandboxLauncher,
|
||||
build_pod_manifest,
|
||||
@@ -46,6 +49,9 @@ _MANIFEST_KW = {
|
||||
"workspace": "/home/omnigent/workspace",
|
||||
}
|
||||
|
||||
# Minimal valid host_config exercised by the injection tests below.
|
||||
_HOST_CONFIG: dict[str, object] = {"providers": {"litellm": {"kind": "gateway"}}}
|
||||
|
||||
|
||||
# ── pure manifest / rendering tests (no SDK) ────────────────
|
||||
|
||||
@@ -91,6 +97,103 @@ def test_build_pod_manifest_without_repo_has_no_clone() -> None:
|
||||
assert "git clone" not in script
|
||||
|
||||
|
||||
def test_build_pod_manifest_host_config_is_written_by_init_container() -> None:
|
||||
"""host_config rides the init container script, after mkdir/clone, before the host."""
|
||||
manifest = build_pod_manifest(
|
||||
**{**_MANIFEST_KW, "clone_dir": "/home/omnigent/workspace/repo"},
|
||||
repo_url="https://github.com/org/repo.git",
|
||||
host_config=_HOST_CONFIG,
|
||||
)
|
||||
script = manifest["spec"]["initContainers"][0]["command"][2]
|
||||
write_command = render_host_config_write_command(_HOST_CONFIG)
|
||||
assert write_command in script
|
||||
# Ordering within the script: workspace prep and clone come first.
|
||||
assert script.index("mkdir -p") < script.index("git clone") < script.index(write_command)
|
||||
# The main container is untouched — the write happens before the host boots.
|
||||
assert write_command not in manifest["spec"]["containers"][0]["command"][2]
|
||||
|
||||
|
||||
def test_build_pod_manifest_forwards_config_home_to_init_container() -> None:
|
||||
"""Init and host resolve injected config under the same configured directory."""
|
||||
manifest = build_pod_manifest(
|
||||
**{
|
||||
**_MANIFEST_KW,
|
||||
"env_literals": {
|
||||
"OMNIGENT_CONFIG_HOME": "/home/omnigent/custom-config",
|
||||
"PLAIN_CONFIG": "host-only",
|
||||
},
|
||||
},
|
||||
host_config=_HOST_CONFIG,
|
||||
)
|
||||
|
||||
init_env = manifest["spec"]["initContainers"][0]["env"]
|
||||
host_env = manifest["spec"]["containers"][0]["env"]
|
||||
assert init_env == [
|
||||
{"name": "HOME", "value": "/home/omnigent"},
|
||||
{
|
||||
"name": "OMNIGENT_CONFIG_HOME",
|
||||
"value": "/home/omnigent/custom-config",
|
||||
},
|
||||
]
|
||||
assert {entry["name"] for entry in host_env} >= {
|
||||
"OMNIGENT_CONFIG_HOME",
|
||||
"PLAIN_CONFIG",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"config_home",
|
||||
["/tmp/elsewhere", "/home/omnigent-other", "/home/omnigent/../tmp", "../etc"],
|
||||
)
|
||||
def test_build_pod_manifest_rejects_config_home_outside_home_dir(config_home: str) -> None:
|
||||
"""
|
||||
Init and host share only the HOME emptyDir, so a config dir that resolves
|
||||
outside it would make the injected config invisible to the host. Fail the
|
||||
launch loudly — including paths that only escape after normalizing ``..``.
|
||||
"""
|
||||
with pytest.raises(ValueError, match=r"OMNIGENT_CONFIG_HOME.*must resolve under"):
|
||||
build_pod_manifest(
|
||||
**{**_MANIFEST_KW, "env_literals": {"OMNIGENT_CONFIG_HOME": config_home}},
|
||||
host_config=_HOST_CONFIG,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"config_home",
|
||||
["/home/omnigent", "/home/omnigent/", "/home/omnigent/cfg", "cfg", "relative/dir", ".", ""],
|
||||
)
|
||||
def test_build_pod_manifest_accepts_config_home_at_or_under_home_dir(config_home: str) -> None:
|
||||
"""
|
||||
A dir at or under HOME (absolute or relative to the shared workingDir) is on
|
||||
the shared volume — allowed. An empty value is treated as unset by the
|
||||
writer, so it is forwarded without validation. All are forwarded to init.
|
||||
"""
|
||||
manifest = build_pod_manifest(
|
||||
**{**_MANIFEST_KW, "env_literals": {"OMNIGENT_CONFIG_HOME": config_home}},
|
||||
host_config=_HOST_CONFIG,
|
||||
)
|
||||
assert {"name": "OMNIGENT_CONFIG_HOME", "value": config_home} in manifest["spec"][
|
||||
"initContainers"
|
||||
][0]["env"]
|
||||
|
||||
|
||||
def test_build_pod_manifest_config_home_outside_home_dir_ok_without_host_config() -> None:
|
||||
"""Without host_config the init container writes nothing, so the path is moot."""
|
||||
manifest = build_pod_manifest(
|
||||
**{**_MANIFEST_KW, "env_literals": {"OMNIGENT_CONFIG_HOME": "/tmp/elsewhere"}},
|
||||
)
|
||||
init_env = manifest["spec"]["initContainers"][0]["env"]
|
||||
assert {"name": "OMNIGENT_CONFIG_HOME", "value": "/tmp/elsewhere"} in init_env
|
||||
|
||||
|
||||
def test_build_pod_manifest_without_host_config_has_no_config_write() -> None:
|
||||
"""No host_config → the init container only preps the workspace."""
|
||||
manifest = build_pod_manifest(**_MANIFEST_KW)
|
||||
script = manifest["spec"]["initContainers"][0]["command"][2]
|
||||
assert "config.yaml" not in script
|
||||
assert "python3 -c" not in script
|
||||
|
||||
|
||||
def test_build_pod_manifest_token_rides_secret_ref_not_the_spec() -> None:
|
||||
"""The launch token is referenced via secretKeyRef, never written into the spec."""
|
||||
manifest = build_pod_manifest(**_MANIFEST_KW)
|
||||
@@ -408,6 +511,33 @@ def test_launch_host_cleans_up_on_create_failure(fake_core: _FakeCore) -> None:
|
||||
assert "omnigent-pod-3" in fake_core.deleted_pods
|
||||
|
||||
|
||||
def test_launch_host_invalid_config_home_fails_before_creating_secret(
|
||||
fake_core: _FakeCore, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
"""
|
||||
An out-of-HOME config dir must fail while the manifest is built — before the
|
||||
token Secret is created — so no credential-bearing Secret is orphaned.
|
||||
"""
|
||||
monkeypatch.setenv("OMNIGENT_CONFIG_HOME", "/tmp/outside")
|
||||
launcher = KubernetesSandboxLauncher(
|
||||
in_cluster=True,
|
||||
namespace="omnigent-sandboxes",
|
||||
secret_name="omnigent-creds",
|
||||
env=["OMNIGENT_CONFIG_HOME"],
|
||||
)
|
||||
with pytest.raises(ValueError, match=r"OMNIGENT_CONFIG_HOME.*must resolve under"):
|
||||
launcher.start_host(
|
||||
"omnigent-pod-x",
|
||||
token=_TOKEN,
|
||||
host_id="host_x",
|
||||
host_name="managed-x",
|
||||
server_url="http://srv.example.com",
|
||||
host_config=_HOST_CONFIG,
|
||||
)
|
||||
assert "create_secret" not in fake_core.calls
|
||||
assert fake_core.created_secrets == []
|
||||
|
||||
|
||||
def test_launch_host_fast_fails_on_clone_failure_with_log_tail(
|
||||
fake_core: _FakeCore,
|
||||
) -> None:
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
from typing import ClassVar
|
||||
|
||||
@@ -11,6 +13,7 @@ from fastapi import FastAPI, HTTPException
|
||||
from httpx import ASGITransport, AsyncClient
|
||||
|
||||
from omnigent.db.utils import now_epoch
|
||||
from omnigent.onboarding.sandboxes.base import render_host_config_write_command
|
||||
from omnigent.onboarding.sandboxes.e2b import managed_token_ttl_s as e2b_managed_token_ttl_s
|
||||
from omnigent.runtime.agent_cache import AgentCache
|
||||
from omnigent.server.app import create_app
|
||||
@@ -58,6 +61,7 @@ def _injected_config(
|
||||
*,
|
||||
server_url: str = "https://srv.example.com",
|
||||
token_ttl_s: int = 3600,
|
||||
host_config: dict[str, object] | None = None,
|
||||
) -> ManagedSandboxConfig:
|
||||
"""
|
||||
Build a config that injects *fake* through the launcher-factory seam
|
||||
@@ -66,12 +70,14 @@ def _injected_config(
|
||||
:param fake: The launcher every launch should use.
|
||||
:param server_url: Server URL the sandbox host dials back to.
|
||||
:param token_ttl_s: Launch-token lifetime in seconds.
|
||||
:param host_config: In-sandbox config.yaml content to forward, or ``None``.
|
||||
:returns: A ready :class:`ManagedSandboxConfig`.
|
||||
"""
|
||||
return ManagedSandboxConfig(
|
||||
server_url=server_url,
|
||||
launcher_factory=lambda: fake,
|
||||
token_ttl_s=token_ttl_s,
|
||||
host_config=host_config,
|
||||
)
|
||||
|
||||
|
||||
@@ -552,6 +558,126 @@ def test_parse_kubernetes_without_section_defaults(monkeypatch: pytest.MonkeyPat
|
||||
assert fake.resources is None
|
||||
|
||||
|
||||
def test_parse_host_config_threads_verbatim_without_resolving_secrets(
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
"""
|
||||
A valid host_config lands on the parsed config verbatim, and its
|
||||
``api_key_ref: env:`` reference is NOT resolved at parse time — the
|
||||
variable names sandbox environment, not server environment, so parsing
|
||||
must succeed with the variable unset on the server.
|
||||
"""
|
||||
monkeypatch.delenv("LITELLM_API_KEY", raising=False)
|
||||
monkeypatch.delenv("OMNIGENT_LITELLM_API_KEY", raising=False)
|
||||
host_config = {
|
||||
"providers": {
|
||||
"litellm": {
|
||||
"kind": "gateway",
|
||||
"default": ["pi"],
|
||||
"openai": {
|
||||
"base_url": "http://litellm.litellm.svc.cluster.local/v1",
|
||||
"api_key_ref": "env:LITELLM_API_KEY",
|
||||
"wire_api": "chat",
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
cfg = parse_sandbox_config(
|
||||
{"provider": "modal", "server_url": "https://s.example.com", "host_config": host_config}
|
||||
)
|
||||
|
||||
assert cfg is not None
|
||||
assert cfg.host_config == host_config
|
||||
|
||||
|
||||
def test_parse_absent_host_config_is_none() -> None:
|
||||
"""No host_config key → nothing forwarded, existing configs unchanged."""
|
||||
cfg = parse_sandbox_config({"provider": "modal", "server_url": "https://s.example.com"})
|
||||
assert cfg is not None
|
||||
assert cfg.host_config is None
|
||||
|
||||
|
||||
def test_parse_host_config_null_providers_fails_loud() -> None:
|
||||
"""
|
||||
An explicit ``providers: null`` fails parse. Left through, the sandbox
|
||||
merge would write ``providers: null`` over any existing block and the
|
||||
harness would silently fall back to its own login — the exact
|
||||
degradation this parse exists to stop.
|
||||
"""
|
||||
with pytest.raises(ValueError, match=r"sandbox\.host_config\.providers"):
|
||||
parse_sandbox_config(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s.example.com",
|
||||
"host_config": {"providers": None},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_parse_host_config_duplicate_default_fails_loud() -> None:
|
||||
"""Duplicate defaults fail at server startup, before sandbox launch."""
|
||||
provider = {
|
||||
"kind": "gateway",
|
||||
"default": ["pi"],
|
||||
"openai": {
|
||||
"base_url": "https://gateway.example.com/v1",
|
||||
"api_key_ref": "env:GATEWAY_API_KEY",
|
||||
},
|
||||
}
|
||||
|
||||
with pytest.raises(
|
||||
ValueError,
|
||||
match=r"sandbox\.host_config\.providers.*multiple providers.*'pi' family",
|
||||
):
|
||||
parse_sandbox_config(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s.example.com",
|
||||
"host_config": {
|
||||
"providers": {
|
||||
"first": provider,
|
||||
"second": provider,
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_parse_host_config_inline_api_key_fails_loud() -> None:
|
||||
"""Literal provider credentials cannot ride in the managed host config."""
|
||||
with pytest.raises(ValueError, match=r"api_key_ref: env:VAR"):
|
||||
parse_sandbox_config(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s.example.com",
|
||||
"host_config": {
|
||||
"providers": {
|
||||
"openai": {
|
||||
"kind": "key",
|
||||
"openai": {
|
||||
"base_url": "https://api.openai.com/v1",
|
||||
"api_key": "sk-inline-secret",
|
||||
},
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_parse_host_config_lossy_json_key_collision_fails_loud() -> None:
|
||||
"""JSON key coercion cannot silently collapse distinct config entries."""
|
||||
with pytest.raises(ValueError, match=r"JSON-serializable"):
|
||||
parse_sandbox_config(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s.example.com",
|
||||
"host_config": {"metadata": {1: "integer", "1": "string"}},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("kubernetes_block", "expected_fragment"),
|
||||
[
|
||||
@@ -774,6 +900,36 @@ def test_parse_kubernetes_invalid_block_fails_loud(
|
||||
{"provider": "openshell", "server_url": "https://s", "openshell": {"cluster": " "}},
|
||||
"sandbox.openshell.cluster",
|
||||
),
|
||||
# host_config present but malformed (provider-agnostic top-level key).
|
||||
(
|
||||
{"provider": "modal", "server_url": "https://s", "host_config": "providers: {}"},
|
||||
"sandbox.host_config",
|
||||
),
|
||||
(
|
||||
{"provider": "modal", "server_url": "https://s", "host_config": {"providers": "x"}},
|
||||
"sandbox.host_config.providers",
|
||||
),
|
||||
# An invalid provider entry (bad kind) is caught by the same parser
|
||||
# omnigent itself uses — inside the sandbox this would degrade
|
||||
# silently, so parse time is the only loud failure point.
|
||||
(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s",
|
||||
"host_config": {"providers": {"litellm": {"kind": "bogus"}}},
|
||||
},
|
||||
"sandbox.host_config.providers",
|
||||
),
|
||||
# yaml.safe_load turns an unquoted date into datetime.date, which the
|
||||
# per-launch json.dumps cannot take — must fail startup, not launches.
|
||||
(
|
||||
{
|
||||
"provider": "modal",
|
||||
"server_url": "https://s",
|
||||
"host_config": {"last_rotated": datetime.date(2024, 1, 1)},
|
||||
},
|
||||
"JSON-serializable",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_parse_invalid_config_fails_loud(raw: object, expected_fragment: str) -> None:
|
||||
@@ -1025,6 +1181,143 @@ async def test_launch_success_registers_host_and_returns_workspace(db_uri: str)
|
||||
assert fake.terminated == []
|
||||
|
||||
|
||||
async def test_launch_materializes_host_config_before_host_start(db_uri: str) -> None:
|
||||
"""
|
||||
A configured host_config is written into the sandbox strictly BEFORE
|
||||
``omnigent host`` starts — the whole point of the injection is that the
|
||||
host boots with its providers already on disk.
|
||||
"""
|
||||
host_store = HostStore(db_uri)
|
||||
|
||||
def _register(invocation: HostStartInvocation) -> None:
|
||||
host_store.upsert_on_connect(
|
||||
host_id=invocation.host_id,
|
||||
name=invocation.host_name,
|
||||
owner=_OWNER,
|
||||
)
|
||||
|
||||
fake = FakeSandboxLauncher(on_host_start=_register)
|
||||
host_config: dict[str, object] = {"providers": {"litellm": {"kind": "gateway"}}}
|
||||
|
||||
await launch_managed_host(
|
||||
config=_injected_config(fake, host_config=host_config),
|
||||
owner=_OWNER,
|
||||
host_store=host_store,
|
||||
)
|
||||
|
||||
write_index = fake.commands.index(render_host_config_write_command(host_config))
|
||||
host_index = next(i for i, cmd in enumerate(fake.commands) if "omnigent host --server" in cmd)
|
||||
assert write_index < host_index
|
||||
|
||||
|
||||
async def test_resume_rematerializes_host_config_before_host_restart(db_uri: str) -> None:
|
||||
"""
|
||||
Waking a dormant sandbox re-runs the config write before re-execing the
|
||||
host — resume_managed_host bypasses _arm_and_start_host, so this is a
|
||||
distinct wiring point, and re-materializing is what lets an operator's
|
||||
host_config change land on the next wake without a new sandbox.
|
||||
"""
|
||||
host_store = HostStore(db_uri)
|
||||
|
||||
def _register(invocation: HostStartInvocation) -> None:
|
||||
host_store.upsert_on_connect(
|
||||
host_id=invocation.host_id,
|
||||
name=invocation.host_name,
|
||||
owner=_OWNER,
|
||||
)
|
||||
|
||||
fake = FakeSandboxLauncher(on_host_start=_register, can_resume=True)
|
||||
host_config: dict[str, object] = {"providers": {"litellm": {"kind": "gateway"}}}
|
||||
config = _injected_config(fake, host_config=host_config)
|
||||
|
||||
result = await launch_managed_host(config=config, owner=_OWNER, host_store=host_store)
|
||||
host_store.set_offline(result.host_id)
|
||||
commands_before = len(fake.commands)
|
||||
|
||||
await resume_managed_host(result.host_id, host_store, config)
|
||||
|
||||
assert fake.resumed == ["sb-fake-1"]
|
||||
resumed_commands = fake.commands[commands_before:]
|
||||
write_index = resumed_commands.index(render_host_config_write_command(host_config))
|
||||
host_index = next(
|
||||
i for i, cmd in enumerate(resumed_commands) if "omnigent host --server" in cmd
|
||||
)
|
||||
assert write_index < host_index
|
||||
|
||||
|
||||
async def test_launch_without_host_config_writes_no_config(db_uri: str) -> None:
|
||||
"""No host_config → the launch issues no config-write command at all."""
|
||||
host_store = HostStore(db_uri)
|
||||
|
||||
def _register(invocation: HostStartInvocation) -> None:
|
||||
host_store.upsert_on_connect(
|
||||
host_id=invocation.host_id,
|
||||
name=invocation.host_name,
|
||||
owner=_OWNER,
|
||||
)
|
||||
|
||||
fake = FakeSandboxLauncher(on_host_start=_register)
|
||||
|
||||
await launch_managed_host(config=_injected_config(fake), owner=_OWNER, host_store=host_store)
|
||||
|
||||
assert not any(cmd.startswith("python3 -c") for cmd in fake.commands)
|
||||
|
||||
|
||||
async def test_launch_without_host_config_supports_legacy_start_host_signature(
|
||||
db_uri: str,
|
||||
) -> None:
|
||||
"""
|
||||
A deployment-injected launcher whose ``start_host`` override predates the
|
||||
``host_config`` parameter keeps launching when no host_config is set —
|
||||
the kwarg is omitted entirely rather than passed as ``None``.
|
||||
"""
|
||||
host_store = HostStore(db_uri)
|
||||
|
||||
def _register(invocation: HostStartInvocation) -> None:
|
||||
host_store.upsert_on_connect(
|
||||
host_id=invocation.host_id,
|
||||
name=invocation.host_name,
|
||||
owner=_OWNER,
|
||||
)
|
||||
|
||||
class _LegacySignatureLauncher(FakeSandboxLauncher):
|
||||
"""Overrides start_host with the pre-host_config explicit signature."""
|
||||
|
||||
def start_host(
|
||||
self,
|
||||
sandbox_id: str,
|
||||
*,
|
||||
token: str,
|
||||
host_id: str,
|
||||
host_name: str,
|
||||
server_url: str,
|
||||
repo_url: str | None = None,
|
||||
repo_branch: str | None = None,
|
||||
repo_name: str | None = None,
|
||||
on_stage: Callable[[str], None] | None = None,
|
||||
) -> str:
|
||||
return super().start_host(
|
||||
sandbox_id,
|
||||
token=token,
|
||||
host_id=host_id,
|
||||
host_name=host_name,
|
||||
server_url=server_url,
|
||||
repo_url=repo_url,
|
||||
repo_branch=repo_branch,
|
||||
repo_name=repo_name,
|
||||
on_stage=on_stage,
|
||||
)
|
||||
|
||||
fake = _LegacySignatureLauncher(on_host_start=_register)
|
||||
|
||||
result = await launch_managed_host(
|
||||
config=_injected_config(fake), owner=_OWNER, host_store=host_store
|
||||
)
|
||||
|
||||
[start] = fake.host_starts
|
||||
assert result.host_id == start.host_id
|
||||
|
||||
|
||||
async def test_launch_with_injected_custom_launcher(db_uri: str) -> None:
|
||||
"""
|
||||
The embedding seam end to end: a deployment-defined launcher (a
|
||||
@@ -1293,6 +1586,7 @@ class _EntrypointFakeLauncher(FakeSandboxLauncher):
|
||||
repo_url: str | None = None,
|
||||
repo_branch: str | None = None,
|
||||
repo_name: str | None = None,
|
||||
host_config: dict[str, object] | None = None,
|
||||
on_stage=None,
|
||||
) -> str:
|
||||
"""Record the call, prove the token already resolves, and connect."""
|
||||
|
||||
Reference in New Issue
Block a user