Files
Zecheng Zhang 50a68fc0d4 feat(dsh): stream background output, sandbox facts, spill, and the runtime reach marker (#796)
* refactor(runtime): replace confined with a three-value reach marker

A runtime now declares reach: vfs | process | remote, stating whether
the workspace dispatch gate is its only door or the code can act
around it (host process doors, another machine). The default is
process, the no-promise claim, so a custom runtime must narrow its
reach explicitly. The dsh sandbox claim reads the aggregate: every
runtime at vfs means workspace-write, anything wider means no claim.
Mirrored in Python (RuntimeReach in runtime/types.py) with reach
declared on monty, quickjs, wasi, vfs, local, and RemoteSandbox.

* fix(pyodide): seal the js module so the guest has no host door

Pyodide's default exposes the host globalThis as the `js` module,
which under Node handed guest code js.process (host env, confirmed
reading HOME) and js.fetch (network) — doors around the workspace
bridge that made the runtime's reach='vfs' claim false and
contradicted its own 'no network' docstring. Pass a null-prototype
jsglobals to loadPyodide: `import js` still resolves but the host
globals are unreachable through it, while pyodide's internals (which
capture their globals at load time) and the FS bridge are unaffected.
Pinned by jsglobals.test.ts.

* feat(dsh): stamp ShellSandboxInfo on run results and process handles

When the world is fully workspace-bound (vfsOnly), run() and start()
now fill dsh's optional sandbox field: mode workspace-write,
enforcement 'full' (the VFS gate is unbypassable, so unlike an OS
sandbox on an old kernel there is no promised effect it fails to
govern), denied false (mirage has no out-of-band denial channel; a
refused write fails in-band as an ordinary command error), and
runnerFailed false (the executor is the runner). The process handle
stamps it on settle. Omitted when any runtime reaches beyond the
workspace.

* feat(dsh): stream background command output through a JobConsole

Adds a public ExecuteOptions.sink: pass a JobConsole and the line's
output streams into it as each statement finishes, instead of being
returned whole (the result then carries only the exit code). This
reuses the executor's existing internal sink mechanism, so a compound
line flushes per statement and stdout/stderr keep their channels.

MirageShellProcess is rewritten over that seam: start() runs the
command with a console as its sink and a follow loop drains it into
the read buffer, so readOutput() delivers output incrementally and
stdout/stderr interleave in order (stderr opened by a marker) rather
than stderr being concatenated at the end. The unread backlog is
bounded to stdoutMaxBytes (tail kept, lossy flagged) so a reader that
never drains cannot grow it without limit.

* feat(dsh): spill the full stream to a workspace file on overrun

Adds an opt-in spillDir config. When a background command's streamed
output overruns its delta budget, the full stdout and stderr are
written to files under that workspace directory and readOutput()
points at them (stdoutSpillPath/stderrSpillPath), so a reader can
recover what the delta dropped by reading the spill through the same
VFS. Memory stays bounded: each channel buffers only until the first
overrun, then flushes to its file and appends from there. A write
failure (no writable mount at the path) disables the sink and leaves
the paths undefined, the honest 'no safe path' answer. Default unset,
so nothing spills unless a deployment asks for it.

* docs(dsh): custom backends, background streaming, and the reach model

Corrects the sandbox-claim wording to the reach model (workspace-write
when every runtime reaches only the vfs, dropped when one reaches the
host), and adds a Custom backends section (registerResourceFactory,
host-side before the workspace builds) and a Background commands
section (per-statement streaming, bounded backlog, spillDir).

* test(dsh,core): satisfy lint on the streaming tests

Narrow spill paths with an explicit guard instead of a non-null
assertion (forbidden in the dsh package), and drop the now-unnecessary
ExecuteResult casts the sink overload already implies.

* style: prettier formatting on the streaming changes

* fix(core): drain a buffered line into the sink

A sink only saw output the command-tree walk emitted, so a whole-line
runtime, the syntax gate, a policy denial and a failed line all answered
with bytes in hand that a streaming caller never read. executeLine now
moves any buffered result into the console on every path, in one place
rather than five, and the result stays empty as it already did when the
line streamed.

* fix(dsh): bound the console store, make the spill dir idempotent

Capping the delta did not bound memory: reading a chunk advances a
cursor but frees nothing, so an uncapped store held every chunk of a
noisy background command for the life of the process. The store now
carries a retention budget, and the drain reports a trimmed chunk as
lossy and stops the spill, since a file missing the middle of a stream
is worse than no file.

The spill directory is created through ensureDirPath, which walks the
ancestors and accepts a refusal for a directory that now exists, so two
commands overrunning at once do not cost the loser its spill.
2026-08-14 14:19:06 -07:00
..