github-featured/strukto-ai--mirage
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

refactor(runtime): one mount op vocabulary for the typescript sandbox runtimes, and serve pyodide from an Emscripten filesystem (#732)

Browse Source 
* refactor(pyodide): serve mounts from an Emscripten filesystem, not a python shim

Replace the 425-line python source string the runtime injected with a
filesystem mounted through pyodide.FS.mount, so interception moves below
the interpreter instead of rebinding names inside it.

This fixes two bugs the shim shipped. os.open/os.write/os.fdopen and a
bare os.truncate recorded nothing, because the shim rebound 13 names and
those were not among them, so a low-level write applied to guest memory
and was dropped at exit 0. And a cross-mount rename raised errno 18,
which is EDOM under pyodide's musl numbering, so a guest comparing
against errno.EXDEV (75) never matched; the old test asserted the
message string python rendered from that same literal, so it could not
fail.

Every prefix is re-seeded on every run. The conformance rows covering
post-boot seeding only passed before because the shim's lazy backfill
used run_sync, and vitest enables JSPI while production does not.

A file the mount lists but will not serve now becomes a node that
refuses to open with EIO, rather than a hole an append would fill by
replacing the file.

* refactor(pyodide): name the filesystem package vfs, not fs

Mirrors the python side, where the module holding the guest filesystem
is vfs.py beside runtime/vfs.py. Leaving the package as fs/ while its
entry module became vfs.ts would have kept the mixed spelling the
rename is removing.

* refactor(pyodide): name the sentinel and the whence values

Number.MAX_SAFE_INTEGER appeared five times as the has-not-written-yet
sentinel, and llseek compared whence against bare 1 and 2. Both are now
in constants.ts beside the mode bits, matching the python side where
READONLY_HINT moved to wasm/constants.py.

* refactor(runtime): one mount op vocabulary for the typescript sandbox runtimes

New runtime/vfs.ts holds RuntimeVFS, planFlush and the mount routing that
pyodide, quickjs and monty each had their own copy of, mirroring the python
core. BridgeDispatchFn gains APPEND, so a handle that only extended a file
ships its tail: eight 3-byte appends now cost 24 bytes rather than 536, and
the two amplification rows in the conformance suite flip from it.fails to
it. runtime/config.ts takes coerceRuntimeConfig out of base.ts so a typed
config has the same home in both languages.

monty.ts becomes a monty/ package (binding, constants, errors, osaccess,
runtime, vfs) matching the python split, and MontyVFS gains the negative
cache python already had: monty asks whether a path exists on nearly every
guest expression, and each miss was costing a fresh listing.

mirage_bridge.ts splits into vfs/journal.ts and vfs/preload.ts, and
js/mirage_fs.ts becomes js/vfs.ts, so every module holding a guest
filesystem is named vfs. Dead MontyVFS.mountOf dropped in both languages.

* fix(pyodide): refuse a root mount, mount nested prefixes parent-first

Three review findings.

A `/` prefix stripped to the empty string, which Emscripten takes as a
detached pseudo-mount no path reaches: the guest kept reading and writing
MEMFS, and a write reported success the resource never saw. `/` is already
MEMFS's own mount root holding the stdlib, so there is nowhere to put it;
the prefix is now skipped with one warning rather than silently dropped.

prefixes() is longest-first, which is what routing wants and the reverse of
what the mount table wants: with /data/ and /data/inner/ the child mounted
first and the parent then mounted over it, orphaning the child. Unmount
deepest-first, mount shallowest-first.

MontyVFS lives as long as the runtime while python rebuilds its
MirageOSAccess per run, so the negative cache added here outlived the
command it belonged to and hid a file another writer created between two
monty runs. Reset it at the top of run and eval.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
This commit is contained in:
Zecheng Zhang
2026-08-08 20:20:18 -07:00
committed by GitHub
parent d82c6e9478
commit 3b18a436df
49 changed files with 3114 additions and 1748 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/test_typescript.yml
+5 -5
View File
@@ -113,7 +113,7 @@ jobs:
run redis/redis_vfs.ts integ/truth/typescript/redis_vfs.txt
run version/branching.ts integ/truth/version_branching.txt
python-fs-shim:
python-fs:
needs: changes
if: ${{ !cancelled() && (github.event_name != 'pull_request' || needs.changes.outputs.hit == 'true') }}
runs-on: ubuntu-latest
@@ -125,7 +125,7 @@ jobs:
with:
version: 10.32.1
- name: Set up Node 24 (for --experimental-wasm-jspi)
- name: Set up Node 24
uses: actions/setup-node@v7
with:
node-version: "24"
@@ -143,16 +143,16 @@ jobs:
working-directory: typescript
run: pnpm -r --filter './packages/*' build
- name: Python FS shim example (output checked against integ/ truth file)
- name: Python filesystem example (output checked against integ/ truth file)
working-directory: examples/typescript
run: |
node --experimental-wasm-jspi --import tsx/esm pyodide/vfs.ts 2>&1 \
node --import tsx/esm pyodide/vfs.ts 2>&1 \
| bash ../../integ/check_lines.sh ../../integ/truth/typescript/pyodide_vfs.txt
gate:
name: test-typescript-gate
runs-on: ubuntu-latest
needs: [changes, test, python-fs-shim]
needs: [changes, test, python-fs]
if: always()
steps:
- name: Check required jobs
docs/typescript/limitations.mdx
+5 -7
View File
@@ -138,15 +138,13 @@ Mounts that hit third-party APIs (S3, GitHub, Linear, etc.) make `fetch` calls f
- **Pre-signed URLs.** For S3/R2/GCS, generate pre-signed URLs server-side and pass them in. The Mirage browser examples include a Vite dev-server `presigner` plugin as reference.
- **Same-origin proxy.** Stand up a tiny proxy on your own domain that forwards to the upstream API with the right auth headers.
### 4. Python writes need JSPI
### 4. Python sees the mount as of the start of the run
The Python FS shim (`open()` against mounted paths) flushes writes back through an async bridge. That requires [JSPI](https://github.com/WebAssembly/js-promise-integration). Reads of preloaded files still work without it, but `close()` on a write throws `RuntimeError: Cannot stack switch`.
`open()` against a mounted path is served by a filesystem Mirage registers inside the interpreter. Its callbacks are synchronous, so the tree is collected before the script starts and writes are replayed after it ends: an edit made to the resource from elsewhere mid-run is not visible until the next run.
- **Chrome / Edge 137+** (May 2025): works out of the box.
- **Firefox**: behind `javascript.options.wasm_js_promise_integration`.
- **Safari**: not yet shipped.
This needs no [JSPI](https://github.com/WebAssembly/js-promise-integration) and no V8 flag, so it works the same in Chrome, Firefox, Safari and Node.
See [Python FS shim](/typescript/python-fs#runtime-requirements) for the full matrix.
See [Python filesystem](/typescript/python-fs) for the details.
### 5. Some resource drivers remain Node-only
@@ -175,7 +173,7 @@ Those HTTP drivers still need a browser-reachable endpoint with the correct CORS
| OPFS reads/writes within quota | Browser | ✅ works |
| OPFS over quota | Browser | ❌ `QuotaExceededError`; check `navigator.storage.estimate()` |
| HTTP-backed resources without CORS allow-list | Browser | ❌ blocked; use PKCE, presigned URLs, or a same-origin proxy |
| Python `open()` writes back through mount | Browser | ✅ with JSPI (Chrome 137+); ❌ otherwise |
| Python `open()` writes back through mount | Browser | ✅ on any engine |
| Postgres through `NeonPgDriver` | Browser | ✅ over HTTP; endpoint must allow the browser origin |
| MongoDB through `HttpMongoDriver` | Browser | ✅ through an HTTP proxy; direct MongoDB sockets are unavailable |
| Node-only resources (SSH, LanceDB, Email, Disk, Redis, Hugging Face, GridFS, FUSE) | Browser | ❌ Node APIs or raw TCP required |
docs/typescript/python-fs.mdx
+32 -37
View File
@@ -1,10 +1,10 @@
---
title: Python FS shim
title: Python filesystem
icon: file-binary
description: Use Python's open(), os.listdir, pathlib, PIL, pandas, directly against any Mirage mount.
---
Python code inside `ws.execute('python3 ...')` can `open()`, `os.listdir()`, `pathlib.Path()` etc. against any registered Mirage mount. The shim routes those calls through Mirage's mount layer (RAM, S3, Linear, GDocs, Slack, anything you've registered).
Python code inside `ws.execute('python3 ...')` can `open()`, `os.listdir()`, `pathlib.Path()` etc. against any registered Mirage mount. Mirage mounts a filesystem of its own inside the interpreter, below the syscall boundary, so those calls reach Mirage's mount layer (RAM, S3, Linear, GDocs, Slack, anything you've registered).
```python
# inside python3
@@ -22,14 +22,14 @@ open('/ram/notes.txt', 'w').write('hello') # writes flush back to R
open('/linear/teams/eng/team.json').read()
open('/gdocs/owned/MyDoc.gdoc.json').read()
```
The first read fetches via the resource and caches in MEMFS. Subsequent reads are sync.
Every mounted prefix is collected before the run, so reads are sync and cost no round trip.
</Tab>
<Tab title="Write a file">
```python
open('/ram/out.txt', 'w').write('hello')
open('/s3/icon.png', 'wb').write(png_bytes)
```
`close()` marks the path dirty; when the script finishes, the final bytes flush back through the resource's `write` op (one write per path, however many times the script reopened it).
`close()` records the write; when the script finishes, the bytes go back through the resource's `write` op, or its `append` op when the handle only extended the file.
</Tab>
<Tab title="List a directory">
```python
@@ -41,7 +41,7 @@ open('/ram/notes.txt', 'w').write('hello') # writes flush back to R
for msg_dir in os.listdir('/gmail/INBOX/2026-05-03/'):
print(msg_dir)
```
The first `listdir` of a synthesized path triggers a one-time bridge fetch.
A synthesized path is materialized when the prefix is collected, before the run.
</Tab>
<Tab title="pathlib & glob">
```python
@@ -85,33 +85,35 @@ open('/ram/notes.txt', 'w').write('hello') # writes flush back to R
| Case | Why | Workaround |
|---|---|---|
| **C extensions calling `fopen` directly**, `sqlite3.connect('/ram/db.sqlite')`, `h5py.File('/ram/x.h5')` | They bypass Python's `open()` and the shim never sees them | Use a local copy, or stream bytes via stdin |
| **External edits show up live**, someone else edits the GDoc while you're reading | Once a path is in MEMFS the shim doesn't re-fetch it | Currently no API; must `unmount` + `addMount` |
| **Huge mounts**, preloading a 100GB S3 bucket | Every byte lives in MEMFS until close | Mount a narrower prefix |
| **External edits show up mid-run**, someone else edits the GDoc while your script runs | The tree is collected before the run and served synchronously after that | Each run re-collects, so the next one sees the edit |
| **Huge mounts**, collecting a 100GB S3 bucket | Every byte is held in memory for the run | Mount a narrower prefix |
| **Symlinks**, `os.symlink` inside the guest | Links are namespace state in Mirage, so no mount op can carry one | Refused with `EPERM` rather than silently lost at the end of the run |
| **Concurrent writers**, two Python processes write the same path | Last `close()` wins, no conflict detection | Out of scope for v1 |
| **Browser-only resources from Node**, OPFS | OPFS isn't a thing in Node | Run in a browser-side Mirage |
## How it works (one paragraph)
Pyodide's in-memory FS (MEMFS) is the sync facade. Before each run, Mirage walks any newly mounted prefix and copies files into MEMFS. Python reads hit MEMFS directly, fast, no network. On a **miss** (path not yet in MEMFS but the prefix is registered), the shim catches the error, calls back through the bridge via JSPI to fetch it, populates MEMFS, and retries. Writes are intercepted on `close()`, which marks the path dirty in MEMFS; after the script returns, Mirage flushes each dirty path's final bytes back through Mirage's `write` op (no JSPI involved, and a flush failure lands on stderr with a nonzero exit).
Mirage registers its own Emscripten filesystem at each mount prefix, so every spelling of an operation (`open`, `os.open`, `pathlib`, a C extension's `fopen`, `shutil`'s fd-relative walk) arrives as the same callback. Emscripten's callbacks are synchronous, so nothing reaches the mount inline: before each run Mirage collects every mounted prefix into the filesystem's node table, and reads are served from there. Writes are recorded on a journal in the order the guest performed them, and replayed against the mounts after the script returns, where awaiting is free. A handle that only extended a file replays as an `append`, so an append loop ships its tail rather than the whole file each time. A failed replay lands on stderr with a nonzero exit, and stops the rest of the journal rather than applying entries whose prerequisite never landed.
```
Python open() / listdir()
Python open() / os.open() / C fopen()
│
▼
Pyodide MEMFS ──hit──► bytes
│
miss
│
▼
run_sync(_mirage_bridge.list/fetch) ← the one JSPI-dependent hop
│
▼
populate MEMFS, retry
Python close() ──► mark dirty ──► script ends ──► flush to mount
Mirage filesystem callbacks (sync)
│ │
reads writes
│ │
▼ ▼
node table mutation journal
(collected │
before the run) script ends
│
▼
replay against mounts
```
Nothing is patched inside the interpreter, and none of it needs JSPI.
## Quick start (TypeScript)
```ts
@@ -150,35 +152,28 @@ new Workspace({}, { python: { autoLoadFromImports: false } })
| S3, R2, GCS, OCI, Supabase | ✅ (narrow the prefix to keep the working set small) |
| Linear, GDocs, GSheets, GSlides, GDrive | ✅ |
| GitHub, Redis | ✅ |
| Gmail (including synthesized date dirs) | ✅ (lazy-fetched on first access) |
| Gmail (including synthesized date dirs) | ✅ (materialized when the prefix is collected) |
| Slack | ⚠️ preloads channel histories; large workspaces may be slow |
## Runtime requirements
None: preloaded reads, mounts added between runs, and writes all work on any engine. [JSPI](https://github.com/WebAssembly/js-promise-integration) only powers the shim's **lazy backfill**, the mid-run fetch of a path that appeared on the mount after its prefix was preloaded. With JSPI available that miss is filled in place; without it the miss stays a `FileNotFoundError` (a `console.warn: mirage lazy: ...` names the real reason).
None. Reads, mounts added between runs, and writes all work on any engine, with no V8 flag and no [JSPI](https://github.com/WebAssembly/js-promise-integration). That is the point of collecting the tree before the run and replaying writes after it: a filesystem callback never has to suspend, so nothing depends on stack switching, which no shipping Node enables by default and Safari does not implement.
Where JSPI is available:
- **Chrome / Edge 137+** (May 2025): out of the box
- **Firefox**: behind `javascript.options.wasm_js_promise_integration`
- **Node**: pass `--experimental-wasm-jspi` (vitest config already does this; V8 flags cannot ride `NODE_OPTIONS`)
- **Cloudflare Workers**: Python Workers with JSPI runtime
To run examples with the lazy backfill enabled:
To run the example:
```bash
node --experimental-wasm-jspi --import tsx/esm examples/typescript/pyodide/vfs.ts
node --import tsx/esm examples/typescript/pyodide/vfs.ts
```
## Errors you might see
| Symptom | What it means |
|---|---|
| `FileNotFoundError [Errno 44]` | Path isn't in MEMFS and lazy fetch failed too. Check the preceding `console.warn: mirage lazy: ...` for the real reason (auth, network, bad path). |
| `console.warn: mirage lazy: list <path> failed` | The lazy backfill called `_mirage_bridge.list` and got rejected. Followed by `FileNotFoundError`. |
| `python3: failed to flush <path> to mount: ...` on stderr, exit 1 | The script finished but a dirty file could not be written back to its mount (auth, read-only mount, network). MEMFS still holds the bytes. |
| `RuntimeError: WebAssembly stack switching not supported` in a lazy warn | JSPI not enabled, so a mid-run backfill could not suspend. Reads of preloaded content and all writes are unaffected. |
| `console.warn: mirage preload: skipping <path>` | A single entry failed during initial preload, others continued. Usually harmless (synthetic listing entries). |
| `FileNotFoundError [Errno 44]` | The path was not in the tree collected before the run. A callback cannot go looking for more, so this is a genuine absence at collection time. |
| `python3: failed to <op> <path> on mount: ...` on stderr, exit 1 | The script finished but a recorded mutation could not be applied (auth, read-only mount, network). |
| `python3: skipped N later mutation(s) after that failure` | The journal is a sequence, not a set, so the replay stops at the first failure rather than applying entries whose prerequisite never landed. |
| `OSError [Errno 5]` on opening a file you can see | The mount listed the file but would not serve its bytes, so its real content is unknown and any handle is refused rather than guessing. |
| `console.warn: mirage preload: cannot read <path>` | One entry failed while the prefix was collected, others continued. The file is marked unreadable rather than left looking absent. |
## See also
docs/typescript/python.mdx
+11 -15
View File
@@ -137,24 +137,20 @@ const png = await ws.fs.readFile('/ram/icon.png')
### How it works
- **Eager preload**: `addMount(prefix, ...)` walks the resource and populates Pyodide's MEMFS at `prefix`. Subsequent reads from Python are sync and fast.
- **Flush on close**: when Python `close()`s a file under a mounted prefix, the bytes are flushed back through the workspace bridge.
- **Python `open()` and friends**: `open()`, `pathlib.Path.write_text()`, `numpy.save`, `PIL.Image.save`, `pandas.to_csv`. Anything that ultimately calls Python-level `open()` works.
- **C extensions calling `fopen` directly**: see only the preloaded MEMFS snapshot (sqlite3, h5py). Most data-science libs use Python `open` and just work; native FFI database drivers don't.
- **Its own filesystem**: Mirage registers an Emscripten filesystem at each mount prefix, below the interpreter's syscall boundary. Every spelling of an operation arrives as the same callback, so nothing inside Python is patched.
- **Collected before the run**: each prefix is walked into the filesystem's node table before the script starts, so reads are sync and cost no round trip.
- **Replayed after it**: writes are recorded in guest order and applied to the mounts once the script returns. A handle that only extended a file replays as an `append`.
- **Every spelling**: `open()`, `os.open`, `pathlib`, `shutil`, `numpy.save`, `PIL.Image.save`, `pandas.to_csv`, and C extensions calling `fopen` (sqlite3, h5py) all reach the mount.
### Runtime requirements
The shim uses [JSPI](https://github.com/WebAssembly/js-promise-integration) (JavaScript Promise Integration) so sync Python calls can drive async JS bridge ops.
None. No V8 flag, no [JSPI](https://github.com/WebAssembly/js-promise-integration), no stack switching: a filesystem callback never suspends, because the reads it serves were collected before the run and the writes it takes are replayed after.
- **Browser**: Chrome 137+ (May 2025); Firefox behind `javascript.options.wasm_js_promise_integration`.
- **Node**: 24+ with `--experimental-wasm-jspi` (the vitest config in this repo sets it).
- **No JSPI**: reads of preloaded files still work, but `close()` on a write under a mounted prefix throws `RuntimeError: Cannot stack switch`.
### What doesn't work
### What doesn't work with the shim
- C extensions calling `fopen` directly (sqlite3, h5py): they only see the preloaded MEMFS state. Don't read or modify mount-backed databases through these.
- Stale listings: changes made to the underlying resource from outside this workspace aren't picked up until the next `addMount` cycle.
- Concurrent writers: last-flush wins; no conflict detection.
- Live external edits: a change made to the resource from outside mid-run is not seen until the next run re-collects the prefix.
- Concurrent writers: last write wins; no conflict detection.
- Symlinks: links are namespace state in Mirage, so `os.symlink` is refused with `EPERM` rather than accepted and lost at the end of the run.
## What you cannot do
@@ -204,7 +200,7 @@ PYEOF`)
| `sys.modules` fresh per call | shared within workspace |
| True CPU parallelism within one workspace | serialized; use separate workspaces |
| `select()` / `poll()` / `fcntl()` on stdin | no real fds in WASM |
| `open('/<mount>/...')` inside Python | via FS shim, eager preload + flush on close |
| `open('/<mount>/...')` inside Python | via Mirage's own filesystem, collected before the run + replayed after |
| `pip install` at runtime | pre-bundle instead |
| Native CPython fallback | always Pyodide |
| Browser support | Chrome 137+, Node 24+ with JSPI |
| Browser support | any engine (no JSPI, no V8 flag) |
typescript/packages/core/src/index.ts
+14 -1
View File
@@ -510,7 +510,20 @@ export { LanguageRuntime } from './runtime/language.ts'
export { PythonRuntime } from './runtime/python/base.ts'
export { JsRuntime } from './runtime/js/base.ts'
export { bindCommands, DEFAULT_ENTRIES, runtimeBindingsFor, VFSRuntime } from './runtime/table.ts'
export { EvalError } from './runtime/errors.ts'
export {
coerceRuntimeConfig,
HOME_CONFIG_KEYS,
type HomeConfig,
type RuntimeConfig,
} from './runtime/config.ts'
export { CrossMountError, EvalError } from './runtime/errors.ts'
export {
planFlush,
RuntimeVFS,
type FlushKind,
type VFSEntry,
type VFSStat,
} from './runtime/vfs.ts'
export {
EVALUATOR,
isEvaluator,
typescript/packages/core/src/runtime/base.ts
+3 -24
View File
@@ -12,31 +12,10 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { coerceRuntimeConfig, type RuntimeConfig } from './config.ts'
import { ScriptSource, type PolicyScript } from './policy/types.ts'
import type { RuntimeOptions } from './types.ts'
/**
* A constructor's config option as the runtime's own config, mirroring
* Python's RuntimeConfig.coerce: keys outside the runtime's list fail
* loud (Python gets this from the dataclass raising TypeError; a TS
* object spread would silently swallow a typo key without it).
*/
function coerceRuntimeConfig<C extends object>(
value: C | undefined,
keys: readonly string[],
label = 'runtime',
): C {
const config = value ?? ({} as C)
for (const key of Object.keys(config)) {
if (!keys.includes(key)) {
const known =
keys.length > 0 ? `expected: ${keys.map((k) => `'${k}'`).join(', ')}` : 'none allowed'
throw new Error(`unknown ${label} config key '${key}' (${known})`)
}
}
return { ...config }
}
/**
* An engine the workspace can route commands or whole lines to.
*
@@ -63,11 +42,11 @@ export abstract class Runtime {
abstract readonly name: string
readonly captures: readonly string[]
/** The runtime's coerced implementation knobs. */
config: object
config: RuntimeConfig
script?: PolicyScript
constructor(
options: RuntimeOptions<object> = {},
options: RuntimeOptions<RuntimeConfig> = {},
defaultCaptures: readonly string[] = [],
configKeys: readonly string[] = [],
) {
typescript/packages/core/src/runtime/config.ts
+69
View File
@@ -0,0 +1,69 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
/**
* How a runtime's engine is set up: its implementation knobs.
*
* Every runtime is constructed the same way (captures, config,
* script); what differs between runtimes lives here. In yaml this is
* the runtime entry's `config` block, mirroring a mount's.
*
* The base names no field, because no knob is shared by every tier.
* Python gets its fail-loud check from the dataclass (a field the
* runtime does not have is a TypeError); TypeScript has no runtime
* view of an interface, so each runtime hands the base its own key
* list and coerceRuntimeConfig enforces it.
*/
export type RuntimeConfig = object
/**
* Config for runtimes whose engine lives in a directory or binary.
*
* Args:
* home: where the engine is (a wasm build dir, an interpreter
* path). Absent falls back to the runtime's own environment
* variable, then its built-in default.
*/
export interface HomeConfig extends RuntimeConfig {
home?: string
}
export const HOME_CONFIG_KEYS: readonly string[] = ['home']
/**
* A constructor's config option as the runtime's own config, mirroring
* Python's RuntimeConfig.coerce: keys outside the runtime's list fail
* loud (Python gets this from the dataclass raising TypeError; a TS
* object spread would silently swallow a typo key without it).
*
* Args:
* value: the caller's config block, or undefined for the defaults.
* keys: the field names this runtime accepts.
* label: what to call the config in the failure message.
*/
export function coerceRuntimeConfig<C extends RuntimeConfig>(
value: C | undefined,
keys: readonly string[],
label = 'runtime',
): C {
const config = value ?? ({} as C)
for (const key of Object.keys(config)) {
if (!keys.includes(key)) {
const known =
keys.length > 0 ? `expected: ${keys.map((k) => `'${k}'`).join(', ')}` : 'none allowed'
throw new Error(`unknown ${label} config key '${key}' (${known})`)
}
}
return { ...config }
}
typescript/packages/core/src/runtime/conformance.test.ts
+38 -34
View File
@@ -18,7 +18,7 @@ import { RAMResource } from '../resource/ram/ram.ts'
import { MountMode } from '../types.ts'
import { getTestParser, stderrStr, stdoutStr } from '../workspace/fixtures/workspace_fixture.ts'
import { Workspace } from '../workspace/workspace.ts'
import { MontyRuntime } from './python/monty.ts'
import { MontyRuntime } from './python/monty/index.ts'
import { PyodideRuntime } from './python/pyodide.ts'
import { QuickJsRuntime } from './js/quickjs.ts'
import type { BridgeDispatchFn, RunArgs } from './types.ts'
@@ -50,7 +50,6 @@ const MONTY_OPEN_UNSUPPORTED =
// The workspace bridge has no APPEND op, so every append close
// re-flushes the whole file: n appends ship O(n^2) bytes.
const APPEND_AMPLIFIED = 'the bridge has no append op: each close re-flushes the whole file'
interface Row {
capability: string
@@ -478,6 +477,15 @@ function makeCountingBridge(seed: Record<string, string>): CountingBridge {
files.set(path, bytes === undefined ? new Uint8Array() : new Uint8Array(bytes))
return Promise.resolve(undefined)
}
if (op === 'APPEND') {
const base = files.get(path) ?? new Uint8Array()
const tail = bytes ?? new Uint8Array()
const next = new Uint8Array(base.length + tail.length)
next.set(base)
next.set(tail, base.length)
files.set(path, next)
return Promise.resolve(undefined)
}
if (op === 'STAT') {
const hit = files.get(path)
if (hit !== undefined) return Promise.resolve({ size: hit.length, isDir: false, mtimeMs: 0 })
@@ -515,10 +523,14 @@ function makeCountingBridge(seed: Record<string, string>): CountingBridge {
}
return Promise.resolve(undefined)
}
// Both spellings count: the question is how many bytes crossed the
// transport, and a runtime that ships tails is exactly the one being
// measured. Counting WRITE alone would score a working append as 0.
const isMutation = (op: string): boolean => op === 'WRITE' || op === 'APPEND'
const mutationBytes = () =>
ops.filter(([op]) => op === 'WRITE').reduce((total, [, , size]) => total + size, 0)
ops.filter(([op]) => isMutation(op)).reduce((total, [, , size]) => total + size, 0)
const mutationOps = () =>
ops.filter(([op]) => op === 'WRITE').map(([op, path]) => `${op} ${path}`)
ops.filter(([op]) => isMutation(op)).map(([op, path]) => `${op} ${path}`)
return { dispatch, files, mutationBytes, mutationOps }
}
@@ -550,35 +562,27 @@ describe('append ships only the deltas', () => {
120_000,
)
it.fails(
`pyodide [${APPEND_AMPLIFIED}]`,
async () => {
const counting = makeCountingBridge({ '/data/log.txt': 'S'.repeat(64) })
const rt = new PyodideRuntime()
rt.attach(counting.dispatch, () => ['/data/'])
const result = await rt.run(runArgs(APPEND_LOOP_PY))
await rt.close()
expect(result.exitCode).toBe(0)
const dec = new TextDecoder()
expect(dec.decode(counting.files.get('/data/log.txt'))).toBe('S'.repeat(64) + 'xyz'.repeat(8))
expect(counting.mutationBytes(), counting.mutationOps().join(', ')).toBe(24)
},
120_000,
)
it('pyodide', async () => {
const counting = makeCountingBridge({ '/data/log.txt': 'S'.repeat(64) })
const rt = new PyodideRuntime()
rt.attach(counting.dispatch, () => ['/data/'])
const result = await rt.run(runArgs(APPEND_LOOP_PY))
await rt.close()
expect(result.exitCode).toBe(0)
const dec = new TextDecoder()
expect(dec.decode(counting.files.get('/data/log.txt'))).toBe('S'.repeat(64) + 'xyz'.repeat(8))
expect(counting.mutationBytes(), counting.mutationOps().join(', ')).toBe(24)
}, 120_000)
it.fails(
`quickjs [${APPEND_AMPLIFIED}]`,
async () => {
const counting = makeCountingBridge({ '/data/log.txt': 'S'.repeat(64) })
const rt = new QuickJsRuntime()
rt.attach(counting.dispatch, () => ['/data/'])
const result = await rt.run(runArgs(APPEND_LOOP_JS))
await rt.close()
expect(result.exitCode).toBe(0)
const dec = new TextDecoder()
expect(dec.decode(counting.files.get('/data/log.txt'))).toBe('S'.repeat(64) + 'xyz'.repeat(8))
expect(counting.mutationBytes(), counting.mutationOps().join(', ')).toBe(24)
},
120_000,
)
it('quickjs', async () => {
const counting = makeCountingBridge({ '/data/log.txt': 'S'.repeat(64) })
const rt = new QuickJsRuntime()
rt.attach(counting.dispatch, () => ['/data/'])
const result = await rt.run(runArgs(APPEND_LOOP_JS))
await rt.close()
expect(result.exitCode).toBe(0)
const dec = new TextDecoder()
expect(dec.decode(counting.files.get('/data/log.txt'))).toBe('S'.repeat(64) + 'xyz'.repeat(8))
expect(counting.mutationBytes(), counting.mutationOps().join(', ')).toBe(24)
}, 120_000)
})
typescript/packages/core/src/runtime/errors.ts
+25
View File
@@ -28,3 +28,28 @@ export class EvalError extends Error {
this.syntax = options.syntax ?? false
}
}
/**
* A rename whose two ends do not live on the same mount.
*
* The dispatcher picks the mount from the source and addresses the
* destination against that same backend, so applying one would drop
* the source and write the target into the wrong store.
*
* Deliberately carries no errno. The condition is decided once, in
* RuntimeVFS.rename, and each encoder maps it to the number its own
* reference implementation answers: pathlib says EXDEV, while a WASI
* guest sees ENOENT because each mount is its own preopen. Unifying
* those two is a separate decision from writing the rule down once.
*/
export class CrossMountError extends Error {
readonly src: string
readonly dst: string
constructor(src: string, dst: string) {
super(`cross-mount rename: ${src} -> ${dst}`)
this.name = 'CrossMountError'
this.src = src
this.dst = dst
}
}
typescript/packages/core/src/runtime/js/quickjs.ts
+13 -17
View File
@@ -13,14 +13,15 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { CommandTimeoutError } from '../../commands/builtin/utils/limit.ts'
import { HOME_CONFIG_KEYS } from '../config.ts'
import { EvalError } from '../errors.ts'
import { JsRuntime } from './base.ts'
import { EVALUATOR, type Evaluator } from '../mixin.ts'
import type { EvalResult, EvalValue, RunArgs, RunResult, RuntimeOptions } from '../types.ts'
import { createMirageBridge, type MirageBridge } from '../python/mirage_bridge.ts'
import { RuntimeVFS } from '../vfs.ts'
import type { BridgeDispatchFn } from '../types.ts'
import { QUICKJS_RUNTIME } from './interface.ts'
import { installMirageFs, MIRAGE_FS_BOOTSTRAP } from './mirage_fs.ts'
import { installMirageFs, MIRAGE_FS_BOOTSTRAP } from './vfs.ts'
import { QuickJsUnavailableError } from './types.ts'
import type {
QuickJSAsyncContext,
@@ -151,10 +152,9 @@ globalThis.std = {
};
`
// quickjs-emscripten bundles its own wasm, so the `home` config key
// (for parity with the Python quickjs runtime, which locates
// quickjs-emscripten bundles its own wasm, so the HomeConfig `home`
// key (for parity with the Python quickjs runtime, which locates
// qjs-wasi.wasm) has nothing to locate here and is ignored.
const QUICKJS_CONFIG_KEYS: readonly string[] = ['home']
// The asyncify variant is used so `std.open`/`os.readdir` can suspend
// the guest while a workspace-mount read or write awaits the dispatch,
@@ -167,7 +167,7 @@ export class QuickJsRuntime extends JsRuntime implements Evaluator {
private listMounts: () => string[] = () => []
constructor(options: RuntimeOptions = {}) {
super(options, QUICKJS_CONFIG_KEYS)
super(options, HOME_CONFIG_KEYS)
}
override attach(dispatch: BridgeDispatchFn, listMounts: () => string[]): void {
@@ -193,11 +193,9 @@ export class QuickJsRuntime extends JsRuntime implements Evaluator {
const timedOut = this.installInterrupt(runtime, args.signal, args.timeoutSeconds)
try {
this.installGlobals(ctx, args, out, err, exit)
const bridge: MirageBridge | null =
this.workspaceBridge !== null
? createMirageBridge(this.workspaceBridge, this.listMounts)
: null
installMirageFs(ctx, bridge)
const vfs =
this.workspaceBridge !== null ? new RuntimeVFS(this.workspaceBridge, this.listMounts) : null
installMirageFs(ctx, vfs)
const boot = ctx.evalCode(BOOTSTRAP + MIRAGE_FS_BOOTSTRAP, 'mirage:bootstrap')
if (boot.error) {
@@ -278,12 +276,10 @@ export class QuickJsRuntime extends JsRuntime implements Evaluator {
})
// Same filesystem surface as run(): an attached workspace serves
// std.open/os.readdir, so a JS policy script can read mounted
// content (the python evaluator gets this via run()'s WasmVFS).
const bridge: MirageBridge | null =
this.workspaceBridge !== null
? createMirageBridge(this.workspaceBridge, this.listMounts)
: null
installMirageFs(ctx, bridge)
// content (the python evaluator gets this via run()'s RuntimeVFS).
const vfs =
this.workspaceBridge !== null ? new RuntimeVFS(this.workspaceBridge, this.listMounts) : null
installMirageFs(ctx, vfs)
const boot = ctx.evalCode(BOOTSTRAP + MIRAGE_FS_BOOTSTRAP, 'mirage:bootstrap')
if (boot.error) {
boot.error.dispose()
typescript/packages/core/src/runtime/js/mirage_fs.ts → typescript/packages/core/src/runtime/js/vfs.ts
+38 -35
View File
@@ -12,7 +12,7 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { MirageBridge } from '../python/mirage_bridge.ts'
import { NO_WRITE, type RuntimeVFS, type VFSStat } from '../vfs.ts'
import type { QuickJSAsyncContext, QuickJSHandle } from 'quickjs-emscripten'
const ENC = new TextEncoder()
@@ -56,14 +56,20 @@ interface GuestFile {
pos: number
dirty: boolean
writable: boolean
// What close needs to plan the flush: the length this handle opened
// over, and the lowest offset it wrote at. An append-mode handle only
// ever writes at or past baseLen, so its close ships the tail alone.
baseLen: number
lowWrite: number
}
// The `std.open`/`os.readdir` surface that qjs-wasi exposes natively,
// synthesized here over the workspace bridge so quickjs-emscripten
// matches it. Whole-file buffering mirrors the Python `WasiFs`: open
// fetches the bytes (or starts empty), the byte-level calls touch the
// in-memory buffer, and close flushes a dirty buffer back through
// dispatch. Only open, close, and readdir cross the async bridge, so
// synthesized here over the runtime's mount vocabulary so
// quickjs-emscripten matches it. Whole-file buffering mirrors the Python
// `WasiFs`: open fetches the bytes (or starts empty), the byte-level
// calls touch the in-memory buffer, and close hands the buffer to
// `RuntimeVFS.flush`, which ships a tail when the handle only extended
// the file. Only open, close, and readdir cross the async boundary, so
// they are asyncified host functions (the guest suspends until the
// dispatch resolves); the byte-level calls are synchronous.
//
@@ -102,6 +108,7 @@ function isWritable(mode: string): boolean {
}
function writeAt(file: GuestFile, bytes: Uint8Array): void {
file.lowWrite = Math.min(file.lowWrite, file.pos)
const end = file.pos + bytes.length
if (end > file.buf.length) {
const grown = new Uint8Array(end)
@@ -115,26 +122,20 @@ function writeAt(file: GuestFile, bytes: Uint8Array): void {
/**
* Install the `std.open`/`os.readdir` host functions on an asyncified
* quickjs context, backed by the workspace bridge. A null bridge (no
* quickjs context, backed by the runtime vfs. A null vfs (no
* workspace mounts wired) still installs the surface, but every open
* and readdir fails cleanly — `std.open` returns null and `os.readdir`
* reports ENOENT — so guest code sees an empty filesystem rather than a
* missing global.
*
* @param ctx - the asyncified quickjs context
* @param bridge - the workspace bridge, or null when no mounts are wired
* @param vfs - the runtime's mount vocabulary, or null when no mounts are wired
*/
export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge | null): void {
export function installMirageFs(ctx: QuickJSAsyncContext, vfs: RuntimeVFS | null): void {
const table = new Map<number, GuestFile>()
let nextFd = 1
const mountOf = (path: string): string | null => {
if (bridge === null) return null
for (const p of bridge.prefixes()) {
if (path === p.slice(0, -1) || path.startsWith(p)) return p
}
return null
}
const mountOf = (path: string): string | null => (vfs === null ? null : vfs.mountOf(path))
const underMount = (path: string): boolean => mountOf(path) !== null
@@ -156,7 +157,7 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
defineAsync('__mirage_open', async (pathH, modeH) => {
const path = ctx.getString(pathH)
const mode = ctx.getString(modeH)
if (bridge === null || !underMount(path)) return ctx.newNumber(-1)
if (vfs === null || !underMount(path)) return ctx.newNumber(-1)
const truncate = mode.includes('w')
const append = mode.includes('a')
const writable = isWritable(mode)
@@ -164,19 +165,19 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
let existed = false
if (!truncate) {
try {
buf = await bridge.fetch(path)
buf = await vfs.read(path)
existed = true
} catch {
if (!writable) return ctx.newNumber(-1)
}
}
// Truncate or create writes through the bridge at open, mirroring
// Truncate or create writes through the mount at open, mirroring
// the Python runtime: this enforces write modes (a read-only mount
// or a read-narrowed session throws here, so the guest gets null)
// and establishes the file before the buffered writes.
if (truncate || (writable && !existed)) {
try {
await bridge.flush(path, buf)
await vfs.write(path, buf)
} catch {
return ctx.newNumber(-1)
}
@@ -188,6 +189,8 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
pos: append ? buf.length : 0,
dirty: false,
writable,
baseLen: buf.length,
lowWrite: NO_WRITE,
})
return ctx.newNumber(fd)
})
@@ -196,8 +199,8 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
const file = table.get(ctx.getNumber(fdH))
if (file === undefined) return ctx.undefined
table.delete(ctx.getNumber(fdH))
if (file.dirty && file.writable && bridge !== null) {
await bridge.flush(file.path, file.buf)
if (file.dirty && file.writable && vfs !== null) {
await vfs.flush(file.path, file.baseLen, file.lowWrite, file.buf)
}
return ctx.undefined
})
@@ -206,12 +209,12 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
const path = ctx.getString(pathH)
const names: string[] = []
let errno = 0
if (bridge === null || !underMount(path)) {
if (vfs === null || !underMount(path)) {
errno = ENOENT
} else {
try {
const prefix = path.endsWith('/') ? path : path + '/'
for (const entry of await bridge.list(prefix)) {
for (const entry of await vfs.readdir(prefix)) {
const rel = entry.path.replace(/\/$/, '').slice(prefix.length)
if (rel.length > 0 && !rel.includes('/')) names.push(rel)
}
@@ -288,13 +291,13 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
// files and empty directories; os.stat answers [obj, errno].
defineAsync('__mirage_remove', async (pathH) => {
const path = ctx.getString(pathH)
if (bridge === null || !underMount(path)) return ctx.newNumber(-ENOENT)
if (vfs === null || !underMount(path)) return ctx.newNumber(-ENOENT)
try {
const st = await bridge.stat(path)
const st = await vfs.stat(path)
if (st.isDir) {
await bridge.rmdir(path)
await vfs.rmdir(path)
} else {
await bridge.unlink(path)
await vfs.unlink(path)
}
return ctx.newNumber(0)
} catch (err) {
@@ -304,9 +307,9 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
defineAsync('__mirage_mkdir', async (pathH) => {
const path = ctx.getString(pathH)
if (bridge === null || !underMount(path)) return ctx.newNumber(-ENOENT)
if (vfs === null || !underMount(path)) return ctx.newNumber(-ENOENT)
try {
await bridge.mkdir(path)
await vfs.mkdir(path)
return ctx.newNumber(0)
} catch (err) {
return ctx.newNumber(-wasiErrno(err))
@@ -316,14 +319,14 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
defineAsync('__mirage_rename', async (srcH, dstH) => {
const src = ctx.getString(srcH)
const dst = ctx.getString(dstH)
if (bridge === null || !underMount(src) || !underMount(dst)) return ctx.newNumber(-ENOENT)
if (vfs === null || !underMount(src) || !underMount(dst)) return ctx.newNumber(-ENOENT)
// The dispatcher addresses the rename's endpoints against the
// source's mount, so a cross-mount pair would land inside the
// wrong tree; the real engine answers -44 (pinned live: each
// mount is its own preopen and the destination never resolves).
if (mountOf(src) !== mountOf(dst)) return ctx.newNumber(-ENOENT)
try {
await bridge.rename(src, dst)
await vfs.rename(src, dst)
return ctx.newNumber(0)
} catch (err) {
return ctx.newNumber(-wasiErrno(err))
@@ -332,13 +335,13 @@ export function installMirageFs(ctx: QuickJSAsyncContext, bridge: MirageBridge |
defineAsync('__mirage_stat', async (pathH) => {
const path = ctx.getString(pathH)
let st = null as Awaited<ReturnType<MirageBridge['stat']>> | null
let st: VFSStat | null = null
let errno = 0
if (bridge === null || !underMount(path)) {
if (vfs === null || !underMount(path)) {
errno = ENOENT
} else {
try {
st = await bridge.stat(path)
st = await vfs.stat(path)
} catch (err) {
errno = wasiErrno(err)
}
typescript/packages/core/src/runtime/python/loader.ts
+5 -1
View File
@@ -12,6 +12,7 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { ErrnoCodes, FSHost, FSType } from './vfs/types.ts'
import { PyodideUnavailableError } from './types.ts'
const noopIo = (): void => undefined
@@ -27,11 +28,13 @@ function envHome(): string | null {
return typeof value === 'string' && value !== '' ? value : null
}
interface PyodideFS {
interface PyodideFS extends FSHost {
mkdirTree: (path: string, mode?: number) => void
writeFile: (path: string, data: Uint8Array | string) => void
readFile: (path: string, opts?: { encoding?: 'binary' | 'utf8' }) => Uint8Array | string
unlink?: (path: string) => void
mount: (type: FSType, opts: Record<string, unknown>, mountpoint: string) => unknown
unmount: (mountpoint: string) => void
}
export interface PyodideInterface {
@@ -48,6 +51,7 @@ export interface PyodideInterface {
unregisterJsModule?: (name: string) => void
setInterruptBuffer?: (buffer: Int32Array | Uint8Array) => void
FS: PyodideFS
ERRNO_CODES: ErrnoCodes
}
function isNode(): boolean {
typescript/packages/core/src/runtime/python/mirage_bridge.ts
-285
View File
@@ -1,285 +0,0 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { isMissingPath } from '../../utils/errors.ts'
import type { BridgeDispatchFn } from '../types.ts'
export interface MirageEntry {
path: string
size: number
isDir: boolean
}
export interface FSLike {
mkdirTree(path: string): void
writeFile(path: string, bytes: Uint8Array): void
}
interface MirageStat {
size: number
isDir: boolean
mtimeMs: number
}
/**
* One guest mutation, recorded in the order the script performed it.
* `write` and `append` carry their bytes because the drain runs after the
* script returns, when MEMFS holds only the final state: an atomic-write
* (write a temp file, rename it into place) would otherwise replay as a
* read of a path the rename already moved.
*/
export type MirageMutation =
| { readonly kind: 'write'; readonly path: string; readonly bytes: Uint8Array }
| { readonly kind: 'append'; readonly path: string; readonly bytes: Uint8Array }
| { readonly kind: 'mkdir'; readonly path: string }
| { readonly kind: 'unlink'; readonly path: string }
| { readonly kind: 'rmdir'; readonly path: string }
| { readonly kind: 'rename'; readonly path: string; readonly dst: string }
export interface MirageBridge {
fetch(path: string): Promise<Uint8Array>
flush(path: string, bytes: Uint8Array): Promise<void>
list(path: string): Promise<MirageEntry[]>
stat(path: string): Promise<MirageStat>
unlink(path: string): Promise<void>
mkdir(path: string): Promise<void>
rmdir(path: string): Promise<void>
rename(src: string, dst: string): Promise<void>
/**
* Live view of the workspace mount prefixes (trailing-slash normalized).
* The fs shim consults this on every intercepted call, so the mount
* registry stays the single source of truth: nothing is pushed into the
* interpreter when mounts are added or removed.
*/
prefixes(): string[]
/**
* Record one guest mutation. Every mark is deliberately synchronous:
* the guest's close(), os.mkdir() and os.rename() run inside sync WASM
* frames where awaiting a bridge op needs JSPI stack switching, which
* most engines do not enable. The guest records here and the runtime
* replays after the script returns, where awaiting is free.
*
* Args:
* path: mount-prefixed path the mutation names.
* bytes: the whole file for a write, the new tail for an append.
*/
markWrite(path: string, bytes: Uint8Array): void
markAppend(path: string, bytes: Uint8Array): void
markMkdir(path: string): void
markUnlink(path: string): void
markRmdir(path: string): void
markRename(src: string, dst: string): void
/** Drain the journal: every mutation in guest order, cleared. */
takeMutations(): MirageMutation[]
}
function concatBytes(head: Uint8Array, tail: Uint8Array): Uint8Array {
const out = new Uint8Array(head.length + tail.length)
out.set(head, 0)
out.set(tail, head.length)
return out
}
export function createMirageBridge(
dispatch: BridgeDispatchFn,
listMounts: () => string[] = () => [],
): MirageBridge {
const journal: MirageMutation[] = []
return {
prefixes() {
return listMounts().map((p) => (p.endsWith('/') ? p : p + '/'))
},
markWrite(path, bytes) {
// A guest buffer handed over by pyodide can be a view into WASM
// memory, which relocates when the heap grows; copy on arrival so
// the journal owns bytes that stay valid until the drain.
const owned = new Uint8Array(bytes)
const last = journal[journal.length - 1]
if (last?.kind === 'write' && last.path === path) {
journal[journal.length - 1] = { kind: 'write', path, bytes: owned }
return
}
journal.push({ kind: 'write', path, bytes: owned })
},
markAppend(path, bytes) {
const owned = new Uint8Array(bytes)
const last = journal[journal.length - 1]
if (last?.kind === 'append' && last.path === path) {
journal[journal.length - 1] = {
kind: 'append',
path,
bytes: concatBytes(last.bytes, owned),
}
return
}
journal.push({ kind: 'append', path, bytes: owned })
},
markMkdir(path) {
journal.push({ kind: 'mkdir', path })
},
markUnlink(path) {
journal.push({ kind: 'unlink', path })
},
markRmdir(path) {
journal.push({ kind: 'rmdir', path })
},
markRename(src, dst) {
journal.push({ kind: 'rename', path: src, dst })
},
takeMutations() {
return journal.splice(0, journal.length)
},
async fetch(path) {
const out = await dispatch('READ', path)
if (!(out instanceof Uint8Array)) {
throw new TypeError(`mirage bridge: READ ${path} expected Uint8Array, got ${typeof out}`)
}
return out
},
async flush(path, bytes) {
const out = await dispatch('WRITE', path, bytes)
if (out !== undefined) {
throw new TypeError(`mirage bridge: WRITE ${path} expected void, got ${typeof out}`)
}
},
async stat(path) {
const out = await dispatch('STAT', path)
const st = out as MirageStat | null
if (
st === null ||
typeof st !== 'object' ||
typeof st.size !== 'number' ||
typeof st.isDir !== 'boolean' ||
typeof st.mtimeMs !== 'number'
) {
throw new TypeError(`mirage bridge: STAT ${path} bad shape`)
}
return st
},
async unlink(path) {
await dispatch('UNLINK', path)
},
async mkdir(path) {
await dispatch('MKDIR', path)
},
async rmdir(path) {
await dispatch('RMDIR', path)
},
async rename(src, dst) {
await dispatch('RENAME', src, undefined, dst)
},
async list(path) {
const out = await dispatch('LIST', path)
if (!Array.isArray(out)) {
throw new TypeError(`mirage bridge: LIST ${path} expected array`)
}
for (const e of out) {
if (
e === null ||
typeof e !== 'object' ||
typeof (e as MirageEntry).path !== 'string' ||
typeof (e as MirageEntry).size !== 'number' ||
typeof (e as MirageEntry).isDir !== 'boolean'
) {
throw new TypeError(`mirage bridge: LIST ${path} bad entry shape`)
}
}
return out as MirageEntry[]
},
}
}
/**
* Extend a mount file by `tail`.
*
* Read-modify-write, because no transport op appends yet: the whole file
* goes back over the wire per call, which is why the append-amplification
* conformance row stays marked. It also leaves the append non-atomic
* against a concurrent writer, the same lost-update hazard whole-file
* flushing already carries; a transport append op closes both at once.
*
* Only a confirmed absence starts from an empty base, since an append
* may create the file. Every other read failure propagates: writing the
* tail on its own over a file that exists but is momentarily unreadable
* would replace content this run never saw.
*
* Args:
* bridge: the mount bridge to read and write through.
* path: mount path being extended.
* tail: bytes the guest appended.
*/
async function appendOnMount(bridge: MirageBridge, path: string, tail: Uint8Array): Promise<void> {
let base: Uint8Array = new Uint8Array()
try {
base = await bridge.fetch(path)
} catch (err) {
if (!isMissingPath(err)) throw err
}
await bridge.flush(path, concatBytes(base, tail))
}
/**
* Replay one recorded guest mutation against the mounts.
*
* Args:
* bridge: the mount bridge to apply through.
* mutation: the journal entry to apply.
*/
export async function applyMutation(bridge: MirageBridge, mutation: MirageMutation): Promise<void> {
switch (mutation.kind) {
case 'write':
return bridge.flush(mutation.path, mutation.bytes)
case 'append':
return appendOnMount(bridge, mutation.path, mutation.bytes)
case 'mkdir':
return bridge.mkdir(mutation.path)
case 'unlink':
return bridge.unlink(mutation.path)
case 'rmdir':
return bridge.rmdir(mutation.path)
case 'rename':
return bridge.rename(mutation.path, mutation.dst)
}
}
async function preloadEntry(fs: FSLike, bridge: MirageBridge, entry: MirageEntry): Promise<void> {
if (entry.isDir) {
fs.mkdirTree(entry.path)
const next = entry.path.endsWith('/') ? entry.path : entry.path + '/'
try {
await preloadInto(fs, bridge, next)
} catch (err) {
console.warn(
`mirage preload: skipping subtree ${next}: ${err instanceof Error ? err.message : String(err)}`,
)
}
return
}
try {
const bytes = await bridge.fetch(entry.path)
fs.writeFile(entry.path, bytes)
} catch (err) {
console.warn(
`mirage preload: skipping ${entry.path}: ${err instanceof Error ? err.message : String(err)}`,
)
}
}
export async function preloadInto(fs: FSLike, bridge: MirageBridge, prefix: string): Promise<void> {
const prefixWithSlash = prefix.endsWith('/') ? prefix : prefix + '/'
const prefixWithoutSlash = prefixWithSlash.slice(0, -1)
fs.mkdirTree(prefixWithoutSlash)
const entries = await bridge.list(prefixWithSlash)
await Promise.all(entries.map((entry) => preloadEntry(fs, bridge, entry)))
}
typescript/packages/core/src/runtime/python/mirage_fs_shim.test.ts
-385
View File
@@ -1,385 +0,0 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { beforeAll, beforeEach, describe, expect, it } from 'vitest'
import { loadPyodideRuntime, type PyodideInterface } from './loader.ts'
import {
applyMutation,
createMirageBridge,
type MirageBridge,
type MirageEntry,
} from './mirage_bridge.ts'
import type { BridgeDispatchFn } from '../types.ts'
import { MIRAGE_FS_SHIM_PY } from './mirage_fs_shim.ts'
interface Call {
op: string
path: string
bytes?: Uint8Array
}
// The runtime's post-run drain: the guest only records mutations, and
// they are applied host-side where awaiting the bridge needs no JSPI.
async function drain(bridge: MirageBridge): Promise<void> {
for (const mutation of bridge.takeMutations()) {
await applyMutation(bridge, mutation)
}
}
describe('mirage_fs_shim', () => {
let py: PyodideInterface
let bridge: MirageBridge
const calls: Call[] = []
const mounts: string[] = []
const preloaded = new Map<string, Uint8Array>()
const lazyFiles = new Map<string, Uint8Array>()
const lazyListings = new Map<string, MirageEntry[]>()
const lazyListErrors = new Set<string>()
function preload(path: string, bytes: Uint8Array): void {
preloaded.set(path, bytes)
py.FS.writeFile(path, bytes)
}
function seedLazyFile(path: string, bytes: Uint8Array): void {
lazyFiles.set(path, bytes)
}
function seedLazyListing(dir: string, entries: MirageEntry[]): void {
const norm = dir.endsWith('/') ? dir : dir + '/'
lazyListings.set(norm, entries)
}
beforeAll(async () => {
py = await loadPyodideRuntime()
const dispatch: BridgeDispatchFn = (op, path, bytes) => {
const entry: Call = bytes ? { op, path, bytes: new Uint8Array(bytes) } : { op, path }
calls.push(entry)
if (op === 'WRITE') return Promise.resolve(undefined)
if (op === 'READ') {
const lazy = lazyFiles.get(path)
if (lazy !== undefined) return Promise.resolve(lazy)
return Promise.resolve(preloaded.get(path) ?? new Uint8Array())
}
if (lazyListErrors.has(path)) {
return Promise.reject(new Error(`mock LIST failure: ${path}`))
}
return Promise.resolve(lazyListings.get(path) ?? [])
}
bridge = createMirageBridge(dispatch, () => mounts)
py.registerJsModule('_mirage_bridge', bridge)
await py.runPythonAsync(MIRAGE_FS_SHIM_PY)
py.FS.mkdirTree('/ram')
py.FS.mkdirTree('/tmp')
}, 60_000)
beforeEach(() => {
calls.length = 0
mounts.length = 0
preloaded.clear()
lazyFiles.clear()
lazyListings.clear()
lazyListErrors.clear()
bridge.takeMutations()
})
it('flushes a binary write to the bridge on close', async () => {
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/hello.txt', 'wb') as f:
f.write(b'world')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/hello.txt')
expect(new TextDecoder().decode(w0.bytes)).toBe('world')
})
it('does not flush writes outside mounted prefixes', async () => {
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/tmp/x.txt', 'wb') as f:
f.write(b'unbridged')
`)
await drain(bridge)
expect(calls.filter((c) => c.op === 'WRITE')).toHaveLength(0)
})
it('text mode write also flushes', async () => {
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/x.txt', 'w') as f:
f.write('hello')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/x.txt')
expect(new TextDecoder().decode(w0.bytes)).toBe('hello')
})
it('append mode lands the tail on top of the mount content', async () => {
preload('/ram/log.txt', new TextEncoder().encode('a'))
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/log.txt', 'ab') as f:
f.write(b'b')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/log.txt')
expect(new TextDecoder().decode(w0.bytes)).toBe('ab')
})
it('removing a prefix from the live mount view stops flushing', async () => {
mounts.push('/ram/')
mounts.length = 0
await py.runPythonAsync(`
with open('/ram/x.txt', 'wb') as f:
f.write(b'nope')
`)
await drain(bridge)
expect(calls.filter((c) => c.op === 'WRITE')).toHaveLength(0)
})
it('reads of preloaded files do not call the bridge', async () => {
preload('/ram/data.bin', new Uint8Array([1, 2, 3]))
mounts.push('/ram/')
await py.runPythonAsync(`
_read_result = open('/ram/data.bin', 'rb').read()
`)
const bytes = py.globals.get('_read_result') as Uint8Array | { toJs?: () => Uint8Array } | null
let arr: Uint8Array
if (bytes instanceof Uint8Array) {
arr = bytes
} else if (
bytes !== null &&
typeof (bytes as { toJs?: () => Uint8Array }).toJs === 'function'
) {
arr = (bytes as { toJs: () => Uint8Array }).toJs()
} else {
throw new Error(`unexpected read_result shape: ${typeof bytes}`)
}
expect(Array.from(arr)).toEqual([1, 2, 3])
expect(calls.filter((c) => c.op === 'READ')).toHaveLength(0)
})
it('rejects path traversal — /ram/../etc/x is not in /ram/', async () => {
mounts.push('/ram/')
let opened = true
try {
await py.runPythonAsync(`
f = open('/ram/../etc/x', 'wb')
f.write(b'x')
f.close()
`)
} catch {
opened = false
}
void opened
await drain(bridge)
expect(calls.filter((c) => c.op === 'WRITE' && c.path === '/ram/../etc/x')).toHaveLength(0)
expect(calls.filter((c) => c.op === 'WRITE' && c.path === '/etc/x')).toHaveLength(0)
})
it('repeated open/close cycles drain to one flush with the final content', async () => {
mounts.push('/ram/')
await py.runPythonAsync(`
for i in range(5):
with open('/ram/loop.txt', 'ab') as f:
f.write(b'x')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/loop.txt')
expect(new TextDecoder().decode(w0.bytes)).toBe('xxxxx')
})
it('r+b mode flushes the modified content on close', async () => {
preload('/ram/data.bin', new Uint8Array([1, 2, 3, 4]))
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/data.bin', 'r+b') as f:
f.seek(2)
f.write(b'\\x99\\x99')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/data.bin')
expect(Array.from(w0.bytes)).toEqual([1, 2, 0x99, 0x99])
})
it('shim is idempotent — loading twice still works', async () => {
await py.runPythonAsync(MIRAGE_FS_SHIM_PY)
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/x', 'wb') as f:
f.write(b'y')
`)
await drain(bridge)
const writes = calls.filter((c) => c.op === 'WRITE')
expect(writes).toHaveLength(1)
const w0 = writes[0]
if (w0?.bytes === undefined) throw new Error('unreachable')
expect(w0.path).toBe('/ram/x')
expect(new TextDecoder().decode(w0.bytes)).toBe('y')
})
it('os.listdir lazy-fetches paths not in MEMFS but available via the bridge', async () => {
seedLazyListing('/ram/lazy/', [
{ path: '/ram/lazy/file.txt', size: 10, isDir: false },
{ path: '/ram/lazy/sub', size: 0, isDir: true },
])
seedLazyFile('/ram/lazy/file.txt', new TextEncoder().encode('lazy-bytes'))
mounts.push('/ram/')
await py.runPythonAsync(`
import os
_lazy_entries = sorted(os.listdir('/ram/lazy'))
`)
const entries = py.globals.get('_lazy_entries') as { toJs?: () => string[] } | string[] | null
let arr: string[]
if (Array.isArray(entries)) arr = entries
else if (entries !== null && typeof (entries as { toJs?: () => string[] }).toJs === 'function')
arr = (entries as { toJs: () => string[] }).toJs()
else throw new Error('unexpected _lazy_entries shape')
expect(arr).toEqual(['file.txt', 'sub'])
expect(calls.filter((c) => c.op === 'LIST' && c.path === '/ram/lazy/')).toHaveLength(1)
})
it('open() in read mode lazy-fetches when MEMFS misses', async () => {
seedLazyListing('/ram/lazyread/', [{ path: '/ram/lazyread/note.txt', size: 5, isDir: false }])
seedLazyFile('/ram/lazyread/note.txt', new TextEncoder().encode('hello-lazy'))
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/lazyread/note.txt', 'rb') as f:
_lazy_read = f.read()
`)
const data = py.globals.get('_lazy_read') as Uint8Array | { toJs?: () => Uint8Array } | null
let bytes: Uint8Array
if (data instanceof Uint8Array) bytes = data
else if (data !== null && typeof (data as { toJs?: () => Uint8Array }).toJs === 'function')
bytes = (data as { toJs: () => Uint8Array }).toJs()
else throw new Error('unexpected _lazy_read shape')
expect(new TextDecoder().decode(bytes)).toBe('hello-lazy')
expect(
calls.filter((c) => c.op === 'READ' && c.path === '/ram/lazyread/note.txt'),
).toHaveLength(1)
})
it('os.stat lazy-backfills', async () => {
seedLazyListing('/ram/lazystat/', [{ path: '/ram/lazystat/data.bin', size: 3, isDir: false }])
seedLazyFile('/ram/lazystat/data.bin', new Uint8Array([7, 8, 9]))
mounts.push('/ram/')
await py.runPythonAsync(`
import os
_lazy_size = os.stat('/ram/lazystat/data.bin').st_size
`)
const sizeProxy = py.globals.get('_lazy_size') as number | { valueOf?: () => number } | null
const size = typeof sizeProxy === 'number' ? sizeProxy : Number(sizeProxy)
expect(size).toBe(3)
})
it('lazy backfill is cached — second call does not re-hit the bridge', async () => {
seedLazyListing('/ram/cached/', [{ path: '/ram/cached/a.txt', size: 1, isDir: false }])
seedLazyFile('/ram/cached/a.txt', new TextEncoder().encode('A'))
mounts.push('/ram/')
await py.runPythonAsync(`
import os
os.listdir('/ram/cached')
os.listdir('/ram/cached')
with open('/ram/cached/a.txt', 'rb') as f:
_again = f.read()
`)
expect(calls.filter((c) => c.op === 'LIST' && c.path === '/ram/cached/')).toHaveLength(1)
expect(calls.filter((c) => c.op === 'READ' && c.path === '/ram/cached/a.txt')).toHaveLength(1)
})
it('records mutations a MEMFS miss would otherwise drop', async () => {
// Nothing here exists in MEMFS: the mount is the authority, and a
// path this interpreter never preloaded must not read as absent.
mounts.push('/ram/')
await py.runPythonAsync(`
import os
os.mkdir('/ram/fresh')
os.remove('/ram/gone.txt')
os.rmdir('/ram/olddir')
`)
expect(bridge.takeMutations()).toEqual([
{ kind: 'mkdir', path: '/ram/fresh' },
{ kind: 'unlink', path: '/ram/gone.txt' },
{ kind: 'rmdir', path: '/ram/olddir' },
])
})
it('an append records only the tail, never the buffer it started from', async () => {
mounts.push('/ram/')
await py.runPythonAsync(`
with open('/ram/unseen.txt', 'ab') as f:
f.write(b'tail')
`)
const mutations = bridge.takeMutations()
expect(mutations).toHaveLength(1)
const only = mutations[0]
if (only?.kind !== 'append') throw new Error('expected an append')
expect(only.path).toBe('/ram/unseen.txt')
expect(new TextDecoder().decode(only.bytes)).toBe('tail')
})
it('a rename across two mounts raises EXDEV and records nothing', async () => {
mounts.push('/ram/', '/other/')
let raised = ''
try {
await py.runPythonAsync(`
import os
os.rename('/ram/a.txt', '/other/a.txt')
`)
} catch (err) {
raised = err instanceof Error ? err.message : String(err)
}
expect(raised).toMatch(/Invalid cross-device link/)
expect(bridge.takeMutations()).toEqual([])
})
it('lazy backfill failure surfaces as FileNotFoundError', async () => {
lazyListErrors.add('/ram/missing/')
lazyListErrors.add('/ram/')
mounts.push('/ram/')
let raised = false
try {
await py.runPythonAsync(`
import os
os.listdir('/ram/missing')
`)
} catch (err) {
raised = true
const msg = err instanceof Error ? err.message : String(err)
expect(msg).toMatch(/FileNotFoundError|No such file/)
}
expect(raised).toBe(true)
})
})
typescript/packages/core/src/runtime/python/mirage_fs_shim.ts
-439
View File
@@ -1,439 +0,0 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
export const MIRAGE_FS_SHIM_PY = `
import builtins
import io
import js
import os
import sys
import types
from pyodide.ffi import run_sync, to_js
import _mirage_bridge as _mb
_already_patched = getattr(builtins.open, '_mirage_patched', False)
if _already_patched:
_open = builtins.open._mirage_original_open
_io_open = io.open
_shim = sys.modules['_mirage_fs_shim']
_listdir = _shim._original_listdir
_stat = _shim._original_stat
_scandir = _shim._original_scandir
_mkdir = _shim._original_mkdir
_rmdir = _shim._original_rmdir
_unlink = _shim._original_unlink
_rename = _shim._original_rename
_lstat = _shim._original_lstat
else:
_open = builtins.open
_io_open = io.open
_listdir = os.listdir
_stat = os.stat
_scandir = os.scandir
_mkdir = os.mkdir
_rmdir = os.rmdir
_unlink = os.unlink
_rename = os.rename
_lstat = os.lstat
def _under_prefix(path):
if not isinstance(path, str):
return False
for p in _mb.prefixes():
if path.startswith(p):
return True
return False
# The mount serving a path, longest prefix first, or None. Two paths
# belong to the same mount only when this agrees for both, which is what
# the cross-mount rename guard needs. prefixes() is a sync JS call, so
# this costs no stack switching.
def _mount_of(path):
best = None
for p in _mb.prefixes():
if path == p.rstrip('/') or path.startswith(p):
if best is None or len(p) > len(best):
best = p
return best
def _is_writable_mode(mode):
for c in ('w', 'a', '+', 'x'):
if c in mode:
return True
return False
def _truncates(mode):
return 'w' in mode or 'x' in mode
def _read_raw(path):
try:
with _open(path, 'rb') as f:
return f.read()
except OSError as exc:
js.console.warn('mirage flush: read back ' + path + ' failed: ' + str(exc))
return b''
# close() only records on the bridge (a sync JS call): awaiting the async
# op from these sync WASM frames would need JSPI stack switching, which
# most engines do not enable. The runtime replays the journal after the
# script returns.
#
# An append-mode handle records only the bytes it wrote, never the whole
# file. That is what makes append safe on a file MEMFS has not seen: the
# mount keeps whatever it already held and gains the tail, instead of
# being overwritten by a buffer that started empty.
class _FlushOnClose(io.FileIO):
def __init__(self, path, mode='r', closefd=True, opener=None):
super().__init__(path, mode=mode, closefd=closefd, opener=opener)
self._mirage_path = os.fspath(path)
self._mirage_dirty = False
self._mirage_append = 'a' in mode
self._mirage_delta = bytearray()
def write(self, b):
n = super().write(b)
if n:
self._mirage_dirty = True
if self._mirage_append:
self._mirage_delta += bytes(b)[:n]
return n
def writelines(self, lines):
# IOBase.writelines calls self.write per line, so the delta is
# already accumulated above; this only records the dirty edge.
super().writelines(lines)
self._mirage_dirty = True
def truncate(self, size=None):
out = super().truncate(size)
self._mirage_dirty = True
# Truncation rewrites history the tail-only record cannot express,
# so fall back to shipping the whole file for this handle.
self._mirage_append = False
return out
def close(self):
was_dirty = self._mirage_dirty and not self.closed
path = self._mirage_path
append = self._mirage_append
delta = bytes(self._mirage_delta)
super().close()
if not was_dirty:
return
if append:
_mb.markAppend(path, to_js(delta))
else:
_mb.markWrite(path, to_js(_read_raw(path)))
def _strip_bt(mode):
return mode.replace('b', '').replace('t', '')
def _entry_path(e):
return getattr(e, 'path', None) or e['path']
def _entry_is_dir(e):
val = getattr(e, 'isDir', None)
if val is None:
val = e['isDir']
return bool(val)
def _to_bytes(data):
if isinstance(data, (bytes, bytearray)):
return bytes(data)
to_py = getattr(data, 'to_py', None)
if to_py is not None:
converted = to_py()
if isinstance(converted, (bytes, bytearray)):
return bytes(converted)
return bytes(converted)
return bytes(data)
# The lazy backfill below is the shim's one JSPI-dependent surface:
# run_sync suspends the WASM stack while the async bridge op settles, and
# raises RuntimeError on engines without stack switching. Both callers
# treat that as a miss (warn, return None), so without JSPI reads fall
# back to whatever the host preloaded into MEMFS before the run.
def _list_bridge(target):
try:
return run_sync(_mb.list(target))
except BaseException as exc:
js.console.warn('mirage lazy: list ' + target + ' failed: ' + str(exc))
return None
def _fetch_bridge(path):
try:
return run_sync(_mb.fetch(path))
except BaseException as exc:
js.console.warn('mirage lazy: fetch ' + path + ' failed: ' + str(exc))
return None
def _exists_raw(path):
try:
_stat(path)
return True
except (FileNotFoundError, NotADirectoryError):
return False
except OSError:
return False
def _makedirs_raw(path):
if path == '' or path == '/' or _exists_raw(path):
return
parent = os.path.dirname(path)
if parent and parent != path:
_makedirs_raw(parent)
try:
_mkdir(path)
except FileExistsError:
pass
except OSError as exc:
js.console.warn('mirage lazy: mkdir ' + path + ' failed: ' + str(exc))
def _populate_entries(entries):
for e in entries:
ep = _entry_path(e)
if not isinstance(ep, str) or ep == '':
continue
if _entry_is_dir(e):
_makedirs_raw(ep)
else:
if not _exists_raw(ep):
data = _fetch_bridge(ep)
if data is None:
continue
parent = os.path.dirname(ep)
if parent:
_makedirs_raw(parent)
try:
with _open(ep, 'wb') as f:
f.write(_to_bytes(data))
except OSError as exc:
js.console.warn('mirage lazy: write ' + ep + ' failed: ' + str(exc))
def _backfill(path):
if not _under_prefix(path):
return False
norm = os.path.normpath(path)
as_dir = norm if norm.endswith('/') else norm + '/'
entries = _list_bridge(as_dir)
if entries is not None and len(entries) > 0:
_populate_entries(entries)
return True
parent = os.path.dirname(norm)
if parent == '' or parent == norm:
return entries is not None
parent_dir = parent if parent.endswith('/') else parent + '/'
if not _under_prefix(parent_dir):
return entries is not None
parent_entries = _list_bridge(parent_dir)
if parent_entries is None:
return entries is not None
_populate_entries(parent_entries)
return True
# Absolute, because every mount test below compares against absolute
# prefixes: guest code that chdirs into a mount and then names a relative
# path ('child' after os.chdir('/data')) would otherwise miss every
# prefix and mutate only MEMFS. abspath resolves against the guest cwd
# and leaves an already-absolute path at what normpath would give.
def _normalize_path(path):
if isinstance(path, int):
return None
sp = path if isinstance(path, str) else os.fspath(path)
if isinstance(sp, bytes):
sp = sp.decode()
return os.path.abspath(sp)
def _patched_listdir(path='.'):
sp = _normalize_path(path)
if sp is None:
return _listdir(path)
try:
return _listdir(sp)
except FileNotFoundError:
if _backfill(sp):
return _listdir(sp)
raise
def _patched_stat(path, *, dir_fd=None, follow_symlinks=True):
sp = _normalize_path(path)
if sp is None:
return _stat(path, dir_fd=dir_fd, follow_symlinks=follow_symlinks)
try:
return _stat(sp, dir_fd=dir_fd, follow_symlinks=follow_symlinks)
except FileNotFoundError:
if _backfill(sp):
return _stat(sp, dir_fd=dir_fd, follow_symlinks=follow_symlinks)
raise
def _patched_lstat(path, *, dir_fd=None):
sp = _normalize_path(path)
if sp is None:
return _lstat(path, dir_fd=dir_fd)
try:
return _lstat(sp, dir_fd=dir_fd)
except FileNotFoundError:
if _backfill(sp):
return _lstat(sp, dir_fd=dir_fd)
raise
def _patched_scandir(path='.'):
sp = _normalize_path(path)
if sp is None:
return _scandir(path)
try:
return _scandir(sp)
except FileNotFoundError:
if _backfill(sp):
return _scandir(sp)
raise
# The mutation patches below apply to MEMFS first, so the rest of the run
# sees its own change, and then record on the bridge. They never consult
# the bridge inline, so unlike the lazy backfill they need no JSPI.
#
# A MEMFS miss is not an error here and must not be raised: the mount is
# the authority, and this interpreter only ever preloads a prefix once, so
# anything a shell command created since is legitimately absent from MEMFS.
# Refusing on that basis is exactly the bug this patch set removes. A path
# that truly exists nowhere fails when the journal is replayed, which puts
# the message on stderr and the run's exit code at 1.
def _patched_mkdir(path, mode=0o777, *, dir_fd=None):
sp = _normalize_path(path)
if sp is None or dir_fd is not None:
return _mkdir(path, mode, dir_fd=dir_fd)
norm = sp
if not _under_prefix(norm):
return _mkdir(path, mode, dir_fd=dir_fd)
_makedirs_raw(os.path.dirname(norm))
_mkdir(norm, mode)
_mb.markMkdir(norm)
def _patched_rmdir(path, *, dir_fd=None):
sp = _normalize_path(path)
if sp is None or dir_fd is not None:
return _rmdir(path, dir_fd=dir_fd)
norm = sp
if not _under_prefix(norm):
return _rmdir(path, dir_fd=dir_fd)
try:
_rmdir(norm)
except FileNotFoundError:
pass
_mb.markRmdir(norm)
def _patched_unlink(path, *, dir_fd=None):
sp = _normalize_path(path)
if sp is None or dir_fd is not None:
return _unlink(path, dir_fd=dir_fd)
norm = sp
if not _under_prefix(norm):
return _unlink(path, dir_fd=dir_fd)
try:
_unlink(norm)
except FileNotFoundError:
pass
_mb.markUnlink(norm)
def _patched_rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None):
ssp = _normalize_path(src)
dsp = _normalize_path(dst)
if ssp is None or dsp is None or src_dir_fd is not None or dst_dir_fd is not None:
return _rename(src, dst, src_dir_fd=src_dir_fd, dst_dir_fd=dst_dir_fd)
s = ssp
d = dsp
if not _under_prefix(s) and not _under_prefix(d):
return _rename(src, dst)
# The dispatcher picks the mount from the source and addresses the
# destination against that same backend, so a cross-mount rename would
# drop the source and write into the wrong store. EXDEV is the POSIX
# answer, and the one that tells a caller to copy instead.
if _mount_of(s) != _mount_of(d):
raise OSError(18, 'Invalid cross-device link', s, None, d)
_makedirs_raw(os.path.dirname(d))
try:
_rename(s, d)
except FileNotFoundError:
pass
_mb.markRename(s, d)
def _patched_open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None):
if isinstance(file, int):
return _open(file, mode, buffering, encoding, errors, newline, closefd, opener)
sp = file if isinstance(file, str) else os.fspath(file)
if isinstance(sp, bytes):
sp = sp.decode()
sp = os.path.abspath(sp)
under = _under_prefix(sp)
writable = _is_writable_mode(mode)
# A non-truncating writable open ('a', 'r+') keeps what the file
# already holds, so it needs the same backfill a read does; only 'w'
# and 'x' start from nothing. Without this an update-in-place opened
# against a MEMFS miss would build its buffer from empty.
if under and (not writable or not _truncates(mode)) and not _exists_raw(sp):
_backfill(sp)
if under and writable:
binary_mode = _strip_bt(mode) or 'r'
if 'b' in mode:
return _FlushOnClose(sp, mode=binary_mode, closefd=closefd, opener=opener)
raw = _FlushOnClose(sp, mode=binary_mode, closefd=closefd, opener=opener)
line_buffering = buffering == 1
return io.TextIOWrapper(raw, encoding=encoding, errors=errors, newline=newline, line_buffering=line_buffering)
return _open(file, mode, buffering, encoding, errors, newline, closefd, opener)
if not _already_patched:
_patched_open._mirage_patched = True
_patched_open._mirage_original_open = _open
builtins.open = _patched_open
io.open = _patched_open
os.listdir = _patched_listdir
os.stat = _patched_stat
os.scandir = _patched_scandir
# Rebinding these six covers the whole family: os.makedirs calls the
# module's own mkdir, pathlib's Path.mkdir/unlink/rename and
# shutil.rmtree/move all reach the mutation through os, so no
# per-spelling patch list is needed and none can fall out of step.
os.mkdir = _patched_mkdir
os.rmdir = _patched_rmdir
os.unlink = _patched_unlink
os.remove = _patched_unlink
os.rename = _patched_rename
os.replace = _patched_rename
os.lstat = _patched_lstat
# Nothing reached over the bridge can act on a directory fd: the
# mount ops address paths. Saying so keeps a caller that chooses
# between a path-based and an fd-relative implementation on the one
# these patches can see.
os.supports_dir_fd = frozenset()
# shutil is already imported when the shim installs, so it cached
# that answer from the unpatched os and rmtree would still walk with
# openat/unlinkat, invisible here. Correct the cached gate. If
# upstream renames it, the rmtree conformance row goes red rather
# than dropping the mutation silently.
_shutil = sys.modules.get('shutil')
if _shutil is not None and hasattr(_shutil, '_use_fd_functions'):
_shutil._use_fd_functions = False
_mod = types.ModuleType('_mirage_fs_shim')
_mod._original_listdir = _listdir
_mod._original_stat = _stat
_mod._original_scandir = _scandir
_mod._original_mkdir = _mkdir
_mod._original_rmdir = _rmdir
_mod._original_unlink = _unlink
_mod._original_rename = _rename
_mod._original_lstat = _lstat
_mod._backfill = _backfill
sys.modules['_mirage_fs_shim'] = _mod
`
typescript/packages/core/src/runtime/python/monty/binding.ts
+57
View File
@@ -0,0 +1,57 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { MISSING_PACKAGE_HINT } from './constants.ts'
export class MontyUnavailableError extends Error {
constructor(message: string, options?: { cause?: unknown }) {
super(message, options)
this.name = 'MontyUnavailableError'
}
}
// Structural views of @pydantic/monty so its types never leak into our
// public .d.ts (the package is an optional peer dependency). Python
// gets the same isolation from a try/except ImportError block that
// leaves the names None.
export interface MontySessionLike {
readonly workerPid?: number
feedRun(code: string, options?: Record<string, unknown>): Promise<unknown>
close(): Promise<void>
}
export interface MontyPoolLike {
checkout(options?: Record<string, unknown>): Promise<MontySessionLike>
close(): Promise<void>
}
export interface MontyModuleLike {
Monty: { create(options?: Record<string, unknown>): Promise<MontyPoolLike> }
NOT_HANDLED: symbol
MontySyntaxError: new (...args: never[]) => Error
MontyRuntimeError: new (...args: never[]) => Error
}
export interface MontyDisplayableError extends Error {
display?: (format?: string) => string
}
/** Import the optional binding, failing loud when it is absent. */
export async function loadMontyModule(): Promise<MontyModuleLike> {
try {
return (await import('@pydantic/monty')) as unknown as MontyModuleLike
} catch (err) {
throw new MontyUnavailableError(MISSING_PACKAGE_HINT, { cause: err })
}
}
typescript/packages/core/src/runtime/python/monty/constants.ts
+28
View File
@@ -0,0 +1,28 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
export const MISSING_PACKAGE_HINT =
"monty runtime requires the '@pydantic/monty' package — install it or select the pyodide runtime"
// argv[0] when the caller named no program of its own.
export const DEFAULT_PROG = 'main.py'
// One-shot eval is bounded like quickjs's: nothing above the runtime
// can stop a hung guest, so the runtime owns its own interrupt.
export const EVAL_INTERRUPT_SECONDS = 10
// A console tells "keep typing" from "this is broken" by matching
// monty's own traceback wording, so a version that rephrases either
// line turns every continuation into an error at the prompt.
export const INCOMPLETE_MARKERS = ['unexpected EOF', 'Expected an indented block'] as const
typescript/packages/core/src/runtime/python/monty/errors.ts
+72
View File
@@ -0,0 +1,72 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { MontyDisplayableError } from './binding.ts'
// fs error codes -> the python exception the guest should catch, with
// CPython's errno message shape.
const CODE_TO_GUEST_EXC = {
ENOENT: { name: 'FileNotFoundError', errno: 2, phrase: 'No such file or directory' },
EISDIR: { name: 'IsADirectoryError', errno: 21, phrase: 'Is a directory' },
ENOTDIR: { name: 'NotADirectoryError', errno: 20, phrase: 'Not a directory' },
EACCES: { name: 'PermissionError', errno: 13, phrase: 'Permission denied' },
EEXIST: { name: 'FileExistsError', errno: 17, phrase: 'File exists' },
EXDEV: { name: 'OSError', errno: 18, phrase: 'Invalid cross-device link' },
} as const
export type GuestCode = keyof typeof CODE_TO_GUEST_EXC
function isGuestCode(code: string | undefined): code is GuestCode {
return code !== undefined && code in CODE_TO_GUEST_EXC
}
/** The traceback monty renders for one of its own errors. */
export function displayError(err: unknown): string {
const e = err as MontyDisplayableError
if (typeof e.display === 'function') return e.display('traceback')
return e instanceof Error ? e.message : String(err)
}
/**
* Build the guest-side exception for one fs code, in CPython's message
* shape.
*
* Args:
* code: the fs error code, e.g. ENOENT.
* path: the path the operation names.
* target: rename's destination, when there is one.
*/
export function guestError(code: GuestCode, path: string, target?: string): Error {
const mapped = CODE_TO_GUEST_EXC[code]
const where = target === undefined ? `'${path}'` : `'${path}' -> '${target}'`
const guest = new Error(`[Errno ${String(mapped.errno)}] ${mapped.phrase}: ${where}`)
guest.name = mapped.name
return guest
}
/**
* Re-throw a mount failure under its python exception name: the monty
* binding raises `err.name` as the matching guest exception type
* (PYTHON_EXC_NAMES), so agent code can `except FileNotFoundError`
* exactly as it does on the python host.
*
* Args:
* err: whatever the mount op rejected with.
* path: the path the operation names.
*/
export function asGuestError(err: unknown, path: string): unknown {
const code = (err as { code?: string }).code
if (!isGuestCode(code)) return err
return guestError(code, path)
}
typescript/packages/core/src/runtime/python/monty/index.ts
+18
View File
@@ -0,0 +1,18 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
export { MontyUnavailableError } from './binding.ts'
export { MirageOSAccess } from './osaccess.ts'
export { MontyRuntime } from './runtime.ts'
export { MontyVFS } from './vfs.ts'
typescript/packages/core/src/runtime/python/monty/osaccess.test.ts
+117
View File
@@ -0,0 +1,117 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it, vi } from 'vitest'
import type { BridgeDispatchFn } from '../../types.ts'
import { RuntimeVFS } from '../../vfs.ts'
import { MirageOSAccess } from './index.ts'
import { MontyVFS } from './vfs.ts'
const NOT_HANDLED = Symbol('NOT_HANDLED')
function accessOn(
dispatch: BridgeDispatchFn,
env: Record<string, string> = {},
mounts: string[] = ['/ram'],
): MirageOSAccess {
return new MirageOSAccess(NOT_HANDLED, env, new MontyVFS(new RuntimeVFS(dispatch, () => mounts)))
}
const noop = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
describe('MirageOSAccess environment', () => {
it('answers os.getenv from the run environment, with the caller default on a miss', () => {
const access = accessOn(noop, { HOME: '/root' })
expect(access.handle('os.getenv', ['HOME'])).toBe('/root')
expect(access.handle('os.getenv', ['NOPE'])).toBeNull()
expect(access.handle('os.getenv', ['NOPE', 'fallback'])).toBe('fallback')
})
it('misses on an inherited property rather than leaking a host function', () => {
// The guest picks the key, so `toString` must not resolve.
expect(accessOn(noop, {}).handle('os.getenv', ['toString'])).toBeNull()
})
it('hands os.environ a copy, so a mutating guest cannot reach the session env', () => {
const env = { A: '1' }
const out = accessOn(noop, env).handle('os.environ', []) as Record<string, string>
expect(out).toEqual({ A: '1' })
out.A = 'tampered'
expect(env.A).toBe('1')
})
it('answers the environment doors even with no workspace attached', () => {
const access = new MirageOSAccess(NOT_HANDLED, { A: '1' }, null)
expect(access.handle('os.getenv', ['A'])).toBe('1')
expect(access.handle('Path.read_text', ['/ram/x'])).toBe(NOT_HANDLED)
})
})
describe('MirageOSAccess declining', () => {
it('declines a path outside every mount, so monty serves it from its own tree', () => {
expect(accessOn(noop).handle('Path.read_bytes', ['/tmp/x'])).toBe(NOT_HANDLED)
})
it('declines an operation it does not implement', () => {
expect(accessOn(noop).handle('Path.chmod', ['/ram/x'])).toBe(NOT_HANDLED)
})
it('declines a rename whose destination is outside the workspace', () => {
// Declining beats half-applying: monty then moves it in its own tree.
expect(accessOn(noop).handle('Path.rename', ['/ram/x', '/tmp/y'])).toBe(NOT_HANDLED)
})
it('accepts a path object as well as a string', () => {
expect(accessOn(noop).handle('Path.mkdir', [{ path: '/ram/d' }])).not.toBe(NOT_HANDLED)
})
})
describe('MirageOSAccess path operations', () => {
it('decodes read_text and leaves read_bytes raw', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(new TextEncoder().encode('hi')))
const access = accessOn(dispatch)
expect(await access.handle('Path.read_text', ['/ram/x'])).toBe('hi')
expect(await access.handle('Path.read_bytes', ['/ram/x'])).toEqual(
new TextEncoder().encode('hi'),
)
})
it('iterdir yields the entry paths', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([
{ path: '/ram/d/a', size: 1, isDir: false },
{ path: '/ram/d/sub', size: 0, isDir: true },
]),
)
expect(await accessOn(dispatch).handle('Path.iterdir', ['/ram/d'])).toEqual([
'/ram/d/a',
'/ram/d/sub',
])
})
it('answers the exists family as booleans, never as a rejection', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op, path) => {
if (op === 'LIST' && path === '/ram/d/') {
return Promise.resolve([{ path: '/ram/d/a', size: 1, isDir: false }])
}
return Promise.reject(Object.assign(new Error('gone'), { code: 'ENOENT' }))
})
const access = accessOn(dispatch)
expect(await access.handle('Path.is_dir', ['/ram/d'])).toBe(true)
expect(await access.handle('Path.is_file', ['/ram/d/a'])).toBe(true)
expect(await access.handle('Path.exists', ['/ram/d/a'])).toBe(true)
expect(await access.handle('Path.is_file', ['/ram/d/nope'])).toBe(false)
expect(await access.handle('Path.exists', ['/ram/nope/deep'])).toBe(false)
})
})
typescript/packages/core/src/runtime/python/monty/osaccess.ts
+127
View File
@@ -0,0 +1,127 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { MontyVFS } from './vfs.ts'
function pathArg(value: unknown): string | null {
if (typeof value === 'string') return value
if (value !== null && typeof value === 'object' && 'path' in value) {
const p = (value as { path: unknown }).path
return typeof p === 'string' ? p : null
}
return null
}
/**
* Monty's OS door, serving the guest's `os` and `pathlib` calls.
*
* This is monty's tier of the interception taxonomy: the binding calls
* one host callback per operation and takes back either a value (or a
* promise of one) or NOT_HANDLED, in which case monty falls back to
* its own in-memory tree. That is the whole seam, which is why this
* side is a value-returning table rather than the subclass Python's
* binding hands over, and why the runtime needs no interpreter patch.
*
* Declining is load-bearing, not an error path: a path under no mount
* must reach monty's own tree, so `/tmp` still behaves like `/tmp`.
*
* Args:
* notHandled: the binding's NOT_HANDLED sentinel.
* env: the run's environment, readable both ways python's monty
* spells it (`os.getenv` and `os.environ`).
* vfs: the mount view, or null when no workspace is attached.
*/
export class MirageOSAccess {
private readonly notHandled: symbol
private readonly env: Record<string, string>
private readonly vfs: MontyVFS | null
constructor(notHandled: symbol, env: Record<string, string>, vfs: MontyVFS | null) {
this.notHandled = notHandled
this.env = env
this.vfs = vfs
}
readonly handle = (name: string, args: unknown[]): unknown => {
if (name === 'os.getenv') {
// hasOwn, not `in`: the guest picks the key, so a name like
// `toString` must miss instead of leaking a host function.
const key = String(args[0])
if (Object.hasOwn(this.env, key)) return this.env[key]
return args.length > 1 ? args[1] : null
}
if (name === 'os.environ') {
// The engine asks for the whole mapping as one call; a plain
// object arrives in the guest as a dict, so `.get`, `[...]`,
// `in`, iteration and len all work, and a missing key raises
// KeyError. Declining instead raised "'os.environ' is not
// supported in this environment", which made a program written
// against the python host fail here (integ/runtime caught it).
// A copy, like python's OSAccess(environ=dict(environ)): a
// guest that mutates it cannot reach the session's own env.
return { ...this.env }
}
// Everything below serves a path through the mounts; the env doors
// above need no mount.
const vfs = this.vfs
if (vfs === null) return this.notHandled
const path = pathArg(args[0])
if (path === null || !vfs.serves(path)) return this.notHandled
switch (name) {
case 'Path.read_bytes':
return vfs.read(path)
case 'Path.read_text':
return vfs.read(path).then((b) => new TextDecoder().decode(b))
case 'Path.write_bytes':
case 'Path.write_text':
return vfs.write(path, args[1])
case 'Path.mkdir':
return vfs.mkdir(path)
case 'Path.rmdir':
return vfs.rmdir(path)
case 'Path.unlink':
return vfs.unlink(path)
case 'Path.rename': {
const dst = pathArg(args[1])
// A destination outside the workspace has no mount to rename
// into; decline rather than half-apply the move.
if (dst === null || !vfs.serves(dst)) return this.notHandled
return vfs.rename(path, dst)
}
case 'Path.iterdir':
return vfs.readdir(path).then((entries) => entries.map((e) => e.path))
case 'Path.is_dir':
return vfs.readdir(path).then(
() => true,
() => false,
)
case 'Path.is_file':
return vfs.entryFor(path).then(
(e) => e !== null && !e.isDir,
() => false,
)
case 'Path.exists':
return vfs.entryFor(path).then(
(e) => e !== null,
() =>
vfs.readdir(path).then(
() => true,
() => false,
),
)
default:
return this.notHandled
}
}
}
typescript/packages/core/src/runtime/python/monty.test.ts → typescript/packages/core/src/runtime/python/monty/runtime.test.ts
+22 -10
View File
@@ -13,14 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { afterAll, describe, expect, it } from 'vitest'
import type { BridgeDispatchFn } from '../types.ts'
import { MontyRuntime } from './monty.ts'
import { PyodideRuntime } from './pyodide.ts'
import { buildRuntime } from '../table.ts'
import { getTestParser } from '../../workspace/fixtures/workspace_fixture.ts'
import { RAMResource } from '../../resource/ram/ram.ts'
import { MountMode } from '../../types.ts'
import { Workspace } from '../../workspace/workspace.ts'
import type { BridgeDispatchFn } from '../../types.ts'
import { MontyRuntime } from './index.ts'
import { PyodideRuntime } from '../pyodide.ts'
import { buildRuntime } from '../../table.ts'
import { getTestParser } from '../../../workspace/fixtures/workspace_fixture.ts'
import { RAMResource } from '../../../resource/ram/ram.ts'
import { MountMode } from '../../../types.ts'
import { Workspace } from '../../../workspace/workspace.ts'
function makeBridge(seed: Record<string, Uint8Array>): {
dispatch: BridgeDispatchFn
@@ -275,6 +275,18 @@ describe('MontyRuntime', () => {
expect(files.has('/s3/a.txt')).toBe(false)
}, 30_000)
it('unlink after rename reaches the mount', async () => {
const { dispatch, mutations, files } = makeBridge({ '/s3/a.txt': new Uint8Array([1]) })
const rt = make(dispatch)
const result = await run(
rt,
"from pathlib import Path\nPath('/s3/a.txt').rename('/s3/b.txt')\nPath('/s3/b.txt').unlink()",
)
expect(result.exitCode).toBe(0)
expect(mutations).toEqual(['RENAME /s3/a.txt /s3/b.txt', 'UNLINK /s3/b.txt'])
expect(files.has('/s3/b.txt')).toBe(false)
}, 30_000)
it('rename carries both paths to the bridge', async () => {
const { dispatch, mutations, files } = makeBridge({ '/s3/a.txt': new Uint8Array([1]) })
const rt = make(dispatch)
@@ -483,8 +495,8 @@ describe('Workspace with the monty runtime', () => {
describe('monty unavailable', () => {
it('handlePython maps MontyUnavailableError to exit 127', async () => {
const { handlePython } = await import('../../workspace/executor/python/handle.ts')
const { MontyUnavailableError } = await import('./monty.ts')
const { handlePython } = await import('../../../workspace/executor/python/handle.ts')
const { MontyUnavailableError } = await import('./index.ts')
const runtime = {
name: 'monty',
captures: ['python3', 'python'],
typescript/packages/core/src/runtime/python/monty.ts → typescript/packages/core/src/runtime/python/monty/runtime.ts
+40 -309
View File
@@ -12,53 +12,24 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { CommandTimeoutError } from '../../commands/builtin/utils/limit.ts'
import { PythonRuntime } from './base.ts'
import { EvalError } from '../errors.ts'
import { EVALUATOR, type Evaluator } from '../mixin.ts'
import type { EvalResult, EvalValue, RunArgs, RunResult, RuntimeOptions } from '../types.ts'
import type { BridgeDispatchFn } from '../types.ts'
import { MONTY_RUNTIME } from './interface.ts'
export class MontyUnavailableError extends Error {
constructor(message: string, options?: { cause?: unknown }) {
super(message, options)
this.name = 'MontyUnavailableError'
}
}
// Structural views of @pydantic/monty so its types never leak into our
// public .d.ts (the package is an optional peer dependency).
interface MontySessionLike {
readonly workerPid?: number
feedRun(code: string, options?: Record<string, unknown>): Promise<unknown>
close(): Promise<void>
}
interface MontyPoolLike {
checkout(options?: Record<string, unknown>): Promise<MontySessionLike>
close(): Promise<void>
}
interface MontyModuleLike {
Monty: { create(options?: Record<string, unknown>): Promise<MontyPoolLike> }
NOT_HANDLED: symbol
MontySyntaxError: new (...args: never[]) => Error
MontyRuntimeError: new (...args: never[]) => Error
}
interface MontyDisplayableError extends Error {
display?: (format?: string) => string
}
interface MirageEntryLike {
path: string
isDir: boolean
}
// One-shot eval is bounded like quickjs's: nothing above the runtime
// can stop a hung guest, so the runtime owns its own interrupt.
const EVAL_INTERRUPT_SECONDS = 10
import { CommandTimeoutError } from '../../../commands/builtin/utils/limit.ts'
import { PythonRuntime } from '../base.ts'
import { EvalError } from '../../errors.ts'
import { EVALUATOR, type Evaluator } from '../../mixin.ts'
import type { EvalResult, EvalValue, RunArgs, RunResult, RuntimeOptions } from '../../types.ts'
import type { BridgeDispatchFn, PrefixSource } from '../../types.ts'
import { RuntimeVFS } from '../../vfs.ts'
import { MONTY_RUNTIME } from '../interface.ts'
import {
loadMontyModule,
type MontyModuleLike,
type MontyPoolLike,
type MontySessionLike,
} from './binding.ts'
import { DEFAULT_PROG, EVAL_INTERRUPT_SECONDS, INCOMPLETE_MARKERS } from './constants.ts'
import { displayError } from './errors.ts'
import { MirageOSAccess } from './osaccess.ts'
import { MontyVFS } from './vfs.ts'
const INTERRUPTED = Symbol('interrupted')
@@ -90,12 +61,6 @@ function killWorker(pid: number | undefined): void {
}
}
function displayError(err: unknown): string {
const e = err as MontyDisplayableError
if (typeof e.display === 'function') return e.display('traceback')
return e instanceof Error ? e.message : String(err)
}
/**
* Fold Monty's value shapes into the EvalValue contract: the worker
* hands python dicts back as Map, but EvalValue promises str-keyed
@@ -131,7 +96,8 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
readonly name = MONTY_RUNTIME
readonly [EVALUATOR] = true as const
private workspaceBridge: BridgeDispatchFn | null = null
private listMounts: () => string[] = () => []
private listMounts: PrefixSource = () => []
private vfs: MontyVFS | null = null
private module: MontyModuleLike | null = null
private pool: MontyPoolLike | null = null
private poolPromise: Promise<MontyPoolLike> | null = null
@@ -141,10 +107,11 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
super(options)
}
override attach(dispatch: BridgeDispatchFn, listMounts: () => string[]): void {
override attach(dispatch: BridgeDispatchFn, listMounts: PrefixSource): void {
if (this.workspaceBridge === null) {
this.workspaceBridge = dispatch
this.listMounts = listMounts
this.vfs = new MontyVFS(new RuntimeVFS(dispatch, listMounts))
}
}
@@ -246,7 +213,7 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
if (stream === 'stderr') err.push(text)
else out.push(text)
},
os: this.buildOsCallback(module, {}),
os: new MirageOSAccess(module.NOT_HANDLED, {}, this.perRunVfs()).handle,
}
const enc = new TextEncoder()
// One-shot evals get the quickjs-style 10s bound (the policy layer
@@ -280,8 +247,7 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
const trace = displayError(caught)
// Console continuation, not a broken program: the source
// merely stopped early (an open block or unclosed suite).
const incomplete =
trace.includes('unexpected EOF') || trace.includes('Expected an indented block')
const incomplete = INCOMPLETE_MARKERS.some((marker) => trace.includes(marker))
if (opts.session !== undefined) {
if (incomplete) {
return {
@@ -334,6 +300,19 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
this.poolPromise = null
}
/**
* The mount view for one command, with its negative cache cleared.
*
* python builds a fresh `MirageOSAccess` per run, so its absence
* cache never outlives a command; this view is attached once, so it
* has to be told. Without the reset a path a shell command created
* between two monty commands would stay invisible to the second.
*/
private perRunVfs(): MontyVFS | null {
this.vfs?.reset()
return this.vfs
}
private async ensurePool(): Promise<MontyPoolLike> {
if (this.pool !== null) return this.pool
this.poolPromise ??= this.loadPool()
@@ -347,15 +326,7 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
}
private async loadModule(): Promise<MontyModuleLike> {
if (this.module !== null) return this.module
try {
this.module = (await import('@pydantic/monty')) as unknown as MontyModuleLike
} catch (err) {
throw new MontyUnavailableError(
"monty runtime requires the '@pydantic/monty' package — install it or select the pyodide runtime",
{ cause: err },
)
}
this.module ??= await loadMontyModule()
return this.module
}
@@ -370,12 +341,12 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
const options: Record<string, unknown> = {
// argv[0] is the program's own name when the caller has one (a
// CLI install's head word), else the interpreter's placeholder.
inputs: { argv: [args.prog ?? 'main.py', ...args.args], stdin: args.stdin },
inputs: { argv: [args.prog ?? DEFAULT_PROG, ...args.args], stdin: args.stdin },
printCallback: (stream: 'stdout' | 'stderr', text: string) => {
if (stream === 'stderr') err.push(text)
else out.push(text)
},
os: this.buildOsCallback(module, args.env),
os: new MirageOSAccess(module.NOT_HANDLED, args.env, this.perRunVfs()).handle,
}
try {
await session.feedRun(code, options)
@@ -396,244 +367,4 @@ export class MontyRuntime extends PythonRuntime implements Evaluator {
exitCode: 0,
}
}
private buildOsCallback(
module: MontyModuleLike,
env: Record<string, string>,
): (name: string, args: unknown[]) => unknown {
const bridge = this.workspaceBridge
const notHandled = module.NOT_HANDLED
return (name: string, args: unknown[]): unknown => {
if (name === 'os.getenv') {
// hasOwn, not `in`: the guest picks the key, so a name like
// `toString` must miss instead of leaking a host function.
const key = String(args[0])
if (Object.hasOwn(env, key)) return env[key]
return args.length > 1 ? args[1] : null
}
if (name === 'os.environ') {
// The engine asks for the whole mapping as one call; a plain
// object arrives in the guest as a dict, so `.get`, `[...]`,
// `in`, iteration and len all work, and a missing key raises
// KeyError. Declining instead raised "'os.environ' is not
// supported in this environment", which made a program written
// against the python host fail here (integ/runtime caught it).
// A copy, like python's OSAccess(environ=dict(environ)): a
// guest that mutates it cannot reach the session's own env.
return { ...env }
}
// Everything below serves a path through the workspace bridge;
// the env doors above need no mount.
if (bridge === null) return notHandled
const path = pathArg(args[0])
if (path === null) return notHandled
if (!this.underWorkspaceMount(path)) return notHandled
switch (name) {
case 'Path.read_bytes':
return readBytes(bridge, path)
case 'Path.read_text':
return readBytes(bridge, path).then((b) => new TextDecoder().decode(b))
case 'Path.write_bytes':
case 'Path.write_text':
return writeBack(bridge, path, args[1])
case 'Path.mkdir':
return mutate(bridge, 'MKDIR', path)
case 'Path.rmdir':
return mutate(bridge, 'RMDIR', path)
case 'Path.unlink':
return mutate(bridge, 'UNLINK', path)
case 'Path.rename': {
const dst = pathArg(args[1])
// A destination outside the workspace has no mount to rename
// into; decline rather than half-apply the move.
if (dst === null || !this.underWorkspaceMount(dst)) return notHandled
// The dispatcher picks the mount from the source alone and
// reads the destination against that same backend, so a
// cross-mount rename would drop the source and write the
// target into the wrong store. POSIX answers this with
// EXDEV, which is also what tells a caller to copy instead.
if (this.mountOf(path) !== this.mountOf(dst)) {
return Promise.reject(guestError('EXDEV', path, dst))
}
return mutate(bridge, 'RENAME', path, dst)
}
case 'Path.iterdir':
return listEntries(bridge, path).then((entries) => entries.map((e) => e.path))
case 'Path.is_dir':
return listEntries(bridge, path).then(
() => true,
() => false,
)
case 'Path.is_file':
return entryFor(bridge, path).then(
(e) => e !== null && !e.isDir,
() => false,
)
case 'Path.exists':
return entryFor(bridge, path).then(
(e) => e !== null,
() =>
listEntries(bridge, path).then(
() => true,
() => false,
),
)
default:
return notHandled
}
}
}
/**
* True when `path` may be serviced by the workspace bridge. An empty
* live view means no scoping: every path routes to the bridge.
*/
private underWorkspaceMount(path: string): boolean {
const prefixes = this.listMounts()
if (prefixes.length === 0) return true
return prefixes.some((p) => {
const norm = p.endsWith('/') ? p : p + '/'
return path.startsWith(norm) || path === norm.slice(0, -1)
})
}
/**
* The mount prefix serving `path`, longest match first, or null when
* none does. Two paths belong to the same mount only when this
* agrees for both, which is what a rename needs to know.
*/
private mountOf(path: string): string | null {
let best: string | null = null
for (const p of this.listMounts()) {
const norm = p.endsWith('/') ? p : p + '/'
if (path.startsWith(norm) || path === norm.slice(0, -1)) {
if (best === null || norm.length > best.length) best = norm
}
}
return best
}
}
function pathArg(value: unknown): string | null {
if (typeof value === 'string') return value
if (value !== null && typeof value === 'object' && 'path' in value) {
const p = (value as { path: unknown }).path
return typeof p === 'string' ? p : null
}
return null
}
// fs error codes -> the python exception the guest should catch, with
// CPython's errno message shape.
const CODE_TO_GUEST_EXC = {
ENOENT: { name: 'FileNotFoundError', errno: 2, phrase: 'No such file or directory' },
EISDIR: { name: 'IsADirectoryError', errno: 21, phrase: 'Is a directory' },
ENOTDIR: { name: 'NotADirectoryError', errno: 20, phrase: 'Not a directory' },
EACCES: { name: 'PermissionError', errno: 13, phrase: 'Permission denied' },
EEXIST: { name: 'FileExistsError', errno: 17, phrase: 'File exists' },
EXDEV: { name: 'OSError', errno: 18, phrase: 'Invalid cross-device link' },
} as const
type GuestCode = keyof typeof CODE_TO_GUEST_EXC
function isGuestCode(code: string | undefined): code is GuestCode {
return code !== undefined && code in CODE_TO_GUEST_EXC
}
/**
* Build the guest-side exception for one fs code, in CPython's message
* shape. `target` renders rename's two-path form.
*
* Args:
* code: the fs error code, e.g. ENOENT.
* path: the path the operation names.
* target: rename's destination, when there is one.
*/
function guestError(code: GuestCode, path: string, target?: string): Error {
const mapped = CODE_TO_GUEST_EXC[code]
const where = target === undefined ? `'${path}'` : `'${path}' -> '${target}'`
const guest = new Error(`[Errno ${String(mapped.errno)}] ${mapped.phrase}: ${where}`)
guest.name = mapped.name
return guest
}
/**
* Re-throw a bridge failure under its python exception name: the monty
* binding raises `err.name` as the matching guest exception type
* (PYTHON_EXC_NAMES), so agent code can `except FileNotFoundError`
* exactly as it does on the python host.
*/
function asGuestError(err: unknown, path: string): unknown {
const code = (err as { code?: string }).code
if (!isGuestCode(code)) return err
return guestError(code, path)
}
/**
* Run one mutating bridge op, translating a coded failure the way the
* read and write helpers do. Without this the raw workspace error
* reaches monty with `name` still `Error`, so guest code cannot catch
* the `FileNotFoundError` or `FileExistsError` the operation implies.
*
* Args:
* bridge: the workspace dispatch callable.
* op: the mutation to run.
* path: the path the operation names.
* dst: rename's destination.
*/
async function mutate(
bridge: BridgeDispatchFn,
op: 'MKDIR' | 'RMDIR' | 'UNLINK' | 'RENAME',
path: string,
dst?: string,
): Promise<null> {
try {
await bridge(op, path, undefined, dst)
} catch (caught) {
throw asGuestError(caught, path)
}
return null
}
async function readBytes(bridge: BridgeDispatchFn, path: string): Promise<Uint8Array> {
let data: unknown
try {
data = await bridge('READ', path)
} catch (caught) {
throw asGuestError(caught, path)
}
if (data instanceof Uint8Array) return data
throw new Error(`monty bridge: READ ${path} expected bytes`)
}
async function writeBack(bridge: BridgeDispatchFn, path: string, data: unknown): Promise<number> {
const bytes =
data instanceof Uint8Array
? data
: new TextEncoder().encode(typeof data === 'string' ? data : '')
try {
await bridge('WRITE', path, bytes)
} catch (caught) {
throw asGuestError(caught, path)
}
return typeof data === 'string' ? data.length : bytes.length
}
async function listEntries(bridge: BridgeDispatchFn, path: string): Promise<MirageEntryLike[]> {
const prefix = path.endsWith('/') ? path : path + '/'
let out: unknown
try {
out = await bridge('LIST', prefix)
} catch (caught) {
throw asGuestError(caught, path)
}
if (!Array.isArray(out)) throw new Error(`monty bridge: LIST ${prefix} expected array`)
return out as MirageEntryLike[]
}
async function entryFor(bridge: BridgeDispatchFn, path: string): Promise<MirageEntryLike | null> {
const slash = path.lastIndexOf('/')
const parent = slash <= 0 ? '/' : path.slice(0, slash)
const entries = await listEntries(bridge, parent)
return entries.find((e) => e.path === path || e.path === path + '/') ?? null
}
typescript/packages/core/src/runtime/python/monty/vfs.test.ts
+179
View File
@@ -0,0 +1,179 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it, vi } from 'vitest'
import type { BridgeDispatchFn } from '../../types.ts'
import { RuntimeVFS } from '../../vfs.ts'
import { MontyVFS } from './index.ts'
function viewOn(dispatch: BridgeDispatchFn, mounts: string[] = ['/ram']): MontyVFS {
return new MontyVFS(new RuntimeVFS(dispatch, () => mounts))
}
describe('MontyVFS scoping', () => {
const noop = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
it('serves a path under a mount and declines one outside every mount', () => {
const vfs = viewOn(noop)
expect(vfs.serves('/ram/x')).toBe(true)
expect(vfs.serves('/ram')).toBe(true)
expect(vfs.serves('/tmp/x')).toBe(false)
})
it('serves everything when no mounts are wired', () => {
// No scoping rather than no service: the runtime is attached but the
// workspace has no prefixes, so nothing should be withheld.
expect(viewOn(noop, []).serves('/tmp/x')).toBe(true)
})
})
describe('MontyVFS guest errors', () => {
it('names a missing file the way python spells it, so except catches', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.reject(Object.assign(new Error('gone'), { code: 'ENOENT' })),
)
await expect(viewOn(dispatch).read('/ram/x')).rejects.toMatchObject({
name: 'FileNotFoundError',
message: "[Errno 2] No such file or directory: '/ram/x'",
})
})
it('names an existing directory FileExistsError on mkdir', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.reject(Object.assign(new Error('there'), { code: 'EEXIST' })),
)
await expect(viewOn(dispatch).mkdir('/ram/d')).rejects.toMatchObject({
name: 'FileExistsError',
})
})
it('leaves an unmapped failure alone rather than inventing an errno', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.reject(new Error('transport down')))
await expect(viewOn(dispatch).read('/ram/x')).rejects.toThrow(/transport down/)
})
it('spells a cross-mount rename EXDEV, which is what tells a caller to copy', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await expect(viewOn(dispatch, ['/ram', '/s3']).rename('/ram/x', '/s3/x')).rejects.toMatchObject(
{
name: 'OSError',
message: "[Errno 18] Invalid cross-device link: '/ram/x' -> '/s3/x'",
},
)
expect(dispatch).not.toHaveBeenCalled()
})
})
describe('MontyVFS values', () => {
it('reports the character count for text and the byte count for bytes', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
const vfs = viewOn(dispatch)
expect(await vfs.write('/ram/x', 'héllo')).toBe(5)
expect(await vfs.write('/ram/y', new Uint8Array([1, 2, 3]))).toBe(3)
})
it('lists a directory through its slash-terminated prefix', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([{ path: '/ram/d/a', size: 1, isDir: false }]),
)
const entries = await viewOn(dispatch).readdir('/ram/d')
expect(dispatch).toHaveBeenCalledWith('LIST', '/ram/d/')
expect(entries).toHaveLength(1)
})
it('finds a path through its parent listing, and answers null for a miss', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([{ path: '/ram/a', size: 1, isDir: false }]),
)
const vfs = viewOn(dispatch)
expect(await vfs.entryFor('/ram/a')).toMatchObject({ isDir: false })
expect(await vfs.entryFor('/ram/nope')).toBeNull()
})
})
describe('MontyVFS negative cache', () => {
const listing = () =>
vi.fn<BridgeDispatchFn>((op) => {
if (op === 'LIST') return Promise.resolve([{ path: '/ram/a', size: 1, isDir: false }])
return Promise.resolve(undefined)
})
it('remembers an absence, so a repeated probe costs no second listing', async () => {
// Monty asks whether a path exists on nearly every guest
// expression, so the second miss must not reach the mount.
const dispatch = listing()
const vfs = viewOn(dispatch)
expect(await vfs.entryFor('/ram/nope')).toBeNull()
expect(await vfs.entryFor('/ram/nope')).toBeNull()
expect(dispatch.mock.calls.filter((c) => c[0] === 'LIST')).toHaveLength(1)
})
it('answers a remembered absence from read without dispatching', async () => {
const dispatch = listing()
const vfs = viewOn(dispatch)
await vfs.entryFor('/ram/nope')
await expect(vfs.read('/ram/nope')).rejects.toMatchObject({ name: 'FileNotFoundError' })
expect(dispatch.mock.calls.filter((c) => c[0] === 'READ')).toHaveLength(0)
})
it('forgets the absence once the path is written, so the guest sees its own write', async () => {
const dispatch = listing()
const vfs = viewOn(dispatch)
expect(await vfs.entryFor('/ram/new.txt')).toBeNull()
await vfs.write('/ram/new.txt', 'hi')
await vfs.entryFor('/ram/new.txt')
expect(dispatch.mock.calls.filter((c) => c[0] === 'LIST')).toHaveLength(2)
})
it('remembers a path it removed, and forgets a rename destination', async () => {
const dispatch = listing()
const vfs = viewOn(dispatch)
await vfs.unlink('/ram/a')
expect(await vfs.entryFor('/ram/a')).toBeNull()
expect(dispatch.mock.calls.filter((c) => c[0] === 'LIST')).toHaveLength(0)
await vfs.rename('/ram/b', '/ram/a')
expect(await vfs.entryFor('/ram/a')).toMatchObject({ isDir: false })
})
it('forgets across commands, so another writer between runs is seen', async () => {
// python builds a fresh MirageOSAccess per run; this view is
// attached once, so the runtime resets it at the top of each
// command. Without that a shell command's file stays invisible.
let created = false
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve(created ? [{ path: '/ram/late.txt', size: 1, isDir: false }] : []),
)
const vfs = viewOn(dispatch)
expect(await vfs.entryFor('/ram/late.txt')).toBeNull()
created = true
expect(await vfs.entryFor('/ram/late.txt')).toBeNull()
vfs.reset()
expect(await vfs.entryFor('/ram/late.txt')).toMatchObject({ isDir: false })
})
it('does not remember a transport failure as an absence', async () => {
// "I could not reach the mount" is not "there is nothing here";
// caching it would hide the file for the rest of the run.
let down = true
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (down) return Promise.reject(new Error('transport down'))
if (op === 'READ') return Promise.resolve(new TextEncoder().encode('back'))
return Promise.resolve(undefined)
})
const vfs = viewOn(dispatch)
await expect(vfs.read('/ram/x')).rejects.toThrow(/transport down/)
down = false
expect(await vfs.read('/ram/x')).toEqual(new TextEncoder().encode('back'))
})
})
typescript/packages/core/src/runtime/python/monty/vfs.ts
+186
View File
@@ -0,0 +1,186 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { CrossMountError } from '../../errors.ts'
import type { RuntimeVFS, VFSEntry } from '../../vfs.ts'
import { asGuestError, guestError } from './errors.ts'
// The three ways a mount says "there is nothing here", as opposed to
// "I could not reach it": only these may be remembered as absence.
const ABSENT_NAMES = new Set(['FileNotFoundError', 'IsADirectoryError', 'NotADirectoryError'])
function isAbsence(err: unknown): boolean {
return err instanceof Error && ABSENT_NAMES.has(err.name)
}
/**
* Monty's mount view: the shared core, spelled the way the binding
* needs it, plus a negative cache.
*
* Three things the core deliberately does not do live here. Every
* failure is re-thrown under its python exception name, because the
* binding turns `err.name` into the guest exception type and agent
* code catches `FileNotFoundError`, not a bare Error. A path outside
* every mount is *declined* rather than failed, so monty falls back to
* its own in-memory tree; the core's `mountOf` answers null for both
* "no mounts wired" and "not under one", which are different questions
* here. And a path the mount already answered "not there" for is
* remembered, because monty asks whether a path exists on nearly every
* guest expression and each miss otherwise costs a fresh listing;
* every mutation keeps the cache honest.
*
* Args:
* core: the shared op vocabulary.
*/
export class MontyVFS {
private readonly core: RuntimeVFS
private readonly missing = new Set<string>()
constructor(core: RuntimeVFS) {
this.core = core
}
/**
* Forget every remembered absence.
*
* The cache may only span one command. This object lives as long as
* the runtime, but python builds its `MirageOSAccess` (and with it
* the `_missing` set this mirrors) once per run, so a file a shell
* command creates between two monty commands has to be visible to
* the second one. Callers reset at the top of `run` and each `eval`.
*/
reset(): void {
this.missing.clear()
}
/**
* True when `path` may be serviced by the mounts. An empty live view
* means no scoping: every path routes to the workspace.
*/
serves(path: string): boolean {
const prefixes = this.core.prefixes()
if (prefixes.length === 0) return true
return this.core.mountOf(path) !== null
}
async read(path: string): Promise<Uint8Array> {
if (this.missing.has(path)) throw guestError('ENOENT', path)
try {
return await this.core.read(path)
} catch (caught) {
throw this.absent(path, asGuestError(caught, path))
}
}
async write(path: string, data: unknown): Promise<number> {
const bytes =
data instanceof Uint8Array
? data
: new TextEncoder().encode(typeof data === 'string' ? data : '')
try {
await this.core.write(path, bytes)
} catch (caught) {
throw asGuestError(caught, path)
}
this.missing.delete(path)
return typeof data === 'string' ? data.length : bytes.length
}
async mkdir(path: string): Promise<null> {
const out = await this.mutate(path, () => this.core.mkdir(path))
this.missing.delete(path)
return out
}
async rmdir(path: string): Promise<null> {
const out = await this.mutate(path, () => this.core.rmdir(path))
this.missing.add(path)
return out
}
async unlink(path: string): Promise<null> {
const out = await this.mutate(path, () => this.core.unlink(path))
this.missing.add(path)
return out
}
/**
* Rename within one mount, spelling a cross-mount pair as EXDEV.
*
* POSIX answers EXDEV for a rename across filesystems, which is also
* what tells a caller to copy instead. Monty ships no `shutil`, so
* its own code has to write that fallback by hand; the runtimes with
* a real stdlib get it from `shutil.move`, which retries on exactly
* this errno.
*
* Args:
* src: the rename source.
* dst: the rename destination.
*/
async rename(src: string, dst: string): Promise<null> {
try {
await this.core.rename(src, dst)
} catch (caught) {
if (caught instanceof CrossMountError) throw guestError('EXDEV', src, dst)
throw asGuestError(caught, src)
}
this.missing.add(src)
this.missing.delete(dst)
return null
}
/** The directory's entries. Throws when it is not a directory. */
async readdir(path: string): Promise<VFSEntry[]> {
const prefix = path.endsWith('/') ? path : path + '/'
try {
return await this.core.readdir(prefix)
} catch (caught) {
throw asGuestError(caught, path)
}
}
/** The parent's entry for `path`, or null when the parent lacks one. */
async entryFor(path: string): Promise<VFSEntry | null> {
if (this.missing.has(path)) return null
const slash = path.lastIndexOf('/')
const parent = slash <= 0 ? '/' : path.slice(0, slash)
const entries = await this.readdir(parent)
const found = entries.find((e) => e.path === path || e.path === path + '/') ?? null
if (found === null) this.missing.add(path)
return found
}
/**
* Remember `path` as absent when `err` says it is, then hand the
* error back for throwing. A transport failure is not an absence,
* so only the three fs codes that mean "nothing here" are cached.
*
* Args:
* path: the path the operation named.
* err: the guest-shaped error the op failed with.
*/
private absent(path: string, err: unknown): unknown {
if (isAbsence(err)) this.missing.add(path)
return err
}
private async mutate(path: string, run: () => Promise<void>): Promise<null> {
try {
await run()
} catch (caught) {
throw asGuestError(caught, path)
}
return null
}
}
typescript/packages/core/src/runtime/python/mount.test.ts
+69 -7
View File
@@ -32,6 +32,15 @@ function makeBridge(): {
files.set(path, normalizedBytes ?? new Uint8Array())
return Promise.resolve(undefined)
}
if (op === 'APPEND') {
const base = files.get(path) ?? new Uint8Array()
const tail = normalizedBytes ?? new Uint8Array()
const next = new Uint8Array(base.length + tail.length)
next.set(base)
next.set(tail, base.length)
files.set(path, next)
return Promise.resolve(undefined)
}
if (op === 'READ') return Promise.resolve(files.get(path) ?? new Uint8Array())
const prefix = path
const entries: { path: string; size: number; isDir: boolean }[] = []
@@ -130,6 +139,55 @@ describe('PyodideRuntime mount visibility', () => {
await rt.close()
}, 60_000)
it('a root mount is refused rather than mounted at nothing', async () => {
// `/` is already MEMFS's mount root, so Emscripten answers EBUSY;
// the empty mountpoint it used to compute mounts a detached
// filesystem, and the guest then reads and writes MEMFS while a
// write reports success the resource never sees.
const { dispatch, calls } = makeBridge()
const warnings: string[] = []
const warn = console.warn
console.warn = (msg: unknown) => warnings.push(String(msg))
const rt = new PyodideRuntime()
rt.attach(dispatch, () => ['/'])
try {
const result = await rt.run({
code: `open('/out.txt', 'wb').write(b'data')`,
args: [],
env: {},
stdin: new Uint8Array(),
})
expect(result.exitCode).toBe(0)
expect(calls.filter((c) => c.op === 'WRITE')).toHaveLength(0)
expect(warnings.some((w) => w.includes("cannot serve a mount at '/'"))).toBe(true)
// Reported once, not once per run.
await rt.run({ code: 'pass', args: [], env: {}, stdin: new Uint8Array() })
expect(warnings.filter((w) => w.includes('cannot serve a mount'))).toHaveLength(1)
} finally {
console.warn = warn
await rt.close()
}
}, 60_000)
it('a nested prefix stays reachable under its parent mount', async () => {
// prefixes() is longest-first; mounting in that order puts /data
// over the /data/inner mounted a moment earlier and orphans it.
const { dispatch, files } = makeBridge()
files.set('/data/outer.txt', new TextEncoder().encode('OUTER'))
files.set('/data/inner/deep.txt', new TextEncoder().encode('DEEP'))
const rt = new PyodideRuntime()
rt.attach(dispatch, () => ['/data/', '/data/inner/'])
const result = await rt.run({
code: "print(open('/data/outer.txt').read(), open('/data/inner/deep.txt').read())",
args: [],
env: {},
stdin: new Uint8Array(),
})
expect(result.exitCode).toBe(0)
expect(new TextDecoder().decode(result.stdout)).toContain('OUTER DEEP')
await rt.close()
}, 60_000)
it('a failed flush surfaces on stderr and flips a clean exit to 1', async () => {
const dispatch: BridgeDispatchFn = (op) => {
if (op === 'WRITE') return Promise.reject(new Error('mount is read-only'))
@@ -195,14 +253,19 @@ describe('PyodideRuntime mount visibility', () => {
await rt.close()
}, 60_000)
it('an unreadable base fails the append rather than replacing the file', async () => {
it('an unreadable base refuses the open rather than replacing the file', async () => {
const writes: Uint8Array[] = []
// The mount lists the file, so it exists, but will not hand over its
// content. Leaving it out of the guest's tree would read as absence,
// and the append would then ship its tail as the whole file.
const dispatch: BridgeDispatchFn = (op, path, bytes) => {
if (op === 'READ' && path === '/ram/log.txt') {
return Promise.reject(new Error('backend unavailable'))
}
if (op === 'READ') return Promise.resolve(new Uint8Array())
if (op === 'LIST') return Promise.resolve([])
if (op === 'LIST') {
return Promise.resolve([{ path: '/ram/log.txt', size: 4, isDir: false }])
}
if (op === 'WRITE' && bytes !== undefined) writes.push(new Uint8Array(bytes))
return Promise.resolve(undefined)
}
@@ -214,12 +277,11 @@ describe('PyodideRuntime mount visibility', () => {
env: {},
stdin: new Uint8Array(),
})
// An error that is not a confirmed absence must not be read as an
// empty base, or the tail alone would overwrite the file.
expect(writes).toHaveLength(0)
expect(new TextDecoder().decode(result.stderr ?? new Uint8Array())).toContain(
'failed to append /ram/log.txt',
)
// The refusal reaches the guest at the call site now, rather than
// surfacing after the run as a failed replay.
const stderr = new TextDecoder().decode(result.stderr ?? new Uint8Array())
expect(stderr).toContain('OSError')
expect(result.exitCode).toBe(1)
await rt.close()
}, 60_000)
typescript/packages/core/src/runtime/python/nojspi.test.ts
+7
View File
@@ -150,6 +150,13 @@ describe('PyodideRuntime without JSPI', () => {
files.set(path, new Uint8Array(bytes))
writes.push({ path, text: new TextDecoder().decode(bytes) })
}
if (op === 'APPEND' && bytes !== undefined) {
const base = files.get(path) ?? new Uint8Array()
const next = new Uint8Array(base.length + bytes.length)
next.set(base)
next.set(bytes, base.length)
files.set(path, next)
}
return undefined
}
const rt = new PyodideRuntime()
typescript/packages/core/src/runtime/python/pyodide.ts
+107 -41
View File
@@ -26,14 +26,12 @@ import type {
} from '../types.ts'
import { createPyodideInterrupter, type PyodideInterrupter } from './interrupt.ts'
import { loadPyodideRuntime, type PyodideInterface } from './loader.ts'
import {
applyMutation,
createMirageBridge,
preloadInto,
type MirageBridge,
} from './mirage_bridge.ts'
import type { BridgeDispatchFn } from '../types.ts'
import { MIRAGE_FS_SHIM_PY } from './mirage_fs_shim.ts'
import { RuntimeVFS } from '../vfs.ts'
import { applyMutation, createJournal, type MutationJournal } from './vfs/journal.ts'
import { preloadInto } from './vfs/preload.ts'
import { MirageFs } from './vfs/vfs.ts'
import { MirageFsSeed } from './vfs/seed.ts'
import { PYTHON_EVAL_WRAPPER, PYTHON_REPL_WRAPPER, PYTHON_WRAPPER } from './wrapper.ts'
import { PYODIDE_RUNTIME } from './interface.ts'
@@ -72,6 +70,32 @@ function reviveEvalValue(_key: string, value: unknown): unknown {
return bytes
}
// Mount prefixes are normalized with a trailing slash; an Emscripten
// mountpoint is a directory path, so it carries none. The root prefix
// is the one that strips to nothing, and nothing is what makes it
// unmountable here: see `servable`.
function mountpointOf(prefix: string): string {
return prefix.endsWith('/') ? prefix.slice(0, -1) : prefix
}
/**
* Whether this runtime can mount `prefix` at all.
*
* Everything but the workspace root can. `/` cannot: it is already
* MEMFS's own mount root, holding the interpreter's stdlib, and
* Emscripten answers EBUSY to a second mount there. Left to itself
* `mountpointOf` hands back the empty string, which mounts a detached
* pseudo-filesystem no path reaches, so the guest keeps reading and
* writing MEMFS and a write reports success while the resource never
* sees it.
*
* Args:
* prefix: a slash-terminated mount prefix.
*/
function servable(prefix: string): boolean {
return mountpointOf(prefix) !== ''
}
function runtimeEnv(): Record<string, string> {
const env: Record<string, string> = {}
const proc = (globalThis as { process?: { env?: Record<string, string | undefined> } }).process
@@ -143,8 +167,12 @@ export class PyodideRuntime extends PythonRuntime implements Evaluator {
private readonly denyPackages: ReadonlySet<string>
private listMounts: () => string[] = () => []
private readonly home: string | null
private bridge: MirageBridge | null = null
private readonly preloadedPrefixes = new Set<string>()
private vfs: RuntimeVFS | null = null
private readonly journal: MutationJournal = createJournal()
private readonly mounted = new Set<string>()
// Prefixes this runtime cannot mount, remembered so the refusal is
// reported once rather than on every run.
private readonly refused = new Set<string>()
// The guest executes on this event loop, so only the watchdog-backed
// interrupt buffer can stop a busy loop (see interrupt.ts); null
// where SharedArrayBuffer/workers are unavailable (runs unbounded).
@@ -271,8 +299,8 @@ export class PyodideRuntime extends PythonRuntime implements Evaluator {
}
this.pyodide = null
this.initPromise = null
this.bridge = null
this.preloadedPrefixes.clear()
this.vfs = null
this.mounted.clear()
this.interrupter?.close()
this.interrupter = null
this.interrupterTried = false
@@ -288,8 +316,8 @@ export class PyodideRuntime extends PythonRuntime implements Evaluator {
private async ensureLoaded(): Promise<PyodideInterface> {
if (this.pyodide !== null) {
if (this.bootstrapPromise !== null) await this.bootstrapPromise
await this.wireBridgeIfNeeded(this.pyodide)
await this.preloadNewPrefixes(this.pyodide)
this.wireBridgeIfNeeded()
await this.syncMounts(this.pyodide)
await this.wireInterruptIfNeeded(this.pyodide)
return this.pyodide
}
@@ -310,38 +338,76 @@ export class PyodideRuntime extends PythonRuntime implements Evaluator {
})()
await this.bootstrapPromise
}
await this.wireBridgeIfNeeded(this.pyodide)
await this.preloadNewPrefixes(this.pyodide)
this.wireBridgeIfNeeded()
await this.syncMounts(this.pyodide)
await this.wireInterruptIfNeeded(this.pyodide)
return this.pyodide
}
private async wireBridgeIfNeeded(pyodide: PyodideInterface): Promise<void> {
if (this.workspaceBridge === null || this.bridge !== null) return
const bridge = createMirageBridge(this.workspaceBridge, this.listMounts)
pyodide.registerJsModule('_mirage_bridge', bridge)
await pyodide.runPythonAsync(MIRAGE_FS_SHIM_PY)
this.bridge = bridge
private wireBridgeIfNeeded(): void {
if (this.workspaceBridge === null || this.vfs !== null) return
this.vfs = new RuntimeVFS(this.workspaceBridge, this.listMounts)
}
/**
* Hydrate MEMFS for mount prefixes not yet preloaded, before every run.
* This is the host-side half of read visibility: a mount added after
* boot gets its tree here, where awaiting the bridge is free, instead
* of relying on the shim's lazy backfill, whose run_sync needs JSPI.
* Content changes under an already-preloaded prefix still ride the
* lazy path (a full re-list per run would be one API sweep per mount).
* Rebuild the guest's mount table from the workspace's, before every
* run.
*
* Every prefix is re-seeded, not just a newly added one, because the
* filesystem's callbacks are synchronous and so this is the only place
* that can await the bridge at all. A stale snapshot is not merely a
* missed read: a file the snapshot lacks looks like a new file, so
* `open(path, 'a')` would start from an empty buffer and the flush
* would replace content the guest never saw. Re-seeding is what keeps
* that impossible without JSPI, which no shipping Node has and Safari
* does not implement.
*
* The cost is one LIST plus one READ per file per run. Narrowing that
* to what actually changed needs a per-entry change stamp the bridge
* does not carry yet; until it does, correctness is the side to err on.
*
* A prefix the workspace has dropped is unmounted, so a removed mount
* stops being visible instead of lingering for the interpreter's life.
*/
private async preloadNewPrefixes(pyodide: PyodideInterface): Promise<void> {
if (this.bridge === null) return
for (const prefix of this.bridge.prefixes()) {
if (this.preloadedPrefixes.has(prefix)) continue
// Record only after success: preloadInto's one throw path is the
// top-level LIST (per-entry failures warn and continue), so a
// transient failure surfaces loudly and retries on the next run
// instead of poisoning the prefix for the runtime's lifetime.
await preloadInto(pyodide.FS, this.bridge, prefix)
this.preloadedPrefixes.add(prefix)
private async syncMounts(pyodide: PyodideInterface): Promise<void> {
const vfs = this.vfs
if (vfs === null) return
const all = vfs.prefixes()
for (const prefix of all) {
if (servable(prefix) || this.refused.has(prefix)) continue
this.refused.add(prefix)
console.warn(
`mirage: the ${PYODIDE_RUNTIME} runtime cannot serve a mount at ` +
`'${prefix}', because that is the interpreter's own filesystem ` +
`root; python will not see it`,
)
}
// `prefixes()` is longest-first, which is the order routing wants
// and the reverse of the order the mount table wants: mounting
// /data over an existing /data/inner orphans the child. So unmount
// deepest-first and mount shallowest-first.
const wanted = new Set(all.filter(servable))
for (const prefix of [...this.mounted].sort((a, b) => b.length - a.length)) {
if (wanted.has(prefix)) continue
pyodide.FS.unmount(mountpointOf(prefix))
this.mounted.delete(prefix)
}
for (const prefix of [...wanted].sort((a, b) => a.length - b.length)) {
// Collect before touching the mount table: a failed LIST then
// leaves the previous snapshot serving rather than an empty mount,
// and the prefix retries on the next run.
const seed = new MirageFsSeed()
await preloadInto(seed, vfs, prefix)
const mountpoint = mountpointOf(prefix)
if (this.mounted.has(prefix)) pyodide.FS.unmount(mountpoint)
const fs = new MirageFs(pyodide.FS, pyodide.ERRNO_CODES, this.journal, mountpoint)
pyodide.FS.mkdirTree(mountpoint)
pyodide.FS.mount(fs.type, {}, mountpoint)
// After the mount, never inside it: Emscripten assigns the root's
// `mount` only once `type.mount()` has returned, and a node built
// before that inherits an undefined one.
fs.seed(seed)
this.mounted.add(prefix)
}
}
@@ -353,15 +419,15 @@ export class PyodideRuntime extends PythonRuntime implements Evaluator {
* way it ran. Returns one message per failure for the caller's stderr.
*/
private async drainMutations(): Promise<string[]> {
const bridge = this.bridge
if (bridge === null) return []
const vfs = this.vfs
if (vfs === null) return []
const failures: string[] = []
const pending = bridge.takeMutations()
const pending = this.journal.takeMutations()
for (let i = 0; i < pending.length; i++) {
const mutation = pending[i]
if (mutation === undefined) continue
try {
await applyMutation(bridge, mutation)
await applyMutation(vfs, mutation)
} catch (err) {
const detail = err instanceof Error ? err.message : String(err)
failures.push(`python3: failed to ${mutation.kind} ${mutation.path} on mount: ${detail}`)
typescript/packages/core/src/runtime/python/vfs/constants.ts
+25
View File
@@ -0,0 +1,25 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
const S_IFDIR = 0o040000
const S_IFREG = 0o100000
export const DIR_MODE = S_IFDIR | 0o777
export const FILE_MODE = S_IFREG | 0o666
export const BLKSIZE = 4096
// llseek's whence, which Emscripten passes through as the raw number.
export const SEEK_CUR = 1
export const SEEK_END = 2
typescript/packages/core/src/runtime/python/vfs/errors.test.ts
+54
View File
@@ -0,0 +1,54 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it } from 'vitest'
import { errnoError, fsError } from './errors.ts'
import type { ErrnoCodes, FSHost } from './types.ts'
class FakeErrnoError extends Error {
readonly errno: number
constructor(errno: number) {
super(`errno ${String(errno)}`)
this.errno = errno
}
}
// musl's numbering, which is what pyodide reports and what an errno
// literal copied from Linux would get wrong (EXDEV is 75 here, and 18
// is EDOM rather than the cross-device link everyone remembers).
const CODES: ErrnoCodes = { ENOENT: 44, EPERM: 63, EINVAL: 28, EIO: 29 }
const HOST = { ErrnoError: FakeErrnoError } as unknown as FSHost
describe('fsError', () => {
it('names the condition rather than numbering it', () => {
const err = fsError('EIO', 'mount went away')
expect((err as { code?: string }).code).toBe('EIO')
expect(err.message).toBe('mount went away')
})
})
describe('errnoError', () => {
it('answers in the numbering the interpreter reports', () => {
expect((errnoError(HOST, CODES, 'ENOENT') as FakeErrnoError).errno).toBe(44)
expect((errnoError(HOST, CODES, 'EPERM') as FakeErrnoError).errno).toBe(63)
expect((errnoError(HOST, CODES, 'EINVAL') as FakeErrnoError).errno).toBe(28)
expect((errnoError(HOST, CODES, 'EIO') as FakeErrnoError).errno).toBe(29)
})
it('builds the constructor the kernel recognizes, not a bare Error', () => {
expect(errnoError(HOST, CODES, 'EIO')).toBeInstanceOf(FakeErrnoError)
})
})
typescript/packages/core/src/runtime/python/vfs/errors.ts
+50
View File
@@ -0,0 +1,50 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { ErrnoCodes, FSHost } from './types.ts'
/**
* The conditions this filesystem can report, named rather than
* numbered. Same split as `fuse/errors.ts`: the layer that decides a
* call failed names the condition, and the adapter turns the name into
* whatever number its kernel interface uses.
*/
export type FsErrorCode = 'ENOENT' | 'EPERM' | 'EINVAL' | 'EIO'
/**
* Build an error naming a POSIX condition, for failures the host sees.
*
* Args:
* code: POSIX name of the condition.
* message: what failed, for a human reading the stack.
*/
export function fsError(code: FsErrorCode, message: string): Error {
return Object.assign(new Error(message), { code })
}
/**
* Build the error a guest syscall should fail with.
*
* Both halves come from the running interpreter: the constructor so the
* kernel recognizes it as an errno rather than a crash, and the number
* so it is musl's and not a literal that happens to be right on Linux.
*
* Args:
* host: the Emscripten FS namespace supplying `ErrnoError`.
* codes: that interpreter's errno table (`pyodide.ERRNO_CODES`).
* code: POSIX name of the condition to report.
*/
export function errnoError(host: FSHost, codes: ErrnoCodes, code: FsErrorCode): Error {
return new host.ErrnoError(codes[code])
}
typescript/packages/core/src/runtime/python/vfs/journal.ts
+130
View File
@@ -0,0 +1,130 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { concatBytes, type RuntimeVFS } from '../../vfs.ts'
/**
* One guest mutation, recorded in the order the script performed it.
* `write` and `append` carry their bytes because the drain runs after the
* script returns, when the filesystem holds only the final state: an
* atomic-write (write a temp file, rename it into place) would otherwise
* replay as a read of a path the rename already moved.
*/
export type MirageMutation =
| { readonly kind: 'write'; readonly path: string; readonly bytes: Uint8Array }
| { readonly kind: 'append'; readonly path: string; readonly bytes: Uint8Array }
| { readonly kind: 'mkdir'; readonly path: string }
| { readonly kind: 'unlink'; readonly path: string }
| { readonly kind: 'rmdir'; readonly path: string }
| { readonly kind: 'rename'; readonly path: string; readonly dst: string }
/**
* The write-ahead log a pyodide guest records into.
*
* Every mark is deliberately synchronous: the guest's close(), os.mkdir()
* and os.rename() run inside sync WASM frames where awaiting a mount op
* needs JSPI stack switching, which most engines do not enable. The guest
* records here and the runtime replays after the script returns, where
* awaiting is free. This is the one thing pyodide needs that the other
* runtimes do not: quickjs suspends at the call and monty runs the op on
* its own worker.
*/
export interface MutationJournal {
/**
* Args:
* path: mount-prefixed path the mutation names.
* bytes: the whole file for a write, the new tail for an append.
*/
markWrite(path: string, bytes: Uint8Array): void
markAppend(path: string, bytes: Uint8Array): void
markMkdir(path: string): void
markUnlink(path: string): void
markRmdir(path: string): void
markRename(src: string, dst: string): void
/** Drain the journal: every mutation in guest order, cleared. */
takeMutations(): MirageMutation[]
}
export function createJournal(): MutationJournal {
const journal: MirageMutation[] = []
return {
markWrite(path, bytes) {
// A guest buffer handed over by pyodide can be a view into WASM
// memory, which relocates when the heap grows; copy on arrival so
// the journal owns bytes that stay valid until the drain.
const owned = new Uint8Array(bytes)
const last = journal[journal.length - 1]
if (last?.kind === 'write' && last.path === path) {
journal[journal.length - 1] = { kind: 'write', path, bytes: owned }
return
}
journal.push({ kind: 'write', path, bytes: owned })
},
markAppend(path, bytes) {
const owned = new Uint8Array(bytes)
const last = journal[journal.length - 1]
if (last?.kind === 'append' && last.path === path) {
journal[journal.length - 1] = {
kind: 'append',
path,
bytes: concatBytes(last.bytes, owned),
}
return
}
journal.push({ kind: 'append', path, bytes: owned })
},
markMkdir(path) {
journal.push({ kind: 'mkdir', path })
},
markUnlink(path) {
journal.push({ kind: 'unlink', path })
},
markRmdir(path) {
journal.push({ kind: 'rmdir', path })
},
markRename(src, dst) {
journal.push({ kind: 'rename', path: src, dst })
},
takeMutations() {
return journal.splice(0, journal.length)
},
}
}
/**
* Replay one recorded guest mutation against the mounts.
*
* Args:
* vfs: the runtime's mount vocabulary to apply through.
* mutation: the journal entry to apply.
*/
export async function applyMutation(vfs: RuntimeVFS, mutation: MirageMutation): Promise<void> {
switch (mutation.kind) {
case 'write':
return vfs.write(mutation.path, mutation.bytes)
// The journal recorded only the tail, so the whole file is not
// available here; RuntimeVFS.append reads the base itself when the
// mount has no append op.
case 'append':
return vfs.append(mutation.path, mutation.bytes)
case 'mkdir':
return vfs.mkdir(mutation.path)
case 'unlink':
return vfs.unlink(mutation.path)
case 'rmdir':
return vfs.rmdir(mutation.path)
case 'rename':
return vfs.rename(mutation.path, mutation.dst)
}
}
typescript/packages/core/src/runtime/python/mirage_bridge.test.ts → typescript/packages/core/src/runtime/python/vfs/preload.test.ts
+14 -83
View File
@@ -13,78 +13,9 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it, vi } from 'vitest'
import { createMirageBridge, preloadInto } from './mirage_bridge.ts'
import type { BridgeDispatchFn } from '../types.ts'
describe('createMirageBridge', () => {
it('forwards fetch to dispatch READ and returns bytes', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(new Uint8Array([1, 2, 3])))
const b = createMirageBridge(dispatch)
const out = await b.fetch('/ram/x.txt')
expect(dispatch).toHaveBeenCalledWith('READ', '/ram/x.txt')
expect(Array.from(out)).toEqual([1, 2, 3])
})
it('forwards flush to dispatch WRITE with bytes and resolves void', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
const b = createMirageBridge(dispatch)
await b.flush('/ram/x.txt', new Uint8Array([9, 9]))
const call = dispatch.mock.calls[0]
if (call === undefined) throw new Error('unreachable')
const [op, path, bytes] = call
if (bytes === undefined) throw new Error('unreachable')
expect(op).toBe('WRITE')
expect(path).toBe('/ram/x.txt')
expect(Array.from(bytes)).toEqual([9, 9])
})
it('forwards list to dispatch LIST and returns entries', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([
{ path: '/ram/a.txt', size: 4, isDir: false },
{ path: '/ram/sub', size: 0, isDir: true },
]),
)
const b = createMirageBridge(dispatch)
const entries = await b.list('/ram/')
expect(entries).toHaveLength(2)
expect(entries[0]).toEqual({ path: '/ram/a.txt', size: 4, isDir: false })
})
it('rethrows dispatch errors', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.reject(new Error('boom')))
const b = createMirageBridge(dispatch)
await expect(b.fetch('/x')).rejects.toThrow(/boom/)
})
it('throws TypeError when READ returns non-Uint8Array', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve('not bytes' as unknown as Uint8Array),
)
await expect(createMirageBridge(dispatch).fetch('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when LIST returns non-array', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve({ not: 'array' } as unknown as never[]),
)
await expect(createMirageBridge(dispatch).list('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when LIST entry has bad shape', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([{ path: '/x' }] as unknown as never[]),
)
await expect(createMirageBridge(dispatch).list('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when WRITE dispatch returns non-undefined', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve('unexpected' as unknown))
await expect(createMirageBridge(dispatch).flush('/x', new Uint8Array([1]))).rejects.toThrow(
TypeError,
)
})
})
import { preloadInto } from './preload.ts'
import { RuntimeVFS } from '../../vfs.ts'
import type { BridgeDispatchFn } from '../../types.ts'
interface FakeFS {
mkdirTree(path: string): void
@@ -123,7 +54,7 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram/')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')
expect(fs._dirs.has('/ram')).toBe(true)
expect(new TextDecoder().decode(fs._files.get('/ram/a.txt'))).toBe('hello')
const bbin = fs._files.get('/ram/b.bin')
@@ -141,14 +72,14 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram/')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')
expect(fs._dirs.has('/ram/sub')).toBe(true)
const ctxt = fs._files.get('/ram/sub/c.txt')
if (ctxt === undefined) throw new Error('unreachable')
expect(Array.from(ctxt)).toEqual([7])
})
it('is idempotent: re-running overwrites with the bridge content', async () => {
it('is idempotent: re-running overwrites with the mount content', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op, path) => {
if (op === 'LIST' && path === '/ram/')
return Promise.resolve([{ path: '/ram/x', size: 1, isDir: false }])
@@ -156,10 +87,10 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
const bridge = createMirageBridge(dispatch)
await preloadInto(fs, bridge, '/ram/')
const vfs = new RuntimeVFS(dispatch)
await preloadInto(fs, vfs, '/ram/')
fs.writeFile('/ram/x', new Uint8Array([99]))
await preloadInto(fs, bridge, '/ram/')
await preloadInto(fs, vfs, '/ram/')
const x = fs._files.get('/ram/x')
if (x === undefined) throw new Error('unreachable')
expect(Array.from(x)).toEqual([42])
@@ -170,7 +101,7 @@ describe('preloadInto', () => {
Promise.resolve(op === 'LIST' ? [] : new Uint8Array()),
)
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram/')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')
expect(fs._dirs.has('/ram')).toBe(true)
expect(fs._files.size).toBe(0)
})
@@ -181,7 +112,7 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram')
expect(fs._dirs.has('/ram')).toBe(true)
expect(dispatch).toHaveBeenCalledWith('LIST', '/ram/')
})
@@ -200,7 +131,7 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram/')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')
const ok = fs._files.get('/ram/ok.txt')
if (ok === undefined) throw new Error('unreachable')
expect(Array.from(ok)).toEqual([1, 2])
@@ -223,7 +154,7 @@ describe('preloadInto', () => {
return Promise.reject(new Error(`unexpected ${op} ${path}`))
})
const fs = makeFakeFS()
await preloadInto(fs, createMirageBridge(dispatch), '/ram/')
await preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')
const ok = fs._files.get('/ram/ok.txt')
if (ok === undefined) throw new Error('unreachable')
expect(Array.from(ok)).toEqual([7])
@@ -235,7 +166,7 @@ describe('preloadInto', () => {
it('lets the top-level LIST error propagate', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.reject(new Error('top-level boom')))
const fs = makeFakeFS()
await expect(preloadInto(fs, createMirageBridge(dispatch), '/ram/')).rejects.toThrow(
await expect(preloadInto(fs, new RuntimeVFS(dispatch), '/ram/')).rejects.toThrow(
/top-level boom/,
)
})
typescript/packages/core/src/runtime/python/vfs/preload.ts
+69
View File
@@ -0,0 +1,69 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { RuntimeVFS, VFSEntry } from '../../vfs.ts'
export interface FSLike {
mkdirTree(path: string): void
writeFile(path: string, bytes: Uint8Array): void
/**
* Note a listed file whose content could not be fetched. A target that
* does not implement this simply omits the file, which is only safe
* when nothing will write to it.
*/
markUnreadable?(path: string): void
}
async function preloadEntry(fs: FSLike, vfs: RuntimeVFS, entry: VFSEntry): Promise<void> {
if (entry.isDir) {
fs.mkdirTree(entry.path)
const next = entry.path.endsWith('/') ? entry.path : entry.path + '/'
try {
await preloadInto(fs, vfs, next)
} catch (err) {
console.warn(
`mirage preload: skipping subtree ${next}: ${err instanceof Error ? err.message : String(err)}`,
)
}
return
}
try {
const bytes = await vfs.read(entry.path)
fs.writeFile(entry.path, bytes)
} catch (err) {
// The mount listed it, so it exists; we just cannot serve it. Say so
// rather than leaving a hole the guest would read as absence and an
// append would fill by replacing the file.
fs.markUnreadable?.(entry.path)
console.warn(
`mirage preload: cannot read ${entry.path}: ${err instanceof Error ? err.message : String(err)}`,
)
}
}
/**
* Copy one mount prefix into a synchronous target.
*
* Args:
* fs: the tree collector to fill.
* vfs: the runtime's mount vocabulary to read through.
* prefix: the mount prefix to walk.
*/
export async function preloadInto(fs: FSLike, vfs: RuntimeVFS, prefix: string): Promise<void> {
const prefixWithSlash = prefix.endsWith('/') ? prefix : prefix + '/'
const prefixWithoutSlash = prefixWithSlash.slice(0, -1)
fs.mkdirTree(prefixWithoutSlash)
const entries = await vfs.readdir(prefixWithSlash)
await Promise.all(entries.map((entry) => preloadEntry(fs, vfs, entry)))
}
typescript/packages/core/src/runtime/python/vfs/seed.ts
+51
View File
@@ -0,0 +1,51 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { FSLike } from './preload.ts'
/**
* A mount's tree, collected in memory so it can be served synchronously.
*
* `preloadInto` walks the bridge and writes through this interface; the
* result seeds a `NodeTree` once its filesystem has mounted. The split
* exists because fetching is async and every filesystem callback is not.
*/
export class MirageFsSeed implements FSLike {
readonly dirs: string[] = []
readonly files = new Map<string, Uint8Array>()
readonly unreadable = new Set<string>()
mkdirTree(path: string): void {
this.dirs.push(path)
}
writeFile(path: string, bytes: Uint8Array): void {
this.files.set(path, bytes)
}
/**
* Note a file the mount listed but would not hand over.
*
* Leaving it out of the tree entirely would be unsafe: the guest would
* see no file, and `open(path, 'a')` would build its buffer from empty
* and replace content this run never read. A node that refuses to open
* says the same thing without risking the file.
*
* Args:
* path: guest-absolute path that could not be fetched.
*/
markUnreadable(path: string): void {
this.unreadable.add(path)
}
}
typescript/packages/core/src/runtime/python/vfs/tree.ts
+183
View File
@@ -0,0 +1,183 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { DIR_MODE, FILE_MODE } from './constants.ts'
import { fsError } from './errors.ts'
import type { MirageFsSeed } from './seed.ts'
import type { FSNode, NodeHost, NodeOps, StreamOps } from './types.ts'
/**
* The node table for one mount prefix.
*
* Everything about *which* nodes exist and what they are called lives
* here: creating them, naming them, moving them, and translating between
* the guest's absolute paths and positions in this tree. It records no
* mutations and reports no errno, so the filesystem above it is left with
* only the semantics of each call.
*/
export class NodeTree {
private readonly host: NodeHost
private readonly nodeOps: NodeOps
private readonly streamOps: StreamOps
private readonly prefix: string
private root: FSNode | null = null
/**
* Args:
* host: the Emscripten FS namespace, for `createNode` and the mode
* predicates.
* prefix: the mount prefix this tree serves, trailing slash optional.
* nodeOps: op table every node created here must carry.
* streamOps: stream op table every node created here must carry.
*/
constructor(host: NodeHost, prefix: string, nodeOps: NodeOps, streamOps: StreamOps) {
this.host = host
this.prefix = prefix.endsWith('/') ? prefix.slice(0, -1) : prefix
this.nodeOps = nodeOps
this.streamOps = streamOps
}
/** Build the root. Emscripten calls this through `FSType.mount`. */
mount(): FSNode {
this.root = this.makeNode(null, '/', DIR_MODE)
return this.root
}
/**
* Populate the tree from a collected seed.
*
* Must run after `FS.mount`, never inside it: `FSNode` copies `mount`
* from its parent, and Emscripten assigns the root's only once
* `type.mount()` has returned. Seeding early leaves every nested node
* with an undefined mount, and two undefined mounts compare equal,
* which silently defeats the kernel's cross-mount rename check.
*
* Args:
* seed: tree collected from the bridge before the run.
*/
seed(seed: MirageFsSeed): void {
for (const dir of seed.dirs) {
const rel = this.relative(dir)
if (rel !== null) this.ensureDir(rel)
}
for (const path of seed.unreadable) {
const node = this.placeFile(path)
if (node !== null) node.unreadable = true
}
for (const [path, bytes] of seed.files) {
const node = this.placeFile(path)
if (node === null) continue
node.contents = bytes
node.usedBytes = bytes.length
}
}
/**
* Create a node and file it under its parent.
*
* Args:
* parent: directory to file it under, null only for the root.
* name: the child's name.
* mode: type and permission bits.
*/
makeNode(parent: FSNode | null, name: string, mode: number): FSNode {
const node = this.host.createNode(parent, name, mode, 0)
node.node_ops = this.nodeOps
node.stream_ops = this.streamOps
if (this.host.isDir(mode)) node.children = new Map()
else {
node.contents = new Uint8Array(0)
node.usedBytes = 0
}
node.atime = node.mtime = node.ctime = Date.now()
if (parent !== null) {
parent.children ??= new Map()
parent.children.set(name, node)
}
return node
}
childOf(parent: FSNode, name: string): FSNode | undefined {
return parent.children?.get(name)
}
childNames(node: FSNode): string[] {
return [...(node.children?.keys() ?? [])]
}
detach(parent: FSNode, name: string): void {
parent.children?.delete(name)
}
/**
* Move a node to a new parent and name.
*
* Args:
* node: the node being moved.
* newDir: its new parent directory.
* newName: its new name.
*/
move(node: FSNode, newDir: FSNode, newName: string): void {
node.parent.children?.delete(node.name)
newDir.children ??= new Map()
newDir.children.set(newName, node)
node.parent = newDir
node.name = newName
}
/** The guest-absolute path of a node, which is what the journal names. */
pathOf(node: FSNode): string {
const parts: string[] = []
let cur = node
while (cur.parent !== cur) {
parts.unshift(cur.name)
cur = cur.parent
}
return parts.length === 0 ? this.prefix : this.prefix + '/' + parts.join('/')
}
private rootNode(): FSNode {
// Only reachable from seed(), which the host calls: a guest syscall
// always arrives on a node, and no node exists before mount().
if (this.root === null) throw fsError('EIO', 'mirage fs: seeded before it was mounted')
return this.root
}
private relative(path: string): string | null {
if (path === this.prefix) return ''
if (!path.startsWith(this.prefix + '/')) return null
return path.slice(this.prefix.length + 1).replace(/\/+$/, '')
}
private ensureDir(rel: string): FSNode {
let cur = this.rootNode()
if (rel === '') return cur
for (const part of rel.split('/')) {
if (part === '') continue
const next = cur.children?.get(part)
cur = next ?? this.makeNode(cur, part, DIR_MODE)
}
return cur
}
private placeFile(path: string): FSNode | null {
const rel = this.relative(path)
if (rel === null) return null
const cut = rel.lastIndexOf('/')
const name = cut < 0 ? rel : rel.slice(cut + 1)
if (name === '') return null
const parent = cut <= 0 ? this.rootNode() : this.ensureDir(rel.slice(0, cut))
return this.makeNode(parent, name, FILE_MODE)
}
}
typescript/packages/core/src/runtime/python/vfs/types.ts
+131
View File
@@ -0,0 +1,131 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
/**
* Emscripten's errno numbering, which is musl's and not Linux's: EXDEV
* is 75 here and 18 is EDOM. Always read the numbers off the running
* interpreter (`pyodide.ERRNO_CODES`) rather than writing literals.
*/
export interface ErrnoCodes {
readonly ENOENT: number
readonly EPERM: number
readonly EINVAL: number
readonly EIO: number
}
export interface FSNode {
id: number
name: string
mode: number
rdev: number
parent: FSNode
mount: unknown
atime: number
mtime: number
ctime: number
node_ops: NodeOps
stream_ops: StreamOps
/**
* A Map, never a plain object: filenames come from the mount, so a
* child called `__proto__` or `constructor` would never become an own
* property and would resolve to something off Object.prototype.
*/
children?: Map<string, FSNode>
contents?: Uint8Array
usedBytes?: number
unreadable?: boolean
}
export interface FSStream {
node: FSNode
position: number
baseLen?: number
lowWrite?: number
}
export interface FSAttr {
dev: number
ino: number
mode: number
nlink: number
uid: number
gid: number
rdev: number
size: number
atime: Date
mtime: Date
ctime: Date
blksize: number
blocks: number
}
export interface SetAttr {
mode?: number
atime?: number
mtime?: number
ctime?: number
size?: number
}
export interface NodeOps {
getattr(node: FSNode): FSAttr
setattr(node: FSNode, attr: SetAttr): void
lookup(parent: FSNode, name: string): FSNode
mknod(parent: FSNode, name: string, mode: number, rdev: number): FSNode
rename(node: FSNode, newDir: FSNode, newName: string): void
unlink(parent: FSNode, name: string): void
rmdir(parent: FSNode, name: string): void
readdir(node: FSNode): string[]
symlink(parent: FSNode, name: string, target: string): FSNode
}
export interface StreamOps {
open(stream: FSStream): void
close(stream: FSStream): void
read(
stream: FSStream,
buffer: Uint8Array,
offset: number,
length: number,
position: number,
): number
write(
stream: FSStream,
buffer: Uint8Array,
offset: number,
length: number,
position: number,
): number
llseek(stream: FSStream, offset: number, whence: number): number
}
/** The node-building slice of the Emscripten FS namespace. */
export interface NodeHost {
createNode(parent: FSNode | null, name: string, mode: number, rdev: number): FSNode
isDir(mode: number): boolean
isFile(mode: number): boolean
}
/**
* `NodeHost` plus the error constructor. Only the adapter takes this
* one: the node table below it cannot reach `ErrnoError`, so it cannot
* accidentally answer in an interpreter's private numbering.
*/
export interface FSHost extends NodeHost {
ErrnoError: new (errno: number) => Error
}
export interface FSType {
mount(mount: unknown): FSNode
}
typescript/packages/core/src/runtime/python/vfs/vfs.test.ts
+326
View File
@@ -0,0 +1,326 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { beforeAll, beforeEach, describe, expect, it } from 'vitest'
import { loadPyodideRuntime, type PyodideInterface } from '../loader.ts'
import { RuntimeVFS } from '../../vfs.ts'
import { applyMutation, createJournal, type MutationJournal } from './journal.ts'
import { preloadInto } from './preload.ts'
import { MirageFs } from './vfs.ts'
import { MirageFsSeed } from './seed.ts'
import type { BridgeDispatchFn } from '../../types.ts'
const enc = new TextEncoder()
const dec = new TextDecoder()
interface Call {
op: string
path: string
bytes?: Uint8Array
}
describe('MirageFs', () => {
let py: PyodideInterface
let vfs: RuntimeVFS
let journal: MutationJournal
const calls: Call[] = []
const mounts: string[] = []
const store = new Map<string, Uint8Array>()
const unreadable = new Set<string>()
let counter = 0
// Mirrors what PyodideRuntime.syncMounts does, including collecting the
// seed through preloadInto rather than hand-building one, so the tests
// exercise the real producer of every node they then read. Seeding after
// the mount is load-bearing (an FSNode copies `mount` from its parent,
// and Emscripten assigns the root's only once type.mount() returned).
async function mountPrefix(prefix: string): Promise<void> {
mounts.push(prefix)
const seed = new MirageFsSeed()
await preloadInto(seed, vfs, prefix)
const mountpoint = prefix.slice(0, -1)
const fs = new MirageFs(py.FS, py.ERRNO_CODES, journal, mountpoint)
py.FS.mkdirTree(mountpoint)
py.FS.mount(fs.type, {}, mountpoint)
fs.seed(seed)
}
// The runtime's post-run drain, applied host-side where awaiting the
// bridge needs no JSPI.
async function drain(): Promise<void> {
for (const mutation of journal.takeMutations()) await applyMutation(vfs, mutation)
}
beforeAll(async () => {
py = await loadPyodideRuntime()
const dispatch: BridgeDispatchFn = (op, path, bytes, dst) => {
calls.push(bytes ? { op, path, bytes: new Uint8Array(bytes) } : { op, path })
if (op === 'READ') {
if (unreadable.has(path)) return Promise.reject(new Error('backend unavailable'))
const found = store.get(path)
if (found === undefined) return Promise.reject(new Error(`no such file: ${path}`))
return Promise.resolve(found)
}
if (op === 'WRITE' && bytes !== undefined) store.set(path, new Uint8Array(bytes))
if (op === 'APPEND' && bytes !== undefined) {
const base = store.get(path) ?? new Uint8Array()
const next = new Uint8Array(base.length + bytes.length)
next.set(base)
next.set(bytes, base.length)
store.set(path, next)
}
if (op === 'UNLINK') store.delete(path)
if (op === 'RENAME' && dst !== undefined) {
const moved = store.get(path)
if (moved === undefined) return Promise.reject(new Error(`no such file: ${path}`))
store.delete(path)
store.set(dst, moved)
}
if (op === 'LIST') {
return Promise.resolve(
[...store.keys()]
.filter((k) => k.startsWith(path))
.map((k) => ({ path: k, size: store.get(k)?.length ?? 0, isDir: false })),
)
}
return Promise.resolve(undefined)
}
vfs = new RuntimeVFS(dispatch, () => mounts)
journal = createJournal()
}, 60_000)
beforeEach(() => {
calls.length = 0
mounts.length = 0
store.clear()
unreadable.clear()
journal.takeMutations()
counter += 1
})
function prefix(): string {
return `/m${String(counter)}/`
}
it('serves a seeded file to the guest', async () => {
const p = prefix()
store.set(`${p}seed.txt`, enc.encode('ORIGINAL'))
await mountPrefix(p)
await py.runPythonAsync(`_out = open('${p}seed.txt').read()`)
expect(py.globals.get('_out')).toBe('ORIGINAL')
})
it('records only the tail for an append, and does not clobber the mount', async () => {
const p = prefix()
store.set(`${p}log.txt`, enc.encode('BASE'))
await mountPrefix(p)
await py.runPythonAsync(`
with open('${p}log.txt', 'a') as f:
f.write('+more')
`)
const mutations = journal.takeMutations()
expect(mutations).toHaveLength(1)
const only = mutations[0]
if (only?.kind !== 'append') throw new Error(`expected an append, got ${String(only?.kind)}`)
expect(dec.decode(only.bytes)).toBe('+more')
})
// The shim this replaced patched builtins.open but never os.open, so a
// low-level write applied to the guest's memory and was dropped on the
// floor: exit 0, mount unchanged.
it('records a low-level os.open write', async () => {
const p = prefix()
await mountPrefix(p)
await py.runPythonAsync(`
import os
fd = os.open('${p}low.txt', os.O_WRONLY | os.O_CREAT)
os.write(fd, b'LOWLEVEL')
os.close(fd)
`)
await drain()
expect(dec.decode(store.get(`${p}low.txt`) ?? new Uint8Array())).toBe('LOWLEVEL')
})
it('records a bare os.truncate, which opens no handle at all', async () => {
const p = prefix()
store.set(`${p}t.txt`, enc.encode('12345678'))
await mountPrefix(p)
await py.runPythonAsync(`
import os
os.truncate('${p}t.txt', 3)
`)
await drain()
expect(dec.decode(store.get(`${p}t.txt`) ?? new Uint8Array())).toBe('123')
})
it('carries a file that is created and never written', async () => {
const p = prefix()
await mountPrefix(p)
await py.runPythonAsync(`
from pathlib import Path
Path('${p}empty.txt').touch()
`)
await drain()
expect(store.has(`${p}empty.txt`)).toBe(true)
expect(store.get(`${p}empty.txt`)?.length).toBe(0)
})
it('records a shutil.rmtree in post order, through its fd-relative walk', async () => {
const p = prefix()
store.set(`${p}tree/a.txt`, enc.encode('a'))
store.set(`${p}tree/b/c.txt`, enc.encode('c'))
await mountPrefix(p)
await py.runPythonAsync(`
import shutil
shutil.rmtree('${p}tree')
`)
const kinds = journal.takeMutations().map((m) => `${m.kind} ${m.path.slice(p.length)}`)
expect(kinds).toEqual([
'unlink tree/a.txt',
'unlink tree/b/c.txt',
'rmdir tree/b',
'rmdir tree',
])
})
it('keeps the write-temp-then-rename idiom in order', async () => {
const p = prefix()
await mountPrefix(p)
await py.runPythonAsync(`
import os
with open('${p}tmp.part', 'w') as f:
f.write('ATOMIC')
os.rename('${p}tmp.part', '${p}final.txt')
`)
await drain()
expect(dec.decode(store.get(`${p}final.txt`) ?? new Uint8Array())).toBe('ATOMIC')
expect(store.has(`${p}tmp.part`)).toBe(false)
})
it('refuses a cross-mount rename with a real EXDEV the guest can match', async () => {
const a = prefix()
const b = `/other${String(counter)}/`
store.set(`${a}sub/x.txt`, enc.encode('X'))
store.set(`${b}sub/y.txt`, enc.encode('Y'))
await mountPrefix(a)
await mountPrefix(b)
// Nested, not top level: a node seeded before its mount is assigned
// inherits an undefined one, and two undefined mounts compare equal,
// which silently lets the kernel's cross-mount check through.
await py.runPythonAsync(`
import errno, os
try:
os.rename('${a}sub/x.txt', '${b}sub/x.txt')
_res = 'NO ERROR'
except OSError as e:
_res = 'EXDEV' if e.errno == errno.EXDEV else f'wrong errno {e.errno}'
`)
expect(py.globals.get('_res')).toBe('EXDEV')
expect(journal.takeMutations()).toEqual([])
})
it('resolves a relative path against the guest cwd', async () => {
const p = prefix()
await mountPrefix(p)
await py.runPythonAsync(`
import os
os.chdir('${p}')
with open('rel.txt', 'w') as f:
f.write('RELATIVE')
`)
await drain()
expect(dec.decode(store.get(`${p}rel.txt`) ?? new Uint8Array())).toBe('RELATIVE')
})
it('stops serving a prefix once it is unmounted', async () => {
const p = prefix()
store.set(`${p}gone.txt`, enc.encode('here'))
await mountPrefix(p)
await py.runPythonAsync(`_before = open('${p}gone.txt').read()`)
expect(py.globals.get('_before')).toBe('here')
py.FS.unmount(p.slice(0, -1))
await py.runPythonAsync(`
import os
_after = os.listdir('${p}')
`)
expect((py.globals.get('_after') as { length: number }).length).toBe(0)
})
it('does not record a write that traverses back out of the mount', async () => {
const p = prefix()
await mountPrefix(p)
// The escape is the kernel's to resolve, not a prefix test of ours:
// `<mount>/../escaped.txt` never reaches this filesystem at all, so
// it lands in the guest's own memory and touches no mount.
await py.runPythonAsync(`
with open('${p}../escaped.txt', 'w') as f:
f.write('ESCAPED')
`)
await drain()
expect([...store.keys()]).toEqual([])
expect(calls.filter((c) => c.op === 'WRITE')).toEqual([])
})
it('refuses to open a listed file the mount would not hand over', async () => {
const p = prefix()
// The mount lists it, so it exists; it just will not serve it. The
// unreadable mark has to come from preloadInto reacting to that, not
// from the test setting it by hand.
store.set(`${p}locked.txt`, enc.encode('SECRET'))
unreadable.add(`${p}locked.txt`)
await mountPrefix(p)
await py.runPythonAsync(`
import errno
try:
open('${p}locked.txt', 'a').write('tail')
_errno = 0
except OSError as e:
_errno = e.errno
`)
// EIO, not ENOENT: absence and unreadable must stay distinguishable,
// since only absence makes an empty base safe to build a write on.
expect(py.globals.get('_errno')).toBe(py.ERRNO_CODES.EIO)
expect(journal.takeMutations()).toEqual([])
})
// Filenames are the mount's to choose, so the child table is keyed by a
// Map. On a plain object these names reach Object.prototype instead of
// an own property, and the file either vanishes or resolves to junk.
it('serves files whose names collide with object prototype keys', async () => {
const p = prefix()
store.set(`${p}__proto__`, enc.encode('PROTO'))
store.set(`${p}constructor`, enc.encode('CTOR'))
await mountPrefix(p)
await py.runPythonAsync(`
import os
_names = ','.join(sorted(os.listdir('${p}')))
_proto = open('${p}__proto__').read()
_ctor = open('${p}constructor').read()
`)
expect(py.globals.get('_names')).toBe('__proto__,constructor')
expect(py.globals.get('_proto')).toBe('PROTO')
expect(py.globals.get('_ctor')).toBe('CTOR')
})
it('reports the seeded size through os.stat', async () => {
const p = prefix()
store.set(`${p}sized.txt`, enc.encode('123456789'))
await mountPrefix(p)
await py.runPythonAsync(`
import os
_size = os.stat('${p}sized.txt').st_size
`)
expect(py.globals.get('_size')).toBe(9)
})
})
typescript/packages/core/src/runtime/python/vfs/vfs.ts
+257
View File
@@ -0,0 +1,257 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { NO_WRITE, planFlush } from '../../vfs.ts'
import { BLKSIZE, SEEK_CUR, SEEK_END } from './constants.ts'
import { errnoError } from './errors.ts'
import type { MutationJournal } from './journal.ts'
import type { MirageFsSeed } from './seed.ts'
import { NodeTree } from './tree.ts'
import type {
ErrnoCodes,
FSAttr,
FSHost,
FSNode,
FSStream,
FSType,
NodeOps,
SetAttr,
StreamOps,
} from './types.ts'
/**
* An Emscripten filesystem backed by a mirage mount.
*
* This is the pyodide runtime's interception layer. Mounting it at a mount
* prefix puts mirage below the guest's syscall boundary, where every
* spelling of an operation (`open`, `os.open`, `pathlib`, `shutil`'s
* fd-relative walk) arrives as the same callback. Nothing is patched
* inside the interpreter.
*
* Callbacks are synchronous because Emscripten's are, so they never reach
* the mounts inline: reads are served from the tree `seed` filled before
* the run, and writes are recorded on the mutation journal for the runtime
* to replay after it. That is the same contract the guest had before, and
* the reason it needs no JSPI.
*
* The node table lives in `NodeTree` and the flush decision in
* `planFlush`; what is left here is one method per Emscripten callback,
* plus the errno translation only this layer performs.
*/
export class MirageFs {
readonly type: FSType
private readonly host: FSHost
private readonly errno: ErrnoCodes
private readonly journal: MutationJournal
private readonly tree: NodeTree
/**
* Args:
* host: the Emscripten FS namespace (`pyodide.FS`).
* errno: Emscripten's errno table (`pyodide.ERRNO_CODES`).
* journal: the write-ahead log guest mutations are recorded on.
* prefix: the mount prefix this filesystem serves.
*/
constructor(host: FSHost, errno: ErrnoCodes, journal: MutationJournal, prefix: string) {
this.host = host
this.errno = errno
this.journal = journal
const nodeOps: NodeOps = {
getattr: this.getattr.bind(this),
setattr: this.setattr.bind(this),
lookup: this.lookup.bind(this),
mknod: this.mknod.bind(this),
rename: this.rename.bind(this),
unlink: this.unlink.bind(this),
rmdir: this.rmdir.bind(this),
readdir: this.readdir.bind(this),
symlink: this.symlink.bind(this),
}
const streamOps: StreamOps = {
open: this.streamOpen.bind(this),
close: this.streamClose.bind(this),
read: this.streamRead.bind(this),
write: this.streamWrite.bind(this),
llseek: this.llseek.bind(this),
}
this.tree = new NodeTree(host, prefix, nodeOps, streamOps)
this.type = { mount: this.tree.mount.bind(this.tree) }
}
/**
* Populate the tree. Must run after `FS.mount`; see `NodeTree.seed`.
*
* Args:
* seed: tree collected from the mounts before the run.
*/
seed(seed: MirageFsSeed): void {
this.tree.seed(seed)
}
private getattr(node: FSNode): FSAttr {
const size = this.host.isDir(node.mode) ? BLKSIZE : (node.usedBytes ?? 0)
return {
dev: 1,
ino: node.id,
mode: node.mode,
nlink: 1,
uid: 0,
gid: 0,
rdev: 0,
size,
atime: new Date(node.atime),
mtime: new Date(node.mtime),
ctime: new Date(node.ctime),
blksize: BLKSIZE,
blocks: Math.ceil(size / BLKSIZE),
}
}
private setattr(node: FSNode, attr: SetAttr): void {
if (attr.mode !== undefined) node.mode = attr.mode
if (attr.atime !== undefined) node.atime = attr.atime
if (attr.mtime !== undefined) node.mtime = attr.mtime
if (attr.ctime !== undefined) node.ctime = attr.ctime
if (attr.size === undefined) return
const old = node.contents ?? new Uint8Array(0)
const next = new Uint8Array(attr.size)
next.set(old.subarray(0, Math.min(old.length, attr.size)))
node.contents = next
node.usedBytes = attr.size
// A resize rewrites history, so it can only ship whole. Recording here
// rather than at close is what makes a bare `os.truncate(path, n)`,
// which opens no handle at all, reach the mount.
this.journal.markWrite(this.tree.pathOf(node), next)
}
private lookup(parent: FSNode, name: string): FSNode {
const found = this.tree.childOf(parent, name)
// A miss is a genuine ENOENT: the seed is everything the host could
// reach before the run, and a callback cannot await the mounts to go
// looking for more.
if (found === undefined) throw errnoError(this.host, this.errno, 'ENOENT')
return found
}
private mknod(parent: FSNode, name: string, mode: number, rdev: number): FSNode {
const node = this.tree.makeNode(parent, name, mode)
node.rdev = rdev
const path = this.tree.pathOf(node)
if (this.host.isDir(mode)) this.journal.markMkdir(path)
// An empty write is what carries a file that is created and never
// written (`Path.touch()`, `open(p,'w').close()`) through to the
// mount. A later close for the same path coalesces over it.
else this.journal.markWrite(path, new Uint8Array(0))
return node
}
private rename(node: FSNode, newDir: FSNode, newName: string): void {
const from = this.tree.pathOf(node)
this.tree.move(node, newDir, newName)
this.journal.markRename(from, this.tree.pathOf(node))
}
private unlink(parent: FSNode, name: string): void {
const path = this.tree.pathOf(parent) + '/' + name
this.tree.detach(parent, name)
this.journal.markUnlink(path)
}
private rmdir(parent: FSNode, name: string): void {
const path = this.tree.pathOf(parent) + '/' + name
this.tree.detach(parent, name)
this.journal.markRmdir(path)
}
private readdir(node: FSNode): string[] {
return ['.', '..', ...this.tree.childNames(node)]
}
private symlink(): FSNode {
// Links are namespace state in mirage, not backend state, so no mount
// op can express one. Refusing is honest; MEMFS would accept it and
// the link would vanish at the end of the run.
throw errnoError(this.host, this.errno, 'EPERM')
}
private streamOpen(stream: FSStream): void {
// The mount listed this file but would not serve it, so its real
// length and content are unknown. Any handle at all is refused,
// because a write through one could only guess at what it replaces.
if (stream.node.unreadable === true) throw errnoError(this.host, this.errno, 'EIO')
stream.baseLen = stream.node.usedBytes ?? 0
stream.lowWrite = NO_WRITE
}
private streamClose(stream: FSStream): void {
const low = stream.lowWrite ?? NO_WRITE
if (low === NO_WRITE) return
const node = stream.node
const buf = (node.contents ?? new Uint8Array(0)).subarray(0, node.usedBytes ?? 0)
const [kind, bytes] = planFlush(stream.baseLen ?? 0, low, buf)
stream.lowWrite = NO_WRITE
const path = this.tree.pathOf(node)
if (kind === 'append') this.journal.markAppend(path, bytes)
else this.journal.markWrite(path, bytes)
}
private streamRead(
stream: FSStream,
buffer: Uint8Array,
offset: number,
length: number,
position: number,
): number {
const node = stream.node
const used = node.usedBytes ?? 0
if (position >= used) return 0
const size = Math.min(used - position, length)
buffer.set((node.contents ?? new Uint8Array(0)).subarray(position, position + size), offset)
return size
}
private streamWrite(
stream: FSStream,
buffer: Uint8Array,
offset: number,
length: number,
position: number,
): number {
if (length === 0) return 0
const node = stream.node
const need = position + length
let contents = node.contents ?? new Uint8Array(0)
if (contents.length < need) {
const grown = new Uint8Array(need)
grown.set(contents)
contents = grown
node.contents = grown
}
contents.set(buffer.subarray(offset, offset + length), position)
node.usedBytes = Math.max(node.usedBytes ?? 0, need)
node.mtime = node.ctime = Date.now()
stream.lowWrite = Math.min(stream.lowWrite ?? NO_WRITE, position)
return length
}
private llseek(stream: FSStream, offset: number, whence: number): number {
let position = offset
if (whence === SEEK_CUR) position += stream.position
else if (whence === SEEK_END && this.host.isFile(stream.node.mode)) {
position += stream.node.usedBytes ?? 0
}
if (position < 0) throw errnoError(this.host, this.errno, 'EINVAL')
return position
}
}
typescript/packages/core/src/runtime/sandbox/config.ts
+3 -1
View File
@@ -12,6 +12,8 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { RuntimeConfig } from '../config.ts'
/**
* How the sandbox machine is built: the fields every provider has.
*
@@ -21,7 +23,7 @@
* honor fails loud at construction. In yaml this is the runtime
* entry's `config` block, mirroring a mount's.
*/
export interface SandboxConfig {
export interface SandboxConfig extends RuntimeConfig {
/** Environment set in the sandbox. */
env?: Record<string, string>
}
typescript/packages/core/src/runtime/table.test.ts
+1 -1
View File
@@ -22,7 +22,7 @@ import {
runtimeBindingsFor,
VFSRuntime,
} from './table.ts'
import { MontyRuntime } from './python/monty.ts'
import { MontyRuntime } from './python/monty/index.ts'
import { PyodideRuntime } from './python/pyodide.ts'
import { QuickJsRuntime } from './js/quickjs.ts'
typescript/packages/core/src/runtime/table.ts
+1 -1
View File
@@ -15,7 +15,7 @@
import { Runtime } from './base.ts'
import { isLineExecutor, type LineExecutor } from './mixin.ts'
import { QuickJsRuntime } from './js/quickjs.ts'
import { MontyRuntime } from './python/monty.ts'
import { MontyRuntime } from './python/monty/index.ts'
import { PyodideRuntime } from './python/pyodide.ts'
import type { RuntimeOptions } from './types.ts'
typescript/packages/core/src/runtime/types.ts
+3 -2
View File
@@ -12,6 +12,7 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { RuntimeConfig } from './config.ts'
import type { PolicyScript } from './policy/types.ts'
/**
@@ -28,7 +29,7 @@ export type RuntimeLanguage = 'python' | 'js'
* imports no workspace module (Python's DispatchFn in runtime/types).
*/
export type BridgeDispatchFn = (
op: 'READ' | 'WRITE' | 'LIST' | 'STAT' | 'UNLINK' | 'MKDIR' | 'RMDIR' | 'RENAME',
op: 'READ' | 'WRITE' | 'APPEND' | 'LIST' | 'STAT' | 'UNLINK' | 'MKDIR' | 'RMDIR' | 'RENAME',
path: string,
bytes?: Uint8Array,
dst?: string,
@@ -132,7 +133,7 @@ export interface EvalResult {
}
/** Constructor options every runtime accepts (a yaml entry's keys). */
export interface RuntimeOptions<C extends object = Record<string, unknown>> {
export interface RuntimeOptions<C extends RuntimeConfig = Record<string, unknown>> {
/**
* Commands this runtime claims, overriding the class default; ["*"]
* claims every line for a line-executing runtime.
typescript/packages/core/src/runtime/vfs.test.ts
+246
View File
@@ -0,0 +1,246 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it, vi } from 'vitest'
import { enotsup } from '../utils/errors.ts'
import { CrossMountError } from './errors.ts'
import type { BridgeDispatchFn } from './types.ts'
import { planFlush, RuntimeVFS } from './vfs.ts'
const enc = new TextEncoder()
describe('planFlush', () => {
it('ships a tail when the handle only extended the file', () => {
expect(planFlush(3, 3, enc.encode('abcXYZ'))).toEqual(['append', enc.encode('XYZ')])
})
it('ships the whole file when history was rewritten', () => {
expect(planFlush(3, 0, enc.encode('ZZZdef'))).toEqual(['write', enc.encode('ZZZdef')])
})
it('ships the whole file for a new one', () => {
expect(planFlush(0, 0, enc.encode('fresh'))).toEqual(['write', enc.encode('fresh')])
})
it('ships the whole file when the buffer shrank', () => {
expect(planFlush(6, 6, enc.encode('abc'))).toEqual(['write', enc.encode('abc')])
})
})
describe('RuntimeVFS transport', () => {
it('forwards read to dispatch READ and returns bytes', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(new Uint8Array([1, 2, 3])))
const out = await new RuntimeVFS(dispatch).read('/ram/x.txt')
expect(dispatch).toHaveBeenCalledWith('READ', '/ram/x.txt')
expect(Array.from(out)).toEqual([1, 2, 3])
})
it('forwards write to dispatch WRITE with bytes and resolves void', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await new RuntimeVFS(dispatch).write('/ram/x.txt', new Uint8Array([9, 9]))
const call = dispatch.mock.calls[0]
if (call === undefined) throw new Error('unreachable')
const [op, path, bytes] = call
if (bytes === undefined) throw new Error('unreachable')
expect(op).toBe('WRITE')
expect(path).toBe('/ram/x.txt')
expect(Array.from(bytes)).toEqual([9, 9])
})
it('forwards readdir to dispatch LIST and returns entries', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([
{ path: '/ram/a.txt', size: 4, isDir: false },
{ path: '/ram/sub', size: 0, isDir: true },
]),
)
const entries = await new RuntimeVFS(dispatch).readdir('/ram/')
expect(entries).toHaveLength(2)
expect(entries[0]).toEqual({ path: '/ram/a.txt', size: 4, isDir: false })
})
it('rethrows dispatch errors', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.reject(new Error('boom')))
await expect(new RuntimeVFS(dispatch).read('/x')).rejects.toThrow(/boom/)
})
it('throws TypeError when READ returns non-Uint8Array', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve('not bytes' as unknown as Uint8Array),
)
await expect(new RuntimeVFS(dispatch).read('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when LIST returns non-array', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve({ not: 'array' } as unknown as never[]),
)
await expect(new RuntimeVFS(dispatch).readdir('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when LIST entry has bad shape', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() =>
Promise.resolve([{ path: '/x' }] as unknown as never[]),
)
await expect(new RuntimeVFS(dispatch).readdir('/x')).rejects.toThrow(TypeError)
})
it('throws TypeError when WRITE dispatch returns non-undefined', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve('unexpected' as unknown))
await expect(new RuntimeVFS(dispatch).write('/x', new Uint8Array([1]))).rejects.toThrow(
TypeError,
)
})
it('throws TypeError when STAT returns a bad shape', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve({ size: 1 }))
await expect(new RuntimeVFS(dispatch).stat('/x')).rejects.toThrow(TypeError)
})
})
describe('RuntimeVFS routing', () => {
const noop = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
it('normalizes prefixes to a trailing slash, longest first', () => {
const vfs = new RuntimeVFS(noop, () => ['/a', '/a/deep/', '/b'])
expect(vfs.prefixes()).toEqual(['/a/deep/', '/a/', '/b/'])
})
it('picks the longest matching mount, and the prefix itself counts', () => {
const vfs = new RuntimeVFS(noop, () => ['/a', '/a/deep'])
expect(vfs.mountOf('/a/deep/x')).toBe('/a/deep/')
expect(vfs.mountOf('/a/deep')).toBe('/a/deep/')
expect(vfs.mountOf('/a/x')).toBe('/a/')
expect(vfs.mountOf('/elsewhere')).toBeNull()
})
it('answers no mount when none are wired', () => {
expect(new RuntimeVFS(noop).mountOf('/a/x')).toBeNull()
})
it('refuses a rename whose ends are on different mounts', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
const vfs = new RuntimeVFS(dispatch, () => ['/a', '/b'])
await expect(vfs.rename('/a/x', '/b/x')).rejects.toThrow(CrossMountError)
expect(dispatch).not.toHaveBeenCalled()
})
it('dispatches a rename within one mount', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await new RuntimeVFS(dispatch, () => ['/a']).rename('/a/x', '/a/y')
expect(dispatch).toHaveBeenCalledWith('RENAME', '/a/x', undefined, '/a/y')
})
})
describe('RuntimeVFS append', () => {
it('ships only the tail when the mount takes an append', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await new RuntimeVFS(dispatch, () => ['/a']).append('/a/x', enc.encode('tail'))
expect(dispatch.mock.calls.map((c) => c[0])).toEqual(['APPEND'])
})
it('writes the whole file the caller supplied when the mount has no append', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(enotsup('s3', 'append', '/a/x'))
return Promise.resolve(undefined)
})
await new RuntimeVFS(dispatch, () => ['/a']).append(
'/a/x',
enc.encode('tail'),
enc.encode('headtail'),
)
const write = dispatch.mock.calls.find((c) => c[0] === 'WRITE')
if (write?.[2] === undefined) throw new Error('unreachable')
expect(new TextDecoder().decode(write[2])).toBe('headtail')
})
it('reads the base itself when no whole file was supplied', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(enotsup('s3', 'append', '/a/x'))
if (op === 'READ') return Promise.resolve(enc.encode('head'))
return Promise.resolve(undefined)
})
await new RuntimeVFS(dispatch, () => ['/a']).append('/a/x', enc.encode('tail'))
const write = dispatch.mock.calls.find((c) => c[0] === 'WRITE')
if (write?.[2] === undefined) throw new Error('unreachable')
expect(new TextDecoder().decode(write[2])).toBe('headtail')
})
it('starts from an empty base when the file is simply absent', async () => {
const missing = Object.assign(new Error('nope'), { code: 'ENOENT' })
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(enotsup('s3', 'append', '/a/x'))
if (op === 'READ') return Promise.reject(missing)
return Promise.resolve(undefined)
})
await new RuntimeVFS(dispatch, () => ['/a']).append('/a/x', enc.encode('tail'))
const write = dispatch.mock.calls.find((c) => c[0] === 'WRITE')
if (write?.[2] === undefined) throw new Error('unreachable')
expect(new TextDecoder().decode(write[2])).toBe('tail')
})
it('propagates a read failure that is not an absence', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(enotsup('s3', 'append', '/a/x'))
if (op === 'READ') return Promise.reject(new Error('transport down'))
return Promise.resolve(undefined)
})
await expect(
new RuntimeVFS(dispatch, () => ['/a']).append('/a/x', enc.encode('tail')),
).rejects.toThrow(/transport down/)
expect(dispatch.mock.calls.some((c) => c[0] === 'WRITE')).toBe(false)
})
it('remembers a mount that declined, so it costs one failed dispatch', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(enotsup('s3', 'append', '/a/x'))
return Promise.resolve(undefined)
})
const vfs = new RuntimeVFS(dispatch, () => ['/a'])
await vfs.append('/a/x', enc.encode('1'), enc.encode('1'))
await vfs.append('/a/y', enc.encode('2'), enc.encode('2'))
expect(dispatch.mock.calls.filter((c) => c[0] === 'APPEND')).toHaveLength(1)
})
it('lets a real append failure propagate instead of writing whole', async () => {
const dispatch = vi.fn<BridgeDispatchFn>((op) => {
if (op === 'APPEND') return Promise.reject(new Error('mount is read-only'))
return Promise.resolve(undefined)
})
await expect(
new RuntimeVFS(dispatch, () => ['/a']).append('/a/x', enc.encode('t'), enc.encode('t')),
).rejects.toThrow(/read-only/)
expect(dispatch.mock.calls.some((c) => c[0] === 'WRITE')).toBe(false)
})
})
describe('RuntimeVFS flush', () => {
it('sends a pure extension as an append', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await new RuntimeVFS(dispatch, () => ['/a']).flush('/a/x', 3, 3, enc.encode('abcXYZ'))
const call = dispatch.mock.calls[0]
if (call?.[2] === undefined) throw new Error('unreachable')
expect(call[0]).toBe('APPEND')
expect(new TextDecoder().decode(call[2])).toBe('XYZ')
})
it('sends a rewrite as a whole-file write', async () => {
const dispatch = vi.fn<BridgeDispatchFn>(() => Promise.resolve(undefined))
await new RuntimeVFS(dispatch, () => ['/a']).flush('/a/x', 3, 0, enc.encode('ZZZdef'))
const call = dispatch.mock.calls[0]
if (call?.[2] === undefined) throw new Error('unreachable')
expect(call[0]).toBe('WRITE')
expect(new TextDecoder().decode(call[2])).toBe('ZZZdef')
})
})
typescript/packages/core/src/runtime/vfs.ts
+267
View File
@@ -0,0 +1,267 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { isMissingOp, isMissingPath } from '../utils/errors.ts'
import { CrossMountError } from './errors.ts'
import type { BridgeDispatchFn, PrefixSource } from './types.ts'
export type FlushKind = 'append' | 'write'
// The lowest offset a handle has written at, before it writes anything.
// A sentinel rather than a null because every write takes the minimum of
// it and the new offset, and "nothing yet" has to lose that comparison.
export const NO_WRITE = Number.MAX_SAFE_INTEGER
/** One directory entry as the mounts report it. */
export interface VFSEntry {
path: string
size: number
isDir: boolean
}
/** One path's metadata, in the shape every guest encoder needs. */
export interface VFSStat {
size: number
isDir: boolean
mtimeMs: number
}
export function concatBytes(head: Uint8Array, tail: Uint8Array): Uint8Array {
const out = new Uint8Array(head.length + tail.length)
out.set(head, 0)
out.set(tail, head.length)
return out
}
/**
* Decide what a closing whole-file buffer owes the mount.
*
* Every encoder buffers a whole file and has to answer the same
* question at close: did this handle only add to the end, or did it
* rewrite what was already there? Only the first can travel as a
* delta, and answering "write" always is what makes an append loop
* quadratic.
*
* Args:
* baseLen: length the file had when the handle opened.
* lowWrite: lowest offset this handle wrote at, or the base length
* when it never wrote below the end.
* buf: the handle's whole buffer.
*
* Returns:
* ['append', tail] when the handle only extended the file, else
* ['write', whole buffer].
*/
export function planFlush(
baseLen: number,
lowWrite: number,
buf: Uint8Array,
): [FlushKind, Uint8Array] {
if (baseLen > 0 && lowWrite >= baseLen && buf.length >= baseLen) {
return ['append', buf.slice(baseLen)]
}
return ['write', buf.slice()]
}
/**
* The mount-facing op vocabulary a sandboxed runtime encodes into.
*
* One instruction set (read/write/append/stat/readdir/unlink/mkdir/
* rmdir/rename), one routing table, one place that knows an append may
* have to become a whole-file write. Encoders hold one of these; they
* never inherit it, because a monty encoder is the binding's own `os`
* callback and a quickjs encoder is a table of host functions.
*
* The surface is async, unlike Python's: a JS guest either suspends at
* the call (quickjs asyncify) or records the mutation and replays it
* after the run (pyodide), so nothing here has to block a worker
* thread the way the Python runtimes do.
*
* Args:
* dispatch: the workspace op dispatch this runtime was attached to.
* mountPrefixes: live view of the workspace mount prefixes; the
* default answers no mounts, so routing questions answer null.
*/
export class RuntimeVFS {
private readonly dispatch: BridgeDispatchFn
private readonly mountPrefixes: PrefixSource
private readonly noAppend = new Set<string>()
constructor(dispatch: BridgeDispatchFn, mountPrefixes: PrefixSource = () => []) {
this.dispatch = dispatch
this.mountPrefixes = mountPrefixes
}
/**
* The workspace mount prefixes, longest first, trailing-slash
* normalized. Longest first is what makes mountOf's first match the
* right one when one mount nests inside another.
*/
prefixes(): string[] {
const out = this.mountPrefixes().map((p) => (p.endsWith('/') ? p : p + '/'))
return out.sort((a, b) => b.length - a.length)
}
/** The mount prefix serving `path`, longest match first, or null. */
mountOf(path: string): string | null {
for (const prefix of this.prefixes()) {
if (path === prefix.slice(0, -1) || path.startsWith(prefix)) return prefix
}
return null
}
async read(path: string): Promise<Uint8Array> {
const out = await this.dispatch('READ', path)
if (!(out instanceof Uint8Array)) {
throw new TypeError(`runtime vfs: READ ${path} expected Uint8Array, got ${typeof out}`)
}
return out
}
async write(path: string, bytes: Uint8Array): Promise<void> {
const out = await this.dispatch('WRITE', path, bytes)
if (out !== undefined) {
throw new TypeError(`runtime vfs: WRITE ${path} expected void, got ${typeof out}`)
}
}
async stat(path: string): Promise<VFSStat> {
const out = await this.dispatch('STAT', path)
const st = out as VFSStat | null
if (
st === null ||
typeof st !== 'object' ||
typeof st.size !== 'number' ||
typeof st.isDir !== 'boolean' ||
typeof st.mtimeMs !== 'number'
) {
throw new TypeError(`runtime vfs: STAT ${path} bad shape`)
}
return st
}
async readdir(path: string): Promise<VFSEntry[]> {
const out = await this.dispatch('LIST', path)
if (!Array.isArray(out)) {
throw new TypeError(`runtime vfs: LIST ${path} expected array`)
}
for (const e of out) {
if (
e === null ||
typeof e !== 'object' ||
typeof (e as VFSEntry).path !== 'string' ||
typeof (e as VFSEntry).size !== 'number' ||
typeof (e as VFSEntry).isDir !== 'boolean'
) {
throw new TypeError(`runtime vfs: LIST ${path} bad entry shape`)
}
}
return out as VFSEntry[]
}
async unlink(path: string): Promise<void> {
await this.dispatch('UNLINK', path)
}
async mkdir(path: string): Promise<void> {
await this.dispatch('MKDIR', path)
}
async rmdir(path: string): Promise<void> {
await this.dispatch('RMDIR', path)
}
/**
* Rename within one mount.
*
* Args:
* src: guest-absolute source path.
* dst: guest-absolute destination path.
*
* Throws:
* CrossMountError: the two ends resolve to different mounts.
*/
async rename(src: string, dst: string): Promise<void> {
if (this.mountOf(src) !== this.mountOf(dst)) throw new CrossMountError(src, dst)
await this.dispatch('RENAME', src, undefined, dst)
}
/**
* Extend `path` by `tail`, falling back to a whole-file write.
*
* `append` is optional per backend (S3 registers `write` and
* `rename` without it), so a mount that declines is remembered: the
* fallback then costs one failed dispatch per mount rather than one
* per call.
*
* The fallback needs the whole file. An encoder that already holds
* it (monty's in-memory tree, a closing file handle) passes it; one
* that does not (pyodide's mutation replay, which recorded only the
* tail) omits it and the fallback reads the base first. Only a
* confirmed absence starts from an empty base, since an append may
* create the file — every other read failure propagates, because
* writing the tail alone over a file that exists but is momentarily
* unreadable would replace content this run never saw.
*
* Args:
* path: guest-absolute virtual path.
* tail: only the newly appended bytes.
* whole: the file's full content, when the caller has it.
*/
async append(path: string, tail: Uint8Array, whole?: Uint8Array): Promise<void> {
if (await this.appendDelta(path, tail)) return
if (whole !== undefined) {
await this.write(path, whole)
return
}
let base: Uint8Array = new Uint8Array()
try {
base = await this.read(path)
} catch (err) {
if (!isMissingPath(err)) throw err
}
await this.write(path, concatBytes(base, tail))
}
private async appendDelta(path: string, tail: Uint8Array): Promise<boolean> {
const mount = this.mountOf(path) ?? path
if (this.noAppend.has(mount)) return false
try {
await this.dispatch('APPEND', path, tail)
} catch (err) {
if (!isMissingOp(err, 'append')) throw err
this.noAppend.add(mount)
return false
}
return true
}
/**
* Send a closing handle's buffer as a delta when it can be one.
*
* Args:
* path: guest-absolute virtual path.
* baseLen: length the file had when the handle opened.
* lowWrite: lowest offset this handle wrote at.
* buf: the handle's whole buffer.
*/
async flush(path: string, baseLen: number, lowWrite: number, buf: Uint8Array): Promise<void> {
const [kind, payload] = planFlush(baseLen, lowWrite, buf)
if (kind === 'write') {
await this.write(path, payload)
return
}
await this.append(path, payload, buf)
}
}
typescript/packages/core/src/workspace/executor/python/handle.ts
+1 -1
View File
@@ -21,7 +21,7 @@ import { IOResult, materialize } from '../../../io/types.ts'
import { PathSpec } from '../../../types.ts'
import type { DispatchFn } from '../cross_mount.ts'
import { ExecutionNode } from '../../types.ts'
import { MontyUnavailableError } from '../../../runtime/python/monty.ts'
import { MontyUnavailableError } from '../../../runtime/python/monty/index.ts'
import { PyodideUnavailableError } from '../../../runtime/python/types.ts'
type Result = [ByteSource | null, IOResult, ExecutionNode]
typescript/packages/core/src/workspace/runtime_policy.test.ts
+1 -1
View File
@@ -26,7 +26,7 @@ import {
import type { EvalResult } from '../runtime/types.ts'
import { POLICY_EVAL_TIMEOUT, evaluatorOf, runtimeForLanguage } from '../runtime/policy/index.ts'
import type { RunArgs, RunResult } from '../runtime/types.ts'
import { MontyRuntime } from '../runtime/python/monty.ts'
import { MontyRuntime } from '../runtime/python/monty/index.ts'
import { QuickJsRuntime } from '../runtime/js/quickjs.ts'
import {
DenyResult,
typescript/packages/core/src/workspace/workspace.ts
+10 -3
View File
@@ -54,9 +54,9 @@ import type { ProvisionResult } from '../provision/types.ts'
import { WorkspaceFS } from './fs.ts'
import type { MountEntry } from './mount/mount.ts'
import { MountRegistry } from './mount/registry.ts'
import type { MirageEntry } from '../runtime/python/mirage_bridge.ts'
import type { VFSEntry } from '../runtime/vfs.ts'
import type { BridgeDispatchFn } from '../runtime/types.ts'
import { MontyUnavailableError } from '../runtime/python/monty.ts'
import { MontyUnavailableError } from '../runtime/python/monty/index.ts'
import { scriptStringError, type Runtime, type RuntimeEntry } from '../runtime/base.ts'
import { isEvaluator } from '../runtime/mixin.ts'
import type { EvalResult } from '../runtime/types.ts'
@@ -335,6 +335,13 @@ export class Workspace {
await this.dispatch('write', path, [buf])
return undefined
}
case 'APPEND': {
if (bytes === undefined) throw new Error('APPEND op requires bytes')
const buf =
bytes instanceof Uint8Array ? bytes : new Uint8Array(bytes as ArrayLike<number>)
await this.dispatch('append', path, [buf])
return undefined
}
case 'STAT': {
const st = (await this.dispatch('stat', path)) as FileStat
const isDir = st.type === FileType.DIRECTORY
@@ -362,7 +369,7 @@ export class Workspace {
case 'LIST': {
const entries = ((await this.dispatch('readdir', path)) as string[] | null) ?? []
return await Promise.all(
entries.map(async (entry): Promise<MirageEntry> => {
entries.map(async (entry): Promise<VFSEntry> => {
// Backends that mark directories with a trailing slash
// skip the stat; unmarked entries (e.g. RAM) need one to
// learn dir-ness.
typescript/packages/node/src/runtime/python/local.ts
+4 -3
View File
@@ -14,15 +14,16 @@
import { type ChildProcess, spawn } from 'node:child_process'
import {
HOME_CONFIG_KEYS,
PythonRuntime,
registerRuntime,
type HomeConfig,
type RunArgs,
type RunResult,
type RuntimeOptions,
} from '@struktoai/mirage-core'
const LOCAL_HOME_ENV = 'MIRAGE_LOCAL_HOME'
const LOCAL_CONFIG_KEYS: readonly string[] = ['home']
/**
* Run Python code on a host interpreter as a subprocess.
@@ -41,8 +42,8 @@ export class LocalRuntime extends PythonRuntime {
private readonly children = new Set<ChildProcess>()
constructor(options: RuntimeOptions = {}) {
super(options, LOCAL_CONFIG_KEYS)
const home = (this.config as { home?: string }).home
super(options, HOME_CONFIG_KEYS)
const home = (this.config as HomeConfig).home
const chosen = home !== undefined && home !== '' ? home : process.env[LOCAL_HOME_ENV]
this.python = chosen !== undefined && chosen !== '' ? chosen : 'python3'
}