main
226 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Zecheng Zhang
|
984f79e2fe |
fix(resource): a generic mount asks to be handed back at load
buildMountArgs consults no registry and substitutes a RAMResource for
any mount it was not given, so the bare `{type}` state GenericResource
inherited restored a custom backend as an empty directory instead of
refusing. Every config-backed TypeScript resource already sets
needs_override for exactly this reason; a GenericResource always holds a
live accessor, so it always sets it too. copy() is unaffected either
way, since it passes the live resource through, and the flag is what
makes it do so.
python writes the key as well, where its own loader ignores it (that one
rebuilds the class from the registry) but a TypeScript reader of a
python snapshot does not.
|
||
|
Zecheng Zhang
|
98fcdcb2e2 |
feat(resource): assemble a TypeScript backend from one CommandIO table
python's GenericResource wires the whole generic command set, the glob resolver and the VFS/FUSE ops from one table, and TypeScript had every ingredient but no class that assembled them, so a custom backend there was still written out by hand. Adds GenericResource<A>, generic over the accessor so the table is checked against the core functions it holds rather than against Accessor. It keeps TypeScript's own wiring style: commands() and ops() return arrays instead of python's register() loop mutating state in the constructor. And there is no sdk.ts, because core's barrel is 99 lines and gated in both directions while the ./* exports map already makes every module importable, so the five names the new example and doc reach for are the only additions to it. Both classes gained sizes_always_known and supports_snapshot. Without the first, no user-written backend could be mounted on FSKit at all, since resolve_backend refuses a resource that cannot size its files. The two one-file examples answer one shared truth file, so the two SDKs cannot drift without a red build. docs/typescript/resource/new.mdx is the twin of the python page, whose stale du_total/du_all is corrected to du. The layout baseline drops to 255: resource/generic was one of the counted divergences. |
||
|
Zecheng Zhang
|
b448775596 |
fix(workspace): the door owns every verb that names a link
Three codex comments on the guest link surface turned out to be one missing rule. The dispatcher already answered symlink, readlink and the unlink of a link; rename, a no-follow stat and the existence check were not there, so each one reached a backend that has never heard of the name. LINK_ENTRY_OPS plus one predicate says which verbs the node table answers, and ln, mv, git and FUSE read the rule instead of keeping their own copy of it. A rename of a link answered ENOENT and moved nothing, on pyodide, wasi, FUSE and os.rename. A no-follow stat answered ENOENT everywhere, so the wasi lstat rebuilt the row from the target string and reported epoch zero: a guest's own utime persisted and stayed invisible. FUSE was a third copy of that mistake, reporting the mount's construction time and the mounting user for every link. And symlink never looked at the name it was given, so ln -s over a live file exited 0 and left the bytes under a link nothing could read, and a guest could bury a mount root. Two more of the same family, found while probing rather than reported: a rename onto an existing link left the link shadowing the file that had just landed, and prepare_mv wrote the node table directly, which made a link rename the one write in the shell no admission policy could see. Both go through the door now, and ln -f is GNU's own algorithm (remove the destination, then link), so it replaces a regular file the way coreutils does. git checkout relied on symlink overwriting a link to retarget one, so restoreEntry removes the old name unconditionally rather than skipping that when the path is already linked. |
||
|
Zecheng Zhang
|
a5cb9730b7 |
feat(runtime): serve symlink, readlink and setattr to guests
A guest could not create a symlink, read one, or stamp a mode or a time. The three verbs reach the name plane rather than a backend, so they work on a mount whose store holds none of them, and wasi, quickjs, pyodide and monty all route through the same ops. Each surface gains only what its engine already has: qjs-wasi has no symlink, readlink or lstat, so the quickjs bootstrap gains os.utimes and nothing else, and pyodide has no os.link. pyodide's journal grew a phantom setattr per created file and split two writes markWrite would have coalesced, because Emscripten finalizes a create through node_ops.setattr with the same shape a guest chmod has. Comparing fields is not enough on its own, since the create genuinely lowers 0o777 to 0o666, so MirageFs.fresh marks the node mknod just made and changedAttrs drops a write that moves nothing. readlink answered EINVAL for every miss; POSIX splits them, and a caller's except FileNotFoundError depends on it. Absence is probed on the failure path only, on both channels a backend can answer on (a prefix store keeps no directory object, so stat misses what readdir lists, and a listing has to be non-empty), through the same admission gate the op would pass, since a policy denying stat must not be reachable through a readlink. A refused channel is not absence: it answers EINVAL, which asserts nothing the policy is withholding. os_patch._link_target swallowed every OSError as "not a link", which would have hidden that ENOENT from os.lstat. |
||
|
Zecheng Zhang
|
2b5c2b42f3 |
feat(permissions): one role per session, mounts narrow instead of gating
A role is now the whole permission document a session runs under. There is no workspace `permissions:` block, no `permissions:` on a mount, and no `extends`, so reading `profiles.<name>` is reading everything that role may do. `mounts:` states infrastructure only. Two rules decide a line, and they are the whole law. A rule naming no path is read by verb, deny before ask, wherever in the document it is written. A rule carrying paths, and every hide, is read by anchor depth: the deeper entry wins, ties break by verb. `decide()` holds both. Naming a mount narrows it; it is not an allowlist. A mount a role does not name keeps its own mode. Keeping a session away from one is a hide, which answers ENOENT rather than a refusal that names what the role cannot see (EACCES on a create, since a silent success would leave a file the session cannot find). Three ways a command still learned a hidden path was there, all fixed above the backend rather than per command: - MountView splits into `descendants` (every mount below a path, for pruning) and `visible_descendants` (session-filtered, for naming), so `tree`, `tar` and `zip` stop drawing or naming a hidden mount while still refusing to cross it. - `du` decided existence from the bound accessor, which knows nothing of hides. It asks the dispatcher first and its content probe now counts only entries the session may see. - TypeScript `readlink` treated only PolicyDenied and EINVAL as silent, so ENOENT printed a raw path. It matches python's `except OSError`. dsh's read-only twin copied only the source session's mount modes, which used to be the allowlist; it now carries the hides too. |
||
|
Zecheng Zhang
|
913f532fe2 | docs: give the ssh runtime page an in-house logo | ||
|
Zecheng Zhang
|
c897f1d3e5 | fix(runtime): default the ssh username and use the agent when no key is named | ||
|
Zecheng Zhang
|
5d6f49bd06 | feat(runtime): add the ssh sandbox provider | ||
|
Zecheng Zhang
|
5d06281df3 | docs: organize sandbox runtime pages | ||
|
Zecheng Zhang
|
1170899c0c | fix(dsh): narrow the read-only twin from the bound session, keep unbound calls one-shot | ||
|
Zecheng Zhang
|
49a6219abb |
fix(dsh): make the providers behave inside a real dsh composition
A working directory dsh resolved on its own machine named nothing in the mirage world, so pwd reported a path the agent could not reach and every relative path failed; the same host path reached ctx.fs as the base for relative file_path arguments. Both now fall back to the configured directory. Because a per-call workdir or env forks a subshell, and dsh sends both on every call, a configured sessionId never persisted anything either: the managed DSH_* snapshot is seeded into the bound session instead of riding along per call. The sandbox policy dsh resolves per call is now enforced rather than reported. Under read-only the shell runs with every mount granted read (the null sink stays writable) and ctx.fs refuses a mutation with FS_SANDBOX_DENIED. MirageFileSystem declares the same workspace-write confinement the executor does, so the two seams over one world stop giving dsh's tool layer opposite answers. A foreground run keeps the output it produced before a timeout or abort instead of returning an empty string, and reports a spill path when truncation dropped bytes. Background output is bounded in bytes rather than re-encoded per chunk (1 MB of output: 562 ms to 1 ms), console retention follows the configured budget, listDir honours its abort signal per entry, a failing declarative mount no longer reaches the process as an unhandled rejection, and runtimes beside an adopted workspace are refused instead of dropped. |
||
|
bytecii
|
15a8dbe043 |
refactor(sandlock): split into a package, excuse the layout divergence, document both runtimes
Codex P1: a python-only module under runtime/python left check_layout_parity.py --strict red (297 against a 296 baseline). Making sandlock a package the way monty already is moves the divergence from a module to a directory, which the exceptions file excuses as a subtree -- the honest shape here, since there is no TypeScript counterpart to mirror and growth inside it is expected rather than drift. Constants and config move out of the runtime module to match monty's and smolvm's layout: constants.py carries the env var, the CLI hint, and the system read set; config.py carries SandlockConfig. Docs: smolvm joins the sandbox provider table and gets its own section on both language pages, covering the refused-state probe and the fact that one machine is one guest, so concurrent lines share a filesystem and a process table. sandlock joins the python runtime table with a section on what "process" reach does and does not buy, and on the environment it deliberately does not inherit. Vendor marks added under the existing docs/images/<name>-logo.svg convention. |
||
|
Zecheng Zhang
|
c504f1dc82 | docs: add a Haystack integration page | ||
|
Zecheng Zhang
|
283da2c8fb |
Merge pull request #820 from strukto-ai/fix/build-resource-sync
fix: make build_resource sync again, hydrate github lazily |
||
|
Zecheng Zhang
|
10fd374287 |
docs(watch): document the push mappers
Push mode showed only the hand-written FileEvent path. It now leads with the three mappers that exist, says why you import one rather than ask the mount for it, and keeps the hand-written version for a backend without one. Slack gets the two traps written down: the day is bucketed in UTC while the client shows local time, and a thread reply belongs to the parent's day because chat.jsonl renders conversations.history, which returns parents only. The matrix drops Slack from planned and gives redis its own row. |
||
|
Zecheng Zhang
|
8575b9b2f4 |
fix(github): hydrate the tree lazily, make build_resource sync again
0.0.5 made build_resource a coroutine so GitHubResource could fetch the repo tree before returning. That fixed a real defect (the two fetches ran in __init__ over a blocking urlopen and froze the daemon's loop) but paid for it with the one function every caller who describes a mount as data comes through: the YAML loader, the daemon's create/load routes, clone, and every embedder reaching it through mirage.sdk. The haystack integration, our first outside consumer, broke on it. Hydrating lazily removes a round trip instead of adding one. Nothing seeded the index at build time, so the first readdir ran ensure_live_index and refetched the whole tree, discarding the one the constructor had just paid for; measured two git/trees calls where one does. GitHubResource now names the repository and contacts nothing, and the tree and default branch arrive through ensure_tree and ensure_default_branch on first use, each behind a lock so concurrent first reads cost one request. readdir and read already hydrated through ensure_live_index; only find, du and grep's narrow read accessor.tree directly and needed wiring. normalize_resources now refuses a non-resource and names the mount. The old failure was 'coroutine' object has no attribute 'set_index', raised two frames away in install_mounts, naming a method the caller never called and no mount. TypeScript stays async: its factory type is uniformly (config) => Promise<Resource> and two of its backends need it. Recorded as a deliberate divergence rather than mirrored. |
||
|
Zecheng Zhang
|
bab44f2502 |
Merge remote-tracking branch 'origin/main' into feat/watch-delta-hooks
# Conflicts: # typescript/packages/browser/src/resource/github/github.ts # typescript/packages/core/src/index.ts # typescript/packages/node/src/core/nextcloud/watch.ts # typescript/packages/node/src/resource/github/github.ts |
||
|
bytecii
|
e677715e96 |
fix(ts): keep the runtime configs exported, declare the Node floor
Two codex findings on #805. P2: node and browser lost Mem0Config, OneDriveConfig and SharePointConfig from their public API. Stage 1 deleted that block from both barrels as a duplicate of `export * from '@struktoai/mirage-core'`, which was true when core exported 1457 names; stage 4 shrank core to 80 and left the premise false. The three resources still arrive through the star -- they are among the 80 -- so only the config types went missing. Both barrels name them module-by-module now. Measured: node and browser each shed ~1356 further names through that star, which is the intended core shrink, and every one is still reachable by module path; these three were the only names their own barrels had ever listed. P1: `with { type: 'json' }` in version.ts parses from Node 20.10, and the tsup build used to inline package.json so any Node could load it. Nothing declared a floor -- no package had an `engines` field at all -- so this would have surfaced as a SyntaxError at import instead of an install warning. All eight published packages now declare `node: ">=20.10.0"` and the three docs lines saying "Node.js 20" say 20.10. Hard-coding the version instead would break a deliberate parity: mirage/version.py reads importlib.metadata for the same reason this reads the shipped package.json. |
||
|
Zecheng Zhang
|
effa4ee1c5 |
Merge remote-tracking branch 'origin/main' into feat/watch-delta-hooks
# Conflicts: # typescript/packages/core/src/resource/onedrive/onedrive.ts # typescript/packages/core/src/resource/sharepoint/sharepoint.ts # typescript/packages/node/src/resource/box/box.ts # typescript/packages/node/src/resource/dropbox/dropbox.ts # typescript/packages/node/src/resource/gdrive/gdrive.ts # typescript/packages/node/src/resource/github/github.ts # typescript/packages/node/src/resource/gridfs/gridfs.ts # typescript/packages/node/src/resource/s3/s3.ts |
||
|
Zecheng Zhang
|
4eef9413f9 |
feat(watch): ship delta_hook for ten more backends
Adds pull change detection to s3, github, dropbox, disk, gridfs, the hf family, ssh, gdrive, graph and box, so eleven resource families now answer "what changed under this root since my checkpoint". Two shared helpers keep a new backend from writing its own loop: synth_dirs builds the directory rows a prefix store implies, and ReaddirWalk descends through a backend's own readdir with a fresh private index per pull. Adding github to the integ battery surfaced two real bugs. Its index was keyed repo-relative while the other eleven backends and CacheManager key mount-absolute, so cache invalidation never reached a github mount at all, silently, since evicting an absent key succeeds. And invalidate_dir drops a row rather than expiring it, so github's readers took a dropped index for real absence. Github now keys mount-absolute like everyone else, its repo-relative path logic reads the git tree on the accessor the way TypeScript's always has, and ensure_live_index refetches when the index holds no listing. Box fingerprinted on modified_at while the same listing already carried sha1, so two writes in one second were indistinguishable. It now prefers sha1. The integ watch battery covers eleven targets at 238 cases, and the disk example is gated in both languages against one shared truth file. |
||
|
Zecheng Zhang
|
1636c74084 |
docs: drop the removed filetype renderers, and clean up dead pages and assets (#807)
Seventeen pages still documented parquet, ORC, HDF5 and feather rendering, which neither language ships any more: the ops are absent from the source, the extras are gone from pyproject, and the two example files the pages linked to do not exist. Ten carried a whole Data Format Support section, the rest advertised cat_parquet and friends inline. Filenames and find examples stay. A .parquet object still sits on these backends and still reads as raw bytes, so only the rendering claims are untrue. Also: home/python.mdx was a lone redirect stub with no nav entry and no inbound links, four images had no reference anywhere in the repo, and four pages carried no description. |
||
|
Zecheng Zhang
|
c32855fd91 |
docs: give each service its own brand icon, and drop the github_ci pages (#806)
The sidebar rendered six databases, three cloud stores and every CLI as the same generic glyph, so the icon carried no information. Thirteen brand marks now stand in: notion, linear, mongodb (also gridfs, which is mongodb's), postgres, qdrant, chroma, redis, supabase, databricks, box, cloudflare for r2, nextcloud and langfuse, applied across the home, python and typescript trees. Twelve come from simple-icons at the pinned 16.28.0 with the brand hex filled in, which is the convention the existing logos already follow; regenerating minio-logo.svg that way reproduces the committed file byte for byte. Notion's brand hex is #000000, so it takes the neutral gray the OpenAI mark already uses rather than vanishing in dark mode. Chroma and langfuse are not in simple-icons and come from each project's own mark instead. Oracle, LanceDB and QingStor keep their generic icons: oracle was removed from simple-icons upstream and the other two publish no usable svg. github_ci is gone from the docs, with its nav entries, its resource matrix row and the resource index bullet. |
||
|
bytecii
|
8564e62221 |
Merge pull request #802 from bytecii/fix/du-sep-dbx-copy-docs
fix(cache): rename evicts dst's own listing (13-backend sweep) + disk Windows separators + databricks copy invalidation + derived-ops docs |
||
|
bytecii
|
cd78c4304c |
fix(cache): rename evicts dst's own listing (13-backend sweep) + disk Windows separators + databricks copy invalidation + derived-ops docs (py+ts)
PR-0e from the restructure audit (bugs 8+9+G5). The new mv-onto-empty-dir integ pin caught the rename write-flavor staleness in disk, ssh, onedrive, and sharepoint; every backend rename now takes the unlink flavor on dst. Dropbox folder-conflict mv and s3/gridfs directory rename are pre-existing divergences excluded from the pin and filed as follow-ups. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Zecheng Zhang
|
53f09f618c |
docs(cli): quote every CLI page description (#804)
gws was left as the only quoted description, which is the shape that invites the next unquoted colon. Quoting the rest makes the frontmatter uniform and immune to it. No parsed value changes. This also re-targets the CLI pages for the docs build. The icons from #801 never went live: that deploy aborted on the gws parse error and dropped all sixteen files with it, and #803 only changed the two gws pages, so the incremental build republished those alone. |
||
|
Zecheng Zhang
|
a3f3cadbb1 |
docs: quote the gws description so the frontmatter parses, and gate it (#803)
The description held an unquoted ": ", which YAML reads as a nested mapping key, so Mintlify failed the whole deploy on both gws pages. Nothing checked frontmatter before CI, so a one-character mistake only surfaced as a failed deploy. check_docs_frontmatter.py parses every page the way the docs build does and runs beside the other repo-root checks. |
||
|
Zecheng Zhang
|
a1a769848e |
docs: fix the README hero snippet, resync the mirrors, and give each CLI page its own icon (#801)
* docs(readme): fix the hero snippet and resync every mirror The hero snippet called ws.command(...), which exists in neither language: registration is the standalone command() plus mount.register. It is replaced with a Python example that mounts ram, redis and slack side by side, captures python with monty, and installs a CLI, all of it run against the published 0.0.5 packages first. Two more corrections. The filetype sentence promised parsed PDF pages, which the filetype removal took away, so it now says a format renders however you register it. DeepSeek Harness joins the coding agents row. The eleven mirrors are regenerated from the root rather than patched, which also closes drift they had accumulated: a stale backend list, the old CLI + daemon integrations line, a missing Grok Build entry and a Codex link pointing at the wrong docs path. * docs(cli): give each CLI page its own icon Every CLI page shared icon: terminal, so the sidebar was nine identical rows. Each now takes the icon its service already uses elsewhere in the docs: slack, discord, github for gh, google for gws, envelope for himalaya, book for ntn and chart-gantt for linear (matching the notion and linear setup pages, since Font Awesome carries no brand mark for either), and git-alt for git. gws and himalaya also get their names spelled GWS and Himalaya; the rest stay lowercase because that is the head word you type. * examples(filetype): register through the public mount accessor The example reached into ws._registry.mount_for, but ws.mount is public and returns the same MountEntry. Output is unchanged, so the CI truth file still matches. |
||
|
Zecheng Zhang
|
5cfb4598b8 |
feat(ts): install a CLI from a file reference in yaml (#798)
* feat(ts): install a CLI from a file reference in yaml
`clis: {tally: {cli: ./tally.mjs:TALLY}}` now resolves in TypeScript, so
a deployment can ship its own program tree without a host program
calling registerCli. Mirrors the `":" in name` branch of Python's
`cli_spec_for`, one layer up: `cliSpecFor` lives in core, which has no
filesystem and is synchronous, so the ref resolves in the config layer.
The referenced file is imported by Node, so .mjs, .js and .ts all work;
a .ts spec rides Node's strip-only type stripping and is refused with a
hint when it uses a construct that needs codegen.
* fix(ts): node's specifier rule for cli refs, and a real spec check
Both from review.
A bare specifier carries slashes in Node (`@scope/pkg`, `pkg/subpath`),
so copying Python's `"/" in source` test read every scoped or subpath
package as a file, rebased it under the config directory and reported it
missing. A path is now what Node calls a path: relative or absolute,
plus a bare filename carrying a module suffix.
A class name is not proof a tree is usable. Dispatch reads subcommands,
aliases and options at every level, so a value that only answers to the
name crashed on the first line an agent typed instead of failing the
create. The shape is checked recursively instead.
|
||
|
Zecheng Zhang
|
50a68fc0d4 |
feat(dsh): stream background output, sandbox facts, spill, and the runtime reach marker (#796)
* refactor(runtime): replace confined with a three-value reach marker A runtime now declares reach: vfs | process | remote, stating whether the workspace dispatch gate is its only door or the code can act around it (host process doors, another machine). The default is process, the no-promise claim, so a custom runtime must narrow its reach explicitly. The dsh sandbox claim reads the aggregate: every runtime at vfs means workspace-write, anything wider means no claim. Mirrored in Python (RuntimeReach in runtime/types.py) with reach declared on monty, quickjs, wasi, vfs, local, and RemoteSandbox. * fix(pyodide): seal the js module so the guest has no host door Pyodide's default exposes the host globalThis as the `js` module, which under Node handed guest code js.process (host env, confirmed reading HOME) and js.fetch (network) — doors around the workspace bridge that made the runtime's reach='vfs' claim false and contradicted its own 'no network' docstring. Pass a null-prototype jsglobals to loadPyodide: `import js` still resolves but the host globals are unreachable through it, while pyodide's internals (which capture their globals at load time) and the FS bridge are unaffected. Pinned by jsglobals.test.ts. * feat(dsh): stamp ShellSandboxInfo on run results and process handles When the world is fully workspace-bound (vfsOnly), run() and start() now fill dsh's optional sandbox field: mode workspace-write, enforcement 'full' (the VFS gate is unbypassable, so unlike an OS sandbox on an old kernel there is no promised effect it fails to govern), denied false (mirage has no out-of-band denial channel; a refused write fails in-band as an ordinary command error), and runnerFailed false (the executor is the runner). The process handle stamps it on settle. Omitted when any runtime reaches beyond the workspace. * feat(dsh): stream background command output through a JobConsole Adds a public ExecuteOptions.sink: pass a JobConsole and the line's output streams into it as each statement finishes, instead of being returned whole (the result then carries only the exit code). This reuses the executor's existing internal sink mechanism, so a compound line flushes per statement and stdout/stderr keep their channels. MirageShellProcess is rewritten over that seam: start() runs the command with a console as its sink and a follow loop drains it into the read buffer, so readOutput() delivers output incrementally and stdout/stderr interleave in order (stderr opened by a marker) rather than stderr being concatenated at the end. The unread backlog is bounded to stdoutMaxBytes (tail kept, lossy flagged) so a reader that never drains cannot grow it without limit. * feat(dsh): spill the full stream to a workspace file on overrun Adds an opt-in spillDir config. When a background command's streamed output overruns its delta budget, the full stdout and stderr are written to files under that workspace directory and readOutput() points at them (stdoutSpillPath/stderrSpillPath), so a reader can recover what the delta dropped by reading the spill through the same VFS. Memory stays bounded: each channel buffers only until the first overrun, then flushes to its file and appends from there. A write failure (no writable mount at the path) disables the sink and leaves the paths undefined, the honest 'no safe path' answer. Default unset, so nothing spills unless a deployment asks for it. * docs(dsh): custom backends, background streaming, and the reach model Corrects the sandbox-claim wording to the reach model (workspace-write when every runtime reaches only the vfs, dropped when one reaches the host), and adds a Custom backends section (registerResourceFactory, host-side before the workspace builds) and a Background commands section (per-statement streaming, bounded backlog, spillDir). * test(dsh,core): satisfy lint on the streaming tests Narrow spill paths with an explicit guard instead of a non-null assertion (forbidden in the dsh package), and drop the now-unnecessary ExecuteResult casts the sink overload already implies. * style: prettier formatting on the streaming changes * fix(core): drain a buffered line into the sink A sink only saw output the command-tree walk emitted, so a whole-line runtime, the syntax gate, a policy denial and a failed line all answered with bytes in hand that a streaming caller never read. executeLine now moves any buffered result into the console on every path, in one place rather than five, and the result stays empty as it already did when the line streamed. * fix(dsh): bound the console store, make the spill dir idempotent Capping the delta did not bound memory: reading a chunk advances a cursor but frees nothing, so an uncapped store held every chunk of a noisy background command for the life of the process. The store now carries a retention budget, and the drain reports a trimmed chunk as lossy and stops the spill, since a file missing the middle of a stream is worse than no file. The spill directory is created through ensureDirPath, which walks the ancestors and accepts a refusal for a directory that now exists, so two commands overrunning at once do not cost the loser its spill. |
||
|
Zecheng Zhang
|
583b2fb545 |
feat(dsh): ship the package as a dsh bundle with declarative mounts (#787)
* feat(dsh): ship the package as a dsh bundle with declarative mounts * fix(dsh): claim workspace-write only for a confined runtime world, recheck aborts after the ready wait * fix(runtime): declare confined on the monty test double |
||
|
Zecheng Zhang
|
8369f04533 |
feat(dsh): session binding for the shell executor (#784)
* feat(dsh): session binding for the shell executor * docs(dsh): session binding |
||
|
Zecheng Zhang
|
f44c6e5c25 | docs(dsh): drop the redundant workspace mode option (#777) | ||
|
Zecheng Zhang
|
829a7e5c29 |
feat(dsh): DeepSeek Harness fs and shell providers over a mirage workspace (#774)
* feat(dsh): DeepSeek Harness fs and shell providers over a mirage workspace * docs(dsh): monty python run and multi-source mounts in the dsh example and page * docs(dsh): tidy the example into sectioned helpers * docs(dsh): trim the page to install, compose, python, one world * docs(dsh): use mirage as the example report owner * docs(dsh): slack-hosted python script example * docs(dsh): rename the slack-hosted script to example.py * fix(dsh): match Slack upload names and newest-first date dirs in the example * fix(dsh): address codex review on lock retention, pre-aborted signals, cyclic links, and byte limits * docs(dsh): generalize the page, compose sample on redis + ram * docs(dsh): keep slack in the compose sample * docs(dsh): state the python3 capture forms * docs(dsh): show the runtimes capture spec with an inline python sample * docs(dsh): monty is the workspace python, capturing python and python3 * docs(dsh): one continuous example, no second workspace * docs(dsh): show monty explicitly capturing python and python3 * docs(dsh): one example end to end — ram + redis + slack, monty capturing python * docs(dsh): drop the one execution world section * docs(dsh): state the in-memory property with measured capacity * docs(dsh): drop the capacity numbers * docs(dsh): drop the in-memory paragraph * docs(dsh): add a harness comparison table * docs(dsh): add concurrency column on a 2 vcpu / 8 gb server * docs(dsh): correct the live python engine footprint to 27 MB * docs(dsh): no em dashes, dsh row name, serving scenario before the table * docs(dsh): tmp mount, redis-bound report, numeric concurrency table with startup * docs(dsh): note the sandbox provisioning cost the cli sessions would add * docs(dsh): sandbox startup cost in the cli startup cells |
||
|
Zecheng Zhang
|
41e43274db |
feat(gh): a GitHub CLI, and the write half of the integ GitHub fake (#768)
* feat(integ/github): repo, contents, issues and compare routes
The fake served the read path the `github` resource mounts -- repo
metadata, git/trees, git/blobs -- plus the Actions dataset `github_ci`
needs. A task that *acts* on a repository had nothing to call.
Adds the routes such a task uses: GET /user; POST /user/repos; DELETE
/repos/{owner}/{repo}; POST .../forks; GET .../branches and
.../branches/{branch}; GET .../commits; GET and PUT .../contents/{path};
GET and POST .../issues; GET .../compare/{basehead}.
Writes are visible to the next read, which is the point -- an issue
filed shows up in the listing, a committed file reads back with its new
bytes and a new blob sha, and the commit is recorded so `compare`
can answer which files moved. `FakeRepo` grows `issues` and `commits`
for that; a repository with no writes yet still reports one synthetic
root commit, so "the latest commit" is answerable before the agent has
done anything.
Three behaviours worth pinning rather than leaving to chance:
- A fork deep-copies. A task that forks an archive repo per run and then
commits to the fork must not write through to the source.
- PUT /contents enforces GitHub's sha rule in both directions: replacing
an existing file without the current blob sha is 409, and supplying
one for a file that does not exist is 422. A task that reads before
writing is doing so for this reason.
- A compare against a base the repository has never seen is 404, not an
empty file list. Answering "nothing changed" to a question about an
unrelated commit is the shape of wrongness that reads as success.
`_commit_list` is newest-first, so `compare` collects the commits
*before* it reaches the base; walking past the base instead would report
the commits the base already contains.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(integ/github): git data API, rename, raw content and GHES paths
The write half a task that builds a repository needs, plus the two shapes
a real client insists on:
- PATCH /repos/{owner}/{repo} renames, carrying the content with it, and
POST /forks takes a name so a fork can be named in one step.
- The git data API -- get commit, create tree, create commit, move ref --
so a multi-file push lands as one commit. A staged tree is invisible
until a ref points at it, which is what git does and what keeps an
abandoned tree from showing up in a read.
- GET /git/ref/{ref}, and trees by commit sha, because a client resolves
a ref to a commit before reading anything.
- /raw/{owner}/{repo}/{ref}/{path}, with a text content type for text, so
a reader does not base64 the whole repository.
- Every route is served under /api/v3 as well as at the root: a client
pointed at a host that is not github.com talks Enterprise.
- GET /search/repositories, and an unrouted fallback that names the
method and path on stderr. Both exist for the same reason: a 404 is an
answer to a real client, so an unimplemented endpoint reads as a
negative result rather than as an error.
- Seeding takes a per-repo default branch, since a template whose branch
is master is graded at that ref by name.
* feat(gh): a GitHub CLI, and the write routes the fake needs behind it
The `github` mount is the read half -- a repository is a tree, so listing
and reading it is `ls` and `cat`. There was no write half at all:
GITHUB_IO carries readdir/read/stat where GDRIVE_IO carries the full set,
so nothing could commit a file, and forking and renaming are account
operations a filesystem has no shape for in any case.
`gh` is that half, spelled as cli.github.com spells it:
gh repo view [OWNER/REPO]
gh repo fork OWNER/REPO [--fork-name NAME]
gh repo rename NEW-NAME -R OWNER/REPO
gh api ENDPOINT [-X METHOD] [-f key=value] [-F key=value]
`rename` takes the new name as the operand and the repository as -R, which
is the reverse of what the shape of the line suggests and is upstream's.
`-f` sends a string and `-F` reads true/false/null/integers as their JSON
types, which is gh's own --raw-field / --field split; a call with no fields
is a GET, one with fields a POST unless -X says otherwise. Committing a
file is `gh api ... -X PUT -f content="$(base64 -w0 f)"`, which is what
real gh and real GitHub require of each other -- there is no file verb to
mimic.
The transport was GET-only, so it grows a `request`, and an empty body (204,
an empty 202) decodes to null rather than throwing on a call that worked.
Fake, in support: DELETE /contents with the same sha rule as the replace,
and GET /commits/{ref}, which is not /git/commits/{sha} -- it takes a branch
name and reports the file list. Both were found by the unrouted logger
during a run: an agent issued 35 deletes, every one 404'd, and it reported
the files as removed, because to a real client a 404 is an answer.
* feat(integ/github): repository metadata, star-sorted search and readme
A task that picks *between* repositories reads their descriptions and star
counts and never opens one, so seeding gains a second mode: --metadata takes
a JSON file keyed by owner/name and creates repositories that are metadata
only. The tree and contents endpoints answer 404 for them, which is what
GitHub says about a repository with no files.
search/repositories honours sort=stars and matches the description as well
as the name, since a repository is found by what it says it does at least as
often as by what it is called. Terms OR rather than AND, which is looser
than GitHub and errs towards showing a caller the row it wants.
GET /repos/{owner}/{repo}/readme, found by the unrouted logger during a run.
* feat(integ/github): branches
The fake kept exactly one branch, which rules out every task whose premise
is a difference between two of them. FakeRepo now holds a file map and a
commit list per branch:
- `files` and `commits` stay bound to the default branch, so every route
that does not name a ref reads what it always read.
- `branch_for` resolves a ref -- a branch name, HEAD, or a commit sha
belonging to one branch's history -- and `tree_of` returns that
branch's files, so a bad ref is a 404 rather than a silent read of the
default branch.
- contents, tree, raw, readme, branches, branch, commits, commit and
git/ref are ref-aware; PUT and DELETE /contents write to the branch
their body names; PATCH /git/refs moves the branch it names.
- Each branch has its own synthetic root commit, so two branches of a
fixture do not share a head.
- A fork copies every branch, not just the default one.
- Seeding takes `into=<branch>`, so a two-branch fixture is two
directories and two --repo flags naming one repository.
The default-branch binding is what keeps this small: 18 call sites read
`repo.files` and none of them had to change.
* feat(integ/github): GET /git/refs/{prefix}
The plural is a different endpoint from the singular: git/ref/<full-ref>
returns one object, git/refs/<prefix> a list of everything beneath it. A
caller picks whichever it expects, so serving only the singular made the
plural read as 'no such ref'. Found by the unrouted logger during a run.
* feat(integ/github): account repo listing, bare /contents, mirror-stable shas
Three things a task that reasons across several repositories needs.
GET /users/{login}/repos and GET /user/repos. A task that asks "is there a
repository for this on my GitHub" answers it here, and a 404 read as "the
account has none" -- a wrong answer rather than an error, which is the
failure mode the unrouted logger exists for.
GET /repos/{owner}/{repo}/contents, with no trailing slash. GitHub serves
both spellings of the root and a caller picks either; only one was routed.
A branch's root commit sha is now derived from its content rather than from
the repository's name. That is what `git clone --mirror` followed by a push
actually gives you -- the same shas -- and a grader that reads an initial
sha from an upstream repository and the latest from a local copy, then asks
what changed between them, depends on it. Two branches still differ exactly
when their trees differ.
* feat(integ/github): POST /git/refs, so a branch can be created
The last write a task can ask for that the fake had no answer to. A new
branch starts as a copy of whatever the base sha resolves to -- which is
what a branch is, another name for one commit and everything reachable from
it -- with its own empty history, so the two do not share future commits.
422 on a ref that is not under refs/heads/, on a name that already exists,
and on a base sha that resolves to nothing.
* feat(integ/github): GET /, the API root
github.com answers the root with a map of endpoint URL templates. Serving
only the endpoints beneath it meant a client probing the root got a 404,
which reads as the host not being there at all rather than as an unfamiliar
API. Found by the unrouted logger during a run.
* feat(integ/github): seed a branch's commit history, not just its tree
Seeding a directory filled a tree and nothing else, so every seeded
branch answered GET /commits with one synthetic root. That is enough for
a task that reads files and wrong for one that asks when something
arrived. `task-tracker` wants the tasks added by the *most recent*
commit on each of fifteen developer branches, and each branch has five
or six -- so a history of one does not make the task hard, it makes it
unanswerable while looking answerable.
--commits owner/name=<file> reads a manifest keyed by branch, each an
array oldest first, and is applied after every tree: a manifest names
branches, and a branch no directory was seeded into would otherwise be
created here as an empty tree with a history, which reads as a branch
whose files were all deleted.
Shas are derived from full_name, branch, date and message rather than
carried, for the same reason the root's is derived from its tree: a
fixture has to answer the same way twice.
Commits now carry commit.author/committer (name, email, date) and
author.login, which is what a client sorting or filtering by date reads,
and the synthetic root carries them too against a fixed epoch -- it
stands for everything before the fixture rather than for a moment, so
"now" would make it sort ahead of the commits it precedes.
GET /commits/{ref} now renders `files` as {filename, status} objects the
way GitHub does. The list endpoint keeps paths, which is also GitHub's
split: `files` is absent from a commit in a list and detailed when one
commit is asked for. _commit_files is shared with the compare route,
which was building the same objects inline.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(github): back the transport with octokit
Wraps @octokit/core behind the unchanged GitHubTransport, so the github
mount and the gh CLI both gain retry and throttling. Handles three
octokit behaviours: {} in a url is a route template that silently eats
the segment, the "METHOD /path" form splits on whitespace, and 204
decodes to '' rather than null. The write throttle is github.com's own
secondary rate limit, so it is off for any other host.
* fix(gh): name the mount its writes invalidate, and parse the host segment
serves was empty, so the executor's post-write cache drop did nothing and
a committed file still read back as its pre-write bytes. parseRepo took
the first two segments of [HOST/]OWNER/REPO, so github.com/acme/tools
resolved to github.com/acme -- a different repository, reported as
success.
* fix(cache): invalidate the index instead of clearing it
A cleared index reads exactly like one that was never filled, so github --
whose index is the whole listing rather than a cache in front of one --
could not tell a cache drop from an empty repository, and ls reported the
mount root missing. IndexCacheStore grows invalidate(), which expires
entries in place, and github refetches once on an EXPIRED lookup. Also
fixes a latent bug with no CLI involved: after the 24h index TTL lapsed a
github mount answered ls with exit 0 and no output. Redis cannot express
stale (an expired key is gone), so it still clears; the comment says why.
* feat(gh): a python gh CLI, mirroring the typescript one
Same four verbs and the same grammar. The python client was GET-only, so
it grows github_request alongside github_get. Layout divergence 305 -> 303.
* test(integ): a gh CLI battery on both hosts
15 cases on a new cli-gh target, covering the write-then-read path through
the mount, the refusals, and the reversed rename grammar. The fake grows
POST /reset, because the cli facet runs both hosts against one process and
writes would otherwise leak between them; it also stops rejecting wrapped
base64, which real GitHub accepts. The unrouted-request logger is now a CI
failure rather than a log line.
* fix(github): probe the parent listing, so an update is not read back stale
The index tracks freshness per directory, never per entry, so get()
never answers EXPIRED and the refill it guarded was unreachable: after a
write invalidated the index the blob's row survived carrying the
pre-write sha, and the read served the old bytes. Create and delete went
through readdir and worked; an update did not.
Readdir on the TypeScript side refilled on any miss, which spent a full
recursive-tree call on every ENOENT and diverged from python. Both now
refill only on EXPIRED.
* test(cache): pin what invalidate keeps that clear discards
A cleared store reads exactly like one that was never filled, which is
what made a github mount report an empty repository. Assert the two
answer differently.
* fix(spec): render an operand's declared name outside the clap dialect
A named slot printed as <text>, so 'man gh api' said <text> where gh
says <endpoint>. argparse prints the dest too. gh is the only spec this
changes; ntn is clap and already rendered it.
* test(integ): cover the gh surface the battery missed
Update-then-read is the case that found the stale-read bug. Also the
help and discovery surface, a leading-slash endpoint, typed fields on a
GET, and a missing endpoint.
* docs(gh): a page per host for the gh CLI
Install, the mount-reads/CLI-acts split, the api rules, and a table of
what differs from gh 2.85.
* fix(github): keep an api field from steering the request
Octokit reads loose parameters off the same object that carries url,
method and headers, so `gh api X -f url=...` retargeted the call
instead of sending the field. The query is spelled into the url and the
body travels as `data`. The python client already separated them.
* fix(cli): let a handler say it did not write
A leaf declares `write` statically because for almost every verb it is
static, but `gh api` carries its method on the line, so a read like
`gh api /user` expired every github mount the install serves. A result
may now report what the spec cannot know.
* feat(gh): expand the owner, repo and branch placeholders
gh's own examples are written with them, so `gh api
repos/{owner}/{repo}/releases` asked for literal braces and 404'd. An
install's repo and branch stand in for the current checkout. Any other
brace pair still reaches the wire, which is gh's behavior too.
* feat(gh): render repo view the way gh renders it
A name line, a description line, then the README, with the separator
omitted when there is none. Probed against gh 2.85. The REST object is
still one `gh api repos/OWNER/REPO` away.
* test(integ): cover placeholders and the new repo view
Also carries a branch on the gh install, which is what {branch}
expands from.
* fix(gh): declare branch on the GhConfig type, and type the test trees
tsc runs in CI but not in pre-commit: GhConfig's zod schema grew
`branch` while its interface did not, and the new github tests built
tree items with a `mode` key the type has no room for.
* refactor(gh): derive GhConfig from its schema instead of declaring it twice
The zod schema is the one doing real work: it validates an install's
config and carries the secretStr marker redaction reads. A hand-written
interface beside it only adds a shape that can drift, which is how
branch reached the schema and not the type.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Zecheng Zhang
|
d9ba2d2460 |
fix(himalaya): file the sender's own copy of a sent message (#761)
* fix(himalaya): file the sender's own copy of a sent message `--send` opened one SMTP conversation and stopped, so mail was delivered but the sender's Sent mailbox stayed empty. A Sent folder is not produced by sending: SMTP keeps no record of itself, and the copy is a second, separate IMAP APPEND that every mail client makes on its own. Four call sites, not one: `message send` plus the route compose, reply and forward share. Upstream v2.0.0 spells that APPEND as `--save <MAILBOX>` on all four verbs, having replaced 1.x's automatic `message.send.save-copy` config. Both are here, because they answer different needs: - `--save <MAILBOX>` is v2 parity. On its own, with no `--send`, it files the message without sending it, which is how a draft is written. - `save_copy` is an account-level default, on, and is mirage's own divergence: an agent that never learned the flag still leaves the record a human sender would. `sent_folder` pins the mailbox. The mailbox is asked for rather than guessed. A server implementing RFC 6154 tags one mailbox \Sent in its LIST reply, which is `[Gmail]/Sent Mail` on Gmail and `Sent Items` on Exchange; failing that the configured name, failing that `Sent`. This is ahead of upstream, whose v2 wizard notes IMAP pins the reserved INBOX alone while it waits on LIST RETURN (SPECIAL-USE) in io-imap. Failure splits on whether a send already happened. A copy that fails after a successful send is a warning on stderr with exit 0, because the message is already gone and a non-zero exit invites a retry that sends it twice. A `--save` that sends nothing fails loudly, because nothing happened yet and retrying is safe. Upstream propagates in both cases. Two bugs found on the way: - aioimaplib joins command arguments with spaces verbatim and quotes nothing, so an unquoted mailbox holding a space arrives as two arguments and the server reads only the first word. That breaks on exactly the two providers above. `quote_mailbox` now covers the APPEND and the pre-existing `select_folder`, which had the same latent bug. imapflow needs none of it, building commands from typed attributes. - The TypeScript test CONFIG was an untyped literal, so a new required field did not fail tsc and those tests passed vacuously through the `!saveCopy` early return. It is annotated now, and a global beforeEach stubs the accessor so no --send test reaches for a real connection. GreenMail hands a new account nothing but an INBOX, where a real provider ships a sent mailbox already made, so both integ seeders create one. Verified against a live GreenMail on both hosts: the email and cli-himalaya targets pass 86 and 74 cases including six new ones, and compose, reply, forward and send each land in Sent with \Seen. Python suite and TypeScript node suite green, pre-commit clean. * fix(email): parse a LIST mailbox as an astring, not as a quoted string Codex review on #761. A LIST mailbox is an astring, so a name needing no quoting may legally arrive bare, and some servers emit it that way. Splitting the line on quotes then reads the hierarchy delimiter as the name: `(\HasNoChildren \Sent) "/" Sent` answered `/`. That splitter predates this branch and already mis-listed every folder on such a server, but this branch is what made it consequential: the name now feeds an APPEND, so a sent copy would have been filed into a mailbox called `/`. The three tokens are walked in order instead. The atom form and the NIL delimiter both parse, and a quoted name is now unescaped rather than cut at the first inner quote, which the splitter also got wrong. Verified against a live GreenMail on both hosts, whose own wire format is the empty-attribute `() "." "INBOX"` shape. |
||
|
bytecii
|
46548b4923 |
refactor(config): name the backend-config rule once, drop the duplicated cache block
Self-review pass over the PR, for special cases that should have been general rules and for shapes that stopped earning their keep: - The s3 exemption was spelled `block.type === 's3'` in two places (skip-camelize, skip-key-check) that encode one rule: a group whose type names a backend carries the backend's config, not ours. Named once as BACKEND_CONFIG_TYPES; a second such backend now joins a set instead of needing both sites found again. - `buildCache` had decayed to an identity cast once the workspace took over building the store, and RamCacheBlock/RedisCacheBlock duplicated core's CacheConfig/RedisCacheConfig field for field. Both deleted; WorkspaceArgs.options.cache narrows to what it now actually carries. - buildCliEntries kept its own copy of the cli key list this PR had just added as CLI_KEYS; the store group names were spelled three times. - absolutizeScripts: isPlainObject guards for the casts it grew, and a comment about running before or after normalization that only one of the two is now true. The key tables are copied by hand from Python's pydantic models, so a field added there would be refused here until someone hit it. Pinned the way rejected.json pins the other direction: integ/fixtures/config/ accepted.json exercises every key of every block and both suites read it. Proved it bites by dropping `root` from STORE_KEYS — the store case fails with `unknown store key \`root\``. Docs showed TypeScript constructing a RedisFileCacheStore beside a declarative `index:`, which is the divergence this PR removed; the snippet is now the config form, matching its own index line and Python. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
bytecii
|
90e564273d |
fix(github): build resources asynchronously, delete the sync HTTP path (#609 T1-J)
`GitHubResource.__init__` fetched the repo's default branch and its
recursive git tree before returning, over `urllib.request.urlopen` — the
only blocking HTTP call in the package. A constructor cannot await, which
is the entire reason that sync client existed.
It is reachable on a live event loop: `server/routers/workspaces.py`'s
`async def load_workspace` calls `build_resource` synchronously, so
mounting a github repo froze the daemon's loop for two GitHub round trips
— every other mount's in-flight I/O and the FUSE queue stalled with it.
Measured against a deliberately slow local GitHub, a 1.24s build let a
50ms heartbeat coroutine tick once; it now ticks 24 times over the same
1.24s.
TypeScript already had the answer, but not the one the audit recorded:
`GitHubResource.open()` there is a no-op, and the work lives in
`static async create` behind a private constructor, with the whole
`ResourceFactory` type declared `(config) => Promise<Resource>`. Mirror
that rather than the `open()` hook — a lazy hook would move the failure
from build time to first path resolution, which is a new divergence, not
a fix.
- `BaseResource.build`: async classmethod factory, default just calls the
constructor. Named `build`, not TypeScript's `create`, because `create`
is already an op name (make an empty file, what `touch` calls) and ops
are served by `__getattr__` — a real `create` on the class shadows every
backend's create op. Caught by the databricks_volume suite.
- `build_resource` is now async and awaits that factory through
`_instantiate`, which falls back to the plain constructor: registered
and entry-point resources need not subclass `BaseResource`.
- `WorkspaceConfig.to_workspace_kwargs` follows, matching TypeScript's
already-async `configToWorkspaceArgs`. `Workspace(**kwargs)` stays sync.
- `GitHubResource.__init__` takes the fetched tree and touches no network;
`GitHubResource.build` does the two fetches over the existing async
`github_get`. `github_get_sync`, `fetch_tree_sync` and
`fetch_default_branch_sync` are deleted.
- New `tests/resource/test_no_blocking_http.py` fails on any `urlopen`,
`urlretrieve` or `requests` import under `mirage/`, including
function-local ones.
- Docs updated on both sides; the TypeScript github pages showed
`new GitHubResource({...})`, which has never compiled against its
private constructor.
Snapshot load is untouched: github's state redacts its token, so
`requires_resource_override` always sends it down the live-override path
and never reaches `_construct_resource`.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Zecheng Zhang
|
7fbe3aedb8 |
fix(notion): delete verb, one trash bit, row cells, and stale docs (#747)
DELETE /v1/blocks/{id} is the only delete verb the public API has and the
only one the MCP tool surface exposes, but the fake had no route for it
and `ntn api` had no DELETE method, so nothing could remove anything.
`archived` is upstream's deprecated alias for `in_trash` and always
returns the same value; the fake stored two columns, so a PATCH of
`archived` left the row in database queries while `ntn pages trash`
(in_trash) worked. Now one stored bit, both spellings on the wire.
A database row's cells now ride in its page.json under `properties`.
Also: trashing a child page moved only the page row, leaving the
child_page block in the parent listing; `datasources query` took its
TSV columns from the schema where upstream takes them from the returned
rows; and two MCP-parity cases pointed at pre-data-source paths, so they
compared two identical errors and asserted nothing.
Docs: both ntn.mdx files documented the pre-#740 grammar (--page flags,
ntn blocks/comments/search). notion.mdx, the TS setup doc and both
examples predated the data-source split; the examples were broken.
|
||
|
Zecheng Zhang
|
3b18a436df |
refactor(runtime): one mount op vocabulary for the typescript sandbox runtimes, and serve pyodide from an Emscripten filesystem (#732)
* refactor(pyodide): serve mounts from an Emscripten filesystem, not a python shim Replace the 425-line python source string the runtime injected with a filesystem mounted through pyodide.FS.mount, so interception moves below the interpreter instead of rebinding names inside it. This fixes two bugs the shim shipped. os.open/os.write/os.fdopen and a bare os.truncate recorded nothing, because the shim rebound 13 names and those were not among them, so a low-level write applied to guest memory and was dropped at exit 0. And a cross-mount rename raised errno 18, which is EDOM under pyodide's musl numbering, so a guest comparing against errno.EXDEV (75) never matched; the old test asserted the message string python rendered from that same literal, so it could not fail. Every prefix is re-seeded on every run. The conformance rows covering post-boot seeding only passed before because the shim's lazy backfill used run_sync, and vitest enables JSPI while production does not. A file the mount lists but will not serve now becomes a node that refuses to open with EIO, rather than a hole an append would fill by replacing the file. * refactor(pyodide): name the filesystem package vfs, not fs Mirrors the python side, where the module holding the guest filesystem is vfs.py beside runtime/vfs.py. Leaving the package as fs/ while its entry module became vfs.ts would have kept the mixed spelling the rename is removing. * refactor(pyodide): name the sentinel and the whence values Number.MAX_SAFE_INTEGER appeared five times as the has-not-written-yet sentinel, and llseek compared whence against bare 1 and 2. Both are now in constants.ts beside the mode bits, matching the python side where READONLY_HINT moved to wasm/constants.py. * refactor(runtime): one mount op vocabulary for the typescript sandbox runtimes New runtime/vfs.ts holds RuntimeVFS, planFlush and the mount routing that pyodide, quickjs and monty each had their own copy of, mirroring the python core. BridgeDispatchFn gains APPEND, so a handle that only extended a file ships its tail: eight 3-byte appends now cost 24 bytes rather than 536, and the two amplification rows in the conformance suite flip from it.fails to it. runtime/config.ts takes coerceRuntimeConfig out of base.ts so a typed config has the same home in both languages. monty.ts becomes a monty/ package (binding, constants, errors, osaccess, runtime, vfs) matching the python split, and MontyVFS gains the negative cache python already had: monty asks whether a path exists on nearly every guest expression, and each miss was costing a fresh listing. mirage_bridge.ts splits into vfs/journal.ts and vfs/preload.ts, and js/mirage_fs.ts becomes js/vfs.ts, so every module holding a guest filesystem is named vfs. Dead MontyVFS.mountOf dropped in both languages. * fix(pyodide): refuse a root mount, mount nested prefixes parent-first Three review findings. A `/` prefix stripped to the empty string, which Emscripten takes as a detached pseudo-mount no path reaches: the guest kept reading and writing MEMFS, and a write reported success the resource never saw. `/` is already MEMFS's own mount root holding the stdlib, so there is nowhere to put it; the prefix is now skipped with one warning rather than silently dropped. prefixes() is longest-first, which is what routing wants and the reverse of what the mount table wants: with /data/ and /data/inner/ the child mounted first and the parent then mounted over it, orphaning the child. Unmount deepest-first, mount shallowest-first. MontyVFS lives as long as the runtime while python rebuilds its MirageOSAccess per run, so the negative cache added here outlived the command it belonged to and hid a file another writer created between two monty runs. Reset it at the top of run and eval. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> |
||
|
bytecii
|
be72d87b95 |
feat: $'...' quoting, git log/show formats, himalaya --attach + byte-exact MIME parity (py+ts) (#723)
* feat: $'...' quoting, git log/show formats, himalaya --attach + byte-exact MIME parity (py+ts) Shell: - $'...' ANSI-C strings decode per bash 5.2 (docker-pinned): full escape table, octal/hex/unicode/control forms, segment-only NUL truncation; $"..." keeps plain double-quote semantics. New shell/escapes decoder, expansion branches, allowlist sites, dollar-quote-aware backtick scan. - Quoted case patterns match like bash: patterns stay nodes and a new expand_case_pattern renders quoting into the fnmatch dialect (quoted segments literal, unquoted globs live, backslash escapes, expansion results live unless double-quoted, no word-splitting). - export/local/declare/readonly accept quoted assignment operands; a bare $ word survives as a literal argument (echo $). Git: - log gains --all (multi-root walk with annotated-tag peeling) and --format/--pretty: oneline/short/medium/full/fuller presets, format:/tformat: templates, ~20 placeholders incl %d decorations in git's exact order; unsupported presets refuse honestly. - show gains --stat (full diff.c show_stats geometry), -s/--no-patch, --name-only, --no-ext-diff, --format; oracle-verified byte-identical to real git in both languages. - commit report counts binary files as files with zero lines (shares diffstat's NUL sniff); fixes a TS-only miss of mode-only changes. Himalaya: - --attach wires up: repeatable path-typed flag (first leaf-level type="path" option; python reads via new FlagView.as_paths), reads through inv.ops.dispatch, multipart/mixed with a content-addressed boundary so both builders emit identical bytes. - TS serialization is now byte-identical to python's EmailMessage.as_bytes(policy=SMTP): new mime.ts ports RFC 2047 encoded words (q/b chooser), unstructured/address header folding, RFC 2231 filename params, and the 7bit/8bit/qp/base64 body selection; fixes threading-header order, empty-subject rendering, and multi-line header refusal. 31 shared pins (integ/fixtures/himalaya/mime_parity.json) asserted by both suites, plus differential fuzzing against the python oracle. Docs list the new git capabilities and remaining deliberate limits; 51 new integ cases across bash/git/himalaya targets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: codex round + generalize pattern quoting, multi-line strings, MIME to core Codex review fixes (all pinned against real git 2.37/2.54, bash 5.2 docker, and CPython 3.12): - git format: empty entries keep their separators under format: and their terminators under tformat: (only an empty template is silent); %xHH emits a raw byte through the shell's byte-escape convention; bare --format gets git's own fatal while bare --pretty stays medium. git show's format: header drops its trailing newline the same way. - ANSI-C \u/\U: values now ride bash's u32toutf8 (UTF-8 locale) - surrogate halves and values past Unicode become raw UTF-8-shaped bytes, 0x80000000 and past produce nothing. The previous "verbatim" rule (and the crash codex flagged) was a C-locale pin artifact. - himalaya: an ASCII attachment filename holding any line break is refused with EmailMessage's exact error; the RFC 2231 path percent-encodes the same characters (new shared parity pin). Review round (generality follow-ups to the case-pattern fix): - expand_case_pattern is now expand_pattern and serves all three pattern-word constructs: case patterns, the [[ == ]] right side (replacing the whole-node right_literal boolean that broke mixed quoting), and parameter-expansion operands. Quoted operand nodes match literally; opaque regex tokens get a lexical scanner honoring single/double/ANSI-C quotes, backslash binds, and live $-refs. escape_glob moved beside GLOB_CHARS in utils/glob_walk (both languages), killing a duplicated constant. - Multi-line double-quoted strings: the newline bytes belong to no tree-sitter token. TypeScript dropped them entirely and python collapsed blank lines; both now re-emit per row step anchored on the quote tokens (leading, trailing and blank lines included). - MIME machinery out of himalaya: mime.ts is now the runtime-agnostic @struktoai/mirage-core utils/mime (TextEncoder + shared base64, no Buffer), exporting the header guard as assertHeaderValue; the fixed extension table lives in utils/filetype as MIME_BY_EXTENSION / mime_type_for beside the FileType maps in both languages. Fixed a silent TS-only bug found in the same file: jpg guessed IMAGE_PNG. python's local drain() replaced by the existing io materialize. Coverage: 40+ new docker/real-git/CPython-pinned unit rows across both languages, 20 new integ cases (git format band, param/test/quoted bash categories, surrogate byte-count), one new shared MIME parity pin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: bytecii <bytecii@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Zecheng Zhang
|
35a4319303 |
fix(monty): port to pydantic-monty 0.0.19 and stop rewriting files on append (#717)
* fix(monty): port to pydantic-monty 0.0.19 and stop rewriting files on append 0.0.19 moved execution into subprocess workers: Monty became a pool, MontyRepl is gone, and run_async moved onto a checked-out session. run() and eval() now drive the pool, and eval() is what the policy layer runs config-borne scripts on, so the pin is exact while the API keeps moving. Cancelling a feed no longer stops a worker, so both paths reclaim the worker pid on CancelledError. Without it a safeguard timeout wedged the line forever and pool teardown blocked uninterruptibly. Also fixes four bridge gaps: mkdir, rename and rmdir never reached the workspace and failed with a bogus ENOENT on paths that existed; piped stdin was discarded, and now binds as a global beside argv; and appends re-sent the whole file every write, which made a write loop quadratic (200 appends shipped 164 KB to build a 1.7 KB file). Monty hands the append hook the new text alone, so it routes to the mount's append op. Documents what a command_safeguard does and does not stop inside a sandbox, where mirage holds only a client and the process keeps running. * fix(monty): bump the ts binding to 0.0.19 and route the write ops Path.mkdir, rmdir, unlink and rename declined in the monty callback, so they applied inside the sandbox's own tree and never reached the mount. The bridge already carried the ops; this wires them up, matching the python runtime. * fix(monty): address review on append fallback, pool race, and cross-mount rename - appends fall back to the whole-file flush on mounts without the optional append op (S3 registers write but not append), remembering the mount so it costs one failed dispatch rather than one per append - the worker pool is cached as a task, so two cold runs cannot each build one and leak the loser's workers past close() - a cancelled eval session hands its checkout back instead of dropping a lease the pool can no longer reach - mkdir forwards parents and answers exist_ok itself - a rename across two mounts raises EXDEV in both languages: the dispatcher picks the mount from the source alone, so crossing would drop the source and write the target into the wrong backend * docs(monty): correct the EXDEV rationale, monty ships no shutil * fix(monty): reject mkdir over a file, and type the mutation errors |
||
|
Zecheng Zhang
|
7fa16fe460 |
fix(pyodide): flush mount writes without JSPI (#720)
* fix(pyodide): flush mount writes without JSPI * chore: retrigger checks * fix(pyodide): address codex review |
||
|
Zecheng Zhang
|
db807f304b |
fix(find): classify walked entries through stat, drop the is_dir_name heuristics (#719)
* fix(find): classify walked entries through stat, drop the is_dir_name heuristics * fix(test): satisfy noUncheckedIndexedAccess in the gmail find fixture * fix(dropbox): walk the native find over a scratch index * test(integ): pin find -type f over attachments and uploads * test(integ): find -type f matches a pdf upload too * ci: retrigger after the Actions outage * fix(email): route the TS find through the generic walk so every flag applies * chore(deps): bump h2 to 4.4.1 for GHSA-6hr6-w5qg-qmwg |
||
|
Zecheng Zhang
|
97356763de |
feat(gws): place creates in the folder scope, and report Google's errors (#715)
* feat(gws): place creates in the folder scope, and report Google's errors Three fixes and a mock-server sweep, all on the Google surface. The gws CLI ignored folder_id, so a create landed in My Drive's root even when the CLI and a gdrive mount shared one config. The editors' create methods carry no parents field at all, so those are moved with a follow-up Drive update; drive files create and copy just default their parents. An explicit parents still wins. Google reports why a call failed in the response body, but the python client raised before reading it, so an agent only ever saw "Bad Request". Typescript already did this correctly. The API and OAuth error shapes are both handled. An account CLI mutates its service by id, which no vfs path can be derived from, so a newly created file had no cache entry to expire and the agent's next ls could not see its own work. Write verbs now drop the listings of the mounts their service backs, declared per spec so a Slack or S3 mount alongside keeps its cache. This corrected two integ goldens that had been asserting the stale listing. The fake Google server grew the Sheets endpoints it never had (values:batchUpdate, batchGet, batchClear, clear, sheets:copyTo, the dimension requests), plus the matching gaps in docs, slides and drive. gws integ coverage is now 35 of 35 leaves. * fix(gws): address codex review on the folder-scope PR Three findings, all real. Cached file bodies survived a CLI write. drop_service_listings cleared the resource index but not the file cache, and all five Google resources cache reads, so a cat after a Docs or Sheets edit kept serving pre-write content without reaching Google. A stale listing hides a create, a stale body hides an edit, and only the listing was being dropped. Adds a prefix-scoped evict_prefix to the file cache contract, implemented on the RAM and Redis stores in both languages, exposed as CacheManager.drop_prefix and called beside the index clear. The function is drop_service_caches now, since listings no longer describes what it does. Redis escapes the prefix before using it as a SCAN MATCH pattern, because a mount path may hold glob metacharacters. Scoped placement never sent supportsAllDrives. Every other Drive helper in the repo sends it, and a folder scope may name a Shared Drive folder, so both the injected parent and the relocation patch would fail there, stranding an already created editor file in My Drive. Both paths send it now, only when mirage is the one injecting. An explicitly empty parents array was silently replaced. The check was a truthiness test, so parents: [] read as absent while parents: ["root"] was honored, which contradicts the documented rule that an explicit array wins. Presence of the key is the test now. Adds unit coverage in both languages for each finding, plus two integ cases pinning the stale body end to end. The integ pair has to span two command lines: apply_io runs once per line, so a cache warmed earlier on the same line is never live for a later read, and a single-line case passes with or without the fix. Documents folder scope and its two divergences from the upstream passthrough in gws.mdx. * test(cache): pin drop_prefix on a root mount A root mount strips to the empty prefix, so the eviction argument is "/" and matches every key. Correct, but non-obvious enough that a reviewer reasoned their way to the wrong answer and back, so both languages now assert it instead of leaving it to be re-derived. |
||
|
Zecheng Zhang
|
9c2053bec7 | feat(cli): author a CLI in code, by pointer, or as a script (#712) | ||
|
Zecheng Zhang
|
67945abf63 |
feat(git): a git CLI over any mount, in both languages (#710)
* feat(git): a git CLI over any mount, in both languages Adds `git` as a builtin CLI (issue #705 item 1): status, log, show, diff, branch, add, reset, commit, checkout. It takes no config, and the repository is read entirely through the mount ops, so a repo on RAM, on disk or on an object store reads the same way. Python goes through dulwich, TypeScript through isomorphic-git backed by a PromiseFsClient over the dispatcher. Status is pinned against the real git binary across 22 repository states and 6 spellings, the mutation verbs are read back by real git and pass `git fsck`, and the rename-similarity score matches dulwich digit for digit. Both run in integ on `git-ram` and `git-disk` in both languages. Also fixes the grep family this leaned on: - grep reads a basic regular expression by default, as POSIX says, with -G for the default and -E for extended. `grep -l` and `grep -rl` compiled their own pattern and still read extended. - zgrep does the same. Its -E flag was read and discarded. - an operand grep could not search exits 2, as GNU does, instead of being flattened to 1. - `grep -l` on a directory reports it rather than walking it. The shared fallback called the recursive walk whenever a failed read turned out to be a directory, which made -l behave like -rl. Supporting pieces: ranged reads (`read_range`) with a read-and-slice fallback and a seeking implementation for disk, PATH-typed group options resolved against the working directory, and git's three unknown-option dialects. * fix(rg): report an unreadable operand as exit 2, like ripgrep CI caught four goldens the grep change had missed, all outside the JSON harness targets I had been running, plus a divergence the change itself introduced. The divergence: TypeScript routes rg through grepGeneric, so rg moved to exit 2 there while python's rg, which has its own generic, stayed at 1. Real ripgrep exits 2 for an operand it could not read, so TypeScript was right and python is brought up to it rather than the other way round. Its single-operand path also let the error escape to the shared handler, which flattens every OSError to exit 1; rg now reports it itself, the same fix grep needed. The rule is now stated once in grep_helper (`exit_code_for` / `exitCodeFor`) and imported by both generics in both languages, instead of living in the grep generic where rg could not reach it. Goldens updated: integ/cross_commands.{py,ts}, the shared observability contract, langfuse and dify. Each pinned exit 1 for a missing operand and carried a comment calling it a deliberate divergence. * fix(git): refuse the three mutations that could lose work Three review findings, each measured against git 2.50.1 before fixing. checkout only compared tracked paths, so a branch holding a file the working tree has untracked wrote its blob straight over it. The file is in no index and no tree, so nothing could see it and nothing could get it back. The untracked set is now part of the conflict check, which needs UNTRACKED_ALL rather than the mode status uses: "normal" collapses a wholly untracked directory to one row, and git names the file inside it. An ignored file stays overwritable, which is git's own split. When both kinds of conflict apply git prints both paragraphs and aborts once, so CheckoutConflictError carries both lists. branch -d deleted a branch HEAD does not contain, dropping the only name pointing at those commits. It now refuses, and -D is added, because -d alone would be a delete with no way to say no. dulwich's can_fast_forward answers exactly this and cannot be used: it asks the repository for its grafts and shallow boundary and a bare BaseRepo raises. Walker is what log already walks with and needs only the object store. add -u ignored its pathspecs and restaged every tracked file, which is how an unrelated edit reaches the next commit. git tells two misses apart and so does this now: a pathspec naming nothing is a fatal about the pathspec, one naming an untracked file is a fatal about git not knowing it, both exit 128. Also two CI failures. The spec dumps were stale for the -G flag added with the zgrep BRE fix. And the git fixture exported its identity as environment variables, which reach only its own commits, so a test that committed into the built repository failed with "Author identity unknown" on any runner with no global identity. It now records the same identity in the repository, leaving the object ids unchanged. Covered by unit tests in both languages and six integ cases across git-ram and git-disk on both hosts. * fix(test): the redis missing-file suite still pinned grep at exit 1 Sibling of the disk suite, which was updated with the exit-2 change. This one needs a live server, so it skips silently without REDIS_URL and the local run never reached it. Only the grep case moves: cat, head, tail and wc all still exit 1 for an operand they could not read, which is the split the comment now records here as it does next door. * fix(grep): read an operand's type from stat, not from how the read failed Four integ jobs caught this across seven backends, and it is one mistake with three faces. Classifying a directory operand *after* a failed read makes the answer depend on what each backend does about reading one, and they disagree: s3, gridfs, hf and nextcloud read a directory path without complaint and hand back nothing, and ssh raises an asyncssh SFTP error that is not an OSError at all, so it escaped the catch entirely and exited 1 with an unattributed "grep: Is a directory". So the operand is now stat-ed before it is read, which is what GNU does and what the -r branch of the same function already did. operand_is_directory had its own version of the same error: a readdir that did not raise counted as a directory, and a prefix store answers readdir for any path at all, returning nothing for one that is not there. Every missing file on those backends therefore read as a directory. The listing must now be non-empty to count, which costs a genuinely empty directory being invisible there, the same divergence du already documents and the safer way round. The catch also widens from three exception types to WALK_ERRORS, the tuple every other operand-tolerant walk in the repo uses, so an errno split cannot make grep abort where tree and grep -r keep going. One test fake resolved stat with `undefined as never` and only worked while nothing read the value. It resolves with a real FileStat now, which is what the postgres backend answers there; the assertion the test exists for is untouched. Verified on gridfs and gridfs-prefix, which reproduce the prefix-store half locally, plus ram, disk and redis for regressions: all five targets on both hosts, 0 failed. * fix(git): three review findings on -C, checkout -b and reset `-C` accepted any path that was not missing, so naming a file inside a repository walked up and ran in the parent instead of failing the way git's chdir does. For a write verb that means mutating a repository the caller never named. It now refuses a non-directory with git's own second wording, "Not a directory". `checkout -b <new> <start>` forced the new branch to HEAD and dropped the start point without a word, so every commit after it landed on the wrong history. The operand is honored, and a start point that is not a commit gets git's sentence naming both it and the branch rather than the generic "ambiguous argument" the same lookup failure produces elsewhere. `reset <operand>` that selected no path unstaged nothing and exited 0, which a script reads as "the index was reset". Two different mistakes reach that point and they now get different fatals. A typo is git's "ambiguous argument". A revision is not: real git resets the index to any commit named there, measured on 2.50.1, so the review's premise that git refuses one is wrong. This build resets from HEAD only, and says which feature is missing instead of claiming a revision it can resolve is unknown. Recorded as a divergence in both git.mdx files. Six integ cases across git-ram and git-disk on both hosts, plus unit tests in both languages. * chore: merge main and regenerate the grep and zgrep specs #713 landed a `pair` field on the option spec while this branch was open. CI builds the PR merged with main, so it regenerated specs carrying the field and compared them against grep.json and zgrep.json, the two files this branch had already rewritten for `-G`, which therefore predate it. Every other spec file came across from main already carrying it. The regeneration also needs a rebuilt dist: gen-specs.ts reads the built package, and against a stale one it dies on an Operand field it does not know, which silently leaves the typescript half unregenerated and shows up as a python/typescript parity divergence rather than as a build error. |
||
|
Zecheng Zhang
|
745044f5d2 |
feat(jq): support the rest of jq's flag surface (#713)
* feat(jq): support the rest of jq's flag surface
Only -r, -c and -s were accepted, with no long forms at all. Adds
-n/-R/-j/-a/-S/-e/-M/-f/-h, every long spelling, --raw-output0, --tab,
--indent, --unbuffered, --arg, --argjson, --rawfile, --slurpfile,
--args, --jsonargs, --stream and --seq, in Python and TypeScript.
Two spec-parser additions carry it: Option(pair=True) for two-token
options like --arg name value, and Operand(text_when=...) so --args
turns later operands into positional strings instead of input files.
Also fixes -s to slurp across every operand rather than per file, and
an empty input to print nothing instead of erroring.
* fix(jq): read inputs as a call, not as the word
The detector matched `.inputs`, `{inputs}`, `$inputs`, a module member
and any string or comment, and a match switches the run into drain mode,
so `jq -c '.inputs'` over a two-document stream printed one line where
jq prints two. Both languages now blank string bodies, comments and
object-shorthand keys before searching for the token, keeping
interpolations as code. $ARGS is read the same way.
The native fixture also tolerates EPIPE on the child's stdin: `jq -n`
exits without reading, which raced the fixture's write and failed the
whole node package on unhandled errors with every test passing.
Adds integ coverage for the rest of the flag surface, including the long
spellings, the refused options and the usage errors.
|
||
|
Zecheng Zhang
|
619be25d32 |
feat(himalaya): align the CLI with the upstream pimalaya grammar (#709)
* feat(himalaya): align the CLI with the upstream pimalaya grammar
- mailbox is -m/--mailbox, message id is a positional operand
- envelope search with upstream's query DSL, replacing list filter flags
- message compose writes RFC 5322 to stdout unless --send; message send
takes raw MIME, so the two chain through a pipe
- upstream aliases: ls, sr, write, new, fwd
core/email/send.* is deleted: the email mount is read-only, so the
composer and the SMTP transport moved into the himalaya package.
Fixes three node-side bugs found while testing:
- sendMail({raw}) needs an explicit envelope; Bcc is stripped by hand
- parseSearchCriteria silently dropped unknown keys, so NOT SEEN matched
every message
- parseImapDate rolled a bad month or day into a wrong date
* fix(himalaya): satisfy the CI eslint rules and stop the yapf/isort import flip
* fix(himalaya): search the Date header and bound envelope pages
Date conditions emit SENTON/SENTBEFORE/SENTSINCE rather than
ON/BEFORE/SINCE, which match the mailbox internal date, and the missing
upstream `before` condition is now parsed.
Envelope listing asked IMAP for every matching uid and fetched headers
for all of them before slicing one page. It now fetches the newest
page * page_size, capped by the account's max_messages.
* test(himalaya): cover every flag in integ, and fix what it found
Grows the himalaya integ suite from 38 to 64 cases: every search
condition, sorter, flag and alias, the composer flags, both quoting
options, stdin bodies, the send operand form, unknown flags and a
missing mailbox.
Three bugs it turned up, all in the email client:
- Five IMAP selects ignored their response, so a missing mailbox left
the session in AUTH and the next command complained about that
instead ('command SEARCH illegal in state AUTH' in python, imapflow's
bare 'Command failed' in typescript). Both now say no such mailbox.
- A refused SEARCH returned an empty uid list, so criteria the server
cannot answer read as 'matched nothing'. It now raises.
- Found because a mutation emitting a malformed date stayed green.
|
||
|
Zecheng Zhang
|
7730eb3c37 |
feat(shell): discover installed CLIs through man, type and which (#707)
* feat(shell): discover installed CLIs through man, type and which
man renders an installed CLI from its own spec, type reports it as a cli, and which is new. Precedence now lives in one lazy layers() generator that serves both route (the winner) and route_all (every layer, which type -a prints).
* refactor(shell): share one option scanner across the bash builtins
command, type and which each hand-rolled the same non-permuting letter
scan. They now share scan_options/last_of, which pins bash's grammar in
one place: a long spelling refuses on its second dash, and a mutually
exclusive group resolves to the last letter typed.
type -f is a filter over the layer list instead of a pop-and-restore on
the session function table, and man takes the install it already looked
up rather than fetching it again.
* fix(shell): keep the layers under a reserved word visible
type -a stopped at the keyword, so a function sharing a reserved word's
name never showed. bash prints both lines (function time { :; }; type -a
time), and mirage's parser lets any reserved word be a function name, so
the shadow was reachable and hidden.
time and coproc leave the keyword table with it: mirage implements
neither, so type called them keywords while running one reported command
not found. which now drops the keyword layer before picking a winner
rather than after, so it reports the function underneath instead of
nothing.
* refactor(shell): split lookup into a package the way route and condition are
types.py holds NameKind, constants.py the consumer and description
tables, classify.py the layer walk, handle.py the two builtins. The
keyword pool moves to the name-pool leaf beside SHELL_NAMES, where the
CLI registry can read it: installing a CLI under a reserved word was
allowed and would never have been reachable, and is now refused.
command.py and lookup build their result triples with the shared
helpers instead of by hand, and man renders its missing-description
placeholder in one place.
* fix(shell): keep command -V's diagnostics when another name resolved
The shared-helper refactor routed the any-found case through ok(),
which drops stderr, so command -V ls nope printed the found line and
swallowed 'command: nope: not found'. bash prints both and exits 0
(pinned), and the TypeScript sibling always did.
All three handlers now build one result with a computed exit code, so
the diagnostics can never ride on the status again, and the mixed case
is a test in both languages.
|
||
|
Zecheng Zhang
|
fc524e1d90 |
feat(cli): slack, discord, ntn and linear as builtin CLI packages (#703)
* feat(cli): slack, discord, ntn and linear as builtin CLI packages Migrate the remaining service bundles to installed CLIs with the settled vocabularies (OpenClaw slack/discord actions, the official Notion CLI grammar as ntn, linear's noun/verb tree), including the new API surface they need (slack pins/emoji/reactions/history, discord edit/delete/ threads/polls, notion page updates). Configs move resource -> core in both languages and the mounts go filesystem-only. chroma-query stays a mount command on purpose: it is path-scoped. Fixes two CLI dispatcher bugs found by the new integ cases in both languages: leaf exceptions now become the command's IOResult (so redirects and ; sequencing work, with the GNU prog: prefix), and argv words re-enter the walk as typed so quoted glob-looking flag values survive. Adds cli-facet integ targets and cases for all four CLIs (108 cases both runners), extends the fake slack/discord/notion servers, and moves CLI usage out of the resource docs into new CLI sections on both tabs. * integ: cover the remaining linear write verbs in the cli facet * feat(cli): linear add-label and set-project resolve names via the issue's team * fix(cli): codex review, standalone thread type and bounded notion search pagination |