Zecheng Zhang
07bca1b503
* fix(archive): tar and zip archive a directory, and -C re-bases the operands after it tar -cf d.tar d exited 1 with "tar: <hostpath>/d: Is a directory": every operand went straight to read_bytes, with no isdir check and no recursion, so the most common tar invocation there is could not work. zip -r <dir> had the identical defect. Both now walk. Create is a two-phase pass in each: decide every member, then write. Both plans are built on one traversal, scan_operand / scanOperand (generic/archive/walk), which merges three sources no single one can see: the backend walk (find's walk_find, so an entry is classified through stat and never by name), the namespace's symlinks, and the mount table. It reports paths, never names, because naming is where the two formats part company; the two things they disagree about in the traversal itself are parameters, so tar passes recurse=True and dereference=-h while zip passes recurse=-r and dereference=not -y. A directory is its own member, so an empty one survives a round trip, and both extractors now mkdir for one. A symlink is a symlink member under tar (SYMTYPE, target in linkname) and under zip -y (mode 0120777). An unreadable operand is reported in virtual path space with the archiver's own wording, because the raw IsADirectoryError was leaking the host path behind a disk mount. tar -C was ignored: `tar -czf /work/out.tgz -C /work/check my_paper` failed as "paths span multiple mounts", the same defect class as unzip -p in #725, because the operand resolved against the session cwd and the router then saw a phantom mount span. -C is not a flag the command reads once, it is a chdir for the operands typed after it, so it is now declared in the spec (CommandSpec.operand_base) and resolved by the one component that walks the line positionally: the parser reports a base per word and the classifier resolves each operand against it. MountView is how a command sees mount boundaries, offered the way LinkView is (name a `mounts` parameter, nothing else). A traversal that renders lines gets this free from the executor's fan-out; one that emits a single binary object cannot, which is why the archivers read the table themselves. A descendant mount is not crossed: the mountpoint stays an entry and its contents are dropped with GNU's --one-file-system wording, since descending would archive by accident what MountRootPolicy now refuses on purpose for tar, zip and cp in a source slot. Semantics pinned against GNU tar 1.35 and Info-ZIP 3.0 on debian:stable-slim, including Info-ZIP's inverted defaults, its anchored -x, its silent leading-slash strip, and "Nothing to do!" exiting 12 with no archive written. * chore(spec): the dumps state what a command declares, not what it defaulted to Both generators emitted every field of every dataclass, so `truncate`, which declares one thing (`-s`/`--size` takes a string), spent 27 lines of spec body restating 21 defaults that read identically in all 93 files. `zip` was 198 lines for five flags, 11 of the 13 keys on each option being defaults. Anything equal to its default is now dropped on both sides. `type` survives even at its default, because what a token is is the first thing a reader looks for, and `"rest": {}` says less than `"rest": {"type": "path"}`. The defaults come from the dataclass fields in python and from a default-constructed instance in typescript, rather than from a table either side could let drift. The two must drop exactly the same keys, and the parity gate reports every command if they do not. This narrows that gate's blast radius rather than widening it. When `operand_base` was missing from gen-specs.ts, python emitted the key in all 93 files and typescript in none, so parity reported ~95 divergences to sift. The same bug now reports one, on tar, which is the only command that sets it. check_spec_parity's option-diff renderer keyed options by `o["long"] or o["short"]`, which raises once an option carries only the spelling it declared. 279 files, 22113 lines of restated defaults gone. * fix(archive): strip the mount prefix by scan, not by a backtracking regex childSpec measured the backend key with `.replace(/^\/+|\/+$/g, '')`. The trailing alternative backtracks on a run of slashes, which CodeQL flags as js/polynomial-redos, and the input is a resource path a mount supplies. utils/slash.ts already has stripSlash, which walks the two ends by charCode and cannot backtrack. Use it. slash.test.ts pins it against the regex it replaces. * fix(archive): the five symlink and mode bugs the codex review found All five reproduced against a live workspace first, then pinned against GNU tar 1.35 and Info-ZIP 3.0 on debian:stable-slim. A symlink operand never reached the planner as a link. tar and zip were absent from NO_FOLLOW_COMMANDS, so the router rewrote the operand through the link table and `tar -cf o.tar link` stored a regular file holding the target's bytes where GNU stores a symlink member of size 0. It also skipped the planner's cross-mount refusal, since by then there was no link left to refuse. Both archivers now lstat, and carry no DEREFERENCE_FLAGS entry on purpose: -h and -y are the planner's to read. Only the last -C was checked. `tar -cf m.tar -C missing x -C good y` reported `x: Cannot stat` and still wrote an archive holding y, where GNU chdirs at each -C and dies at the first it cannot enter. The option accumulates now and the planner walks the list, so the first bad one is fatal and no members are written. Two links to one target were called a loop, and a real loop was not caught at all. Both were the same mistake: an operand-wide `seen` set doing detection the namespace already does properly under a hop limit. Deleting it archives both names, as GNU and Info-ZIP do, and catching the CycleError that resolve raises turns a genuine cycle into one fatal problem per member with GNU's "Too many levels of symbolic links", keeping the directory entry and exiting 2 instead of throwing out of the planner for a bare exit 1. The mount-root refusal denied member selectors. Under -t and -x an operand names something inside the archive, so `tar -tf a.tar data` was refused as busy when `data` happened to spell a mount. Gated on create mode now, dashless first word included. A test asserted the loop bug rather than catching it; it is replaced. Five integ cases cover the lot end to end in both languages. * test(archive): build the test LinkView's stat through FileStat, not a cast
Command Spec Exports
One JSON file per builtin command per implementation, dumped from the live
registries. python/general is the Python surface; typescript/node and
typescript/browser are the two TypeScript package pairings (core + node,
core + browser).
Each file carries the parsed spec (description, epilog, options, positional
operands, rest operand, ignored tokens) plus a _meta block recording which
resources register the command and whether any registration carries a
provision, an aggregate, or the write flag.
_meta.by_resource keys those same facts by the registering resource. The
union flags cannot say which backend carries a provision, so dropping one
backend's provision while another keeps it leaves every union unchanged. The
parity check compares per resource for that reason, and falls back to the
unions only once the per-resource entries agree. Registrations with no
resource (the general commands) are keyed under the empty string.
# Regenerate
./python/.venv/bin/python scripts/gen_specs.py
node --experimental-strip-types typescript/scripts/gen-specs.ts
# Compare the two implementations
./python/.venv/bin/python scripts/check_spec_parity.py
The two CI gates
Spec drift regenerates both trees and fails if the committed JSON moved,
so the checked-in surface always matches the code.
Spec parity runs scripts/check_spec_parity.py, which diffs Python against
TypeScript command by command: every option (including its help text, value
kind, repeatability and shorthand form), every operand, the resource set each
command registers under, and the per-resource metadata. Resources are compared
against the union of the node and browser variants, since Python has no
runtime split.
The comparison walks the union of the keys both sides emit rather than a
fixed field list. Python dumps with asdict(spec), so a new CommandSpec or
_meta field appears in its tree on its own, while gen-specs.ts serializes
through hand-written literals and would not. With an allowlist that asymmetry
was invisible: both drift gates still passed, because each tree regenerated
byte-identically, and parity never looked at the new key. The two TypeScript
variants are also diffed against each other, since Python is compared against
node and nothing else would otherwise read spec/typescript/browser's
non-_meta content.
resources.json
One per implementation tree, beside general/. Registry membership is a
different table from command registration, and only the second was ever
dumped — so a resource could register commands under every backend's _meta
while build_resource / buildResource had no factory for it. That is how
Python shipped without a sharepoint factory and the TypeScript registries
without chroma/dify/lancedb/qdrant while every command spec stayed identical.
Each file records registry (what can be constructed by name) and
command_resources (what registers at least one builtin command). The gate
asserts command_resources ⊆ registry per tree, and that Python's registry
equals the union of the two TypeScript ones. Deliberate omissions — the
workspace-internal history view mount — are declared under
unconstructible_resources.<tree> and stale-checked like every other
exemption. python/tests/resource/test_registry.py and
packages/node/src/resource/registry.test.ts both read these files instead of
re-copying the name list, so neither can pin an omission the way the old
hand-written set pinned SharePoint's.
Divergences that are structural rather than bugs live in
parity_exceptions.json:
resource_expansions— one implementation registers a command once for a name that stands for several resource kinds. Python's HF commands declarehf_bucketsand the datasets/models/spaces resources rebind them at construction, where TypeScript names all four up front.language_only_resources— a backend that exists in only one runtime, such as the browser's OPFS.commands— per-command exemptions.fieldsmutes a whole top-level field;by_resourcenames one resource and one metadata key, so an exemption cannot quietly hide a second divergence on the same command.
The checker fails on a stale exception, so an entry cannot outlive the divergence it documents. An exemption counts as used only when it actually suppresses a live divergence, not merely by being listed.
Keeping the dump complete
gen-specs.ts can only see command groups the package index re-exports, so it
asserts that every *_COMMANDS declared by a builtin command module is
reachable. A backend that defines commands but forgets the re-export used to
drop out of the dump silently; now generation fails.
gen_specs.py has the same hazard from the other direction: a command module
that will not import registers nothing, so a venv missing the optional extras
quietly drops every backend behind them. Regenerating in that state looks like
a legitimate deletion of thousands of committed lines. Generation now names
the modules that failed to import and exits without writing, so run
cd python && uv sync --all-extras --no-extra camel first.