Zecheng Zhang
98fcdcb2e2
python's GenericResource wires the whole generic command set, the glob resolver and the VFS/FUSE ops from one table, and TypeScript had every ingredient but no class that assembled them, so a custom backend there was still written out by hand. Adds GenericResource<A>, generic over the accessor so the table is checked against the core functions it holds rather than against Accessor. It keeps TypeScript's own wiring style: commands() and ops() return arrays instead of python's register() loop mutating state in the constructor. And there is no sdk.ts, because core's barrel is 99 lines and gated in both directions while the ./* exports map already makes every module importable, so the five names the new example and doc reach for are the only additions to it. Both classes gained sizes_always_known and supports_snapshot. Without the first, no user-written backend could be mounted on FSKit at all, since resolve_backend refuses a resource that cannot size its files. The two one-file examples answer one shared truth file, so the two SDKs cannot drift without a red build. docs/typescript/resource/new.mdx is the twin of the python page, whose stale du_total/du_all is corrected to du. The layout baseline drops to 255: resource/generic was one of the counted divergences.
Command Spec Exports
One JSON file per builtin command per implementation, dumped from the live
registries. python/general is the Python surface; typescript/node and
typescript/browser are the two TypeScript package pairings (core + node,
core + browser).
Each file carries the parsed spec (description, epilog, options, positional
operands, rest operand, ignored tokens) plus a _meta block recording which
resources register the command and whether any registration carries a
provision, an aggregate, or the write flag.
_meta.by_resource keys those same facts by the registering resource. The
union flags cannot say which backend carries a provision, so dropping one
backend's provision while another keeps it leaves every union unchanged. The
parity check compares per resource for that reason, and falls back to the
unions only once the per-resource entries agree. Registrations with no
resource (the general commands) are keyed under the empty string.
# Regenerate
./python/.venv/bin/python scripts/gen_specs.py
node --experimental-strip-types typescript/scripts/gen-specs.ts
# Compare the two implementations
./python/.venv/bin/python scripts/check_spec_parity.py
The two CI gates
Spec drift regenerates both trees and fails if the committed JSON moved,
so the checked-in surface always matches the code.
Spec parity runs scripts/check_spec_parity.py, which diffs Python against
TypeScript command by command: every option (including its help text, value
kind, repeatability and shorthand form), every operand, the resource set each
command registers under, and the per-resource metadata. Resources are compared
against the union of the node and browser variants, since Python has no
runtime split.
The comparison walks the union of the keys both sides emit rather than a
fixed field list. Python dumps with asdict(spec), so a new CommandSpec or
_meta field appears in its tree on its own, while gen-specs.ts serializes
through hand-written literals and would not. With an allowlist that asymmetry
was invisible: both drift gates still passed, because each tree regenerated
byte-identically, and parity never looked at the new key. The two TypeScript
variants are also diffed against each other, since Python is compared against
node and nothing else would otherwise read spec/typescript/browser's
non-_meta content.
resources.json
One per implementation tree, beside general/. Registry membership is a
different table from command registration, and only the second was ever
dumped — so a resource could register commands under every backend's _meta
while build_resource / buildResource had no factory for it. That is how
Python shipped without a sharepoint factory and the TypeScript registries
without chroma/dify/lancedb/qdrant while every command spec stayed identical.
Each file records registry (what can be constructed by name) and
command_resources (what registers at least one builtin command). The gate
asserts command_resources ⊆ registry per tree, and that Python's registry
equals the union of the two TypeScript ones. Deliberate omissions — the
workspace-internal history view mount — are declared under
unconstructible_resources.<tree> and stale-checked like every other
exemption. python/tests/resource/test_registry.py and
packages/node/src/resource/registry.test.ts both read these files instead of
re-copying the name list, so neither can pin an omission the way the old
hand-written set pinned SharePoint's.
Two more tables record what a backend does, not just that it exists.
Membership never said whether a mount serves ten-minute-stale listings or
pushes a du down to the API, and both facts were hand-maintained on each
side.
capabilities carries, per registry name, index_ttl, caches_reads,
supports_snapshot and sizes_always_known, plus whether the class overrides
storage_id and statfs (booleans, since the base answers are per-instance
identity and UNKNOWN). Python reads them off the class. TypeScript reads them
from the class declarations (scripts/resource_facts.ts): the twins are
instance fields, so observing them at runtime would mean constructing the
resource, and construction is not inert — buildResource('github', {}) issues
an HTTP request and postgres opens a connection. A value the extractor
cannot read as a literal is dumped verbatim as <expr:…> rather than guessed,
so it surfaces as a real mismatch instead of a plausible default. A browser
factory that exists only to explain that the runtime cannot serve the backend
(lancedb, email) dumps null, and the node entry is the one compared
against Python.
command_io carries, per backend command package, the wired CommandIO slot
names plus local, max_glob_matches and max_du_entries. The adapter's slot
set is a hand-filled literal nothing else reads, so a backend could omit du
or find and fall back to the capped readdir walk — a partial total and an
exit 1 past the cap — while its twin pushed the same query down to the API.
Where the two languages name the package differently, command_io_aliases
maps Python's name onto TypeScript's.
Both tables are diffed node against browser first, then the merged
TypeScript view against Python. Python has no runtime split, so the merge has
to pick one variant and it prefers node; without the first diff that preference
silently discarded a divergence instead of resolving one. It did: the fifteen
browser S3-family resources declared neither sizes_always_known nor
storage_id where their node twins declared both, and Python matched node, so
every gate passed while a browser mv between two mounts of one bucket saw two
separate storages.
Divergences that are structural rather than bugs live in
parity_exceptions.json:
resource_expansions— one implementation registers a command once for a name that stands for several resource kinds. Python's HF commands declarehf_bucketsand the datasets/models/spaces resources rebind them at construction, where TypeScript names all four up front.language_only_resources— a backend that exists in only one runtime, such as the browser's OPFS.commands— per-command exemptions.fieldsmutes a whole top-level field;by_resourcenames one resource and one metadata key, so an exemption cannot quietly hide a second divergence on the same command.resource_capabilitiesandcommand_io— one resource and one key per entry, same rule: an exemption covers the fact it names and nothing else.variant_resource_facts.<table>— the same, for a fact that legitimately differs between the node and browser runtimes rather than between languages.
The checker fails on a stale exception, so an entry cannot outlive the divergence it documents. An exemption counts as used only when it actually suppresses a live divergence, not merely by being listed.
Keeping the dump complete
gen-specs.ts can only see command groups the package index re-exports, so it
asserts that every *_COMMANDS declared by a builtin command module is
reachable. A backend that defines commands but forgets the re-export used to
drop out of the dump silently; now generation fails.
gen_specs.py has the same hazard from the other direction: a command module
that will not import registers nothing, so a venv missing the optional extras
quietly drops every backend behind them. Regenerating in that state looks like
a legitimate deletion of thousands of committed lines. Generation now names
the modules that failed to import and exits without writing, so run
cd python && uv sync --all-extras --no-extra camel first.