github-featured/strukto-ai--mirage
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

fix(archive): tar and zip archive a directory, and -C re-bases the operands after it (#738)

Browse Source 
* fix(archive): tar and zip archive a directory, and -C re-bases the operands after it

tar -cf d.tar d exited 1 with "tar: <hostpath>/d: Is a directory": every
operand went straight to read_bytes, with no isdir check and no
recursion, so the most common tar invocation there is could not work.
zip -r <dir> had the identical defect. Both now walk.

Create is a two-phase pass in each: decide every member, then write.
Both plans are built on one traversal, scan_operand / scanOperand
(generic/archive/walk), which merges three sources no single one can
see: the backend walk (find's walk_find, so an entry is classified
through stat and never by name), the namespace's symlinks, and the
mount table. It reports paths, never names, because naming is where the
two formats part company; the two things they disagree about in the
traversal itself are parameters, so tar passes recurse=True and
dereference=-h while zip passes recurse=-r and dereference=not -y.

A directory is its own member, so an empty one survives a round trip,
and both extractors now mkdir for one. A symlink is a symlink member
under tar (SYMTYPE, target in linkname) and under zip -y (mode 0120777).
An unreadable operand is reported in virtual path space with the
archiver's own wording, because the raw IsADirectoryError was leaking
the host path behind a disk mount.

tar -C was ignored: `tar -czf /work/out.tgz -C /work/check my_paper`
failed as "paths span multiple mounts", the same defect class as unzip
-p in #725, because the operand resolved against the session cwd and
the router then saw a phantom mount span. -C is not a flag the command
reads once, it is a chdir for the operands typed after it, so it is now
declared in the spec (CommandSpec.operand_base) and resolved by the one
component that walks the line positionally: the parser reports a base
per word and the classifier resolves each operand against it.

MountView is how a command sees mount boundaries, offered the way
LinkView is (name a `mounts` parameter, nothing else). A traversal that
renders lines gets this free from the executor's fan-out; one that
emits a single binary object cannot, which is why the archivers read
the table themselves. A descendant mount is not crossed: the mountpoint
stays an entry and its contents are dropped with GNU's
--one-file-system wording, since descending would archive by accident
what MountRootPolicy now refuses on purpose for tar, zip and cp in a
source slot.

Semantics pinned against GNU tar 1.35 and Info-ZIP 3.0 on
debian:stable-slim, including Info-ZIP's inverted defaults, its
anchored -x, its silent leading-slash strip, and "Nothing to do!"
exiting 12 with no archive written.

* chore(spec): the dumps state what a command declares, not what it defaulted to

Both generators emitted every field of every dataclass, so `truncate`,
which declares one thing (`-s`/`--size` takes a string), spent 27 lines
of spec body restating 21 defaults that read identically in all 93
files. `zip` was 198 lines for five flags, 11 of the 13 keys on each
option being defaults.

Anything equal to its default is now dropped on both sides. `type`
survives even at its default, because what a token is is the first
thing a reader looks for, and `"rest": {}` says less than
`"rest": {"type": "path"}`.

The defaults come from the dataclass fields in python and from a
default-constructed instance in typescript, rather than from a table
either side could let drift. The two must drop exactly the same keys,
and the parity gate reports every command if they do not.

This narrows that gate's blast radius rather than widening it. When
`operand_base` was missing from gen-specs.ts, python emitted the key in
all 93 files and typescript in none, so parity reported ~95 divergences
to sift. The same bug now reports one, on tar, which is the only
command that sets it.

check_spec_parity's option-diff renderer keyed options by `o["long"] or
o["short"]`, which raises once an option carries only the spelling it
declared.

279 files, 22113 lines of restated defaults gone.

* fix(archive): strip the mount prefix by scan, not by a backtracking regex

childSpec measured the backend key with `.replace(/^\/+|\/+$/g, '')`.
The trailing alternative backtracks on a run of slashes, which CodeQL
flags as js/polynomial-redos, and the input is a resource path a mount
supplies.

utils/slash.ts already has stripSlash, which walks the two ends by
charCode and cannot backtrack. Use it. slash.test.ts pins it against
the regex it replaces.

* fix(archive): the five symlink and mode bugs the codex review found

All five reproduced against a live workspace first, then pinned against
GNU tar 1.35 and Info-ZIP 3.0 on debian:stable-slim.

A symlink operand never reached the planner as a link. tar and zip were
absent from NO_FOLLOW_COMMANDS, so the router rewrote the operand
through the link table and `tar -cf o.tar link` stored a regular file
holding the target's bytes where GNU stores a symlink member of size 0.
It also skipped the planner's cross-mount refusal, since by then there
was no link left to refuse. Both archivers now lstat, and carry no
DEREFERENCE_FLAGS entry on purpose: -h and -y are the planner's to read.

Only the last -C was checked. `tar -cf m.tar -C missing x -C good y`
reported `x: Cannot stat` and still wrote an archive holding y, where
GNU chdirs at each -C and dies at the first it cannot enter. The option
accumulates now and the planner walks the list, so the first bad one is
fatal and no members are written.

Two links to one target were called a loop, and a real loop was not
caught at all. Both were the same mistake: an operand-wide `seen` set
doing detection the namespace already does properly under a hop limit.
Deleting it archives both names, as GNU and Info-ZIP do, and catching
the CycleError that resolve raises turns a genuine cycle into one fatal
problem per member with GNU's "Too many levels of symbolic links",
keeping the directory entry and exiting 2 instead of throwing out of
the planner for a bare exit 1.

The mount-root refusal denied member selectors. Under -t and -x an
operand names something inside the archive, so `tar -tf a.tar data`
was refused as busy when `data` happened to spell a mount. Gated on
create mode now, dashless first word included.

A test asserted the loop bug rather than catching it; it is replaced.
Five integ cases cover the lot end to end in both languages.

* test(archive): build the test LinkView's stat through FileStat, not a cast
This commit is contained in:
Zecheng Zhang
2026-08-09 07:58:38 -07:00
committed by GitHub
parent 86cf0d515d
commit 07bca1b503
349 changed files with 6919 additions and 22037 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+138 -2
View File
@@ -97,6 +97,68 @@ agent discovers state, the CLI is how it acts.
`file` would promise `type -p` a path that does not exist). `which` prints the
bare name, never a fabricated path.
## Mount boundaries
A mount root is not an ordinary directory, and a mount nested inside
another mount's tree is invisible to the backend that owns the parent:
the child's keys live in a different resource, so the parent's `readdir`
never lists it. Two mechanisms follow from that, and they are separate.
- **`MountView` is how a command sees the boundaries** (`ops/types.py`,
`ops/types.ts`), and it is offered the way `LinkView` is: a command
opts in by naming a `mounts` parameter, `execute_cmd`/`executeCmd`
delivers it only to handlers that do, and there is no list of
boundary-aware commands anywhere. It carries `descendants` (mount
roots strictly under a path), `is_root`, and `root_of`.
A traversal command that renders **lines** does not need it: the
executor's fan-out (`workspace/executor/fanout.py`) already reruns
find/du/tree/grep -r per mount and concatenates the output. A command
whose output is one **binary object** cannot be merged that way, which
is why `tar` reads the boundaries itself.
- **Crossing into a descendant mount is refused, not attempted.** `tar`
and `zip` both keep the mountpoint as a directory entry and drop its
contents with GNU's own `--one-file-system` wording (`<name>/: file is on a different filesystem; not dumped`; Info-ZIP has no message of
its own for this, so zip borrows the wording under its own
`zip warning:` prefix). This is deliberate: descending would archive
by accident exactly what the mount-root refusal below forbids on
purpose.
- **`MountRootPolicy` refuses a mount root in a source slot** for `tar -c`,
`zip` and `cp`, on top of the POSIX EBUSY rules it already enforces
for `rm`/`rmdir`/`mv`/`mkdir`/`touch`/`ln`. Real tar and cp allow it;
mirage does not, because the mount table is the deployment's
configuration and reading a whole backend into one object is neither
what the operand looks like it costs nor something an agent should be
able to do to data it was given a view of. Only **positional** operands
are tested, which is why `CommandContext` carries `operands` beside
`paths`: `tar -xf a.tar -C /mnt` extracts INTO a mount and must stay
legal, while `tar -cf a.tar /mnt` must not. `positional_scopes` /
`positionalScopes` (`executor/command/routing`) is what tells the two
apart, since classification turns every path-shaped word into a
PathSpec whether it filled an operand slot or a flag's value. Mode
matters too: only `tar -c` reads its operands from the filesystem, so
`is_create_mode` / `isCreateMode` gates the refusal. Under `-t` and
`-x` an operand is a member selector matched inside the archive, and
refusing one that happens to spell a mount root would deny an
ordinary listing.
## An option that chdirs: `operand_base`
`tar -C` is not a flag the command reads once, it is a chdir for the path
operands typed **after** it, and it is cumulative (`-C d1 x -C ../d2 y`
reads `d1/x` and `d1/../d2/y`). That is a property of the line, so it is
declared in the spec (`CommandSpec.operand_base` / `operandBase`, tar's
only) and resolved by the one component that walks the line
positionally: `parse_command` / `parseCommand` tracks the base as it
scans and reports it per word as `word_bases` / `wordBases`, which
`classify_parts` then resolves each operand against. Doing it anywhere
later is too late: the classifier has already produced absolute
PathSpecs, and an operand resolved against the wrong base makes the
router see a phantom cross-mount span (which is what
`tar -czf /work/out.tgz -C /work/check my_paper` used to fail as).
Only path operands and the option's own value move; every other
path-valued flag keeps resolving against the session cwd, which is what
GNU does with `-f`.
## Symlinks
Symlinks are **namespace state, not backend state**. The `Namespace` node table
@@ -148,8 +210,15 @@ they bite:
Follow policy is two symmetric tables in `workspace/route/constants.py`, both
read off the raw command line (operand rewriting happens before flag parsing):
`NO_FOLLOW_COMMANDS` lists commands that lstat (`rm`, `mv`, `ln`, `readlink`,
`rmdir`, `unlink`, `stat`, `file`, `du`, `find`), with `DEREFERENCE_FLAGS`
naming the flag that turns following back on (`-L`). `find` states its policy as
`rmdir`, `unlink`, `stat`, `file`, `du`, `find`, `tar`, `zip`), with
`DEREFERENCE_FLAGS` naming the flag that turns following back on (`-L`).
`tar` and `zip` are in that list for a different reason and deliberately carry
no `DEREFERENCE_FLAGS` entry: they dereference too, but their planner has to be
the one doing it. Rewriting the operand in the router hands the planner a
target it can no longer tell was reached through a link, so `tar` stored a
regular file where GNU stores a symlink member, and neither archiver could
apply its own cross-mount refusal or ELOOP wording. `tar -h` and `zip -y` are
read by `scan_operand` instead. `find` states its policy as
a leading `-P`/`-H`/`-L` option instead, last one wins, so it lives in
`LAST_WINS_LINK_OPTIONS`;
`NO_FOLLOW_FLAGS` is the mirror, for a following command that a flag makes lstat
@@ -301,6 +370,73 @@ Invoke the venv's `pre-commit` binary directly (not via `uv --directory python r
were reported as directories, so `find -type f` missed them). Do not
reintroduce name-based classification in a backend; if stat misclassifies an
entry, fix that backend's stat.
- **An archiver walks a directory operand; it does not read it.** `tar`
and `zip` decide every member first (`plan_create` / `planCreate` in
`generic/tar/create.*`, `plan_zip` / `planZip` in
`generic/zip_cmd.*`) and only then write, which is what lets an
exclusion prune a whole subtree and keeps the ordering stable. Both
plans are built on **one traversal**, `scan_operand` / `scanOperand`
(`generic/archive/walk.*`), which merges three sources no single one
can see: the backend walk (reusing find's `walk_find` / `walkFind`, so
an archiver classifies an entry through `stat` exactly as find does,
never by name), the namespace's symlinks, and the mount table. It
reports paths, never names, because naming is exactly where the two
formats disagree; the two things they disagree about in the traversal
itself are parameters (`dereference`, `recurse`), so **a third
archiver adds a caller, not a second walk**. Members are named from
`PathSpec.raw_path`, so `tar -C d x` stores `x`, not `d/x`.
**A directory is its own member**, with GNU's trailing slash and no
content, which is the only record an empty directory leaves and the
reason extraction has to `mkdir` for one. **A symlink is a symlink
member** (`SYMTYPE`, target in `linkname`), never a file of its
target's bytes, unless `-h` says to follow it.
**Two links to one target are not a loop**, and both are archived; the
only loop is one `resolve` refuses to resolve, since the namespace
already walks the chain under a hop limit and raises `CycleError` at
the end of it. That arrives as a fatal `Problem` carrying GNU's
`Too many levels of symbolic links`, reported per member with the
directory entry kept, rather than as an exception that aborts the
plan. **Every `-C` is checked, not just the last**: GNU chdirs at each
one and fails at the first it cannot enter, so the option accumulates
(`multiple=True`) and the planner walks the list.
Two deliberate divergences from GNU, both documented in place:
siblings are sorted rather than emitted in readdir order (the same
choice `du` makes, for the same reason), and a descendant mount is
never crossed (see "Mount boundaries"). Everything else is pinned
against GNU tar 1.35 on `debian:stable-slim`: the leading-slash
warning, `Cowardly refusing to create an empty archive` (exit 2), a
per-operand `Cannot stat` plus one trailer (exit 2, and the other
operands still archive), a `-C` it cannot enter (exit 2, no archive
written), and `archive cannot contain itself; not dumped` (exit 0).
A backend error must never reach the user as itself: an unreadable
operand is reported in virtual path space with tar's wording, because
the raw `IsADirectoryError` leaked the host path behind a disk mount.
- **`zip` is Info-ZIP, which inverts tar's two defaults.** A directory
operand contributes only its own entry unless `-r` says to descend,
and a symlink is *followed* unless `-y` says to store the link, where
tar always descends and always stores unless `-h`. Both are just the
`recurse` / `dereference` arguments to the shared scan. The rest is
pinned against Info-ZIP 3.0 on `debian:stable-slim`: a leading slash
is stripped **in silence** (tar warns, zip does not), `-j` junks to
the basename and drops directory entries entirely, `-x` is
**anchored** on the whole stored name (`d/sub/*` matches, `sub/*` does
not) where tar's `--exclude` is unanchored, an unreachable operand is
`\tzip warning: name not matched: <name>` and does not stop the run,
and a run that matched nothing prints `zip error: Nothing to do!`,
exits **12**, and writes no archive. `-q` silences the warnings but
never that error. Two deliberate divergences: `-x` takes one pattern
per occurrence (mirage's spec has no variadic option value, and
`-x a -x b` says the same thing), and the `adding:` line carries no
`(deflated N%)` suffix, since the ratio depends on the compressor and
would differ between the two languages.
- **The TypeScript `walkFind` answers in mount-relative keys; the Python
`walk_find` answers in virtual paths.** TS's stands in for a backend's
native find op, so a caller that needs virtual paths (tar does, to
name members and compare against mount prefixes) lifts them with
`mountPrefixOf` the way `findGeneric` does. That lift lives once, in
`generic_bind/archive_io.*`, which is where both archivers get their
walk. This asymmetry is real and has bitten once: a unit test on an
unprefixed mount cannot see it, so cover a prefixed mount too.
- **`du` has one backend contract: `size` and `entries`.** Each backend exposes `core/<backend>/du/size.py` (recursive byte total for one path) and `core/<backend>/du/entries.py` (per-file breakdown), wired as `du_size` / `du_entries` on the adapter. `entries` returns `(entries, total)` where entries are **leaf files only, in mount-relative path space, with no summary row**; the generic lifts them onto virtual paths (`to_virtual`, via `mount_prefix_of`) and re-spells them as the operand was typed (`respell_raw`). A backend that returns backend-key paths, or appends its own roll-up row, makes two mounts holding the same filename render identical lines. Do not reintroduce a second shape; the old flat-list `du_multi` contract is gone.
- **`du` prints a line per directory, derived not walked.** GNU prints one line per directory with its recursive total, post-order (children before parents), plus one per file under `-a`. Backends only ever report leaf files, so the generic derives the directory rows by summing each leaf into every ancestor (`rollup`, same name both languages), then emits post-order with siblings sorted. Two deliberate divergences: GNU orders siblings by `readdir` (filesystem-dependent), mirage sorts them; and an empty directory is invisible to mirage because no leaf points at it. Sizes are bytes, not GNU's 1 KiB blocks, since an object store has no block size. `--max-depth` prunes only what is printed, never the walk, because every printed total still covers the whole subtree. Verify changes with the differential harness against `debian:stable-slim`: paths, exit codes and stderr must match GNU exactly.
- **`du` usage errors exit 1, not 2.** `du` is absent from `USAGE_EXIT`, which is correct: GNU du exits 1 for `-s` with `-a` ("cannot both summarize and show all entries"), `-s` with `--max-depth` ("warning: summarizing conflicts with --max-depth=N"), and a bad depth ("invalid maximum depth 'x'"). All three are raised by `parse_flags` / `parseDuFlags` *before* any I/O, mirroring GNU's option-parse order: the depth is parsed as the option is read, so a bad depth wins over the conflict checks. An unreadable operand is not a usage error: GNU names it (`du: cannot access 'x': No such file or directory`), prints every other operand, and exits 1, and still prints `0 total` under `-c` when every operand failed. With no operand at all, du measures the working directory; it never says "missing operand".
integ/unix/arch/tar.json
+311 -6
View File
@@ -31,7 +31,7 @@
"expect": {
"exit": 0,
"stdout": "data/arch/g.txt\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -133,7 +133,7 @@
"expect": {
"exit": 0,
"stdout": "data/arch/g.txt\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -149,7 +149,7 @@
"expect": {
"exit": 0,
"stdout": "data/otar/g.txt\notar-data\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -165,7 +165,7 @@
"expect": {
"exit": 0,
"stdout": "data/otar2/h.txt\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -181,7 +181,7 @@
"expect": {
"exit": 0,
"stdout": "three\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -197,7 +197,7 @@
"expect": {
"exit": 0,
"stdout": "data/otar4/j.txt\n",
"stderr": ""
"stderr": "tar: Removing leading `/' from member names\n"
}
},
{
@@ -231,6 +231,311 @@
"stdout": "",
"stderr": "tar: invalid option -- 'Q'\nTry 'tar --help' for more information."
}
},
{
"id": "arch_tar_create_directory",
"seq": 910063,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/tdir/sub && echo aa | tee /data/tdir/a.txt > /dev/null && echo bb | tee /data/tdir/sub/b.txt > /dev/null && tar -c -v -f /data/tdir.tar -C /data tdir && tar -t -f /data/tdir.tar",
"expect": {
"exit": 0,
"stdout": "tdir/\ntdir/a.txt\ntdir/sub/\ntdir/sub/b.txt\ntdir/\ntdir/a.txt\ntdir/sub/\ntdir/sub/b.txt\n",
"stderr": ""
},
"flags": [
"c",
"C",
"v",
"t"
]
},
{
"id": "arch_tar_dash_c_bases_the_operand",
"seq": 910064,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/tcin/paper && echo p | tee /data/tcin/paper/p.txt > /dev/null && tar -c -z -f /data/tc.tgz -C /data/tcin paper && tar -t -z -f /data/tc.tgz",
"expect": {
"exit": 0,
"stdout": "paper/\npaper/p.txt\n",
"stderr": ""
},
"flags": [
"c",
"C",
"z",
"t"
]
},
{
"id": "arch_tar_directory_round_trips_through_extract",
"seq": 910065,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/trt/sub && echo rt | tee /data/trt/sub/r.txt > /dev/null && tar -c -f /data/trt.tar -C /data trt && tar -x -f /data/trt.tar -C /data/trtout && cat /data/trtout/trt/sub/r.txt",
"expect": {
"exit": 0,
"stdout": "rt\n",
"stderr": ""
},
"flags": [
"c",
"x",
"C"
]
},
{
"id": "arch_tar_exclude_prunes_the_subtree",
"seq": 910066,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/tex/sub && echo a | tee /data/tex/a.txt > /dev/null && echo b | tee /data/tex/sub/b.txt > /dev/null && tar -c -f /data/tex.tar --exclude sub -C /data tex && tar -t -f /data/tex.tar",
"expect": {
"exit": 0,
"stdout": "tex/\ntex/a.txt\n",
"stderr": ""
},
"flags": [
"c",
"C",
"t",
"exclude"
]
},
{
"id": "arch_tar_missing_operand_exits_two",
"seq": 910067,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/tmiss && tar -c -f /data/tmiss.tar -C /data/tmiss nope.txt",
"expect": {
"exit": 2,
"stdout": "",
"stderr": "tar: nope.txt: Cannot stat: No such file or directory\ntar: Exiting with failure status due to previous errors\n"
},
"flags": [
"c",
"C"
]
},
{
"id": "arch_tar_refuses_an_empty_archive",
"seq": 910068,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "tar -c -f /data/tempty.tar",
"expect": {
"exit": 2,
"stdout": "",
"stderr": "tar: Cowardly refusing to create an empty archive\nTry 'tar --help' for more information.\n"
},
"flags": [
"c"
]
},
{
"id": "arch_tar_refuses_a_directory_it_cannot_enter",
"seq": 910069,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "tar -c -f /data/tnodir.tar -C /data/tnodir a.txt",
"expect": {
"exit": 2,
"stdout": "",
"stderr": "tar: /data/tnodir: Cannot open: No such file or directory\ntar: Error is not recoverable: exiting now\n"
},
"flags": [
"c",
"C"
]
},
{
"id": "arch_tar_refuses_a_mount_root_operand",
"seq": 910070,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "tar -c -f /data/tmount.tar /data",
"expect": {
"exit": 2,
"stdout": "",
"stderr": "tar: /data: Cannot open: Device or resource busy\ntar: Error is not recoverable: exiting now\n"
},
"flags": [
"c"
]
},
{
"id": "arch_zip_refuses_a_mount_root_operand",
"seq": 910071,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "zip -r /data/zmount.zip /data",
"expect": {
"exit": 1,
"stdout": "",
"stderr": "zip: cannot read '/data': Device or resource busy\n"
},
"flags": [
"r"
]
},
{
"id": "arch_cp_refuses_a_mount_root_source",
"seq": 910072,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/cpdst && cp -r /data /data/cpdst",
"expect": {
"exit": 1,
"stdout": "",
"stderr": "cp: cannot copy '/data': Device or resource busy\n"
},
"flags": [
"r"
]
},
{
"id": "arch_tar_stores_a_symlink_operand_as_a_link",
"seq": 910085,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/sl && echo alpha > /data/sl/a.txt && ln -s /data/sl/a.txt /data/sl/l && tar -cf /data/sl.tar -C /data/sl l && tar -xf /data/sl.tar -C /data/sl/out 2>/dev/null; readlink /data/sl/l",
"expect": {
"exit": 0,
"stdout": "/data/sl/a.txt\n",
"stderr": ""
},
"flags": [
"c",
"C"
]
},
{
"id": "arch_tar_two_links_to_one_target_are_not_a_loop",
"seq": 910086,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/tl && echo beta > /data/tl/b.txt && ln -s /data/tl/b.txt /data/tl/one && ln -s /data/tl/b.txt /data/tl/two && tar -chf /data/tl.tar -C /data/tl . 2>/dev/null; tar -chf /data/tl2.tar -C /data tl && tar -tf /data/tl2.tar",
"expect": {
"exit": 0,
"stdout": "tl/\ntl/b.txt\ntl/one\ntl/two\n",
"stderr": ""
},
"flags": [
"c",
"h",
"C"
]
},
{
"id": "arch_tar_reports_a_symlink_cycle_per_member",
"seq": 910087,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/cy && ln -s /data/cy/b /data/cy/a && ln -s /data/cy/a /data/cy/b && tar -chf /data/cy.tar -C /data cy; echo exit=$?; tar -tf /data/cy.tar",
"expect": {
"exit": 0,
"stdout": "exit=2\ncy/\n",
"stderr": "tar: cy/a: Cannot stat: Too many levels of symbolic links\ntar: cy/b: Cannot stat: Too many levels of symbolic links\ntar: Exiting with failure status due to previous errors\n"
},
"flags": [
"c",
"h",
"C"
]
},
{
"id": "arch_tar_fails_at_the_first_unenterable_c",
"seq": 910088,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/cgood && echo y > /data/cgood/y.txt && tar -cf /data/cc.tar -C /data/cmissing x -C /data/cgood y.txt; echo exit=$?; ls /data/cc.tar",
"expect": {
"exit": 2,
"stdout": "exit=2\n",
"stderr": "tar: /data/cmissing: Cannot open: No such file or directory\ntar: Error is not recoverable: exiting now\nls: cannot access '/data/cc.tar': No such file or directory\n"
},
"flags": [
"c",
"C"
]
},
{
"id": "arch_tar_listing_is_not_a_mount_root_source",
"seq": 910089,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/ml && echo m > /data/ml/m.txt && tar -cf /data/ml.tar -C /data ml && tar -tf /data/ml.tar data; echo exit=$?",
"expect": {
"exit": 0,
"stdout": "ml/\nml/m.txt\nexit=0\n",
"stderr": ""
},
"flags": [
"c",
"C"
]
}
]
}
integ/unix/arch/zip.json
+222
View File
@@ -33,6 +33,228 @@
"stdout": "gz-data\n",
"stderr": ""
}
},
{
"id": "arch_zip_recurses_a_directory",
"seq": 910073,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zdir/sub && echo one > /data/zdir/a.txt && echo two > /data/zdir/sub/b.txt && zip -r /data/zr.zip /data/zdir && unzip -l /data/zr.zip",
"expect": {
"exit": 0,
"stdout": " adding: data/zdir/\n adding: data/zdir/a.txt\n adding: data/zdir/sub/\n adding: data/zdir/sub/b.txt\n Length Name\n--------- ----\n 0 data/zdir/\n 4 data/zdir/a.txt\n 0 data/zdir/sub/\n 4 data/zdir/sub/b.txt\n",
"stderr": ""
},
"flags": [
"r"
]
},
{
"id": "arch_zip_without_r_stores_only_the_directory",
"seq": 910074,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/znor && echo one > /data/znor/a.txt && zip /data/znor.zip /data/znor && unzip -l /data/znor.zip",
"expect": {
"exit": 0,
"stdout": " adding: data/znor/\n Length Name\n--------- ----\n 0 data/znor/\n",
"stderr": ""
}
},
{
"id": "arch_zip_keeps_an_empty_directory",
"seq": 910075,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zempty/hollow && echo one > /data/zempty/a.txt && zip -q -r /data/ze.zip /data/zempty && mkdir -p /data/zeout && unzip -q -d /data/zeout /data/ze.zip && find /data/zeout -type d | sort",
"expect": {
"exit": 0,
"stdout": "/data/zeout\n/data/zeout/data\n/data/zeout/data/zempty\n/data/zeout/data/zempty/hollow\n",
"stderr": ""
},
"flags": [
"r"
]
},
{
"id": "arch_zip_round_trips_a_tree",
"seq": 910076,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zrt/sub && echo alpha > /data/zrt/a.txt && echo beta > /data/zrt/sub/b.txt && zip -q -r /data/zrt.zip /data/zrt && mkdir -p /data/zrtout && unzip -q -d /data/zrtout /data/zrt.zip && cat /data/zrtout/data/zrt/a.txt /data/zrtout/data/zrt/sub/b.txt",
"expect": {
"exit": 0,
"stdout": "alpha\nbeta\n",
"stderr": ""
},
"flags": [
"r"
]
},
{
"id": "arch_zip_junks_paths_and_drops_directories",
"seq": 910077,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zjunk/sub && echo one > /data/zjunk/a.txt && echo two > /data/zjunk/sub/b.txt && zip -r -j /data/zj.zip /data/zjunk && unzip -l /data/zj.zip",
"expect": {
"exit": 0,
"stdout": " adding: a.txt\n adding: b.txt\n Length Name\n--------- ----\n 4 a.txt\n 4 b.txt\n",
"stderr": ""
},
"flags": [
"r",
"j"
]
},
{
"id": "arch_zip_excludes_a_subtree",
"seq": 910078,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zx/sub && echo one > /data/zx/a.txt && echo two > /data/zx/sub/b.txt && zip -r /data/zx.zip /data/zx -x 'data/zx/sub/*'",
"expect": {
"exit": 0,
"stdout": " adding: data/zx/\n adding: data/zx/a.txt\n",
"stderr": ""
},
"flags": [
"r",
"x"
]
},
{
"id": "arch_zip_warns_on_a_name_it_cannot_match",
"seq": 910079,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zmiss && echo one > /data/zmiss/a.txt && zip /data/zm.zip /data/zmiss/a.txt /data/zmiss/nope.txt",
"expect": {
"exit": 0,
"stdout": " adding: data/zmiss/a.txt\n",
"stderr": "\tzip warning: name not matched: /data/zmiss/nope.txt\n"
}
},
{
"id": "arch_zip_refuses_an_empty_archive",
"seq": 910080,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "zip /data/znothing.zip /data/no-such-file; echo exit=$?; ls /data/znothing.zip",
"expect": {
"exit": 2,
"stdout": "exit=12\n",
"stderr": "\tzip warning: name not matched: /data/no-such-file\n\nzip error: Nothing to do! (/data/znothing.zip)\nls: cannot access '/data/znothing.zip': No such file or directory\n"
}
},
{
"id": "arch_zip_quiet_keeps_the_fatal_error",
"seq": 910081,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "zip -q /data/zq.zip /data/no-such-file",
"expect": {
"exit": 12,
"stdout": "",
"stderr": "\nzip error: Nothing to do! (/data/zq.zip)\n"
},
"flags": [
"q"
]
},
{
"id": "arch_zip_leaves_itself_out",
"seq": 910082,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zself && echo one > /data/zself/a.txt && zip -q -r /data/zself/self.zip /data/zself && unzip -l /data/zself/self.zip",
"expect": {
"exit": 0,
"stdout": " Length Name\n--------- ----\n 0 data/zself/\n 4 data/zself/a.txt\n",
"stderr": ""
},
"flags": [
"r"
]
},
{
"id": "arch_zip_follows_a_symlink_by_default",
"seq": 910083,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zsym && echo linked > /data/zsym/a.txt && ln -s /data/zsym/a.txt /data/zsym/l.txt && zip -q -r /data/zsym.zip /data/zsym && unzip -p /data/zsym.zip 'data/zsym/l.txt'",
"expect": {
"exit": 0,
"stdout": "linked\n",
"stderr": ""
},
"flags": [
"r"
]
},
{
"id": "arch_zip_y_stores_the_link_target_instead",
"seq": 910084,
"targets": [
"ram",
"disk",
"redis",
"opfs"
],
"command": "mkdir -p /data/zsy && echo linked > /data/zsy/a.txt && ln -s /data/zsy/a.txt /data/zsy/l.txt && zip -q -r -y /data/zsy.zip /data/zsy && unzip -p /data/zsy.zip 'data/zsy/l.txt'",
"expect": {
"exit": 0,
"stdout": "/data/zsy/a.txt",
"stderr": ""
},
"flags": [
"r",
"y"
]
}
]
}
python/mirage/commands/builtin/generic/archive/__init__.py
View File
python/mirage/commands/builtin/generic/archive/types.py
+73
View File
@@ -0,0 +1,73 @@
from dataclasses import dataclass
from typing import Literal, TypeAlias
from mirage.types import PathSpec
# What a member is, which is the whole of what an archive records beyond
# its name: a regular file carries bytes, a directory carries none and
# ends in a slash, a symlink carries its target string instead.
MemberKind: TypeAlias = Literal["file", "dir", "link"]
@dataclass(frozen=True, slots=True)
class Entry:
"""One thing found under an operand, before it is named or filtered.
``name_path`` and ``read`` are two different paths whenever a link is
being followed: the member keeps the link's own name while its bytes
come from the target.
Args:
name_path (str): absolute virtual path the member is named
after, before it is respelled and stripped.
kind (MemberKind): file, dir, or link.
target (str): a symlink's target, verbatim as stored.
read (PathSpec | None): where a file's bytes come from.
"""
name_path: str
kind: MemberKind
target: str = ""
read: PathSpec | None = None
@dataclass(frozen=True, slots=True)
class Problem:
"""One thing the scan could not archive, in the order it was met.
Args:
path (str): the absolute virtual path it happened at.
reason (str): why, empty when ``fatal`` says the path could not
be stat'd at all and each archiver has its own wording.
fatal (bool): whether the path was unreachable rather than
merely skipped, which is what decides the exit code.
"""
path: str
reason: str = ""
fatal: bool = False
@dataclass(frozen=True, slots=True)
class Scan:
"""What one operand contributed, in virtual path space.
Nothing here is named yet: naming is where the two archive formats
part company (tar warns about a leading slash, zip strips it in
silence and can junk the path entirely), so the scan reports paths
and the caller spells them.
Args:
entries (tuple[Entry, ...]): the operand and its descendants.
crossings (tuple[str, ...]): virtual path of each descendant
mount whose contents the walk refused to cross into.
problems (tuple[Problem, ...]): what was skipped and why, in
walk order.
missing (bool): whether the operand itself was unreachable, in
which case it contributed no entries.
"""
entries: tuple[Entry, ...] = ()
crossings: tuple[str, ...] = ()
problems: tuple[Problem, ...] = ()
missing: bool = False
python/mirage/commands/builtin/generic/archive/walk.py
+266
View File
@@ -0,0 +1,266 @@
from collections.abc import Awaitable, Callable
from mirage.commands.builtin.generic.archive.types import (Entry, MemberKind,
Problem, Scan)
from mirage.ops.types import LinkView, MountView
from mirage.types import LINK_TARGET_KEY, FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
from mirage.utils.path import CycleError
# A mount boundary is a filesystem boundary, so both archivers stop at
# one and say so in GNU tar's --one-file-system wording. Descending would
# archive by accident exactly what the mount-root refusal forbids on
# purpose.
OTHER_FILESYSTEM = "file is on a different filesystem; not dumped"
# Why a path could not be reached, in GNU's strerror wording. Both ride
# on a fatal Problem; tar prints them after "Cannot stat: " and Info-ZIP
# words every unreachable name the same way, so it ignores the reason.
NO_SUCH = "No such file or directory"
TOO_MANY_LEVELS = "Too many levels of symbolic links"
StatFn = Callable[[PathSpec], Awaitable[FileStat]]
WalkFn = Callable[[PathSpec, str], Awaitable[list[str]]]
DirProbe = Callable[[PathSpec], Awaitable[bool]]
def link_target(stat: FileStat) -> str:
target = stat.extra.get(LINK_TARGET_KEY)
return target if isinstance(target, str) else ""
def child_spec(virtual: str, root: PathSpec) -> PathSpec:
"""A PathSpec for a walked descendant of one operand.
The walk reports absolute virtual paths; reading their bytes needs
the backend key too, which is the virtual path with this mount's
prefix removed.
Args:
virtual (str): the descendant's absolute virtual path.
root (PathSpec): the operand it was walked from.
"""
cut = len(root.virtual.rstrip("/")) - len(root.resource_path.strip("/"))
prefix = root.virtual[:cut].rstrip("/")
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1] or "/",
resource_path=mount_key(virtual, prefix),
raw_path=virtual)
def same_mount(mounts: MountView | None, one: str, other: str) -> bool:
"""Whether two virtual paths are served by the same mount.
Args:
mounts (MountView | None): where the mount boundaries are;
None (outside a workspace) means there is only one mount.
one (str): a virtual path.
other (str): another virtual path.
"""
if mounts is None:
return True
return mounts.root_of(one) == mounts.root_of(other)
async def subtree(
root: PathSpec,
base: str,
name_base: str,
walk: WalkFn,
links: LinkView | None,
mounts: MountView | None,
) -> tuple[list[Entry], list[str]]:
"""Every entry under one directory, named under ``name_base``.
Three sources have to be merged because no single one can see them
all: the backend walk (files and directories), the namespace (its
symlinks, which no backend readdir reports), and the mount table (a
nested mount, whose keys live in another resource entirely).
``base`` and ``name_base`` differ only when a link is being followed:
the walk runs over the target while the members keep the link's own
name.
Args:
root (PathSpec): the operand, for backend keys.
base (str): absolute virtual path actually walked.
name_base (str): absolute virtual path the members are named
under.
walk (WalkFn): subtree listing, by find type.
links (LinkView | None): the namespace's symlink facts.
mounts (MountView | None): where the mount boundaries are.
Returns:
tuple: the entries, and the virtual path of each mount root that
stopped the walk.
"""
walked = child_spec(base, root) if base != root.virtual else root
found: dict[str, tuple[MemberKind, str]] = {}
for virtual in await walk(walked, "d"):
if virtual.rstrip("/") != base:
found[virtual.rstrip("/")] = ("dir", "")
for virtual in await walk(walked, "f"):
found[virtual.rstrip("/")] = ("file", "")
if links is not None:
for virtual, stat in links.subtree(base):
found[virtual.rstrip("/")] = ("link", link_target(stat))
crossings = mounts.descendants(base) if mounts is not None else []
for crossing in crossings:
# The mountpoint itself is still an entry, exactly as GNU's
# --one-file-system keeps the directory and drops its contents.
found[crossing.rstrip("/")] = ("dir", "")
below = [c.rstrip("/") + "/" for c in crossings]
entries: list[Entry] = []
for virtual, (kind, target) in found.items():
if any(virtual.startswith(c) for c in below):
continue
named = name_base.rstrip("/") + virtual[len(base.rstrip("/")):]
entries.append(
Entry(name_path=named,
kind=kind,
target=target,
read=child_spec(virtual, root) if kind == "file" else None))
entries.sort(key=lambda entry: entry.name_path)
return entries, [c.rstrip("/") for c in crossings]
async def follow(
virtual: str,
root: PathSpec,
stat: StatFn,
walk: WalkFn,
links: LinkView | None,
mounts: MountView | None,
recurse: bool,
) -> tuple[list[Entry], list[str], str]:
"""What dereferencing puts in the archive in place of one symlink.
The member keeps the link's own name and takes the target's content,
which is what dereferencing means. Two links resolving to the same
file are not a loop and both are archived; a real loop is whatever
``resolve`` refuses to resolve, since the namespace already walks
the chain under a hop limit and raises ELOOP at the end of it.
Args:
virtual (str): the link's absolute virtual path.
root (PathSpec): the operand, for backend keys.
stat (StatFn): backend stat.
walk (WalkFn): subtree listing, by find type.
links (LinkView | None): the namespace's symlink facts.
mounts (MountView | None): where the mount boundaries are.
recurse (bool): whether a target directory contributes its
contents as well as itself.
Returns:
tuple: the entries, why anything was skipped, and why the link
was unreachable at all (empty when it was reached).
"""
if links is None:
return [], [], ""
try:
target = links.resolve(virtual)
except CycleError:
return [], [], TOO_MANY_LEVELS
if not same_mount(mounts, virtual, target):
return [], [OTHER_FILESYSTEM], ""
spec = child_spec(target, root)
try:
target_stat = await stat(spec)
except (FileNotFoundError, ValueError):
return [], [], NO_SUCH
if target_stat.type != FileType.DIRECTORY:
return [Entry(name_path=virtual, kind="file", read=spec)], [], ""
if not recurse:
return [Entry(name_path=virtual, kind="dir")], [], ""
entries, crossings = await subtree(root, target, virtual, walk, links,
mounts)
reasons = [OTHER_FILESYSTEM] * len(crossings)
return [Entry(name_path=virtual, kind="dir"), *entries], reasons, ""
async def scan_operand(
path: PathSpec,
*,
stat: StatFn,
walk: WalkFn,
links: LinkView | None = None,
mounts: MountView | None = None,
dereference: bool = False,
recurse: bool = True,
) -> Scan:
"""Everything one operand contributes to an archive.
This is the whole of what tar and zip share: which paths go in, what
each one is, and which of them could not be reached. The two formats
disagree about the defaults, not the traversal, so both are
parameters: tar stores a symlink unless ``-h`` says to follow it and
always descends, zip follows unless ``-y`` says otherwise and only
descends under ``-r``.
Args:
path (PathSpec): the operand, glob-resolved and already re-based
by any directory option.
stat (StatFn): backend stat, raising when nothing is there.
walk (WalkFn): subtree listing, by find type.
links (LinkView | None): the namespace's symlink facts.
mounts (MountView | None): where the mount boundaries are.
dereference (bool): archive what a symlink points at rather than
the link.
recurse (bool): whether a directory contributes its contents as
well as itself.
"""
base = path.virtual.rstrip("/") or "/"
entries: list[Entry] = []
crossings: list[str] = []
problems: list[Problem] = []
link_stat = links.stat_at(path.virtual) if links is not None else None
if link_stat is not None and not dereference:
entries.append(
Entry(name_path=base, kind="link", target=link_target(link_stat)))
elif link_stat is not None:
followed, why, unreachable = await follow(base, path, stat, walk,
links, mounts, recurse)
if unreachable:
return Scan(problems=(Problem(path=base,
reason=unreachable,
fatal=True), ),
missing=True)
entries.extend(followed)
problems.extend(Problem(path=base, reason=reason) for reason in why)
else:
try:
root_stat = await stat(path)
except (FileNotFoundError, ValueError):
return Scan(problems=(Problem(path=base,
reason=NO_SUCH,
fatal=True), ),
missing=True)
if root_stat.type != FileType.DIRECTORY:
entries.append(Entry(name_path=base, kind="file", read=path))
else:
entries.append(Entry(name_path=base, kind="dir"))
if recurse:
below, crossings = await subtree(path, base, base, walk, links,
mounts)
entries.extend(below)
if dereference and links is not None:
expanded: list[Entry] = []
for entry in entries:
if entry.kind != "link":
expanded.append(entry)
continue
followed, why, unreachable = await follow(entry.name_path, path,
stat, walk, links,
mounts, recurse)
if unreachable:
problems.append(
Problem(path=entry.name_path,
reason=unreachable,
fatal=True))
continue
expanded.extend(followed)
problems.extend(
Problem(path=entry.name_path, reason=reason) for reason in why)
entries = expanded
return Scan(entries=tuple(entries),
crossings=tuple(crossings),
problems=tuple(problems))
python/mirage/commands/builtin/generic/tar/__init__.py
+9
View File
@@ -1,14 +1,23 @@
from mirage.commands.builtin.generic.tar.constants import (READ_MODES,
WRITE_MODES)
from mirage.commands.builtin.generic.tar.create import (excluded, member_name,
plan_create, pruned)
from mirage.commands.builtin.generic.tar.tar import tar
from mirage.commands.builtin.generic.tar.types import (CompressionSuffix,
CreateResult, Member,
ReadMode, WriteMode)
__all__ = [
"CompressionSuffix",
"CreateResult",
"Member",
"READ_MODES",
"ReadMode",
"WRITE_MODES",
"WriteMode",
"excluded",
"member_name",
"plan_create",
"pruned",
"tar",
]
python/mirage/commands/builtin/generic/tar/create.py
+205
View File
@@ -0,0 +1,205 @@
from mirage.commands.builtin.generic.archive.types import MemberKind
from mirage.commands.builtin.generic.archive.walk import (OTHER_FILESYSTEM,
DirProbe, StatFn,
WalkFn, scan_operand)
from mirage.commands.builtin.generic.tar.types import CreateResult, Member
from mirage.ops.types import LinkView, MountView
from mirage.types import PathSpec
from mirage.utils.fnmatch import fnmatch
from mirage.utils.path import respell_one
# Every diagnostic below is GNU tar 1.35's own wording, pinned on
# debian:stable-slim; only the hint line is mirage's, for the reason
# usage.old_option_error gives (mirage's tar serves no --usage).
USAGE_HINT = "Try 'tar --help' for more information."
EMPTY_ARCHIVE = "tar: Cowardly refusing to create an empty archive"
FATAL_TRAILER = "tar: Error is not recoverable: exiting now"
ERROR_TRAILER = "tar: Exiting with failure status due to previous errors"
LEADING_SLASH = "tar: Removing leading `/' from member names"
SELF_DUMP = "archive cannot contain itself; not dumped"
# The exit GNU gives an operand it could not read, and a -C it could not
# enter. Both are fatal for the whole run, not per-operand.
CREATE_ERROR_EXIT = 2
def _refusal(notices: list[str]) -> CreateResult:
return CreateResult(members=(),
notices=tuple(notices),
exit_code=CREATE_ERROR_EXIT,
write=False)
def excluded(name: str, pattern: str) -> bool:
"""Whether GNU's ``--exclude`` pattern matches this member name.
GNU's exclusion is unanchored: the pattern is tried against the whole
name and against every suffix that starts at a path component, so
``a.txt``, ``d/a.txt`` and ``sub/b.txt`` all match entries under
``d``. Wildcards cross slashes (``*/b.txt`` matches ``d/sub/b.txt``),
which is tar's default for exclusion patterns. A directory's
trailing slash is not part of what the pattern sees. Info-ZIP's
``-x`` is the anchored counterpart, which is why the two are not
shared.
Args:
name (str): the member name, with or without a trailing slash.
pattern (str): the raw ``--exclude`` value.
"""
bare = name.rstrip("/")
if fnmatch(bare, pattern):
return True
cut = bare.find("/")
while cut != -1:
if fnmatch(bare[cut + 1:], pattern):
return True
cut = bare.find("/", cut + 1)
return False
def pruned(names: list[str], pattern: str | None) -> list[str]:
"""Drop excluded names and everything beneath an excluded directory.
GNU does not walk into a directory it excluded, so ``--exclude sub``
takes ``d/sub/`` and ``d/sub/b.txt`` together. Matching each name in
isolation would keep the children of a pruned directory.
Args:
names (list[str]): member names in walk order.
pattern (str | None): the raw ``--exclude`` value, or None.
"""
if pattern is None:
return names
kept: list[str] = []
cut_dirs: list[str] = []
for name in names:
if any(name.startswith(cut) for cut in cut_dirs):
continue
if excluded(name, pattern):
if name.endswith("/"):
cut_dirs.append(name)
continue
kept.append(name)
return kept
def member_name(spelled: str, kind: MemberKind) -> str:
"""The name tar records for a path spelled as the operand was typed.
A leading slash is stripped (tar refuses to store absolute names, and
says so once per run), and a directory carries the trailing slash
that tells an extractor it holds no content.
Args:
spelled (str): the path as the operand spelled it.
kind (MemberKind): what the entry is.
"""
name = spelled.lstrip("/")
if kind == "dir" and name and not name.endswith("/"):
return name + "/"
return name
async def plan_create(
paths: list[PathSpec],
*,
archive: PathSpec,
exclude: str | None,
dereference: bool,
stat: StatFn,
walk: WalkFn,
is_dir: DirProbe,
directories: list[PathSpec] | None = None,
links: LinkView | None = None,
mounts: MountView | None = None,
) -> CreateResult:
"""Decide every member of a new archive, before writing any of it.
One pass per operand, in the order they were typed, each
contributing itself and then its subtree. GNU walks a directory
operand rather than refusing it, and mirage now does too; the one
deliberate divergence is ordering, since GNU emits siblings in
readdir order (filesystem-dependent) and this sorts them, the same
choice ``du`` already documents.
Args:
paths (list[PathSpec]): the operands, glob-resolved and already
re-based by any ``-C``.
archive (PathSpec): the ``-f`` target, so it can be left out of
itself.
exclude (str | None): the raw ``--exclude`` value.
dereference (bool): ``-h``, archive what a symlink points at.
stat (StatFn): backend stat, raising when nothing is there.
walk (WalkFn): subtree listing, by find type.
is_dir (DirProbe): whether a ``-C`` can be entered.
directories (list[PathSpec] | None): every ``-C`` the operands
were based on, in order, checked here because GNU chdirs at
each one before reading anything.
links (LinkView | None): the namespace's symlink facts.
mounts (MountView | None): where the mount boundaries are.
"""
if not paths:
return _refusal([EMPTY_ARCHIVE, USAGE_HINT])
for directory in directories or []:
# GNU chdirs at each -C in turn, before reading a single
# operand, so the FIRST one it cannot enter is fatal for the
# whole run and no members are written. Checking only the last
# would archive the operands that followed a bad earlier one.
if not await is_dir(directory):
return _refusal([
f"tar: {directory.raw_path}: Cannot open: "
"No such file or directory", FATAL_TRAILER
])
members: list[Member] = []
notices: list[str] = []
absolute_seen = False
exit_code = 0
for path in paths:
raw = path.raw_path
base = path.virtual.rstrip("/") or "/"
scan = await scan_operand(path,
stat=stat,
walk=walk,
links=links,
mounts=mounts,
dereference=dereference,
recurse=True)
for problem in scan.problems:
shown = respell_one(problem.path, base, raw)
if not problem.fatal:
notices.append(f"tar: {shown}: {problem.reason}")
continue
notices.append(f"tar: {shown}: Cannot stat: {problem.reason}")
exit_code = CREATE_ERROR_EXIT
if scan.missing:
continue
for crossing in scan.crossings:
shown = member_name(respell_one(crossing, base, raw), "dir")
notices.append(f"tar: {shown}: {OTHER_FILESYSTEM}")
# Every descendant is spelled under the operand's own base, so
# the operand alone decides whether this run stored an absolute
# name and owes GNU's one-per-run warning.
absolute_seen = absolute_seen or raw.startswith("/")
named = [(member_name(respell_one(entry.name_path, base, raw),
entry.kind), entry) for entry in scan.entries]
keep = set(pruned([name for name, _ in named], exclude))
for name, entry in named:
if name not in keep:
continue
read = entry.read
if read is not None and read.virtual == archive.virtual:
notices.append(f"tar: {name}: {SELF_DUMP}")
continue
members.append(
Member(name=name,
kind=entry.kind,
path=entry.read,
target=entry.target))
if absolute_seen:
notices.insert(0, LEADING_SLASH)
if exit_code:
# GNU closes a run that failed an operand with one trailer, after
# everything it did manage to name.
notices.append(ERROR_TRAILER)
return CreateResult(members=tuple(members),
notices=tuple(notices),
exit_code=exit_code)
python/mirage/commands/builtin/generic/tar/tar.py
+84 -30
View File
@@ -2,18 +2,17 @@ import io
import tarfile
from collections.abc import Awaitable, Callable
from mirage.commands.builtin.generic.archive.walk import (DirProbe, StatFn,
WalkFn)
from mirage.commands.builtin.generic.tar.constants import (READ_MODES,
WRITE_MODES)
from mirage.commands.builtin.generic.tar.create import plan_create
from mirage.commands.builtin.generic.tar.types import (CompressionSuffix,
CreateResult, Member,
ReadMode, WriteMode)
from mirage.io.types import ByteSource, IOResult
from mirage.ops.types import LinkView, MountView
from mirage.types import PathSpec
from mirage.utils.fnmatch import fnmatch
def _excluded(name: str, pattern: str) -> bool:
base = name.split("/")[-1]
return fnmatch(name, pattern) or fnmatch(base, pattern)
def _compression_suffix(z: bool, j: bool, J: bool) -> CompressionSuffix:
@@ -34,11 +33,33 @@ def _read_mode(suffix: CompressionSuffix) -> ReadMode:
return READ_MODES[suffix]
def _stderr(lines: list[str]) -> bytes:
return ("\n".join(lines) + "\n").encode() if lines else b""
def _info(member: Member, size: int) -> tarfile.TarInfo:
"""The header for one member, typed the way its kind demands.
Args:
member (Member): the planned entry.
size (int): byte length of the content, 0 for a dir or a link.
"""
info = tarfile.TarInfo(name=member.name)
info.size = size
if member.kind == "dir":
info.type = tarfile.DIRTYPE
info.mode = 0o755
elif member.kind == "link":
info.type = tarfile.SYMTYPE
info.linkname = member.target
info.mode = 0o777
return info
async def _create_archive(
paths: list[PathSpec],
plan: CreateResult,
archive_path: PathSpec,
mode_suffix: CompressionSuffix,
exclude: str | None,
verbose: bool,
read_bytes: Callable[..., Awaitable[bytes]],
write_bytes: Callable[..., Awaitable[None]],
@@ -46,19 +67,18 @@ async def _create_archive(
buf = io.BytesIO()
names: list[str] = []
with tarfile.open(fileobj=buf, mode=_write_mode(mode_suffix)) as tf:
for p in paths:
name = p.virtual.lstrip("/")
if exclude and _excluded(name, exclude):
continue
data = await read_bytes(p)
info = tarfile.TarInfo(name=name)
info.size = len(data)
tf.addfile(info, io.BytesIO(data))
names.append(name)
for member in plan.members:
data = b""
if member.path is not None:
data = await read_bytes(member.path)
tf.addfile(_info(member, len(data)), io.BytesIO(data))
names.append(member.name)
archive = buf.getvalue()
await write_bytes(archive_path, archive)
stdout = ("\n".join(names) + "\n").encode() if verbose and names else None
return stdout, IOResult(writes={archive_path.mount_path: archive})
return stdout, IOResult(writes={archive_path.mount_path: archive},
stderr=_stderr(list(plan.notices)),
exit_code=plan.exit_code)
async def _list_archive(
@@ -69,7 +89,10 @@ async def _list_archive(
data = await read_bytes(archive_path)
with tarfile.open(fileobj=io.BytesIO(data),
mode=_read_mode(mode_suffix)) as tf:
names = tf.getnames()
names = [
member.name + "/" if member.isdir() else member.name
for member in tf.getmembers()
]
return ("\n".join(names) + "\n").encode(), IOResult()
@@ -89,18 +112,29 @@ async def _extract_archive(
with tarfile.open(fileobj=io.BytesIO(data),
mode=_read_mode(mode_suffix)) as tf:
for member in tf.getmembers():
if not member.isfile():
# A symlink member has no bytes to write and no namespace to
# write into from here (links are workspace state, not the
# backend's), so extraction skips it rather than dropping an
# empty file where a link belongs.
if not member.isfile() and not member.isdir():
continue
name_parts = member.name.rstrip("/").split("/")
if strip_n > 0:
name_parts = name_parts[strip_n:]
if not name_parts or name_parts == [""]:
continue
out_path = dest_path.rstrip("/") + "/" + "/".join(name_parts)
if member.isdir():
# A directory member is the only record an empty
# directory leaves, so it has to be recreated even
# though nothing is written inside it.
await mkdir_fn(PathSpec.from_str_path(out_path), parents=True)
names.append(member.name.rstrip("/") + "/")
continue
extracted = tf.extractfile(member)
if not extracted:
continue
content = extracted.read()
name_parts = member.name.split("/")
if strip_n > 0:
name_parts = name_parts[strip_n:]
if not name_parts:
continue
out_path = dest_path.rstrip("/") + "/" + "/".join(name_parts)
parent = out_path.rsplit("/", 1)[0] or "/"
if parent != "/":
await mkdir_fn(PathSpec.from_str_path(parent), parents=True)
@@ -117,6 +151,9 @@ async def tar(
read_bytes: Callable[..., Awaitable[bytes]],
write_bytes: Callable[..., Awaitable[None]],
mkdir_fn: Callable[..., Awaitable[None]],
stat: StatFn,
walk: WalkFn,
is_dir: DirProbe,
c: bool = False,
x: bool = False,
t: bool = False,
@@ -124,20 +161,37 @@ async def tar(
j: bool = False,
J: bool = False,
v: bool = False,
h: bool = False,
f: PathSpec | None = None,
C: PathSpec | None = None,
C: list[PathSpec] | None = None,
strip_components: str | None = None,
exclude: str | None = None,
links: LinkView | None = None,
mounts: MountView | None = None,
) -> tuple[ByteSource | None, IOResult]:
archive = f if f else None
dest_path = C.mount_path if C else "/"
# Only the last -C is a destination; create checks every one.
dest_path = C[-1].mount_path if C else "/"
mode_suffix = _compression_suffix(z, j, J)
strip_n = int(strip_components) if strip_components else 0
if c:
if archive is None:
raise ValueError("tar: -f is required")
return await _create_archive(paths, archive, mode_suffix, exclude, v,
read_bytes, write_bytes)
plan = await plan_create(paths,
archive=archive,
exclude=exclude,
dereference=h,
stat=stat,
walk=walk,
is_dir=is_dir,
directories=C or [],
links=links,
mounts=mounts)
if not plan.write:
return None, IOResult(exit_code=plan.exit_code,
stderr=_stderr(list(plan.notices)))
return await _create_archive(plan, archive, mode_suffix, v, read_bytes,
write_bytes)
if t:
if archive is None:
raise ValueError("tar: -f is required")
python/mirage/commands/builtin/generic/tar/types.py
+49
View File
@@ -1,5 +1,54 @@
from dataclasses import dataclass
from typing import Literal, TypeAlias
from mirage.commands.builtin.generic.archive.types import MemberKind
from mirage.types import PathSpec
CompressionSuffix: TypeAlias = Literal["", ":gz", ":bz2", ":xz"]
WriteMode: TypeAlias = Literal["w", "w:gz", "w:bz2", "w:xz"]
ReadMode: TypeAlias = Literal["r", "r:gz", "r:bz2", "r:xz"]
@dataclass(frozen=True, slots=True)
class Member:
"""One entry the create pass decided to put in the archive.
Choosing every member before writing any of them is what lets an
exclusion prune a whole subtree and the ordering stay stable; the
writer is then a straight loop with no policy left in it.
Args:
name (str): the archive member name, spelled as the operand was
typed. A directory carries GNU's trailing slash.
kind (MemberKind): file, dir, or link.
path (PathSpec | None): where a file's bytes come from; None for
a directory or a symlink, neither of which has content.
target (str): a symlink's target, verbatim as stored; empty for
every other kind.
"""
name: str
kind: MemberKind
path: PathSpec | None = None
target: str = ""
@dataclass(frozen=True, slots=True)
class CreateResult:
"""What one ``tar -c`` pass decided, before anything is written.
Args:
members (tuple[Member, ...]): the entries to write, in order.
notices (tuple[str, ...]): stderr lines, each already carrying
its ``tar: `` prefix and no trailing newline.
exit_code (int): 0, or 2 when an operand could not be read.
write (bool): whether to write an archive at all. False for the
two refusals GNU makes before reading anything (no operands,
and a ``-C`` it cannot enter), which leave no file behind;
an operand it merely failed to stat still writes the rest.
"""
members: tuple[Member, ...]
notices: tuple[str, ...]
exit_code: int
write: bool = True
python/mirage/commands/builtin/generic/unzip.py
+10 -4
View File
@@ -128,17 +128,23 @@ async def unzip(
writes: dict[str, ByteSource] = {}
output_lines: list[str] = []
for info in selected:
entry_name = info.filename.lstrip("/")
out_path = dest.rstrip("/") + "/" + entry_name.rstrip("/")
report_path = (mount_prefix +
out_path) if mount_prefix else out_path
if info.is_dir():
# A directory entry is the only record an empty
# directory leaves, so it has to be recreated even
# though nothing is written inside it.
await mkdir_fn(PathSpec.from_str_path(out_path), parents=True)
if not q:
output_lines.append(f" creating: {report_path}/")
continue
content = zf.read(info)
entry_name = info.filename.lstrip("/")
out_path = dest.rstrip("/") + "/" + entry_name
parent = out_path.rsplit("/", 1)[0] or "/"
if parent != "/":
await mkdir_fn(PathSpec.from_str_path(parent), parents=True)
await write_bytes(PathSpec.from_str_path(out_path), content)
report_path = (mount_prefix +
out_path) if mount_prefix else out_path
writes[out_path] = content
if not q:
output_lines.append(f" inflating: {report_path}")
python/mirage/commands/builtin/generic/zip_cmd.py
+235 -13
View File
@@ -2,9 +2,208 @@ import io
import posixpath
import zipfile
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from mirage.commands.builtin.generic.archive.types import MemberKind
from mirage.commands.builtin.generic.archive.walk import (OTHER_FILESYSTEM,
StatFn, WalkFn,
scan_operand)
from mirage.io.types import ByteSource, IOResult
from mirage.ops.types import LinkView, MountView
from mirage.types import PathSpec
from mirage.utils.fnmatch import fnmatch
from mirage.utils.path import respell_one
# Info-ZIP 3.0's wording, pinned on debian:stable-slim. A warning is
# indented with a tab and -q silences it; the "Nothing to do!" error is
# not a warning and survives -q. Exit 12 is Info-ZIP's ZE_NONE.
WARNING_PREFIX = "\tzip warning: "
# What Info-ZIP calls a path it could not reach. It does not distinguish
# absent from unreadable, and a dangling symlink under the default
# follow prints exactly this too.
NOT_MATCHED = "name not matched: "
NOTHING_TO_DO_EXIT = 12
# Info-ZIP has no mount boundaries to describe, so this borrows GNU
# tar's --one-file-system wording rather than inventing a second one.
CROSSING_REASON = OTHER_FILESYSTEM
# Unix mode bits in the high half of external_attr, which is where
# Info-ZIP puts them and where a symlink entry is told from a file.
DIR_MODE = 0o40755 << 16 | 0x10
FILE_MODE = 0o100644 << 16
LINK_MODE = 0o120777 << 16
@dataclass(frozen=True, slots=True)
class ZipMember:
"""One entry the plan decided to store.
Args:
name (str): the archive entry name; a directory carries the
trailing slash Info-ZIP stores it with.
kind (MemberKind): file, dir, or link.
path (PathSpec | None): where a file's bytes come from.
target (str): a symlink's target, which is its content.
"""
name: str
kind: MemberKind
path: PathSpec | None = None
target: str = ""
@dataclass(frozen=True, slots=True)
class ZipPlan:
"""What one ``zip`` run decided, before anything is written.
Args:
members (tuple[ZipMember, ...]): the entries to store, in order.
warnings (tuple[str, ...]): stderr lines without their prefix.
write (bool): whether to write an archive at all. Info-ZIP
leaves no file behind when nothing matched.
"""
members: tuple[ZipMember, ...]
warnings: tuple[str, ...]
write: bool
def member_name(spelled: str, kind: MemberKind, junk: bool) -> str:
"""The entry name Info-ZIP stores for a path as the operand typed it.
A leading slash is stripped in silence (unlike tar, which warns), a
directory carries a trailing slash, and ``-j`` throws the directory
part away entirely.
Args:
spelled (str): the path as the operand spelled it.
kind (MemberKind): what the entry is.
junk (bool): ``-j``, store the basename only.
"""
name = spelled.lstrip("/")
if junk:
name = posixpath.basename(name.rstrip("/"))
if kind == "dir" and name and not name.endswith("/"):
return name + "/"
return name
def excluded(name: str, patterns: list[str]) -> bool:
"""Whether an Info-ZIP ``-x`` pattern matches this entry name.
Info-ZIP matches the whole stored name, anchored, with wildcards
crossing slashes: ``d/sub/*`` takes ``d/sub/`` and everything under
it, ``*.txt`` takes every ``.txt`` at any depth, and a bare
``b.txt`` matches nothing below the top. That is the opposite of
GNU tar's unanchored ``--exclude``, which is why the two have
separate matchers.
Args:
name (str): the stored entry name, directories slash-terminated.
patterns (list[str]): the raw ``-x`` values.
"""
return any(fnmatch(name, pattern) for pattern in patterns)
async def plan_zip(
paths: list[PathSpec],
*,
archive: PathSpec,
stat: StatFn,
walk: WalkFn,
recurse: bool,
junk: bool,
store_links: bool,
exclude: list[str],
links: LinkView | None = None,
mounts: MountView | None = None,
) -> ZipPlan:
"""Decide every entry of a new archive, before writing any of it.
Info-ZIP's defaults are tar's inverted twice over: a directory
operand contributes only itself unless ``-r`` says to descend, and a
symlink is followed unless ``-y`` says to store the link. Both are
parameters of the shared scan, so the traversal is the same one
``tar -c`` uses.
Args:
paths (list[PathSpec]): the file operands, glob-resolved.
archive (PathSpec): the archive being written, so it is left out
of itself the way Info-ZIP silently leaves it out.
stat (StatFn): backend stat, raising when nothing is there.
walk (WalkFn): subtree listing, by find type.
recurse (bool): ``-r``.
junk (bool): ``-j``.
store_links (bool): ``-y``, store a symlink as a symlink.
exclude (list[str]): the raw ``-x`` values.
links (LinkView | None): the namespace's symlink facts.
mounts (MountView | None): where the mount boundaries are.
"""
members: list[ZipMember] = []
warnings: list[str] = []
for path in paths:
raw = path.raw_path
base = path.virtual.rstrip("/") or "/"
scan = await scan_operand(path,
stat=stat,
walk=walk,
links=links,
mounts=mounts,
dereference=not store_links,
recurse=recurse)
for problem in scan.problems:
shown = respell_one(problem.path, base, raw)
if problem.fatal:
warnings.append(NOT_MATCHED + shown)
else:
warnings.append(f"{shown}: {problem.reason}")
if scan.missing:
continue
for crossing in scan.crossings:
shown = respell_one(crossing, base, raw)
warnings.append(f"{shown}: {CROSSING_REASON}")
for entry in scan.entries:
name = member_name(respell_one(entry.name_path, base, raw),
entry.kind, junk)
if not name or excluded(name, exclude):
continue
# -j has no directory to name, so Info-ZIP drops directory
# entries under it entirely rather than storing bare slashes.
if junk and entry.kind == "dir":
continue
read = entry.read
if read is not None and read.virtual == archive.virtual:
# Info-ZIP never stores the archive it is writing, and
# says nothing about it.
continue
members.append(
ZipMember(name=name,
kind=entry.kind,
path=entry.read,
target=entry.target))
return ZipPlan(members=tuple(members),
warnings=tuple(warnings),
write=bool(members))
def _info(member: ZipMember, size: int) -> zipfile.ZipInfo:
info = zipfile.ZipInfo(filename=member.name)
info.compress_type = zipfile.ZIP_DEFLATED
info.create_system = 3
info.file_size = size
if member.kind == "dir":
info.external_attr = DIR_MODE
info.compress_type = zipfile.ZIP_STORED
elif member.kind == "link":
info.external_attr = LINK_MODE
else:
info.external_attr = FILE_MODE
return info
def _stderr(warnings: tuple[str, ...], quiet: bool) -> bytes:
if quiet:
return b""
return "".join(WARNING_PREFIX + line + "\n" for line in warnings).encode()
async def zip_cmd(
@@ -12,29 +211,52 @@ async def zip_cmd(
*,
read_bytes: Callable[..., Awaitable[bytes]],
write_bytes: Callable[..., Awaitable[None]],
stat: StatFn,
walk: WalkFn,
r: bool = False,
j: bool = False,
q: bool = False,
y: bool = False,
x: list[str] | None = None,
links: LinkView | None = None,
mounts: MountView | None = None,
) -> tuple[ByteSource | None, IOResult]:
if len(paths) < 2:
if not paths:
raise ValueError("zip: usage: zip archive.zip file1 [file2 ...]")
archive_path = paths[0]
file_paths = paths[1:]
plan = await plan_zip(paths[1:],
archive=archive_path,
stat=stat,
walk=walk,
recurse=r,
junk=j,
store_links=y,
exclude=x or [],
links=links,
mounts=mounts)
if not plan.write:
# Info-ZIP writes no archive when nothing matched, and the error
# is not a warning: -q does not silence it.
nothing = f"\nzip error: Nothing to do! ({archive_path.raw_path})\n"
return None, IOResult(exit_code=NOTHING_TO_DO_EXIT,
stderr=_stderr(plan.warnings, q) +
nothing.encode())
buf = io.BytesIO()
output_lines: list[str] = []
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
for p in file_paths:
data = await read_bytes(p)
arcname = posixpath.basename(
p.virtual) if j else p.virtual.lstrip("/")
zf.writestr(arcname, data)
if not q:
output_lines.append(f" adding: {arcname}")
for member in plan.members:
data = b""
if member.kind == "link":
data = member.target.encode()
elif member.path is not None:
data = await read_bytes(member.path)
zf.writestr(_info(member, len(data)), data)
output_lines.append(f" adding: {member.name}")
archive = buf.getvalue()
await write_bytes(archive_path, archive)
stdout = ("\n".join(output_lines) +
"\n").encode() if output_lines else None
return stdout, IOResult(writes={archive_path.mount_path: archive})
stdout = ("\n".join(output_lines) + "\n").encode() if not q else None
return stdout, IOResult(writes={archive_path.mount_path: archive},
stderr=_stderr(plan.warnings, q))
__all__ = ["zip_cmd"]
__all__ = ["plan_zip", "zip_cmd"]
python/mirage/commands/builtin/generic_bind/archive_io.py
+90
View File
@@ -0,0 +1,90 @@
import logging
from functools import partial
from mirage.accessor.base import Accessor
from mirage.cache.index import IndexCacheStore
from mirage.commands.builtin.generic.archive.walk import DirProbe, WalkFn
from mirage.commands.builtin.generic.find import parse_find_args, walk_find
from mirage.commands.builtin.generic_bind.adapter import CommandIO, OperationFn
from mirage.types import FileType, PathSpec
logger = logging.getLogger(__name__)
async def _walk(readdir: OperationFn, stat: OperationFn,
index: IndexCacheStore, path: PathSpec,
find_type: str) -> list[str]:
"""One subtree listing, filtered to files or to directories.
Reuses find's walk so an archiver classifies an entry exactly the
way find does (through stat, never by name). The two calls a
directory operand makes share one readdir cache, so the second is
answered from the index instead of the backend.
Args:
readdir (OperationFn): backend readdir.
stat (OperationFn): backend stat.
index (IndexCacheStore): the per-call cache index.
path (PathSpec): the operand to walk.
find_type (str): "d" or "f".
"""
return await walk_find(path,
readdir=readdir,
stat=stat,
index=index,
args=parse_find_args((), type=find_type))
async def _is_dir(stat: OperationFn, readdir: OperationFn, path: PathSpec,
index: IndexCacheStore) -> bool:
"""Whether a path is a directory an archiver could chdir into.
Two channels, because a stat miss alone is not absence: on a prefix
store a directory is the set of keys under it and nothing answers
stat for it, so a readdir that returns anything is the second and
deciding opinion.
Args:
stat (OperationFn): backend stat.
readdir (OperationFn): backend readdir.
path (PathSpec): the candidate directory.
index (IndexCacheStore): the per-call cache index.
"""
try:
return (await stat(path, index)).type == FileType.DIRECTORY
except (FileNotFoundError, ValueError):
logger.debug("archive: %s does not stat; asking readdir", path.virtual)
try:
return bool(await readdir(path, index))
except (FileNotFoundError, ValueError) as exc:
logger.debug("archive: %s is not a directory on either channel: %r",
path.virtual, exc)
return False
def walk_of(ops: CommandIO, accessor: Accessor,
index: IndexCacheStore) -> WalkFn:
"""The subtree listing tar and zip both walk with.
Args:
ops (CommandIO): the bound backend operations.
accessor (Accessor): the mount's accessor.
index (IndexCacheStore): the per-call cache index.
"""
return partial(_walk, partial(ops.readdir, accessor),
partial(ops.stat, accessor), index)
def is_dir_of(ops: CommandIO, accessor: Accessor,
index: IndexCacheStore) -> DirProbe:
"""The directory probe tar's ``-C`` check uses.
Args:
ops (CommandIO): the bound backend operations.
accessor (Accessor): the mount's accessor.
index (IndexCacheStore): the per-call cache index.
"""
return partial(_is_dir,
partial(ops.stat, accessor),
partial(ops.readdir, accessor),
index=index)
python/mirage/commands/builtin/generic_bind/builders/tar.py
+15 -3
View File
@@ -19,7 +19,10 @@ from mirage.cache.index import NULL_INDEX, IndexCacheStore
from mirage.commands.builtin.generic.tar import tar as generic_tar
from mirage.commands.builtin.generic_bind.adapter import (Builder, CommandIO,
Operation, bound_op)
from mirage.commands.builtin.generic_bind.archive_io import is_dir_of, walk_of
from mirage.commands.spec.types import FlagValue
from mirage.io.types import ByteSource, IOResult
from mirage.ops.types import LinkView, MountView
from mirage.types import PathSpec
@@ -36,12 +39,15 @@ async def tar(
j: bool = False,
J: bool = False,
v: bool = False,
h: bool = False,
f: PathSpec | None = None,
C: PathSpec | None = None,
C: list[PathSpec] | None = None,
strip_components: str | None = None,
exclude: str | None = None,
index: IndexCacheStore = NULL_INDEX,
**flags,
links: LinkView | None = None,
mounts: MountView | None = None,
**flags: FlagValue,
) -> tuple[ByteSource | None, IOResult]:
if not ops.is_mounted(accessor):
raise ValueError("tar: missing operand")
@@ -53,6 +59,9 @@ async def tar(
accessor),
mkdir_fn=partial(ops.require(Operation.MKDIR),
accessor),
stat=bound_op(ops.stat, accessor, index),
walk=walk_of(ops, accessor, index),
is_dir=is_dir_of(ops, accessor, index),
c=c,
x=x,
t=t,
@@ -60,10 +69,13 @@ async def tar(
j=j,
J=J,
v=v,
h=h,
f=f,
C=C,
strip_components=strip_components,
exclude=exclude)
exclude=exclude,
links=links,
mounts=mounts)
BUILDER = Builder('tar',
python/mirage/commands/builtin/generic_bind/builders/zip_cmd.py
+16 -3
View File
@@ -19,7 +19,10 @@ from mirage.cache.index import NULL_INDEX, IndexCacheStore
from mirage.commands.builtin.generic.zip_cmd import zip_cmd as generic_zip
from mirage.commands.builtin.generic_bind.adapter import (Builder, CommandIO,
Operation, bound_op)
from mirage.commands.builtin.generic_bind.archive_io import walk_of
from mirage.commands.spec.types import FlagValue
from mirage.io.types import ByteSource, IOResult
from mirage.ops.types import LinkView, MountView
from mirage.types import PathSpec
@@ -32,10 +35,14 @@ async def zip_cmd(
r: bool = False,
j: bool = False,
q: bool = False,
y: bool = False,
x: list[str] | None = None,
index: IndexCacheStore = NULL_INDEX,
**flags,
links: LinkView | None = None,
mounts: MountView | None = None,
**flags: FlagValue,
) -> tuple[ByteSource | None, IOResult]:
if not ops.is_mounted(accessor) or len(paths) < 2:
if not ops.is_mounted(accessor) or not paths:
raise ValueError("zip: usage: zip archive.zip file1 [file2 ...]")
paths = await ops.resolve_glob(accessor, paths, index)
return await generic_zip(paths,
@@ -43,9 +50,15 @@ async def zip_cmd(
index),
write_bytes=partial(ops.require(Operation.WRITE),
accessor),
stat=partial(ops.stat, accessor, index=index),
walk=walk_of(ops, accessor, index),
r=r,
j=j,
q=q)
q=q,
y=y,
x=x,
links=links,
mounts=mounts)
BUILDER = Builder('zip',
python/mirage/commands/spec/builtin_specs/archive.py
+16 -1
View File
@@ -25,14 +25,22 @@ SPECS: dict[str, CommandSpec] = {
Option(short="-j"),
Option(short="-J"),
Option(short="-v"),
# -h archives what a symlink points at instead of the link.
Option(short="-h"),
Option(short="-f", type="path"),
Option(short="-C", type="path"),
# Every occurrence is kept, in order: GNU chdirs at each
# one and fails at the first it cannot enter, so the
# planner has to see them all, not just the last.
Option(short="-C", type="path", multiple=True),
Option(long="--strip-components", type="str"),
Option(long="--exclude", type="str"),
),
rest=Operand(type="path"),
# `tar xzf a.tgz` is the spelling everyone types.
old_option_style=True,
# -C is a chdir for the operands after it, not a flag the command
# reads once: `tar -cf a.tar -C d x` archives d/x as `x`.
operand_base="-C",
),
'gzip':
CommandSpec(
@@ -69,6 +77,13 @@ SPECS: dict[str, CommandSpec] = {
Option(short="-r"),
Option(short="-j"),
Option(short="-q"),
# -y stores a symlink as a symlink; without it zip archives
# what the link points at, which is tar's -h inverted.
Option(short="-y"),
# Info-ZIP reads -x as a variadic list of patterns; mirage
# takes one per occurrence, since its spec has no variadic
# option value and `-x a -x b` says the same thing.
Option(short="-x", type="str", multiple=True),
),
rest=Operand(type="path"),
),
python/mirage/commands/spec/compile.py
+15
View File
@@ -79,6 +79,9 @@ class CompiledSpec:
numeric_dest (str | None): canonical spelling fed by the
``-<digits>`` shorthand, when one option declares it.
rest_kind (ValueType | None): kind of the rest operand.
base_dest (str | None): canonical spelling of the option that
re-bases the path operands after it (``CommandSpec.
operand_base``, tar's -C).
"""
bool_spellings: frozenset[str] = frozenset()
@@ -102,6 +105,7 @@ class CompiledSpec:
defaults: dict[str, str] = field(default_factory=dict)
numeric_dest: str | None = None
rest_kind: ValueType | None = None
base_dest: str | None = None
def dest_of(self, spelling: str) -> str:
"""Canonical spelling for a typed spelling.
@@ -262,6 +266,16 @@ def compile_spec(spec: CommandSpec) -> CompiledSpec:
long_value_spellings.add(opt.long)
kind_of[opt.long] = opt.type
base_dest: str | None = None
if spec.operand_base is not None:
base_dest = dest.get(spec.operand_base)
if base_dest is None:
raise ValueError(f"operand_base {spec.operand_base!r} is not a "
"declared option")
if kind_by_dest.get(base_dest) != "path" or base_dest in pair_dests:
raise ValueError(f"operand_base {spec.operand_base!r} must be a "
"single-token path option")
# Longest first so an attached match can never be stolen by a
# shorter spelling that happens to prefix it (-name vs -n).
value_spellings.sort(key=len, reverse=True)
@@ -289,4 +303,5 @@ def compile_spec(spec: CommandSpec) -> CompiledSpec:
defaults=defaults,
numeric_dest=numeric_dest,
rest_kind=spec.rest.type if spec.rest is not None else None,
base_dest=base_dest,
)
python/mirage/commands/spec/parser.py
+71 -1
View File
@@ -52,6 +52,45 @@ def _set_value_flag(
flags[name] = value
def _rebase(
flags: dict[str, ParsedFlagValue],
cs: CompiledSpec,
spelling: str,
value: str,
base: str,
) -> str:
"""Fold one option occurrence into the operand base directory.
Called after every value-flag record. Only the spec's declared
``operand_base`` option moves the base, and it moves it the way a
chdir does: relative to wherever the previous occurrence left it, so
``-C d1 ... -C ../d2`` lands in ``d1/../d2``. The resolved absolute
path replaces the raw value in the flag bag, so the later path-flag
pass has nothing left to resolve.
Args:
flags (dict): parsed flag bag, updated in place.
cs (CompiledSpec): compiled spec tables.
spelling (str): dashed spelling as typed.
value (str): the flag's value.
base (str): the base directory in effect before this occurrence.
Returns:
str: the base directory in effect after this occurrence.
"""
if cs.base_dest is None or cs.dest_of(spelling) != cs.base_dest:
return base
moved = resolve_path(value, base)
bag = flags.get(cs.base_dest)
if isinstance(bag, list) and bag:
# An accumulating option already appended the raw value; the
# resolved one replaces it so nothing resolves it twice.
bag[-1] = moved
else:
flags[cs.base_dest] = moved
return moved
def _set_bool_flag(
flags: dict[str, ParsedFlagValue],
cs: CompiledSpec,
@@ -157,6 +196,13 @@ def parse_command(
# the shape heuristic and a path-shaped one (`tar sub/a.tgz`)
# would reach dispatch resolved and unreadable as letters.
word_kinds[0] = "str"
# The directory the next path operand resolves against, and where it
# was for each word already read. It only ever moves for a spec that
# declares operand_base, so every other command records None
# throughout and the classifier keeps using the session cwd.
base = cwd
word_bases: list[str | None] = [None] * len(argv)
raw_bases: list[str] = []
warnings: list[str] = []
invalid_options: list[str] = []
ambiguous_options: list[tuple[str, tuple[str, ...]]] = []
@@ -181,6 +227,7 @@ def parse_command(
if end_of_flags:
raw_args.append(tok)
raw_indices.append(orig_indices[i])
raw_bases.append(base)
i += 1
continue
@@ -221,6 +268,9 @@ def parse_command(
and i + 1 < len(filtered_argv)):
_set_value_flag(flags, cs, etok, filtered_argv[i + 1])
word_kinds[orig_indices[i + 1]] = cs.kind_of[etok]
if cs.dest_of(etok) == cs.base_dest:
word_bases[orig_indices[i + 1]] = base
base = _rebase(flags, cs, etok, filtered_argv[i + 1], base)
i += 2
elif is_pair:
if eq == -1:
@@ -235,12 +285,14 @@ def parse_command(
if eq != -1 and (spelling in cs.long_value_spellings
or spelling in cs.long_optional_spellings):
_set_value_flag(flags, cs, spelling, tok[eq + 1:])
base = _rebase(flags, cs, spelling, tok[eq + 1:], base)
elif etok in cs.long_value_spellings:
# Declared value flag at end of line with no argument.
needs_value_options.append(etok)
elif lenient_dash_operands:
raw_args.append(tok)
raw_indices.append(orig_indices[i])
raw_bases.append(base)
else:
invalid_options.append(tok)
option_error_kinds.append("invalid")
@@ -256,6 +308,7 @@ def parse_command(
for vf in cs.attach_spellings:
if tok.startswith(vf) and len(tok) > len(vf):
_set_value_flag(flags, cs, vf, tok[len(vf):])
base = _rebase(flags, cs, vf, tok[len(vf):], base)
i += 1
matched_optional = True
break
@@ -266,11 +319,15 @@ def parse_command(
if tok == vf and i + 1 < len(filtered_argv):
_set_value_flag(flags, cs, vf, filtered_argv[i + 1])
word_kinds[orig_indices[i + 1]] = cs.kind_of[vf]
if cs.dest_of(vf) == cs.base_dest:
word_bases[orig_indices[i + 1]] = base
base = _rebase(flags, cs, vf, filtered_argv[i + 1], base)
i += 2
matched_value = True
break
if tok.startswith(vf) and len(tok) > len(vf):
_set_value_flag(flags, cs, vf, tok[len(vf):])
base = _rebase(flags, cs, vf, tok[len(vf):], base)
i += 1
matched_value = True
break
@@ -300,6 +357,7 @@ def parse_command(
for name in cluster_bools:
_set_bool_flag(flags, cs, name)
_set_value_flag(flags, cs, vflag, attached)
base = _rebase(flags, cs, vflag, attached, base)
i += 1
continue
if i + 1 < len(filtered_argv):
@@ -307,12 +365,17 @@ def parse_command(
_set_bool_flag(flags, cs, name)
_set_value_flag(flags, cs, vflag, filtered_argv[i + 1])
word_kinds[orig_indices[i + 1]] = cs.kind_of[vflag]
if cs.dest_of(vflag) == cs.base_dest:
word_bases[orig_indices[i + 1]] = base
base = _rebase(flags, cs, vflag, filtered_argv[i + 1],
base)
i += 2
continue
if lenient_dash_operands or NUMERIC_SHORT.match(tok):
raw_args.append(tok)
raw_indices.append(orig_indices[i])
raw_bases.append(base)
elif tok in cs.value_spellings or (mixed is not None
and mixed[2] is None):
# A declared value flag (alone or ending a cluster) with no
@@ -338,6 +401,7 @@ def parse_command(
raw_args.append(tok)
raw_indices.append(orig_indices[i])
raw_bases.append(base)
i += 1
# Declared defaults land as if typed, before choices/required checks
@@ -421,8 +485,13 @@ def parse_command(
else:
kind = overflow_kind
if kind == "path":
classified.append((resolve_path(arg, cwd), "path"))
# Against the base an operand_base option left in effect at
# this position, which is the session cwd for every command
# that declares none.
classified.append((resolve_path(arg, raw_bases[j]), "path"))
raw_operands.append((arg, "path"))
if raw_bases[j] != cwd:
word_bases[raw_indices[j]] = raw_bases[j]
else:
classified.append((arg, kind))
raw_operands.append((arg, kind))
@@ -469,6 +538,7 @@ def parse_command(
text_flag_values=text_flag_values,
warnings=warnings,
word_kinds=word_kinds,
word_bases=word_bases,
invalid_options=invalid_options,
ambiguous_options=ambiguous_options,
option_error_kinds=option_error_kinds,
python/mirage/commands/spec/types.py
+12
View File
@@ -189,6 +189,13 @@ class CommandSpec:
# (`tar xzf a.tgz`). Expanded by expand_old_style before any other
# scanning; see oldstyle.py for the rules and why only tar has it.
old_option_style: bool = False
# The spelling of an option that changes directory for the path
# operands typed AFTER it (tar's -C). Positional and cumulative, the
# way a real chdir is: `tar -cf a.tar -C d1 x -C ../d2 y` reads d1/x
# and d1/../d2/y. Only path operands and the option's own value move;
# every other path-valued flag keeps resolving against the session
# cwd, which is what GNU does with -f.
operand_base: str | None = None
class FlagView:
@@ -310,6 +317,11 @@ class ParsedArgs:
text_flag_values: list[str] = field(default_factory=list)
warnings: list[str] = field(default_factory=list)
word_kinds: list[ValueType | None] = field(default_factory=list)
# Per-position base directory, aligned with word_kinds: the absolute
# path a word resolves against when an operand_base option (tar's -C)
# moved it, and None when the session cwd still applies. Only a spec
# declaring operand_base ever fills this.
word_bases: list[str | None] = field(default_factory=list)
# GNU-shaped option errors, reported (never raised) by the parser:
# undeclared options ('--bogus' or the offending cluster char 'Y'),
# abbreviated longs matching several options (typed prefix, matched
python/mirage/ops/types.py
+35
View File
@@ -28,6 +28,12 @@ StatPath = Callable[[str], Awaitable["FileStat | None"]]
# (GIT_DISCOVERY_ACROSS_FILESYSTEM); crossing it would probe an
# unrelated backend.
MountRoot = Callable[[str], str]
# The mount roots strictly under a virtual path, in prefix order,
# without their trailing slash.
MountDescendants = Callable[[str], list[str]]
# Whether a virtual path IS a mount's root, rather than a directory the
# backend holds.
MountIsRoot = Callable[[str], bool]
# lstat for one path: the link's own stat, None when not a link.
LinkStat = Callable[[str], "FileStat | None"]
# Stat rows for the links directly under a directory, for listings.
@@ -44,6 +50,35 @@ LinkExists = Callable[[str], Awaitable[bool]]
LinkTargetStat = Callable[[str], Awaitable["FileStat | None"]]
@dataclass(frozen=True)
class MountView:
"""Where the mount boundaries are, as one injected object.
A command runs bound to one backend, and that backend cannot see a
mount nested inside its own tree: the child's keys live in another
resource entirely, so the parent's ``readdir`` never lists it. A
walker that must account for the whole subtree therefore has to be
told, the same way ``LinkView`` tells it about symlinks.
Traversal commands that render lines (find, du, grep -r) get this
for free from the executor's fan-out, which reruns them per mount
and concatenates the output. A command whose output is one binary
object (tar, zip) cannot be merged that way, so it reads the
boundaries here and says what it did with them.
A command opts in by naming a ``mounts`` parameter, which is what
makes the dispatcher hand it one.
"""
# Mount roots strictly under a path (a walker: tar, zip).
descendants: MountDescendants
# Whether a path is a mount root itself.
is_root: MountIsRoot
# The mount serving a path, so a walker can tell "still mine" from
# "another backend" before it tries to read something it cannot.
root_of: MountRoot
@dataclass(frozen=True)
class LinkView:
"""The symlink facts a command may consult, as one injected object.
python/mirage/policy/builtin/mount_root.py
+100 -11
View File
@@ -12,8 +12,11 @@
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from collections.abc import Sequence
from mirage.policy.base import Policy
from mirage.policy.types import Action, CommandContext, Deny
from mirage.policy.types import Action, CommandContext, Deny, MountRootQuery
from mirage.types import PathSpec
def has_symlink_flag(argv: tuple[str, ...]) -> bool:
@@ -34,6 +37,38 @@ def has_symlink_flag(argv: tuple[str, ...]) -> bool:
return False
def is_create_mode(argv: tuple[str, ...]) -> bool:
"""Whether a tar line reads the filesystem rather than an archive.
Only ``-c`` makes tar's operands source paths. Under ``-t`` and
``-x`` they are member selectors matched against names inside the
archive, so a selector that happens to spell a mount root is not a
mount at all and refusing it would deny an ordinary listing.
Scanned raw for the same reason :func:`has_symlink_flag` is: the
policy fires before flag parsing. Only the first word may be GNU's
dashless option cluster (``tar cf a.tar d``), so a later bare word
is an operand and cannot turn the mode on.
Args:
argv (tuple[str, ...]): raw argv after the command name.
"""
for i, tok in enumerate(argv):
if not isinstance(tok, str):
continue
if tok == "--create":
return True
if tok.startswith("--"):
continue
if tok.startswith("-"):
if "c" in tok[1:]:
return True
continue
if i == 0 and "c" in tok:
return True
return False
def has_parents_flag(argv: tuple[str, ...]) -> bool:
"""Spot mkdir's -p/--parents by raw token scan.
@@ -53,17 +88,47 @@ def has_parents_flag(argv: tuple[str, ...]) -> bool:
return False
class MountRootPolicy(Policy):
"""The built-in POSIX rule: a mount root is busy, not a directory.
def first_root(query: MountRootQuery,
paths: Sequence[PathSpec]) -> PathSpec | None:
"""The first of these paths that is a mount root, if any.
Mirrors the kernel's refusal to unlink or replace a mountpoint
(EBUSY on Linux), with each command's own GNU message. Fires before
mount resolution and cross-mount routing so the refusal is the same
however the operands span mounts, and before runtime placement so a
routed command is refused identically. MountRegistry seeds it as
the first policy (mount-root semantics belong to the mount layer),
so its exact GNU messages win over user policies by order, not by
privilege.
Args:
query (MountRootQuery): the mount-root oracle.
paths (Sequence[PathSpec]): paths to test, in operand order.
"""
for path in paths:
if query.is_mount_root(path.virtual):
return path
return None
class MountRootPolicy(Policy):
"""The built-in rule: a mount root is not an ordinary directory.
Two rules, one boundary. The first mirrors the kernel's refusal to
unlink or replace a mountpoint (EBUSY on Linux), with each command's
own GNU message: rm, rmdir, mv, mkdir, touch and ln.
The second is mirage's own, and is a deliberate divergence: an
archiver or a recursive copy pointed at a mount root would read an
entire backend into one object. Real tar and cp allow it because a
mountpoint there is just another directory; here the mount table is
the deployment's configuration, and consuming a whole mount is
neither what the operand looks like it costs nor something an agent
should be able to do to data it was merely given a view of. The
refusal names the boundary in each tool's own voice rather than
inventing a mirage error, so a caller sees a filesystem answer.
Only positional operands are tested. tar's ``-C`` and unzip's ``-d``
are destinations to extract INTO, which is ordinary use of a mount,
so reading them here would refuse the safe direction as well.
Fires before mount resolution and cross-mount routing so the refusal
is the same however the operands span mounts, and before runtime
placement so a routed command is refused identically. MountRegistry
seeds it as the first policy (mount-root semantics belong to the
mount layer), so its exact messages win over user policies by order,
not by privilege.
"""
async def pre_command(self, ctx: CommandContext) -> Action | None:
@@ -105,4 +170,28 @@ class MountRootPolicy(Policy):
if has_symlink_flag(ctx.argv) else "link")
return Deny(f"ln: failed to create {kind} "
f"'{ctx.paths[-1].virtual}': File exists\n")
elif cmd == "tar":
# Only -c reads the filesystem; -t and -x match their
# operands against names inside the archive.
root = (first_root(ctx.registry, ctx.operands)
if is_create_mode(ctx.argv) else None)
if root is not None:
return Deny(
f"tar: {root.raw_path}: Cannot open: "
f"Device or resource busy\n"
f"tar: Error is not recoverable: exiting now\n", 2)
elif cmd == "zip":
# The first operand is the archive being written, not a
# source; only what follows it is read.
root = first_root(ctx.registry, ctx.operands[1:])
if root is not None:
return Deny(f"zip: cannot read '{root.raw_path}': "
f"Device or resource busy\n")
elif cmd == "cp":
# The last operand is the destination, and copying INTO a
# mount is ordinary; only the sources are refused.
root = first_root(ctx.registry, ctx.operands[:-1])
if root is not None:
return Deny(f"cp: cannot copy '{root.raw_path}': "
f"Device or resource busy\n")
return None
python/mirage/policy/types.py
+9 -1
View File
@@ -81,7 +81,14 @@ class CommandContext:
Args:
command (str): the command name.
paths (tuple[PathSpec, ...]): positional path operands.
paths (tuple[PathSpec, ...]): every path the line names, the
positional operands first and then the values of any
path-valued flags. What a path-pattern guard matches on.
operands (tuple[PathSpec, ...]): the positional operands alone.
A rule that reads a slot by position (mv's source, ln's
target, tar's files) has to use this: with the flag values
mixed in, ``tar -xf a.tar -C /mnt`` would read the ``-C``
destination as a file being archived.
argv (tuple[str, ...]): raw argv after the command name; the
hook fires before flag parsing, so shorthand flags are raw
tokens.
@@ -94,6 +101,7 @@ class CommandContext:
argv: tuple[str, ...]
cwd: str
registry: MountRootQuery
operands: tuple[PathSpec, ...] = ()
@dataclass(frozen=True, slots=True)
python/mirage/workspace/executor/command/routing.py
+33
View File
@@ -86,6 +86,39 @@ def path_flag_scopes(cmd_name: str, argv: list[str],
]
def positional_scopes(cmd_name: str, argv: list[str], cwd: str,
words: list[str | PathSpec]) -> list[PathSpec]:
"""The path operands a line names positionally, flag values left out.
Classification turns every path-shaped word into a PathSpec,
including the value of a path-valued flag, so the classified word
list cannot tell ``tar -xf a.tar -C /mnt`` (extract INTO a mount)
from ``tar -cf a.tar /mnt`` (archive a whole mount). Only the spec
knows which slot a word filled, so this asks it and keeps the
classified spec for each surviving operand, whose ``raw_path`` is
what a message should name.
Args:
cmd_name (str): command name.
argv (list[str]): the words after the command name, as typed.
cwd (str): working directory the line was typed under.
words (list[str | PathSpec]): the same words, classified.
"""
spec = SPECS.get(cmd_name)
if spec is None:
return [p for p in words if isinstance(p, PathSpec)]
parsed = parse_command(spec, argv, cwd)
by_virtual = {p.virtual: p for p in words if isinstance(p, PathSpec)}
return [
by_virtual.get(
value,
PathSpec(virtual=value,
directory=value,
resource_path="",
raw_path=value)) for value in parsed.paths()
]
def merge_scopes(positional: list[PathSpec],
flag_scopes: list[PathSpec]) -> list[PathSpec]:
"""Combine positional and path-flag scopes, keeping operand order.
python/mirage/workspace/executor/command/run.py
+32 -1
View File
@@ -22,7 +22,7 @@ from mirage.commands.spec.types import FlagValue
from mirage.io import IOResult
from mirage.io.stream import materialize, wrap_cachable_streams
from mirage.io.types import ByteSource
from mirage.ops.types import LinkView
from mirage.ops.types import LinkView, MountView
from mirage.runtime.base import Runtime
from mirage.runtime.policy import PolicyDecision
from mirage.runtime.table import VFSRuntime
@@ -140,6 +140,19 @@ def link_view(namespace: Namespace | None,
dispatch))
def mount_roots_below(registry: MountRegistry, virtual: str) -> list[str]:
"""Mount roots strictly under a path, without the trailing slash.
Args:
registry (MountRegistry): registry holding the mount table.
virtual (str): absolute virtual path to scan beneath.
"""
return [
m.prefix.rstrip("/") or "/"
for m in registry.descendant_mounts(virtual)
]
def mount_root_of(registry: MountRegistry, virtual: str) -> str:
"""The mount prefix serving a virtual path, "/" when none does.
@@ -159,6 +172,23 @@ def mount_root_of(registry: MountRegistry, virtual: str) -> str:
return "/"
def mount_view(registry: MountRegistry) -> MountView:
"""The mount-boundary facts on offer to every command.
Which commands receive it is decided at dispatch by whether the
handler names a ``mounts`` parameter, the same opt-in ``links``
uses, so there is no list of boundary-aware commands to keep in
step.
Args:
registry (MountRegistry): registry holding the mount table.
"""
return MountView(descendants=functools.partial(mount_roots_below,
registry),
is_root=registry.is_mount_root,
root_of=functools.partial(mount_root_of, registry))
async def drop_service_caches(registry: MountRegistry,
serves: tuple[ResourceName, ...]) -> None:
"""Drop cached listings and bodies for the mounts a CLI's service backs.
@@ -321,6 +351,7 @@ async def run_on_mount(
stat_overlay=stat_overlay,
links=links,
stat_path=stat_path,
mounts=mount_view(registry),
)
except UsageError as exc:
# Command-owned usage errors (extra operands, missing patterns)
python/mirage/workspace/expand/argv.py
+8 -1
View File
@@ -26,6 +26,7 @@ from mirage.workspace.expand.classify import classify_parts
from mirage.workspace.expand.globs import resolve_globs
from mirage.workspace.expand.parts import expand_parts
from mirage.workspace.expand.spec_hints import (spec_for_command,
spec_word_bases,
spec_word_kinds)
from mirage.workspace.mount import MountRegistry
from mirage.workspace.route import WordPolicy, route, word_policy
@@ -105,16 +106,22 @@ async def expand_argv(
policy = word_policy(route(name, session, registry))
word_kinds: list[ValueType | None] | None = None
word_bases: list[str | None] | None = None
if policy is WordPolicy.MOUNT:
spec = spec_for_command(name, registry, session.cwd)
if spec:
extra: list[ValueType | None] = ["str"] * (consumed - 1)
word_kinds = extra + spec_word_kinds(spec, expanded[consumed:])
bases = spec_word_bases(spec, expanded[consumed:], session.cwd)
if bases is not None:
head: list[str | None] = [None] * (consumed - 1)
word_bases = head + bases
classified = classify_parts(expanded,
registry,
session.cwd,
word_kinds=word_kinds)
word_kinds=word_kinds,
word_bases=word_bases)
# set -f: glob words become literal paths for every consumer,
# including backend pushdown, so `cat *.txt` looks up a file
# literally named `*.txt` like bash with noglob.
python/mirage/workspace/expand/classify/parts.py
+9 -3
View File
@@ -24,13 +24,16 @@ def classify_parts(
registry: MountRegistry,
cwd: str,
word_kinds: list[ValueType | None] | None = None,
word_bases: list[str | None] | None = None,
) -> list[str | PathSpec]:
"""Classify a list of expanded words.
First element (command name) is never classified as a path.
word_kinds (from CommandSpec, aligned with parts[1:]) decides per
position: TEXT skips classification, PATH classifies even bare
filenames, None falls back to the shape heuristics.
filenames, None falls back to the shape heuristics. word_bases, also
aligned with parts[1:], names the directory a word resolves against
when a chdir option (tar's -C) moved it; None there means the cwd.
"""
if not parts:
return []
@@ -38,10 +41,13 @@ def classify_parts(
for i, w in enumerate(parts[1:]):
kind = (word_kinds[i]
if word_kinds is not None and i < len(word_kinds) else None)
base = (word_bases[i]
if word_bases is not None and i < len(word_bases) else None)
here = base if base is not None else cwd
if kind is not None and kind != "path":
result.append(w)
elif kind == "path":
result.append(classify_bare_path(w, registry, cwd))
result.append(classify_bare_path(w, registry, here))
else:
result.append(classify_word(w, registry, cwd))
result.append(classify_word(w, registry, here))
return result
python/mirage/workspace/expand/spec_hints.py
+23
View File
@@ -67,3 +67,26 @@ def spec_word_kinds(
# nothing to override here: leaving them None sent `find \( ... \)`
# back to the shape heuristic, which read "(" as the bare path "/(".
return list(parse_command(spec, argv, cwd="/").word_kinds)
def spec_word_bases(
spec: CommandSpec,
argv: list[str],
cwd: str,
) -> list[str | None] | None:
"""Per-position base directories for a spec that declares one.
tar's -C is a chdir for the operands typed after it, so those words
are not relative to the session cwd at all. The parser already walks
the line positionally, so it is what says where each word stood;
this asks it, and only for the one command family that can answer
(None everywhere else, so 92 of 93 specs pay nothing).
Args:
spec (CommandSpec): command specification.
argv (list[str]): command arguments (without command name).
cwd (str): the working directory the line was typed under.
"""
if spec.operand_base is None:
return None
return list(parse_command(spec, argv, cwd=cwd).word_bases)
python/mirage/workspace/mount/mount.py
+7 -2
View File
@@ -31,7 +31,7 @@ from mirage.observe.context import (push_mount_prefix, push_revisions,
reset_revisions, with_mount_prefix,
with_revisions)
from mirage.ops.registry import RegisteredOp
from mirage.ops.types import LinkView, StatOverlay, StatPath
from mirage.ops.types import LinkView, MountView, StatOverlay, StatPath
from mirage.policy import resolve_limit
from mirage.resource.base import BaseResource
from mirage.runtime.base import Runtime
@@ -446,6 +446,7 @@ class MountEntry:
stat_overlay: StatOverlay | None = None,
links: LinkView | None = None,
stat_path: StatPath | None = None,
mounts: MountView | None = None,
) -> tuple[ByteSource | None, IOResult]:
"""Execute a command on this mount's resource.
@@ -465,7 +466,10 @@ class MountEntry:
links (LinkView | None): the namespace's symlink facts.
stat_path (StatPath | None): dispatcher-backed stat of one
path, for a traversal command's start point.
All three reach only the handlers that name them as a
mounts (MountView | None): where the mount boundaries are,
for a walker whose output cannot be fanned out and
concatenated (tar, zip).
All four reach only the handlers that name them as a
parameter, so no list of command names is kept here.
"""
extension = get_extension(paths[0].virtual) if paths else None
@@ -533,6 +537,7 @@ class MountEntry:
"stat_overlay": stat_overlay,
"links": links,
"stat_path": stat_path,
"mounts": mounts,
}
offered = {k: v for k, v in offered.items() if v is not None}
if runtime is not None:
python/mirage/workspace/names.py
+9 -1
View File
@@ -70,9 +70,17 @@ JOB_BUILTINS = frozenset({"wait", "fg", "kill", "jobs", "ps"})
# `stat` is here because GNU stat lstats, but it takes -L to dereference
# after all, which route's `dereferences` reads back out of the command
# line; `file`, `du` and `find` are the same shape.
#
# `tar` and `zip` are here for a different reason and deliberately carry
# no DEREFERENCE_FLAGS entry: they dereference too, but their planner
# has to be the one doing it. Rewriting the operand up here would hand
# the planner a target it can no longer tell was reached through a link,
# so `tar` could not store a symlink member at all and neither archiver
# could apply its own cross-mount refusal or ELOOP wording. tar's -h and
# zip's -y are read by the planner instead.
NO_FOLLOW_COMMANDS = frozenset({
"rm", "mv", "ln", "readlink", "rmdir", "unlink", "stat", "file", "du",
"find"
"find", "tar", "zip"
})
SHELL_NAMES = frozenset(str(b) for b in ShellBuiltin) | UNSUPPORTED_BUILTINS
python/mirage/workspace/node/command_dispatch.py
+5 -1
View File
@@ -27,7 +27,8 @@ from mirage.shell.xtrace import trace_command
from mirage.types import PathSpec, Producer, word_text
from mirage.utils.path import CycleError
from mirage.workspace.executor.command import handle_command
from mirage.workspace.executor.command.routing import path_flag_scopes
from mirage.workspace.executor.command.routing import (path_flag_scopes,
positional_scopes)
from mirage.workspace.executor.control import BreakSignal, ContinueSignal
from mirage.workspace.expand import expand_node
from mirage.workspace.expand.argv import Argv, expand_argv
@@ -283,6 +284,9 @@ async def _run_argv(
deny = await registry.policies.pre_command(
CommandContext(command=name,
paths=tuple(scopes),
operands=tuple(
positional_scopes(name, args, session.cwd,
operands)),
argv=tuple(args),
cwd=session.cwd,
registry=registry))
python/tests/commands/builtin/generic/archive/test_walk.py
+236
View File
@@ -0,0 +1,236 @@
from dataclasses import replace
import pytest
from mirage.commands.builtin.generic.archive import walk as aw
from mirage.ops.types import LinkView, MountView
from mirage.types import LINK_TARGET_KEY, FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
from mirage.utils.path import CycleError
def _spec(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True)
class _Tree:
def __init__(self, files: dict[str, bytes], dirs: tuple[str, ...] = ()):
self.files = dict(files)
self.dirs = set(dirs)
async def stat(self, path):
key = path.virtual.rstrip("/") or "/"
if key in self.dirs:
return FileStat(name=key, type=FileType.DIRECTORY)
if key in self.files:
return FileStat(name=key,
type=FileType.TEXT,
size=len(self.files[key]))
raise FileNotFoundError(key)
async def walk(self, path, find_type):
base = path.virtual.rstrip("/") or "/"
pool = self.dirs if find_type == "d" else self.files
return sorted(p for p in pool
if p == base or p.startswith(base.rstrip("/") + "/"))
def _links(entries: dict[str, str]) -> LinkView:
def stat_of(path):
target = entries[path]
return FileStat(name=path,
type=FileType.SYMLINK,
size=len(target),
extra={LINK_TARGET_KEY: target})
async def target_stat(path):
return None
async def exists(path):
return path in entries
return LinkView(
stat_at=lambda p: stat_of(p) if p in entries else None,
children=lambda p: [],
subtree=lambda p: [(k, stat_of(k)) for k in sorted(entries)
if k.startswith(p.rstrip("/") + "/")],
resolve=lambda p: entries.get(p, p),
exists=exists,
target_stat=target_stat,
)
def _cycle(entries: dict[str, str]) -> LinkView:
"""A LinkView whose resolve raises ELOOP, as the namespace does."""
view = _links(entries)
def resolve(path):
raise CycleError(path)
return replace(view, resolve=resolve)
def _mounts(descendants: tuple[str, ...] = (),
roots: tuple[str, ...] = ()) -> MountView:
def root_of(path):
for root in sorted(roots, key=len, reverse=True):
if path == root or path.startswith(root.rstrip("/") + "/"):
return root
return "/"
return MountView(
descendants=lambda p:
[d for d in descendants if d.startswith(p.rstrip("/") + "/")],
is_root=lambda p: p.rstrip("/") in {r.rstrip("/")
for r in roots},
root_of=root_of,
)
async def _scan(tree: _Tree, path: PathSpec, **kwargs):
return await aw.scan_operand(path,
stat=tree.stat,
walk=tree.walk,
**kwargs)
def test_child_spec_strips_the_mount_prefix_from_the_backend_key():
root = _spec("/data/d", "/data")
child = aw.child_spec("/data/d/a.txt", root)
assert child.virtual == "/data/d/a.txt"
assert child.resource_path == "d/a.txt"
def test_same_mount_is_true_without_a_mount_view():
assert aw.same_mount(None, "/a", "/b")
mounts = _mounts(roots=("/", "/m"))
assert aw.same_mount(mounts, "/a", "/b")
assert not aw.same_mount(mounts, "/a", "/m/x")
@pytest.mark.asyncio
async def test_recurse_false_stops_at_the_directory_itself():
tree = _Tree({"/d/a.txt": b"a"}, dirs=("/d", "/d/sub"))
scan = await _scan(tree, _spec("/d"), recurse=False)
assert [e.name_path for e in scan.entries] == ["/d"]
assert scan.entries[0].kind == "dir"
@pytest.mark.asyncio
async def test_recurse_true_reports_the_whole_subtree_sorted():
tree = _Tree({
"/d/a.txt": b"a",
"/d/sub/b.txt": b"b"
},
dirs=("/d", "/d/sub"))
scan = await _scan(tree, _spec("/d"), recurse=True)
assert [e.name_path for e in scan.entries
] == ["/d", "/d/a.txt", "/d/sub", "/d/sub/b.txt"]
@pytest.mark.asyncio
async def test_a_missing_operand_is_one_fatal_problem_and_no_entries():
tree = _Tree({})
scan = await _scan(tree, _spec("/nope"))
assert scan.missing
assert not scan.entries
assert [(p.path, p.fatal) for p in scan.problems] == [("/nope", True)]
@pytest.mark.asyncio
async def test_a_link_is_stored_or_followed_by_the_dereference_flag():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/link.txt": "/d/a.txt"})
stored = await _scan(tree,
_spec("/d"),
links=links,
dereference=False,
recurse=True)
kinds = {e.name_path: e.kind for e in stored.entries}
assert kinds["/d/link.txt"] == "link"
followed = await _scan(tree,
_spec("/d"),
links=links,
dereference=True,
recurse=True)
kinds = {e.name_path: e.kind for e in followed.entries}
assert kinds["/d/link.txt"] == "file"
@pytest.mark.asyncio
async def test_two_links_to_one_target_are_both_archived():
"""Not a loop: GNU tar -h and Info-ZIP both store the two names."""
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/one": "/d/a.txt", "/d/two": "/d/a.txt"})
scan = await _scan(tree,
_spec("/d"),
links=links,
dereference=True,
recurse=True)
assert not scan.problems
names = {e.name_path for e in scan.entries}
assert {"/d/one", "/d/two"} <= names
@pytest.mark.asyncio
async def test_a_real_cycle_is_one_fatal_problem_per_member():
tree = _Tree({}, dirs=("/d", ))
links = _cycle({"/d/a": "/d/b", "/d/b": "/d/a"})
scan = await _scan(tree,
_spec("/d"),
links=links,
dereference=True,
recurse=True)
assert [(p.path, p.reason, p.fatal) for p in scan.problems] == [
("/d/a", aw.TOO_MANY_LEVELS, True),
("/d/b", aw.TOO_MANY_LEVELS, True),
]
# GNU keeps the directory entry and exits 2; it does not abort.
assert [e.name_path for e in scan.entries] == ["/d"]
@pytest.mark.asyncio
async def test_a_dangling_link_is_fatal_with_the_enoent_wording():
tree = _Tree({}, dirs=("/d", ))
links = _links({"/d/bad": "/d/nowhere"})
scan = await _scan(tree,
_spec("/d"),
links=links,
dereference=True,
recurse=True)
assert [(p.reason, p.fatal) for p in scan.problems] == [(aw.NO_SUCH, True)]
@pytest.mark.asyncio
async def test_a_nested_mount_is_reported_and_its_contents_dropped():
tree = _Tree({
"/d/a.txt": b"a",
"/d/nested/deep.txt": b"deep"
},
dirs=("/d", "/d/nested"))
mounts = _mounts(descendants=("/d/nested", ), roots=("/", "/d/nested"))
scan = await _scan(tree, _spec("/d"), mounts=mounts, recurse=True)
assert scan.crossings == ("/d/nested", )
names = [e.name_path for e in scan.entries]
assert names == ["/d", "/d/a.txt", "/d/nested"]
@pytest.mark.asyncio
async def test_a_link_across_a_mount_is_refused_not_followed():
tree = _Tree({"/d/a.txt": b"a"}, dirs=("/d", ))
links = _links({"/d/away": "/m/x.txt"})
mounts = _mounts(roots=("/", "/m"))
scan = await _scan(tree,
_spec("/d"),
links=links,
mounts=mounts,
dereference=True,
recurse=True)
assert [(p.path, p.reason)
for p in scan.problems] == [("/d/away", aw.OTHER_FILESYSTEM)]
python/tests/commands/builtin/generic/test_phase_z.py
-93
View File
@@ -3,10 +3,8 @@ import pytest
from mirage.commands.builtin.generic.diff import diff
from mirage.commands.builtin.generic.jq import jq
from mirage.commands.builtin.generic.patch import patch
from mirage.commands.builtin.generic.tar import tar
from mirage.commands.builtin.generic.tsort import tsort
from mirage.commands.builtin.generic.unzip import unzip
from mirage.commands.builtin.generic.zip_cmd import zip_cmd
from mirage.types import FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
@@ -130,40 +128,6 @@ async def test_jq_no_input():
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_zip_basic():
rb, wb, _, _, store = _make_backend({
"f1.txt": b"hello",
"f2.txt": b"world"
})
out, io = await zip_cmd(
[_spec("out.zip"), _spec("f1.txt"),
_spec("f2.txt")],
read_bytes=rb,
write_bytes=wb)
assert b"adding" in out
assert "out.zip" in store
archive = store["out.zip"]
assert archive.startswith(b"PK")
@pytest.mark.asyncio
async def test_zip_quiet():
rb, wb, _, _, _ = _make_backend({"f.txt": b"x"})
out, _ = await zip_cmd([_spec("o.zip"), _spec("f.txt")],
read_bytes=rb,
write_bytes=wb,
q=True)
assert out is None
@pytest.mark.asyncio
async def test_zip_too_few_paths():
rb, wb, _, _, _ = _make_backend({})
with pytest.raises(ValueError, match="usage"):
await zip_cmd([_spec("only.zip")], read_bytes=rb, write_bytes=wb)
@pytest.mark.asyncio
async def test_unzip_extracts():
import io as _io
@@ -230,63 +194,6 @@ async def test_unzip_pipe_mode():
assert out == b"hello"
@pytest.mark.asyncio
async def test_tar_create_and_list():
rb, wb, _, mk, store = _make_backend({"a.txt": b"alpha", "b.txt": b"beta"})
_, io_res = await tar([_spec("a.txt"), _spec("b.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True,
f=_spec("out.tar"))
assert "/out.tar" in io_res.writes
out, _ = await tar([],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
t=True,
f=_spec("out.tar"))
assert b"a.txt" in out
assert b"b.txt" in out
@pytest.mark.asyncio
async def test_tar_extract():
rb, wb, _, mk, _ = _make_backend({"x.txt": b"data"})
await tar([_spec("x.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True,
f=_spec("a.tar"))
_, io_res = await tar([],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
x=True,
f=_spec("a.tar"),
C=_spec("/out"))
assert any("x.txt" in p for p in io_res.writes)
@pytest.mark.asyncio
async def test_tar_requires_mode():
rb, wb, _, mk, _ = _make_backend({})
with pytest.raises(ValueError, match="-c, -x, or -t"):
await tar([], read_bytes=rb, write_bytes=wb, mkdir_fn=mk)
@pytest.mark.asyncio
async def test_tar_requires_f():
rb, wb, _, mk, _ = _make_backend({"a.txt": b"x"})
with pytest.raises(ValueError, match="-f is required"):
await tar([_spec("a.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True)
@pytest.mark.asyncio
async def test_diff_identical_files():
rb, _, _, _, _ = _make_backend({"a": b"hello\n", "b": b"hello\n"})
python/tests/commands/builtin/generic/test_tar.py
+458
View File
@@ -0,0 +1,458 @@
import io
import tarfile
from dataclasses import replace
import pytest
from mirage.commands.builtin.generic.tar import (excluded, member_name, pruned,
tar)
from mirage.ops.types import LinkView, MountView
from mirage.types import LINK_TARGET_KEY, FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
from mirage.utils.path import CycleError
def _spec(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True)
def _raw(path: str, raw: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True,
raw_path=raw)
class _Tree:
"""A tiny in-memory backend: files by path, directories derived."""
def __init__(self, files: dict[str, bytes], dirs: tuple[str, ...] = ()):
self.files = dict(files)
self.dirs = set(dirs)
for path in files:
parent = path.rsplit("/", 1)[0]
while parent:
self.dirs.add(parent)
parent = parent.rsplit("/", 1)[0] if "/" in parent else ""
async def read_bytes(self, path):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in self.files:
raise FileNotFoundError(key)
return self.files[key]
async def write_bytes(self, path, data):
self.files[path.virtual] = data
async def mkdir(self, path, parents=False):
self.dirs.add(path.virtual.rstrip("/"))
async def stat(self, path):
key = path.virtual.rstrip("/") or "/"
if key in self.dirs:
return FileStat(name=key, type=FileType.DIRECTORY)
if key in self.files:
return FileStat(name=key,
type=FileType.TEXT,
size=len(self.files[key]))
raise FileNotFoundError(key)
async def walk(self, path, find_type):
base = path.virtual.rstrip("/") or "/"
pool = self.dirs if find_type == "d" else self.files
return sorted(p for p in pool
if p == base or p.startswith(base.rstrip("/") + "/"))
async def is_dir(self, path):
return (path.virtual.rstrip("/") or "/") in self.dirs
def _links(entries: dict[str, str]) -> LinkView:
def stat_of(path):
target = entries[path]
return FileStat(name=path,
type=FileType.SYMLINK,
size=len(target),
extra={LINK_TARGET_KEY: target})
async def target_stat(path):
return None
async def exists(path):
return path in entries
return LinkView(
stat_at=lambda p: stat_of(p) if p in entries else None,
children=lambda p: [],
subtree=lambda p: [(k, stat_of(k)) for k in sorted(entries)
if k.startswith(p.rstrip("/") + "/")],
resolve=lambda p: entries.get(p, p),
exists=exists,
target_stat=target_stat,
)
def _cycle(entries: dict[str, str]) -> LinkView:
"""A LinkView whose resolve raises ELOOP, as the namespace does."""
def resolve(path):
raise CycleError(path)
return replace(_links(entries), resolve=resolve)
def _mounts(descendants: tuple[str, ...] = (),
roots: tuple[str, ...] = ()) -> MountView:
def root_of(path):
for root in sorted(roots, key=len, reverse=True):
if path == root or path.startswith(root.rstrip("/") + "/"):
return root
return "/"
return MountView(
descendants=lambda p:
[d for d in descendants if d.startswith(p.rstrip("/") + "/")],
is_root=lambda p: p.rstrip("/") in {r.rstrip("/")
for r in roots},
root_of=root_of,
)
async def _create(tree: _Tree, paths, **flags):
return await tar(paths,
read_bytes=tree.read_bytes,
write_bytes=tree.write_bytes,
mkdir_fn=tree.mkdir,
stat=tree.stat,
walk=tree.walk,
is_dir=tree.is_dir,
**flags)
def _names(archive: bytes) -> list[str]:
with tarfile.open(fileobj=io.BytesIO(archive)) as tf:
return [
member.name + "/" if member.isdir() else member.name
for member in tf.getmembers()
]
def test_excluded_matches_whole_name_and_every_component_suffix():
assert excluded("d/a.txt", "d/a.txt")
assert excluded("d/a.txt", "a.txt")
assert excluded("d/sub/b.txt", "sub/b.txt")
assert excluded("d/sub/b.txt", "*/b.txt")
assert excluded("d/sub/", "sub")
assert not excluded("d/a.txt", "b.txt")
# The pattern is anchored at a component boundary, not mid-name.
assert not excluded("d/abc.txt", "bc.txt")
def test_pruned_takes_the_children_of_an_excluded_directory():
names = ["d/", "d/a.txt", "d/sub/", "d/sub/b.txt"]
assert pruned(names, "sub") == ["d/", "d/a.txt"]
assert pruned(names, "sub/b.txt") == ["d/", "d/a.txt", "d/sub/"]
assert pruned(names, None) == names
def test_member_name_strips_the_leading_slash_and_marks_directories():
assert member_name("/data/d/a.txt", "file") == "data/d/a.txt"
assert member_name("/data/d", "dir") == "data/d/"
assert member_name("d/", "dir") == "d/"
assert member_name("link", "link") == "link"
@pytest.mark.asyncio
async def test_create_walks_a_directory_operand():
tree = _Tree({
"/d/a.txt": b"alpha",
"/d/sub/b.txt": b"beta"
},
dirs=("/d", "/d/sub", "/d/empty"))
out, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
v=True,
f=_spec("/out.tar"))
assert io_res.exit_code == 0
assert out.decode().split() == [
"d/", "d/a.txt", "d/empty/", "d/sub/", "d/sub/b.txt"
]
assert _names(io_res.writes["/out.tar"]) == [
"d/", "d/a.txt", "d/empty/", "d/sub/", "d/sub/b.txt"
]
@pytest.mark.asyncio
async def test_create_keeps_an_empty_directory_as_its_own_member():
tree = _Tree({"/d/a.txt": b"x"}, dirs=("/d", "/d/empty"))
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/out.tar"))
assert "d/empty/" in _names(io_res.writes["/out.tar"])
@pytest.mark.asyncio
async def test_create_names_members_as_the_operand_was_typed():
tree = _Tree({"/base/d/a.txt": b"x"}, dirs=("/base", "/base/d"))
_, io_res = await _create(tree, [_raw("/base/d", "d")],
c=True,
f=_spec("/out.tar"))
assert _names(io_res.writes["/out.tar"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_create_warns_once_about_a_stripped_leading_slash():
tree = _Tree({"/d/a.txt": b"x"}, dirs=("/d", ))
_, io_res = await _create(tree, [_spec("/d")], c=True, f=_spec("/out.tar"))
assert io_res.stderr.decode().count("Removing leading") == 1
assert _names(io_res.writes["/out.tar"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_create_reports_a_missing_operand_and_exits_two():
tree = _Tree({"/d/a.txt": b"x"}, dirs=("/d", ))
_, io_res = await _create(
tree, [_raw("/nope", "nope"), _raw("/d", "d")],
c=True,
f=_spec("/out.tar"))
assert io_res.exit_code == 2
err = io_res.stderr.decode()
assert "tar: nope: Cannot stat: No such file or directory" in err
assert "Exiting with failure status due to previous errors" in err
# GNU still archives every operand it could read.
assert "d/a.txt" in _names(io_res.writes["/out.tar"])
@pytest.mark.asyncio
async def test_create_refuses_an_empty_archive():
tree = _Tree({})
out, io_res = await _create(tree, [], c=True, f=_spec("/out.tar"))
assert out is None
assert io_res.exit_code == 2
assert "Cowardly refusing" in io_res.stderr.decode()
assert not io_res.writes
@pytest.mark.asyncio
async def test_create_refuses_a_directory_it_cannot_enter():
tree = _Tree({"/d/a.txt": b"x"}, dirs=("/d", ))
_, io_res = await _create(tree, [_raw("/nodir/a.txt", "a.txt")],
c=True,
f=_spec("/out.tar"),
C=[_raw("/nodir", "nodir")])
assert io_res.exit_code == 2
err = io_res.stderr.decode()
assert "tar: nodir: Cannot open: No such file or directory" in err
assert "Error is not recoverable: exiting now" in err
assert not io_res.writes
@pytest.mark.asyncio
async def test_create_prunes_an_excluded_subtree():
tree = _Tree({
"/d/a.txt": b"a",
"/d/sub/b.txt": b"b"
},
dirs=("/d", "/d/sub"))
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/out.tar"),
exclude="sub")
assert _names(io_res.writes["/out.tar"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_create_stores_a_symlink_as_a_symlink():
tree = _Tree({"/d/a.txt": b"a"}, dirs=("/d", ))
links = _links({"/d/link.txt": "a.txt"})
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/out.tar"),
links=links)
with tarfile.open(fileobj=io.BytesIO(io_res.writes["/out.tar"])) as tf:
link = tf.getmember("d/link.txt")
assert link.issym()
assert link.linkname == "a.txt"
@pytest.mark.asyncio
async def test_dereference_stores_the_target_content_under_the_link_name():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/link.txt": "/d/a.txt"})
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
h=True,
f=_spec("/out.tar"),
links=links)
with tarfile.open(fileobj=io.BytesIO(io_res.writes["/out.tar"])) as tf:
member = tf.getmember("d/link.txt")
assert not member.issym()
assert tf.extractfile(member).read() == b"alpha"
@pytest.mark.asyncio
async def test_dereference_reports_a_dangling_link_and_exits_two():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/bad": "/d/nope"})
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
h=True,
f=_spec("/out.tar"),
links=links)
assert io_res.exit_code == 2
assert "tar: d/bad: Cannot stat" in io_res.stderr.decode()
@pytest.mark.asyncio
async def test_create_stops_at_a_nested_mount_and_says_so():
tree = _Tree({"/d/a.txt": b"a"}, dirs=("/d", "/d/nested"))
mounts = _mounts(descendants=("/d/nested", ), roots=("/", "/d/nested"))
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/out.tar"),
mounts=mounts)
assert io_res.exit_code == 0
assert ("tar: d/nested/: file is on a different filesystem; not dumped"
in io_res.stderr.decode())
# The mountpoint stays an entry; only its contents are left out.
assert _names(io_res.writes["/out.tar"]) == ["d/", "d/a.txt", "d/nested/"]
@pytest.mark.asyncio
async def test_create_leaves_the_archive_out_of_itself():
tree = _Tree({"/d/a.txt": b"a", "/d/old.tar": b"stale"}, dirs=("/d", ))
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/d/old.tar"))
assert "archive cannot contain itself" in io_res.stderr.decode()
assert _names(io_res.writes["/d/old.tar"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_create_and_list_round_trip_plain_files():
tree = _Tree({"/a.txt": b"alpha", "/b.txt": b"beta"})
_, io_res = await _create(
tree, [_spec("/a.txt"), _spec("/b.txt")], c=True, f=_spec("/out.tar"))
assert "/out.tar" in io_res.writes
out, _ = await _create(tree, [], t=True, f=_spec("/out.tar"))
assert out.decode().split() == ["a.txt", "b.txt"]
@pytest.mark.asyncio
async def test_extract_recreates_directories_including_empty_ones():
tree = _Tree({"/d/a.txt": b"x"}, dirs=("/d", "/d/empty"))
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
f=_spec("/out.tar"))
tree.files["/out.tar"] = io_res.writes["/out.tar"]
_, io_res = await _create(tree, [],
x=True,
f=_spec("/out.tar"),
C=[_spec("/out")])
assert any("d/a.txt" in path for path in io_res.writes)
assert "/out/d/empty" in tree.dirs
@pytest.mark.asyncio
async def test_extract_strips_leading_components():
tree = _Tree({"/deep/d/a.txt": b"x"}, dirs=("/deep", "/deep/d"))
_, io_res = await _create(tree, [_spec("/deep/d")],
c=True,
f=_spec("/out.tar"))
tree.files["/out.tar"] = io_res.writes["/out.tar"]
_, io_res = await _create(tree, [],
x=True,
f=_spec("/out.tar"),
strip_components="2",
C=[_spec("/out")])
assert "/out/a.txt" in io_res.writes
@pytest.mark.asyncio
async def test_requires_a_mode():
tree = _Tree({})
with pytest.raises(ValueError, match="-c, -x, or -t"):
await _create(tree, [])
@pytest.mark.asyncio
async def test_requires_an_archive():
tree = _Tree({"/a.txt": b"x"})
with pytest.raises(ValueError, match="-f is required"):
await _create(tree, [_spec("/a.txt")], c=True)
@pytest.mark.asyncio
async def test_create_fails_at_the_first_unenterable_c_not_the_last():
"""GNU chdirs at each -C, so a bad early one stops the whole run.
Checking only the parsed flag's final value archived the operands
that followed the bad one and named the wrong subject.
"""
tree = _Tree({"/good/y.txt": b"y"}, dirs=("/good", ))
_, io_res = await _create(
tree, [_raw("/good/y.txt", "y.txt")],
c=True,
f=_spec("/out.tar"),
C=[_raw("/missing", "missing"),
_raw("/good", "good")])
assert io_res.exit_code == 2
err = io_res.stderr.decode()
assert "tar: missing: Cannot open: No such file or directory" in err
assert "Error is not recoverable" in err
assert not io_res.writes
@pytest.mark.asyncio
async def test_two_links_to_one_target_are_not_a_loop():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/one": "/d/a.txt", "/d/two": "/d/a.txt"})
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
h=True,
f=_spec("/out.tar"),
links=links)
assert io_res.exit_code == 0
assert io_res.stderr == b""
assert _names(
io_res.writes["/out.tar"]) == ["d/", "d/a.txt", "d/one", "d/two"]
@pytest.mark.asyncio
async def test_a_symlink_cycle_is_reported_per_member_and_keeps_the_directory(
):
tree = _Tree({}, dirs=("/d", ))
links = _cycle({"/d/a": "/d/b", "/d/b": "/d/a"})
_, io_res = await _create(tree, [_raw("/d", "d")],
c=True,
h=True,
f=_spec("/out.tar"),
links=links)
assert io_res.exit_code == 2
err = io_res.stderr.decode()
assert "tar: d/a: Cannot stat: Too many levels of symbolic links" in err
assert "tar: d/b: Cannot stat: Too many levels of symbolic links" in err
# GNU keeps the directory entry rather than aborting the archive.
assert _names(io_res.writes["/out.tar"]) == ["d/"]
@pytest.mark.asyncio
async def test_a_symlink_operand_is_stored_as_a_symlink():
"""The router must not dereference it before the planner sees it."""
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/link": "/d/a.txt"})
_, io_res = await _create(tree, [_raw("/link", "link")],
c=True,
f=_spec("/out.tar"),
links=links)
with tarfile.open(fileobj=io.BytesIO(io_res.writes["/out.tar"])) as tf:
member = tf.getmember("link")
assert member.issym()
assert member.size == 0
assert member.linkname == "/d/a.txt"
python/tests/commands/builtin/generic/test_zip_cmd.py
+305
View File
@@ -0,0 +1,305 @@
import io
import zipfile
import pytest
from mirage.commands.builtin.generic.zip_cmd import (excluded, member_name,
zip_cmd)
from mirage.ops.types import LinkView, MountView
from mirage.types import LINK_TARGET_KEY, FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
def _spec(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True)
def _raw(path: str, raw: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True,
raw_path=raw)
class _Tree:
"""A tiny in-memory backend: files by path, directories derived."""
def __init__(self, files: dict[str, bytes], dirs: tuple[str, ...] = ()):
self.files = dict(files)
self.dirs = set(dirs)
for path in files:
parent = path.rsplit("/", 1)[0]
while parent:
self.dirs.add(parent)
parent = parent.rsplit("/", 1)[0] if "/" in parent else ""
async def read_bytes(self, path):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in self.files:
raise FileNotFoundError(key)
return self.files[key]
async def write_bytes(self, path, data):
self.files[path.virtual] = data
async def stat(self, path):
key = path.virtual.rstrip("/") or "/"
if key in self.dirs:
return FileStat(name=key, type=FileType.DIRECTORY)
if key in self.files:
return FileStat(name=key,
type=FileType.TEXT,
size=len(self.files[key]))
raise FileNotFoundError(key)
async def walk(self, path, find_type):
base = path.virtual.rstrip("/") or "/"
pool = self.dirs if find_type == "d" else self.files
return sorted(p for p in pool
if p == base or p.startswith(base.rstrip("/") + "/"))
def _links(entries: dict[str, str]) -> LinkView:
def stat_of(path):
target = entries[path]
return FileStat(name=path,
type=FileType.SYMLINK,
size=len(target),
extra={LINK_TARGET_KEY: target})
async def target_stat(path):
return None
async def exists(path):
return path in entries
return LinkView(
stat_at=lambda p: stat_of(p) if p in entries else None,
children=lambda p: [],
subtree=lambda p: [(k, stat_of(k)) for k in sorted(entries)
if k.startswith(p.rstrip("/") + "/")],
resolve=lambda p: entries.get(p, p),
exists=exists,
target_stat=target_stat,
)
def _mounts(descendants: tuple[str, ...] = (),
roots: tuple[str, ...] = ()) -> MountView:
def root_of(path):
for root in sorted(roots, key=len, reverse=True):
if path == root or path.startswith(root.rstrip("/") + "/"):
return root
return "/"
return MountView(
descendants=lambda p:
[d for d in descendants if d.startswith(p.rstrip("/") + "/")],
is_root=lambda p: p.rstrip("/") in {r.rstrip("/")
for r in roots},
root_of=root_of,
)
async def _zip(tree: _Tree, paths, **flags):
return await zip_cmd(paths,
read_bytes=tree.read_bytes,
write_bytes=tree.write_bytes,
stat=tree.stat,
walk=tree.walk,
**flags)
def _entries(archive: bytes) -> list[str]:
with zipfile.ZipFile(io.BytesIO(archive)) as zf:
return [info.filename for info in zf.infolist()]
def test_member_name_strips_the_leading_slash_and_marks_directories():
assert member_name("/d/a.txt", "file", False) == "d/a.txt"
assert member_name("/d", "dir", False) == "d/"
assert member_name("/d/sub/b.txt", "file", True) == "b.txt"
assert member_name("link", "link", False) == "link"
def test_excluded_is_anchored_unlike_tars_exclude():
assert excluded("d/sub/b.txt", ["d/sub/*"])
assert excluded("d/sub/", ["d/sub/*"])
assert excluded("d/a.txt", ["*.txt"])
assert excluded("d/sub/b.txt", ["*/b.txt"])
# Info-ZIP matches the whole stored name, so a bare component misses.
assert not excluded("d/sub/b.txt", ["b.txt"])
assert not excluded("d/sub/b.txt", ["sub/*"])
@pytest.mark.asyncio
async def test_recurses_a_directory_operand_under_r():
tree = _Tree({
"/d/a.txt": b"alpha",
"/d/sub/b.txt": b"beta"
},
dirs=("/d", "/d/sub", "/d/empty"))
out, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")], r=True)
assert io_res.exit_code == 0
assert _entries(io_res.writes["/out.zip"]) == [
"d/", "d/a.txt", "d/empty/", "d/sub/", "d/sub/b.txt"
]
assert out.decode().startswith(" adding: d/\n")
@pytest.mark.asyncio
async def test_without_r_a_directory_stores_only_itself():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
_, io_res = await _zip(tree, [_spec("/out.zip"), _raw("/d", "d")])
assert _entries(io_res.writes["/out.zip"]) == ["d/"]
@pytest.mark.asyncio
async def test_directory_entries_carry_no_content():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
_, io_res = await _zip(tree, [_spec("/out.zip"), _raw("/d", "d")], r=True)
with zipfile.ZipFile(io.BytesIO(io_res.writes["/out.zip"])) as zf:
info = zf.getinfo("d/")
assert info.is_dir()
assert info.file_size == 0
@pytest.mark.asyncio
async def test_junk_paths_drops_directories_and_keeps_basenames():
tree = _Tree({
"/d/a.txt": b"alpha",
"/d/sub/b.txt": b"beta"
},
dirs=("/d", "/d/sub"))
_, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")], r=True, j=True)
assert _entries(io_res.writes["/out.zip"]) == ["a.txt", "b.txt"]
@pytest.mark.asyncio
async def test_exclude_pattern_prunes_by_stored_name():
tree = _Tree({
"/d/a.txt": b"alpha",
"/d/sub/b.txt": b"beta"
},
dirs=("/d", "/d/sub"))
_, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")], r=True, x=["d/sub/*"])
assert _entries(io_res.writes["/out.zip"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_follows_a_symlink_by_default():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/link.txt": "/d/a.txt"})
_, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")], r=True, links=links)
with zipfile.ZipFile(io.BytesIO(io_res.writes["/out.zip"])) as zf:
assert zf.read("d/link.txt") == b"alpha"
@pytest.mark.asyncio
async def test_y_stores_a_symlink_as_a_symlink():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
links = _links({"/d/link.txt": "a.txt"})
_, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")],
r=True,
y=True,
links=links)
with zipfile.ZipFile(io.BytesIO(io_res.writes["/out.zip"])) as zf:
info = zf.getinfo("d/link.txt")
assert zf.read(info) == b"a.txt"
assert info.external_attr >> 16 == 0o120777
@pytest.mark.asyncio
async def test_warns_on_a_name_it_cannot_match_but_still_archives_the_rest():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", ))
_, io_res = await _zip(tree, [
_spec("/out.zip"),
_raw("/d/a.txt", "d/a.txt"),
_raw("/nope", "nope")
])
assert io_res.exit_code == 0
assert io_res.stderr.decode() == "\tzip warning: name not matched: nope\n"
assert _entries(io_res.writes["/out.zip"]) == ["d/a.txt"]
@pytest.mark.asyncio
async def test_nothing_to_do_writes_no_archive_and_exits_twelve():
tree = _Tree({})
out, io_res = await _zip(
tree, [_raw("/out.zip", "out.zip"),
_raw("/nope", "nope")])
assert out is None
assert io_res.exit_code == 12
assert not io_res.writes
err = io_res.stderr.decode()
assert err.startswith("\tzip warning: name not matched: nope\n")
assert err.endswith("\nzip error: Nothing to do! (out.zip)\n")
@pytest.mark.asyncio
async def test_quiet_silences_the_warning_but_not_the_fatal_error():
tree = _Tree({})
_, io_res = await _zip(
tree, [_raw("/out.zip", "out.zip"),
_raw("/nope", "nope")], q=True)
assert io_res.stderr.decode() == "\nzip error: Nothing to do! (out.zip)\n"
@pytest.mark.asyncio
async def test_quiet_prints_no_adding_lines():
tree = _Tree({"/a.txt": b"alpha"})
out, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/a.txt", "a.txt")], q=True)
assert out is None
assert _entries(io_res.writes["/out.zip"]) == ["a.txt"]
@pytest.mark.asyncio
async def test_stops_at_a_nested_mount_and_says_so():
tree = _Tree({"/d/a.txt": b"alpha"}, dirs=("/d", "/d/nested"))
mounts = _mounts(descendants=("/d/nested", ), roots=("/", "/d/nested"))
_, io_res = await _zip(
tree, [_spec("/out.zip"), _raw("/d", "d")], r=True, mounts=mounts)
assert io_res.exit_code == 0
assert ("\tzip warning: d/nested: file is on a different filesystem; "
"not dumped\n" in io_res.stderr.decode())
# The mountpoint stays an entry; only its contents are left out.
assert _entries(
io_res.writes["/out.zip"]) == ["d/", "d/a.txt", "d/nested/"]
@pytest.mark.asyncio
async def test_leaves_the_archive_out_of_itself():
tree = _Tree({"/d/a.txt": b"alpha", "/d/old.zip": b"stale"}, dirs=("/d", ))
_, io_res = await _zip(
tree, [_spec("/d/old.zip"), _raw("/d", "d")], r=True)
assert _entries(io_res.writes["/d/old.zip"]) == ["d/", "d/a.txt"]
@pytest.mark.asyncio
async def test_names_members_on_a_prefixed_mount():
tree = _Tree({"/data/d/a.txt": b"alpha"}, dirs=("/data", "/data/d"))
await _zip(
tree,
[_spec("/data/out.zip", "/data"),
_raw("/data/d", "/data/d", "/data")],
r=True)
assert _entries(tree.files["/data/out.zip"]) == ["data/d/", "data/d/a.txt"]
@pytest.mark.asyncio
async def test_requires_an_archive_operand():
tree = _Tree({})
with pytest.raises(ValueError, match="usage"):
await _zip(tree, [])
python/tests/commands/spec/test_parser.py
+41 -2
View File
@@ -637,7 +637,7 @@ def test_tar_old_style_two_value_letters_bind_in_letter_order():
parsed = parse_command(SPECS["tar"], ["xfC", "/data/a.tgz", "/data/out"],
"/")
assert parsed.flags["-f"] == "/data/a.tgz"
assert parsed.flags["-C"] == "/data/out"
assert parsed.flags["-C"] == ["/data/out"]
def test_tar_old_style_value_letter_before_bool_letter():
@@ -670,7 +670,7 @@ def test_tar_old_style_still_accepts_long_options_after_the_cluster():
["xzf", "/data/a.tgz", "--strip-components", "1", "-C", "/data/out"],
"/")
assert parsed.flags["--strip-components"] == "1"
assert parsed.flags["-C"] == "/data/out"
assert parsed.flags["-C"] == ["/data/out"]
def test_old_option_style_is_off_for_every_other_command():
@@ -678,3 +678,42 @@ def test_old_option_style_is_off_for_every_other_command():
parsed = parse_command(SPECS["gzip"], ["dkf"], "/")
assert parsed.paths() == ["/dkf"]
assert parsed.old_option_needs_value is None
def test_operand_base_rebases_the_operands_typed_after_it():
# GNU tar's -C is a chdir for the operands that follow it, so the
# archive (-f) stays relative to the session cwd while the files move.
parsed = parse_command(
SPECS["tar"], ["-czf", "out.tgz", "-C", "/work/check", "my_paper"],
cwd="/home")
assert parsed.paths() == ["/work/check/my_paper"]
assert parsed.flags["-f"] == "/home/out.tgz"
assert parsed.flags["-C"] == ["/work/check"]
def test_operand_base_is_cumulative_like_a_real_chdir():
parsed = parse_command(
SPECS["tar"], ["-cf", "a.tar", "-C", "d1", "x", "-C", "../d2", "y"],
cwd="/work")
assert parsed.paths() == ["/work/d1/x", "/work/d2/y"]
# Every occurrence is kept in order: GNU chdirs at each one.
assert parsed.flags["-C"] == ["/work/d1", "/work/d2"]
def test_operand_base_only_moves_what_follows_it():
parsed = parse_command(
SPECS["tar"], ["-cf", "a.tar", "top.txt", "-C", "/work/e", "e.txt"],
cwd="/work")
assert parsed.paths() == ["/work/top.txt", "/work/e/e.txt"]
def test_operand_base_survives_the_old_style_cluster():
parsed = parse_command(SPECS["tar"], ["czf", "a.tgz", "-C", "sub", "x"],
cwd="/work")
assert parsed.paths() == ["/work/sub/x"]
assert parsed.word_bases[-1] == "/work/sub"
def test_word_bases_are_empty_without_an_operand_base():
parsed = parse_command(SPECS["cat"], ["a.txt"], cwd="/work")
assert parsed.word_bases == [None]
python/tests/commands/test_no_dead_flag_params.py
+1
View File
@@ -48,6 +48,7 @@ DISPATCHER_PARAMS = frozenset({
"stat_overlay",
"links",
"stat_path",
"mounts",
"prefix",
"command",
})
python/tests/policy/builtin/test_mount_root.py
+86 -6
View File
@@ -38,12 +38,15 @@ def _path(virtual: str, raw: str | None = None) -> PathSpec:
def _ctx(command: str,
paths: list[PathSpec],
argv: list[str] | None = None,
registry: MountRegistry | None = None) -> CommandContext:
return CommandContext(command=command,
paths=tuple(paths),
argv=tuple(argv or []),
cwd="/",
registry=registry or _registry())
registry: MountRegistry | None = None,
operands: list[PathSpec] | None = None) -> CommandContext:
return CommandContext(
command=command,
paths=tuple(paths),
operands=tuple(paths if operands is None else operands),
argv=tuple(argv or []),
cwd="/",
registry=registry or _registry())
@pytest.mark.parametrize("cmd,needle", [
@@ -114,3 +117,80 @@ def test_has_parents_flag_spots_the_shorthand_cluster():
assert has_parents_flag(("-pv", ))
assert not has_parents_flag(("--print", ))
assert not has_parents_flag(("x", "-r"))
@pytest.mark.parametrize("cmd,needle", [
("tar", "tar: /data: Cannot open: Device or resource busy"),
("zip", "zip: cannot read '/data': Device or resource busy"),
("cp", "cp: cannot copy '/data': Device or resource busy"),
])
@pytest.mark.asyncio
async def test_whole_mount_archivers_refused(cmd, needle):
# zip's first operand is the archive it writes, cp's last is the
# destination, so each line puts the mount root in a source slot.
operands = {
"tar": [_path("/data")],
"zip": [_path("/out.zip"), _path("/data")],
"cp": [_path("/data"), _path("/dst")],
}[cmd]
argv = ["-cf", "/out.tar"] if cmd == "tar" else []
deny = await MountRootPolicy().pre_command(_ctx(cmd, operands, argv=argv))
assert deny is not None
assert needle in deny.message
@pytest.mark.asyncio
async def test_tar_refusal_names_the_operand_as_typed_and_exits_two():
deny = await MountRootPolicy().pre_command(
_ctx("tar", [_path("/data", raw=".")], argv=["-cf", "/out.tar"]))
assert deny is not None
assert "tar: .: Cannot open" in deny.message
assert "Error is not recoverable" in deny.message
assert deny.exit_code == 2
@pytest.mark.asyncio
async def test_extracting_into_a_mount_root_is_allowed():
# `-C /data` is a path-valued flag, so it reaches paths but never
# operands; refusing it would block the safe direction.
deny = await MountRootPolicy().pre_command(
_ctx("tar",
[_path("/archive.tar"), _path("/data")],
operands=[_path("/archive.tar")]))
assert deny is None
@pytest.mark.asyncio
async def test_copying_into_a_mount_root_is_allowed():
deny = await MountRootPolicy().pre_command(
_ctx("cp", [_path("/src/a.txt"), _path("/data")]))
assert deny is None
@pytest.mark.asyncio
async def test_zip_archive_slot_is_not_a_source():
deny = await MountRootPolicy().pre_command(
_ctx("zip", [_path("/data"), _path("/src/a.txt")]))
assert deny is None
@pytest.mark.parametrize("argv,denied", [
(["-cf", "/a.tar"], True),
(["--create", "-f", "/a.tar"], True),
(["cf", "/a.tar"], True),
(["-tf", "/a.tar"], False),
(["-xf", "/a.tar"], False),
(["xzf", "/a.tar"], False),
(["-xf", "/a.tar", "-C", "/cache"], False),
])
@pytest.mark.asyncio
async def test_only_tar_create_reads_its_operands_from_the_filesystem(
argv, denied):
"""Under -t and -x an operand names a member, not a path.
A selector that happens to spell a mount root is not a mount, so
refusing it would deny an ordinary listing or extraction.
"""
deny = await MountRootPolicy().pre_command(
_ctx("tar", [_path("/data")], argv=argv))
assert (deny is not None) is denied
scripts/check_spec_parity.py
+4 -2
View File
@@ -222,12 +222,14 @@ def describe(diff: str, py: dict[str, Any], ts: dict[str, Any],
return (f" {diff}: python={py['_meta'].get(key)!r} "
f"typescript={ts['_meta'].get(key)!r}")
if diff == "options":
# An option that declares only one spelling carries only that
# key, since the dumps omit anything left at its default.
py_by_name = {
o["long"] or o["short"]: o
o.get("long") or o.get("short"): o
for o in py.get("options", [])
}
ts_by_name = {
o["long"] or o["short"]: o
o.get("long") or o.get("short"): o
for o in ts.get("options", [])
}
lines = [f" {diff}:"]
scripts/gen_specs.py
+50 -2
View File
@@ -17,13 +17,14 @@ import json
import logging
import pkgutil
import sys
from dataclasses import asdict
from dataclasses import MISSING, Field, asdict, fields
from pathlib import Path
from typing import Any
import mirage.commands.builtin
from mirage.commands.config import RegisteredCommand
from mirage.commands.spec import SPECS
from mirage.commands.spec.types import CommandSpec, Operand, Option
from mirage.resource.registry import REGISTRY
logger = logging.getLogger(__name__)
@@ -122,8 +123,55 @@ def _default(o: object) -> object:
raise TypeError(f"unserializable: {type(o)}")
def _default_of(f: Field) -> Any:
if f.default_factory is not MISSING:
return f.default_factory()
return f.default
def _prune(payload: dict[str, Any], cls: type) -> dict[str, Any]:
"""``payload`` without the fields ``cls`` would have defaulted anyway.
A spec dump is a cross-language contract, and restating every
default in all 93 files buries the handful of facts each command
actually declares. The defaults come from the dataclass rather than
a second table, so a field added to a spec type cannot fall out of
step with this. ``type`` survives even at its default, because what
a token *is* is the first thing a reader looks for.
The typescript side prunes against a default-constructed instance
for the same reason; the two must drop exactly the same keys or the
parity gate reports every command.
Args:
payload (dict[str, Any]): one ``asdict`` level, every key
present.
cls (type): the dataclass the payload came from.
"""
kept: dict[str, Any] = {}
for f in fields(cls):
value = payload[f.name]
if f.name != "type" and value == _default_of(f):
continue
kept[f.name] = value
return kept
def _spec_payload(spec: Any) -> dict[str, Any]:
payload = _prune(asdict(spec), CommandSpec)
if "options" in payload:
payload["options"] = [_prune(o, Option) for o in payload["options"]]
if "positional" in payload:
payload["positional"] = [
_prune(p, Operand) for p in payload["positional"]
]
if payload.get("rest") is not None:
payload["rest"] = _prune(payload["rest"], Operand)
return payload
def _emit_one(name: str, spec: Any, rcs: list[RegisteredCommand]) -> None:
payload = asdict(spec)
payload = _spec_payload(spec)
payload["_meta"] = _meta_for(rcs)
path = OUT / f"{name}.json"
path.write_text(
spec/python/general/awk.json
+3 -41
View File
@@ -254,55 +254,20 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
}
],
"positional": [
@@ -310,13 +275,10 @@
"provided_by": [
"-f"
],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/base64.json
+5 -53
View File
@@ -254,78 +254,30 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--decode",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-D",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--wrap",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--ignore-garbage",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/basename.json
+3 -40
View File
@@ -254,61 +254,24 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--multiple",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/bash.json
+11 -127
View File
@@ -8,180 +8,64 @@
"resources": []
},
"description": "Run a command string through Mirage's shell. Only `-c` is meaningful; other flags are accepted and ignored. `bash` and `sh` are aliases.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Read commands from the next argument and execute them.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read commands from stdin instead of from an argument.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Login shell. Mirage does not source profile files.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Interactive flag. Mirage shells are non-interactive.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Exit on first error.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Treat unset variables as errors.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Print commands as they execute.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-x",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Login shell.",
"long": "--login",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Skip rc files.",
"long": "--norc",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) Skip profile files.",
"long": "--noprofile",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "(Ignored) POSIX-conformant mode.",
"long": "--posix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/bc.json
+2 -28
View File
@@ -15,45 +15,19 @@
"resources": []
},
"description": "Arbitrary precision calculator language.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Load the standard math library.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Suppress the welcome banner.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-q",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/cat.json
+10 -120
View File
@@ -261,166 +261,56 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--number",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--number-nonblank",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--show-ends",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-E",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--show-tabs",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-T",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--show-nonprinting",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--show-all",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--squeeze-blank",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/chgrp.json
+4 -56
View File
@@ -7,82 +7,30 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/chmod.json
+3 -44
View File
@@ -7,67 +7,26 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/chown.json
+4 -56
View File
@@ -7,82 +7,30 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/cmp.json
+6 -70
View File
@@ -254,98 +254,34 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/column.json
+3 -43
View File
@@ -254,61 +254,21 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/comm.json
+9 -105
View File
@@ -254,143 +254,47 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-1",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-2",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-3",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--check-order",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--nocheck-order",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--output-delimiter",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--total",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero-terminated",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/cp.json
+11 -146
View File
@@ -100,211 +100,76 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--recursive",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--archive",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--force",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--interactive",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-clobber",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--verbose",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--update",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": false,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--backup",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": false,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-S",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--target-directory",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-target-directory",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-T",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--strip-trailing-slashes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/csplit.json
+8 -98
View File
@@ -107,142 +107,52 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--prefix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--digits",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix-format",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--keep-files",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-k",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--quiet",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--silent",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suppress-matched",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--elide-empty-files",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/curl.json
+10 -115
View File
@@ -15,165 +15,60 @@
"resources": []
},
"description": "Transfer data from or to a server.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Add a custom header to the request.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-H",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set the User-Agent header.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Specify the HTTP request method.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-X",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Send the given data as the request body.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Submit a multipart/form-data field.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Write response body to the given file.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Follow HTTP redirects.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Fail with exit 22 on an HTTP error status.",
"long": "--fail",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Run silently with no progress or messages.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Show errors even when silent.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-S",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/cut.json
+12 -154
View File
@@ -254,211 +254,69 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--fields",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--delimiter",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--characters",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--bytes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-partial",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--complement",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--only-delimited",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-O",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--output-delimiter",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--whitespace-delimited",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero-terminated",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/date.json
+5 -51
View File
@@ -15,77 +15,31 @@
"resources": []
},
"description": "Print or set the system date and time.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Display the time described by the given date string.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Use Coordinated Universal Time (UTC).",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Output date in ISO 8601 format.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-I",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Output date in RFC 5322 email format.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": null
]
}
spec/python/general/df.json
+8 -103
View File
@@ -7,136 +7,41 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-H",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-k",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-T",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-P",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-B",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/diff.json
+8 -94
View File
@@ -254,128 +254,42 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-q",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/dirname.json
+1 -18
View File
@@ -254,31 +254,14 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/du.json
+7 -90
View File
@@ -254,121 +254,38 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--max-depth",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-P",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/echo.json
+2 -31
View File
@@ -7,46 +7,17 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/expand.json
+2 -29
View File
@@ -254,46 +254,19 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--tabs",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--initial",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/expr.json
-7
View File
@@ -15,14 +15,7 @@
"resources": []
},
"description": "Evaluate expressions.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/file.json
+4 -55
View File
@@ -254,76 +254,25 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/find.json
+23 -274
View File
@@ -261,364 +261,113 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [
"(",
")"
],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-name",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-type",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-maxdepth",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-size",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-mtime",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-iname",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-path",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-mindepth",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-P",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-H",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-print",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-print0",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-delete",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-depth",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-prune",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-ls",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-empty",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-or",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-and",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-not",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/fmt.json
+7 -84
View File
@@ -254,121 +254,44 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--width",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--goal",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-g",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--crown-margin",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--prefix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--split-only",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--tagged-paragraph",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--uniform-spacing",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/fold.json
+4 -51
View File
@@ -254,76 +254,29 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--width",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--spaces",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--bytes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--characters",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/grep.json
+23 -301
View File
@@ -261,385 +261,110 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-I",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-E",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-G",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-q",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-H",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-B",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-C",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--color",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--colour",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--line-buffered",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
@@ -648,13 +373,10 @@
"-e",
"-f"
],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/gunzip.json
+4 -55
View File
@@ -107,76 +107,25 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-k",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/gzip.json
+13 -163
View File
@@ -107,211 +107,61 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-k",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-1",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-2",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-3",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-4",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-5",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-6",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-7",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-8",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-9",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/head.json
+6 -73
View File
@@ -261,106 +261,39 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--lines",
"multiple": false,
"numeric_shorthand": true,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--bytes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--quiet",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-q",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--silent",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--verbose",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero-terminated",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/history.json
+8 -94
View File
@@ -17,135 +17,49 @@
]
},
"description": "Show command history for the session.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Clear the command history.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Delete the entry at the given position; negative counts back from the end.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Append the args to the history as a single entry without executing them.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Print the args without storing them.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Append: no-op (file and store are the same).",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read: no-op (file and store are the same).",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Write: no-op (file and store are the same).",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read-new: no-op (file and store are the same).",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/iconv.json
+4 -55
View File
@@ -107,76 +107,25 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/join.json
+14 -164
View File
@@ -254,218 +254,68 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-1",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-2",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--ignore-case",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-j",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero-terminated",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--check-order",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--nocheck-order",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--header",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/jq.json
+24 -254
View File
@@ -254,370 +254,142 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Use null as the single input value",
"long": "--null-input",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read each line as a string instead of JSON",
"long": "--raw-input",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read all inputs into one array",
"long": "--slurp",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Compact instead of pretty-printed output",
"long": "--compact-output",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Output strings without quotes or escapes",
"long": "--raw-output",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Implies -r and writes NUL after each output",
"long": "--raw-output0",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Implies -r and writes no trailing newline",
"long": "--join-output",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-j",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Escape non-ASCII characters in output",
"long": "--ascii-output",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Sort object keys on output",
"long": "--sort-keys",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-S",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set the exit status from the last output",
"long": "--exit-status",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Indent with tabs",
"long": "--tab",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Indent with n spaces (max 7)",
"long": "--indent",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "int",
"value_optional": false
"type": "int"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Disable colored output (already the default)",
"long": "--monochrome-output",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-M",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Accepted for compatibility; output is one buffer",
"long": "--unbuffered",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read the filter from a file",
"long": "--from-file",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read each input as its [path, leaf] events",
"long": "--stream",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read and write RS-delimited JSON text sequences",
"long": "--seq",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set $name to a string value",
"long": "--arg",
"multiple": false,
"numeric_shorthand": false,
"pair": true,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set $name to a JSON value",
"long": "--argjson",
"multiple": false,
"numeric_shorthand": false,
"pair": true,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set $name to a file's contents",
"long": "--rawfile",
"multiple": false,
"numeric_shorthand": false,
"pair": true,
"required": false,
"short": null,
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Set $name to a file's documents, as an array",
"long": "--slurpfile",
"multiple": false,
"numeric_shorthand": false,
"pair": true,
"required": false,
"short": null,
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read the remaining operands as positional string values",
"long": "--args",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Read the remaining operands as positional JSON values",
"long": "--jsonargs",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Show this help and exit",
"long": "--help",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
@@ -625,12 +397,10 @@
"provided_by": [
"-f"
],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [
"--args",
"--jsonargs"
spec/python/general/js.json
+2 -27
View File
@@ -15,45 +15,20 @@
"resources": []
},
"description": "Run JavaScript on a sandboxed quickjs engine.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Evaluate the next argument as a script.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Run as an ES module (top-level import/export/await); .mjs files select this automatically.",
"long": "--module",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/ln.json
+4 -55
View File
@@ -107,76 +107,25 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/look.json
+2 -22
View File
@@ -254,38 +254,18 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/ls.json
+12 -161
View File
@@ -261,211 +261,62 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-a",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-S",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-1",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--color",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": true
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/md5.json
-8
View File
@@ -254,15 +254,7 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/md5sum.json
+10 -122
View File
@@ -254,166 +254,54 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--check",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--binary",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--text",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--tag",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--warn",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--strict",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--ignore-missing",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--status",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--quiet",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/mkdir.json
+3 -49
View File
@@ -107,76 +107,30 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--parents",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--verbose",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--mode",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--context",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-Z",
"short_value": true,
"type": "str",
"value_optional": true
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/mktemp.json
+7 -87
View File
@@ -107,123 +107,43 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--directory",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--tmpdir",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "path",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--dry-run",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--quiet",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-q",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": null
]
}
spec/python/general/mv.json
+10 -136
View File
@@ -100,196 +100,70 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--force",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--interactive",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-clobber",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--verbose",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--update",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-u",
"short_value": false,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--backup",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": false,
"type": "str",
"value_optional": true
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-S",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--target-directory",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-target-directory",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-T",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--exchange",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-copy",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--strip-trailing-slashes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/nl.json
+11 -128
View File
@@ -254,181 +254,64 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--body-numbering",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-b",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--section-delimiter",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--footer-numbering",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--header-numbering",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-h",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--join-blank-lines",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--number-format",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-renumber",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--starting-line-number",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--line-increment",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--number-width",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--number-separator",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/node.json
+2 -27
View File
@@ -15,45 +15,20 @@
"resources": []
},
"description": "Run JavaScript on a sandboxed quickjs engine.",
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": "Evaluate the next argument as a script.",
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": "Run as an ES module (top-level import/export/await); .mjs files select this automatically.",
"long": "--module",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/numfmt.json
+4 -55
View File
@@ -254,76 +254,25 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--to",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--from",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--suffix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--grouping",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/od.json
+4 -50
View File
@@ -254,76 +254,30 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--address-radix",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--skip-bytes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-j",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--read-bytes",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-N",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--format",
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-t",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/paste.json
+3 -40
View File
@@ -254,61 +254,24 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--delimiters",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--serial",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-s",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--zero-terminated",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-z",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/patch.json
+5 -58
View File
@@ -107,83 +107,30 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-p",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-N",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "path"
},
{
"provided_by": [],
"text_when": [],
"type": "path"
}
],
"rest": null
]
}
spec/python/general/printf.json
-9
View File
@@ -7,21 +7,12 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [],
"positional": [
{
"provided_by": [],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/pwd.json
+2 -31
View File
@@ -7,46 +7,17 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-P",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-L",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/python.json
+1 -19
View File
@@ -14,31 +14,13 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/python3.json
+1 -19
View File
@@ -14,31 +14,13 @@
"has_write": false,
"resources": []
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "str"
}
}
spec/python/general/readlink.json
+4 -55
View File
@@ -254,76 +254,25 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/realpath.json
+2 -31
View File
@@ -254,46 +254,17 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/rev.json
-8
View File
@@ -254,15 +254,7 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/rg.json
+19 -243
View File
@@ -261,308 +261,87 @@
"trello"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-n",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-c",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-l",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-w",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-F",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-o",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-H",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-I",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-e",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": true,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "path",
"value_optional": false
"type": "path"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-m",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-A",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-B",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-C",
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--hidden",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--type",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--glob",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": false
"type": "str"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--color",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "str",
"value_optional": true
}
@@ -573,13 +352,10 @@
"-e",
"-f"
],
"text_when": [],
"type": "str"
}
],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/rm.json
+10 -127
View File
@@ -128,166 +128,49 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-r",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-R",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-f",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-d",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-i",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-I",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--preserve-root",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--no-preserve-root",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
},
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": "--one-file-system",
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": null,
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}
spec/python/general/rmdir.json
+1 -19
View File
@@ -100,31 +100,13 @@
"ssh"
]
},
"description": null,
"epilog": null,
"ignore_tokens": [],
"old_option_style": false,
"options": [
{
"choices": [],
"count": false,
"default": null,
"description": null,
"long": null,
"multiple": false,
"numeric_shorthand": false,
"pair": false,
"required": false,
"short": "-v",
"short_value": true,
"type": "bool",
"value_optional": false
"type": "bool"
}
],
"positional": [],
"rest": {
"provided_by": [],
"text_when": [],
"type": "path"
}
}

Some files were not shown because too many files have changed in this diff Show More