* refactor(filetype): remove the bundled format renderers, keep the extension point
mirage shipped renderers for parquet, ORC, feather/arrow/ipc and hdf5/h5, plus
a PDF module that turned out to be entirely dead code: '.pdf' was never in the
factory registry and nothing imported mirage.core.filetype.pdf, so its test
exercised it directly and kept it looking alive.
Both core/filetype/ trees are removed. The dispatch machinery stays, in both
languages and in both places it lives (commands/builtin/filetype_factory and
ops/generic/factory), with empty registries and a comment marking them as the
extension point. A file with an unregistered extension now reads as raw bytes.
Registering a filetype-scoped command still works and is covered:
tests/commands/custom/test_filetype_fns.py registers a '.parquet' handler with
a fake function and asserts dispatch, and test_unregister_removes_all_filetypes
now registers a '.demo' renderer itself rather than leaning on the bundled ones.
Also removed: the parquet/hdf5/pdf extras (and their references from 'all' and
'deepagents'), the hyparquet, hyparquet-writer, apache-arrow and h5wasm
dependencies, integ/resources/columnar, the columnar FileType enum members, the
per-backend filetypeRead declarations that existed only on the TypeScript side,
and the sentence advertising 'cat on .parquet/.orc/.feather returns a formatted
table' from 33 backend and agent prompts, which would otherwise have been
lying to agents.
grep_helper's skip-list and file's MIME map keep their columnar entries: those
are still correct, since the formats remain binary whether or not mirage can
render them.
core/src/ops/generic/factory.ts was reaching NodeJS.ErrnoException through the
columnar packages' type dependencies. core has to work in both runtimes, so it
now uses a structural { code?: string } instead.
Python suite passes, TypeScript core 5458 pass, pre-commit clean.
* refactor(examples): drop the columnar demos alongside the renderers
Deleted, because their whole subject was the removed rendering:
s3_data.py (181 lines built around four columnar constants),
ram_filetypes.ts, box_parquet.ts, dropbox_parquet.ts, ram_parquet.ts.
Also deleted fuse_hooks.py, which was already broken before this change: it
imports mirage.fuse.filetype.data.local.parquet, a module that does not exist
anywhere in the tree.
Edited rather than deleted, since columnar was one section of a broader demo:
gdrive_complex.py, disk.ts, ram_fuse.py, and the openhands README (which
advertised format-aware reads and piped a .parquet through jq).
Everything still referencing parquet only names it in find/ls patterns, which
keeps working: mirage can still list and locate these files, it just no longer
renders them.
* fix(ci): drop the last filetype wiring the removal missed
The nextcloud ops table still declared filetypeRead for feather/hdf5/
parquet, so makeGenericOps threw at module load and every job that
imported @struktoai/mirage-node died before running anything.
Also: regenerate the specs (the crash hid 89 files of drift), drop the
removed pdf/parquet/hdf5 extras from the install matrix, take the
columnar members out of the TS FileType enum and the box/gdrive/dropbox
type guesses so both languages agree again, and stop passing an inert
filetype_read=True from ten Python backends.
* fix(spec): regenerate with every optional backend importable
The previous regeneration ran under the worktree venv, which has no
optional extras installed, so backends that failed to import were
dropped from each spec's resources list. Regenerated with the full
environment: the only remaining change is the emptied filetypes list.
* fix(agents): stop telling models mirage renders columnar formats
The langchain backend mapped parquet/h5/hdf5/feather to text/plain, so
with no renderer registered it handed the model raw binary decoded as
UTF-8 instead of the binary redirect. Those extensions now fall through
to application/octet-stream.
The system prompt, the execute tool description in both languages, and
the README extensibility example all still advertised columnar rendering
as a shipped feature; they now describe it as the extension point it is.
Also drops commands/optional.py, which existed only to soft-import the
removed format helpers and has no callers left.
* refactor(filetype): drop the filetype command factory, keep mount registration
The factory built nine commands per registered extension, but with no
renderers shipping it produced nothing on every one of the ~19 backends
that called it, and its handlers had no test in either language while
the module contract they expected was written down nowhere.
Removes commands/builtin/filetype_factory/ in both languages along with
the filetype_read / filetypeRead op knobs and the reads and provisions
that existed only to feed them. Registration on a mount survives and is
the whole extension point: a command or op carrying a filetype resolves
as (name, filetype) before (name, resource).
examples/{python,typescript}/filetype/ register a .tally renderer end to
end and are gated in CI against integ/truth/*/filetype.txt, so the path
is now exercised rather than asserted. Both emit byte-identical output.
Zecheng Zhang
b117125495