* chore(deps): migrate @xenova/transformers to @huggingface/transformers v4
@xenova/transformers@2.x is deprecated and silently broken on Node 22+
(see #479). The project was renamed to @huggingface/transformers; same
Apache-2.0 license, same code. v4 ships onnxruntime-node/web and sharp
as hard deps, so they're dropped from our optionalDependencies.
Pipeline / RawImage.fromBlob / tolist / text-classification output
shape all unchanged. Three behavior-preserving adjustments needed:
- All 4 pipeline call sites pass { dtype: "q8" }. v4's default on Node
is fp32 (DEFAULT_DEVICE_DTYPE = "fp32"); v2 defaulted to quantized=true.
Without explicit dtype, all 4 sites silently regress (~3.5x larger
download, slower inference). dtype "q8" maps to model_quantized.onnx
per v4's DEFAULT_DTYPE_SUFFIX_MAPPING; file exists in all 3 Xenova
models. This was the regression that prompted the test additions below.
- src/providers/embedding/local.ts: split import try/catch from
pipeline() call so model-load errors (network, missing q8 variant,
disk) propagate with their actual message, not masked as
"Install @huggingface/transformers...".
- src/providers/embedding/{local,clip}.ts: type module from
typeof import("@huggingface/transformers") so PretrainedModelOptions
flows through; drop hand-rolled aliases and @ts-ignore. Cast at
assignment sites (pipeline return union isn't structurally assignable
to our narrow FeatureExtractor / ClipPipeline shapes).
Tests added where coverage was zero (would have caught the dtype
regression):
- test/local-embedding-provider.test.ts (3 tests): unavailable-path
install hint; pipeline called with dtype:q8 + extractor options +
mapped Float32Array result; embedBatch shape.
- test/clip-embedding-provider.test.ts (5 tests): unavailable-path;
text pipeline dtype:q8 + result; embedBatch; embedImage with data:
URL decode; custom model ID propagation.
- test/reranker.test.ts: positive-path using vi.doMock + resetModules.
Other:
- src/huggingface.d.ts deleted (package ships its own types).
- src/xenova.d.ts removed.
- src/providers/embedding/clip.ts: inline single-use DIMENSIONS constant.
- tsdown.config.ts: trim neverBundle list and comment.
- README.md L1267: BGE-small -> Xenova/all-MiniLM-L6-v2 (was always wrong).
- 16 docs: install commands + prose mentions across main README, 11
translations, SECURITY.md, 2 benchmark docs, benchmark script.
- Model IDs (Xenova/all-MiniLM-L6-v2, Xenova/clip-vit-base-patch32,
Xenova/ms-marco-MiniLM-L-6-v2) kept — HF Hub repo names, still valid.
Closes#1095. Fixes#479.
Verified: 1424/1424 tests pass, build clean, tsc clean on migrated files.
* test(embedding): add v4 smoke test, harden import errors, expand CI matrix
Review follow-ups for #1096:
- env-guarded non-mocked smoke test (RUN_HF_SMOKE=1) loading real
Xenova/all-MiniLM-L6-v2, asserts 384 finite dims; skipped by default
- selective ERR_MODULE_NOT_FOUND handling in local/clip providers so real
init errors propagate (checks err.code and err.cause.code to handle
vitest mock-factory wrapping)
- CLIP install hint made embedding-agnostic (loader serves text + image)
- afterEach mock cleanup in doMock-based provider/reranker tests
- CI Node matrix: [20, 22] -> [20, 22, 24, 26] across ubuntu/macos
* refactor(embedding): drop err.cause check, use manual mock for missing-module tests
The .cause branch in the ERR_MODULE_NOT_FOUND check existed only to
accommodate vitest's mock-factory wrapping, not a real Node loader
behavior. Replace it with a manual mock fixture (__mocks__/@huggingface/
transformers.ts) that throws a Node-shaped error at module top-level,
bypassing vitest's factory wrapper so the import rejects with err.code
set directly.
Production code now checks only err.code === 'ERR_MODULE_NOT_FOUND',
matching real Node behavior. Tests verify the same public contract
without coupling production code to the test framework.
Abdullah Alaqeel
8c90741c63