-
fix(lancedb): fix FTS with_position optimize crash + disk bloat (#336)
frostbyte_neo 发布于
2026-07-10 08:54:58 +00:00 FTS indexes built with with_position=True crash lance's optimize/compaction
on lancedb >= 0.32 when merging an unindexed tail (Max offset exceeds length
of values; upstream lance-format/lance#7653). The crash aborts optimize()
including version cleanup, so the index dir grows unbounded until the disk
fills. everos recall is OR-mode BM25 and never does phrase queries, so
positions are never read -- disabling is lossless.- base: default with_position=False
- infra: migrate_fts_indexes() rebuilds pre-fix indexes once at startup + reclaims orphans
- cascade worker: count consecutive optimize failures, escalate warning->error
Fixes #335.
Co-authored-by: zhanghui zhanghui@shanda.com
Co-authored-by: Claude Opus 4.8 (1M context) noreply@anthropic.com下载附件