test(docs): migrate retired BM25 references

This commit is contained in:
Andrey Kumanyaev
2026-08-14 20:02:59 +02:00
parent 079e6cef76
commit 2e3cf10630
7 changed files with 73 additions and 71 deletions
+1 -1
View File
@@ -50,7 +50,7 @@ High-quality parsing 257 languages/grammars through tree-sitter AST analysis, in
- **Zero external dependencies** — single binary, everything in-process. No network, no model download to get started. Install, start daemon, use.
- **Agent integrations (19)** — `gortex init` configures every detected coding assistant on the machine → [docs/agents.md](docs/agents.md)
- **100+ MCP tools, 16 resources, 3 prompts** — symbol lookup, call chains, blast radius, dataflow, clone detection, refactoring, code actions → [docs/mcp.md](docs/mcp.md)
- **Semantic search default-on** — baked GloVe-50d (3.8 MB embedded), hybrid BM25 + vector + RRF, zero deps; opt-in MiniLM / Ollama / OpenAI → [docs/semantic-search.md](docs/semantic-search.md)
- **Semantic search default-on** — baked GloVe-50d (3.8 MB embedded), store-native FTS5/BM25 + vector with adaptive alpha fusion, zero deps; opt-in MiniLM / Ollama / OpenAI → [docs/semantic-search.md](docs/semantic-search.md)
- **Speculative execution** — `preview_edit` / `simulate_chain` answer "what would change if I applied this WorkspaceEdit?" without touching disk
- **Live editor overlays** — push unsaved buffers as a shadow graph; tools read through it. Branching for parallel speculative sessions
- **GCX1 wire format** — published, round-trippable. **An additional 27% tokens vs JSON** at same fidelity → [docs/wire-format.md](docs/wire-format.md)
+20 -20
View File
@@ -27,11 +27,11 @@
# Cases are tiered so per-tier weakness is visible:
#
# tier: exact — symbol-name queries. Tests "find this named
# thing." BM25 should dominate; a retrieval stack
# that can't ace exact tier is broken.
# thing." Store-native lexical search should dominate;
# a retrieval stack that can't ace exact tier is broken.
# tier: concept — natural-language paraphrase queries. Tests
# semantic understanding. Where BM25 starts losing
# to semantic / RRF if the embedder is competent.
# semantic understanding. Where lexical retrieval starts
# losing to vector/adaptive fusion if the embedder is competent.
# tier: multi_hop — relational queries with several valid expected
# IDs (any-hit semantics). Tests graph-aware
# retrieval: "things that handle X" / "extractors
@@ -63,8 +63,8 @@ cases:
- { id: exact-AddNode, tier: exact, query: "AddNode", expected: [internal/graph/graph.go::Graph.AddNode] }
- { id: exact-AtomicWriteFile, tier: exact, query: "AtomicWriteFile", expected: [internal/agents/writer.go::AtomicWriteFile] }
- { id: exact-WriteIfNotExists, tier: exact, query: "WriteIfNotExists", expected: [internal/agents/writer.go::WriteIfNotExists] }
- { id: exact-BM25Backend, tier: exact, query: "BM25Backend", expected: [internal/search/bm25.go::BM25Backend] }
- { id: exact-NewBM25, tier: exact, query: "NewBM25", expected: [internal/search/bm25.go::NewBM25] }
- { id: exact-SymbolSearcherBackend, tier: exact, query: "SymbolSearcherBackend", expected: [internal/search/symbolsearcher_backend.go::SymbolSearcherBackend] }
- { id: exact-NewSymbolSearcherBackend, tier: exact, query: "NewSymbolSearcherBackend", expected: [internal/search/symbolsearcher_backend.go::NewSymbolSearcherBackend] }
- { id: exact-HybridBackend, tier: exact, query: "HybridBackend", expected: [internal/search/hybrid.go::HybridBackend] }
- { id: exact-NewHybrid, tier: exact, query: "NewHybrid", expected: [internal/search/hybrid.go::NewHybrid] }
- { id: exact-alphaFuse, tier: exact, query: "alphaFuse", expected: [internal/search/hybrid.go::alphaFuse] }
@@ -119,10 +119,10 @@ cases:
- { id: concept-reindex-one-file, tier: concept, query: "reindex a single file after an edit", expected: [internal/indexer/indexer.go::Indexer.IndexFile] }
- { id: concept-evict-file, tier: concept, query: "remove all nodes belonging to a file", expected: [internal/graph/graph.go::Graph.EvictFile] }
- { id: concept-evict-repo, tier: concept, query: "drop every node for a repository prefix", expected: [internal/graph/graph.go::Graph.EvictRepo] }
- { id: concept-bm25-index, tier: concept, query: "text search backend with TF-IDF ranking", expected: [internal/search/bm25.go::BM25Backend] }
- { id: concept-hybrid-fuse, tier: concept, query: "combine text and vector search with RRF", expected: [internal/search/hybrid.go::HybridBackend] }
- { id: concept-adaptive-alpha, tier: concept, query: "adaptive alpha weighted reciprocal rank fusion", expected: [internal/search/hybrid.go::alphaFuse] }
- { id: concept-swap-backend, tier: concept, query: "hot-swap the in-memory search backend", expected: [internal/search/swappable.go::Swappable] }
- { id: concept-store-native-text, tier: concept, query: "adapt store-native symbol search into the search backend", expected: [internal/search/symbolsearcher_backend.go::SymbolSearcherBackend] }
- { id: concept-hybrid-fuse, tier: concept, query: "combine store-native text and vector search with adaptive alpha", expected: [internal/search/hybrid.go::HybridBackend] }
- { id: concept-adaptive-alpha, tier: concept, query: "adaptive alpha weighted text and vector fusion", expected: [internal/search/hybrid.go::alphaFuse] }
- { id: concept-swap-backend, tier: concept, query: "atomically replace the vector channel while preserving the live text backend", expected: [internal/search/swappable.go::Swappable.ReplaceHybridVector] }
- { id: concept-glove-embed, tier: concept, query: "built-in GloVe word vector embedder", expected: [internal/embedding/static.go::StaticProvider] }
- { id: concept-mcp-server, tier: concept, query: "MCP server type holding engine and graph", expected: [internal/mcp/server.go::Server] }
- { id: concept-session-state, tier: concept, query: "per-client session activity tracking", expected: [internal/mcp/server.go::sessionState] }
@@ -154,7 +154,7 @@ cases:
- { id: concept-ts-extract, tier: concept, query: "TypeScript language extractor", expected: [internal/parser/languages/typescript.go::TypeScriptExtractor] }
- { id: concept-detect-lang, tier: concept, query: "detect language from file extension", expected: [internal/parser/registry.go::Registry.GetByExtension] }
- { id: concept-register-all, tier: concept, query: "register every language extractor", expected: [internal/parser/languages/register.go::RegisterAll] }
- { id: concept-ranker-bm25, tier: concept, query: "eval ranker adapter for engine search", expected: [internal/eval/recall/rankers.go::EngineRanker] }
- { id: concept-ranker-engine, tier: concept, query: "eval ranker adapter for engine search", expected: [internal/eval/recall/rankers.go::EngineRanker] }
- { id: concept-ranker-rrf, tier: concept, query: "eval ranker adapter for RRF hybrid", expected: [internal/eval/recall/rankers.go::RRFRanker] }
- { id: concept-ranker-semantic, tier: concept, query: "eval ranker for vector-only semantic", expected: [internal/eval/recall/rankers.go::SemanticRanker] }
- { id: concept-ranker-winnow, tier: concept, query: "eval ranker wrapping graph-aware winnow", expected: [internal/eval/recall/rankers.go::WinnowRanker] }
@@ -163,7 +163,7 @@ cases:
- { id: concept-repo-for-file, tier: concept, query: "find which repo contains a given file", expected: [internal/indexer/multi.go::MultiIndexer.RepoForFile] }
- { id: concept-set-embedder, tier: concept, query: "attach embedding provider to the indexer", expected: [internal/indexer/indexer.go::Indexer.SetEmbedder] }
- { id: concept-new-server, tier: concept, query: "construct a new MCP server instance", expected: [internal/mcp/server.go::NewServer] }
- { id: concept-new-hybrid, tier: concept, query: "construct a hybrid BM25+vector backend", expected: [internal/search/hybrid.go::NewHybrid] }
- { id: concept-new-hybrid, tier: concept, query: "construct a hybrid store-native text and vector backend", expected: [internal/search/hybrid.go::NewHybrid] }
- { id: concept-reindex-all, tier: concept, query: "incremental reindex after file changes", expected: [internal/indexer/indexer.go::Indexer.IncrementalReindexPaths] }
- { id: concept-new-vector, tier: concept, query: "HNSW vector index backend", expected: [internal/search/vector.go::VectorBackend] }
- { id: concept-api-embedder, tier: concept, query: "OpenAI-compatible embeddings API provider", expected: [internal/embedding/api.go::APIProvider] }
@@ -229,7 +229,7 @@ cases:
tier: multi_hop
query: "search backend implementations"
expected:
- internal/search/bm25.go::BM25Backend
- internal/search/null.go::NullBackend
- internal/search/symbolsearcher_backend.go::SymbolSearcherBackend
- internal/search/hybrid.go::HybridBackend
- internal/search/vector.go::VectorBackend
@@ -410,15 +410,15 @@ cases:
- internal/config/config.go::Load
- internal/config/config.go::Default
- id: mh-bm25-api
- id: mh-null-backend-api
tier: multi_hop
query: "BM25 backend public API"
query: "null search backend public API"
expected:
- internal/search/bm25.go::BM25Backend.Add
- internal/search/bm25.go::BM25Backend.Remove
- internal/search/bm25.go::BM25Backend.Search
- internal/search/bm25.go::BM25Backend.Count
- internal/search/bm25.go::NewBM25
- internal/search/null.go::NullBackend.Add
- internal/search/null.go::NullBackend.Remove
- internal/search/null.go::NullBackend.Search
- internal/search/null.go::NullBackend.Count
- internal/search/null.go::NewNull
- id: mh-tool-registrations
tier: multi_hop
+37 -37
View File
@@ -45,79 +45,79 @@ cases:
query: IndexFileNozResolve
expected:
- internal/indexer/indexer.go::Indexer.IndexFileNoResolve
- id: exact-IncrementalReindex-typo
- id: exact-IncrementalReindexPaths-typo
tier: exact
query: IncrementalPeindex
query: IncrementalReigdexPaths
expected:
- internal/indexer/indexer.go::Indexer.IncrementalReindexPaths
- id: exact-SetEmbedder-typo
tier: exact
query: SetEmbeeder
query: SetEmbedde
expected:
- internal/indexer/indexer.go::Indexer.SetEmbedder
- id: exact-EvictFile-typo
tier: exact
query: EvictFife
query: EvictFilq
expected:
- internal/graph/graph.go::Graph.EvictFile
- id: exact-EvictRepo-typo
tier: exact
query: EvkctRepo
query: EiictRepo
expected:
- internal/graph/graph.go::Graph.EvictRepo
- id: exact-AddNode-typo
tier: exact
query: AddNode
query: AddoNde
expected:
- internal/graph/graph.go::Graph.AddNode
- id: exact-AtomicWriteFile-typo
tier: exact
query: AtomicnWriteFile
query: AtomicWriteile
expected:
- internal/agents/writer.go::AtomicWriteFile
- id: exact-WriteIfNotExists-typo
tier: exact
query: WrilteIfNotExists
query: WriteIfNotExisks
expected:
- internal/agents/writer.go::WriteIfNotExists
- id: exact-BM25Backend-typo
- id: exact-SymbolSearcherBackend-typo
tier: exact
query: BM25Backejd
query: SymbolSearcherackend
expected:
- internal/search/bm25.go::BM25Backend
- id: exact-NewBM25-typo
- internal/search/symbolsearcher_backend.go::SymbolSearcherBackend
- id: exact-NewSymbolSearcherBackend-typo
tier: exact
query: NenBM25
query: NewSymboSlearcherBackend
expected:
- internal/search/bm25.go::NewBM25
- internal/search/symbolsearcher_backend.go::NewSymbolSearcherBackend
- id: exact-HybridBackend-typo
tier: exact
query: HybridBfckend
query: HybridUackend
expected:
- internal/search/hybrid.go::HybridBackend
- id: exact-NewHybrid-typo
tier: exact
query: NewHybid
query: NtwHybrid
expected:
- internal/search/hybrid.go::NewHybrid
- id: exact-alphaFuse-typo
tier: exact
query: alphaFse
query: agphaFuse
expected:
- internal/search/hybrid.go::alphaFuse
- id: exact-VectorBackend-typo
tier: exact
query: VectorBcakend
query: VectorBackedd
expected:
- internal/search/vector.go::VectorBackend
- id: exact-SearchBackend-typo
tier: exact
query: search.Backeod
query: sewarch.Backend
expected:
- internal/search/search.go::Backend
- id: exact-SearchResult-typo
tier: exact
query: SearchResutl
query: SearchResuzt
expected:
- internal/search/search.go::SearchResult
- id: exact-Swappable-typo
@@ -335,26 +335,26 @@ cases:
query: drop every node for a rpeository prefix
expected:
- internal/graph/graph.go::Graph.EvictRepo
- id: concept-bm25-index-typo
- id: concept-store-native-text-typo
tier: concept
query: text search backend wtih TF-IDF ranking
query: adapt store-native symbol search itno the search backend
expected:
- internal/search/bm25.go::BM25Backend
- internal/search/symbolsearcher_backend.go::SymbolSearcherBackend
- id: concept-hybrid-fuse-typo
tier: concept
query: comlbine text and vector search with RRF
query: combine store-native text and lvector search with adaptive alpha
expected:
- internal/search/hybrid.go::HybridBackend
- id: concept-adaptive-alpha-typo
tier: concept
query: adaptive alpha weighted reciprocal randk fusion
query: adaptive alpha weightded text and vector fusion
expected:
- internal/search/hybrid.go::alphaFuse
- id: concept-swap-backend-typo
tier: concept
query: hot-swap the in-memory searich backend
query: atomically replace the vectior channel while preserving the live text backend
expected:
- internal/search/swappable.go::Swappable
- internal/search/swappable.go::Swappable.ReplaceHybridVector
- id: concept-glove-embed-typo
tier: concept
query: built-in GloVe owrd vector embedder
@@ -510,7 +510,7 @@ cases:
query: register every language extactor
expected:
- internal/parser/languages/register.go::RegisterAll
- id: concept-ranker-bm25-typo
- id: concept-ranker-engine-typo
tier: concept
query: eval rnker adapter for engine search
expected:
@@ -557,7 +557,7 @@ cases:
- internal/mcp/server.go::NewServer
- id: concept-new-hybrid-typo
tier: concept
query: construct a hbyrid BM25+vector backend
query: construct a hybrid stroe-native text and vector backend
expected:
- internal/search/hybrid.go::NewHybrid
- id: concept-reindex-all-typo
@@ -672,7 +672,7 @@ cases:
tier: multi_hop
query: search backend implementatsions
expected:
- internal/search/bm25.go::BM25Backend
- internal/search/null.go::NullBackend
- internal/search/symbolsearcher_backend.go::SymbolSearcherBackend
- internal/search/hybrid.go::HybridBackend
- internal/search/vector.go::VectorBackend
@@ -832,15 +832,15 @@ cases:
expected:
- internal/config/config.go::Load
- internal/config/config.go::Default
- id: mh-bm25-api-typo
- id: mh-null-backend-api-typo
tier: multi_hop
query: BM25 backend pubilc API
query: null seacrh backend public API
expected:
- internal/search/bm25.go::BM25Backend.Add
- internal/search/bm25.go::BM25Backend.Remove
- internal/search/bm25.go::BM25Backend.Search
- internal/search/bm25.go::BM25Backend.Count
- internal/search/bm25.go::NewBM25
- internal/search/null.go::NullBackend.Add
- internal/search/null.go::NullBackend.Remove
- internal/search/null.go::NullBackend.Search
- internal/search/null.go::NullBackend.Count
- internal/search/null.go::NewNull
- id: mh-tool-registrations-typo
tier: multi_hop
query: MCP tool registration functuons
+4 -2
View File
@@ -33,8 +33,10 @@ order of operations."
`parser.ExtractionResult`; `Indexer.processExtraction` writes
them into the `graph.Graph` and accumulates incoming-edge
tracking for the next phase.
4. `Indexer.buildSearchIndex` (the store-native FTS) +
`idx.embedder` (if set) populate the search backends.
4. The backing store owns the symbol FTS corpus: shadow drains bulk-publish it,
direct-store passes rebuild it, and incremental mutations upsert or delete
affected rows. `Indexer.buildSearchIndex` prepares and publishes only the
optional vector corpus through `idx.embedder`.
5. Semantic enrichment (`internal/semantic`) runs LSP / SCIP
providers in parallel; resolved edges get
`Origin=lsp_resolved` for tier filtering.
+1 -1
View File
@@ -34,7 +34,7 @@ The full surface, grouped by concern. Each item links to the deeper reference wh
## Search & navigation
- **Semantic search default-on** — hybrid BM25 + vector with RRF fusion, baked GloVe-50d (~3.8 MB embedded in the binary, top 20k tokens), CPU-only, zero native deps. Large symbols are split into AST-aware windows and de-chunked at query time. Equivalence-class vocabulary expansion bridges `auth ≈ authentication ≈ login` without an LLM. A HITS authority/hub signal feeds the rerank pipeline. A keyword-soup query defense detects degenerate OR-soup *and* operator-free phrasing and skips wasted LLM expansion. Markdown documentation is a first-class corpus with its own retrieval channel and prose-tuned ranking. Opt-in `embedding.provider: local` (Hugot MiniLM-L6-v2) or `api` (Ollama / OpenAI). See [semantic-search.md](semantic-search.md).
- **Semantic search default-on** — store-native FTS5/BM25 + vector with adaptive alpha fusion, baked GloVe-50d (~3.8 MB embedded in the binary, top 20k tokens), CPU-only, zero native deps. Large symbols are split into AST-aware windows and de-chunked at query time. Equivalence-class vocabulary expansion bridges `auth ≈ authentication ≈ login` without an LLM. A HITS authority/hub signal feeds the rerank pipeline. A keyword-soup query defense detects degenerate OR-soup *and* operator-free phrasing and skips wasted LLM expansion. Markdown documentation is a first-class corpus with its own retrieval channel and prose-tuned ranking. Opt-in `embedding.provider: local` (Hugot MiniLM-L6-v2) or `api` (Ollama / OpenAI). See [semantic-search.md](semantic-search.md).
- **Provenance-aware ranking** — the BM25↔vector balance is scored continuously from query shape; edge-resolution provenance attenuates LSP-inflated framework wiring in centrality and rerank; generated files are ranked below a real same-named implementation; the implementation is lifted above its own test; and a post-rerank pass recovers exact embedding cosine. Zero-result identifier queries are auto-decomposed into leaf terms.
- **`context_closure`** — given seed files/symbols, walks the transitive import/dependency closure and packs it under one `token_budget`, ranked by graph distance or seeded random-walk proximity.
- **Code search beyond symbols** — `search_text` is a trigram-indexed literal/regex search; `search_ast` runs structural tree-sitter queries; `analyze kind=sast` is a 190-rule, CWE/OWASP-tagged security scan across 8 languages.
+3 -3
View File
@@ -1,6 +1,6 @@
# Semantic search
**Default-on.** A baked GloVe-50d table (~3.8 MB embedded in the binary, top 20k tokens) gives every install hybrid BM25 + vector search out of the box — no flag, no model download, no native dependency. Reciprocal Rank Fusion blends the two channels, and the BM25↔vector balance is scored *continuously* from the query's shape (identifier density, separators, stopwords) rather than bucketed into a discrete class — so a half-identifier query lands between the symbol and natural-language blends instead of jumping a whole tier. After ranking, an optional pure-cosine refinement pass re-scores the top results with the exact embedding distance the rank-based fusion discards.
**Default-on.** A baked GloVe-50d table (~3.8 MB embedded in the binary, top 20k tokens) gives every install hybrid store-native FTS5/BM25 + vector search out of the box — no flag, no model download, no native dependency. Adaptive alpha-weighted rank fusion blends the two channels, and the text↔vector balance is scored *continuously* from the query's shape (identifier density, separators, stopwords) rather than bucketed into a discrete class — so a half-identifier query lands between the symbol and natural-language blends instead of jumping a whole tier. After ranking, an optional pure-cosine refinement pass re-scores the top results with the exact embedding distance the rank-based fusion discards.
## Configuration
@@ -65,7 +65,7 @@ Centrality (HITS + PageRank) and a dedicated rerank signal weight call/reference
## Keyword-soup defense
Boolean / OR-soup queries (`A OR B OR 'no access' OR …`) — and operator-free keyword lists (`parse decode unmarshal token jwt cache`) and comma-enumerations — defeat embedding retrieval. The query classifier detects all three, skips wasted LLM expansion, and splits the soup into terms fused via the existing BM25 expansion path; a `query_advice` nudge rides on the response. Genuine natural-language questions stay classified as concept. Tune via `search.keyword_soup_rewrite: split | nudge | off`.
Boolean / OR-soup queries (`A OR B OR 'no access' OR …`) — and operator-free keyword lists (`parse decode unmarshal token jwt cache`) and comma-enumerations — defeat embedding retrieval. The query classifier detects all three, skips wasted LLM expansion, and splits the soup into terms fused via the store-native lexical expansion path; a `query_advice` nudge rides on the response. Genuine natural-language questions stay classified as concept. Tune via `search.keyword_soup_rewrite: split | nudge | off`.
## Prose corpus
@@ -111,7 +111,7 @@ Semantic search degrading to text-only (BM25 / FTS5) is always logged — match
- `auto` (default) — skips LLM for identifier queries, expands NL queries
- `on` — forces expansion + rerank
- `off`pure BM25
- `off`store-native FTS5/BM25 only
- `deep` — adds a body-grounded verification pass; +1.54 s; quality is highly model-dependent — unreliable on 3B local models, fine on 7B+ or hosted
See [llm.md](llm.md) for provider configuration.
+7 -7
View File
@@ -7,18 +7,18 @@
// Recall is reported as any-hit set-level recall: a retrieval counts as
// correct at rank K if *any* of the Expected IDs for a case appears in
// the ranker's top-K results. Multiple Expected IDs per case are OK —
// they represent valid alternative targets (e.g. a type and its
// constructor both being reasonable answers to "BM25 backend").
// they represent valid alternative targets (e.g. a backend type and its
// constructor both being reasonable answers to "store-native text search").
//
// Cases are tiered so per-tier weakness is visible:
//
// - exact: symbol-name queries. Tests the basic "can you find a
// named symbol I already know about" case. BM25 should
// dominate here; a retrieval tool that can't ace exact
// tier is broken.
// named symbol I already know about" case. Store-native lexical
// retrieval should dominate here; a retrieval tool that can't ace
// exact tier is broken.
// - concept: natural-language paraphrase queries. Tests semantic
// understanding. This is where BM25 starts losing to
// semantic / RRF.
// understanding. This is where lexical retrieval starts losing to
// vector search and adaptive fusion.
// - multi_hop: relational queries accepting several valid expected
// IDs (any-hit). Tests graph-aware retrieval.
//