The multi-platform smoke run flagged index-cli + index-status as empty/not-ready
even though the binary indexed fine (nodes>0, status ready) — the MCP tool result
wraps its payload as a JSON STRING with escaped quotes (\"nodes\":N), but the
checks grepped for unescaped "nodes":N. Strip backslashes/quotes before matching
and tolerate the nodes= log form. (Binary was healthy; this was a script-parsing
gap the wide-matrix smoke surfaced.)
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- scripts/smoke-invariants.sh: 30-check battery against the PROD binary —
--version/--help, MCP initialize handshake with stdin OPEN (#513), tools/list
(all 14), EVERY tool invocable with valid JSON-RPC + no crash, index→non-empty
graph, malformed-input resilience (bad JSON / empty / huge line / binary /
non-UTF8 / missing path), clean EOF exit, shared-lib resolution, install
dry-run. Bounded waits (read -t / timeout), no sleep loops; msys2-safe.
- .github/workflows/smoke.yml: runs it on the WIDEST runner matrix — ubuntu
22.04+24.04 (x64+arm64; 22.04 = older glibc / AlmaLinux class), macos
14/15/15-intel, windows 2022/2025 + windows-11-arm (experimental). A FAIL on
any platform is a binary a user would receive. workflow_dispatch + qa/smoke-**.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
- _soak.yml: timeout-minutes was 30 (soak-quick) / 45 (asan) while nightly passes
duration_minutes=240 — every 'nightly 4h soak' was silently KILLED at 30 min and
never ran multi-hour. Raise to 300/60 so the soak can actually complete.
- soak-test.sh: add CBM_SOAK_MODE=query-leak (default unchanged). It indexes once
then hammers read-only tools (search_graph/query_graph/trace_path/
get_code_snippet/search_code) with NO reindex/mutation — so index_repository's
cbm_mem_collect never runs to sweep the query-only leak #581 implicates. The
existing RSS ceiling/slope/ratio checks become the #581 detector.
- soak.yml: workflow_dispatch (duration_minutes, mode) + push to qa/soak-** ;
builds the prod binary and runs the soak with timeout = duration + 60, on
ubuntu + macos. Pushing a qa/soak-* branch starts a real multi-hour run.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
A reproduction fails its assertion and returns before cleanup, so LeakSanitizer
flagged benign harness leaks on every red store-level test and _exit()'d,
swallowing the unflushed summary — repro.sh then misread it as a build failure
(Linux only; macOS/Windows have no LSan). Fixes:
- repro.sh: export ASAN_OPTIONS=detect_leaks=0 for the board run. Leak-cleanliness
is not the board's signal (the RED rows are); the #581 leak bug gets a dedicated
RSS-growth test. ASan's real checks (use-after-free, overflow) stay enabled.
- repro_main.c: setvbuf(stdout, _IONBF) so the summary + RED rows survive any
abnormal _exit (sanitizer or crash).
- add tests/repro/repro_harness.h: shared multi-file index + store-query + fork
crash-detector helpers (ported from the proven test_lang_contract.c harness)
for the cross-file / store-level / crash reproduction waves.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Stand up a separate, non-gating reproduce-first suite that holds one RED case
per open bug issue (the redness is the deliverable + the regression guard):
- tests/repro/ + repro_main.c -> `make test-repro` (its own runner with its own
main + counters; deliberately EXCLUDED from ALL_TEST_SRCS so the gating
`make test` / ci-ok required check stays green and PRs are not wedged)
- repro_extraction.c: first reproduction, #554 -- a C++ out-of-line method's
inner CALLS edge must attribute to the class-qualified Method QN, not the
Module; ties the call's enclosing_func_qn to the method definition's own
qualified_name so a class-qualifier drop (the live root cause) fails it
- scripts/repro.sh: build+run the board; a build/link failure fails the job,
while expected test redness is reported as the board state (job stays green)
- .github/workflows/bug-repro.yml: workflow_dispatch (platform filter) + qa/**
push; runs the board on linux x2 / macos x2 / windows so many bug vectors can
be reproduced on many platforms at once
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The MSYS2 CLANG64 toolchain bakes its package-tracker URL
(https://github.com/msys2/MINGW-packages) into the static Windows .exe via
libc++/compiler-rt. The Layer 2 binary string audit flagged it as an
unauthorized URL, failing smoke-windows (both the standard and ui variants)
while Linux smoke stayed green. Add it to ALLOWED_URLS alongside the other
compiler/linker toolchain URLs (gcc.gnu.org, sourceware.org, launchpad).
Add tests/test_security_strings_allowlist.sh as a reproduce-first regression
guard: it reproduces the BLOCKED failure with a binary fixture and includes a
negative control proving a genuinely unauthorized URL is still rejected. Wired
into scripts/test.sh as Step 6.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The UI license enumeration invoked npm by bare name via subprocess,
which fails on Windows (npm is npm.cmd there — WinError 2 in the
Windows UI-archive packaging step). Resolve it through the shell, which
handles platform shims everywhere; the command is a constant string.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
It names license identifiers in its verdict maps by design — the same
documented false-positive category as the other license tooling.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
Every commit must now carry a Signed-off-by trailer matching its
author, certifying the right to submit the change under the project's
MIT license (DCO 1.1, the Linux kernel mechanism). Enforcement is
strict at three layers: a commit-msg hook rejects unsigned commits
locally (scripts/install-git-hooks.sh), and the new DCO workflow
rejects every push and pull request containing one. Merge commits and
bot authors are exempt, matching standard DCO checks.
Signed-off-by: Martin Vogel <martin.vogel@datadice.io>
The license gate now proves itself before running: a --selftest mode
plants an unlicensed vendored file and asserts the structural layer
detects it, so a silently broken gate can no longer pass. The
byte-identity provenance audit gains exit-code semantics (any verdict
outside the accepted set fails) and runs as a blocking step in the
security workflow for both dry runs and releases.
A byte-identity audit of all vendored license files against their
upstream repositories found 113 copies differing only by a missing
trailing newline (vendoring artifact) — replaced with the exact
upstream bytes. Five needed real corrections: fennel carried an
unfilled MIT template although its upstream is CC0-1.0 at the pinned
commit; ron now carries the repo's LICENSE-MIT (dual MIT/Apache
upstream); python, wit, and verilog were refreshed to current upstream
bytes. The manifest's first-party table was also corrected: six of the
twelve grammars are self-maintained forks whose retained upstream
licenses are now byte-verified (assembly's upstream has been deleted
from GitHub; the retained MIT copy is the surviving grant). The audit
tool is kept as scripts/audit-license-provenance.py for future sweeps.
Structural layer: drop the hardcoded component list — any directory
containing vendored source or data files must now be covered by a
license file in itself or an ancestor within the vendored tree, so
newly vendored code without a license fails immediately. New third
layer: the graph-UI npm production tree is resolved (postinstall
scripts disabled) and every bundled package must be on the policy
allow-list; unknown licenses fail. Reclassifies @tailwindcss/vite as a
devDependency — it is a build plugin and was inflating the production
tree with native tooling.
The guard now runs a token-level clone detector (jscpd, with an npx
fallback) comparing internal/cbm/lsp/ C sources against clangd C++ — the
one reference close enough to C to tokenize alike — staging both trees as
a common format and reporting only clone pairs that span them. Catches
structural ports that rewrote strings/identifiers, which the verbatim
string/comment scan misses. clangd verified: no structural clones.
Scans internal/cbm/lsp/ for verbatim string/comment overlap against the
reference language servers (pyright, gopls, tsserver, Roslyn, clangd,
Eclipse JDT LS, kotlin-language-server, rust-analyzer, phpactor). Run
before committing new LSP work; add a new language's upstream reference to
the REFS manifest when implementing it. All 9 current languages verified
with no verbatim overlap.
The with-ui archives now append the per-package license texts of the
frontend bundle's production dependency tree to THIRD_PARTY_NOTICES.md
(platform-specific native build tooling is listed but excluded — its
code never reaches the browser bundle). The Glama check image carries
LICENSE and the notices file alongside the binary. Also renames a
TypeScript LSP test to describe its generic ODM-interface fixture.
The graph-UI HTTP server now owns its listening socket directly, so the
Layer 1 static audit's network-call scan would flag its socket()/bind()
calls. Exempt src/ui/httpd.c — it binds 127.0.0.1 only and is audited
separately by security-ui.sh — while still forbidding raw network calls
in every other source file.
Release archives now carry THIRD_PARTY_NOTICES.md, generated by
scripts/gen-third-party-notices.sh from THIRD_PARTY.md, the grammar
manifest, and the per-component license texts; the Homebrew formula
and AUR PKGBUILD install it alongside the binary. The SBOM gains
per-component license metadata, corrected versions, and the previously
missing vendored libraries. The security workflow gains a
vendored-license scan with an explicit allow-list policy, and the
release workflow exposes a skip_perf input for releases that do not
touch pipeline logic.
Removes the heaviest vendored grammar (66 MB) along with its language
wiring and test fixtures; the supported-language count moves to 158
across README, site, and manifest. The grammar manifest additionally
records the canonical-source decisions for the five
registry-disagreement grammars and the per-directory license files
restored earlier this week.
The web server behind the graph UI is now a purpose-built in-house
module (src/ui/httpd.c): localhost-only listener, strict HTTP/1.1
parsing with fixed request caps, a per-connection receive deadline,
and Connection: close semantics. http_server.c keeps the routing and
handlers, rewritten against the new transport API; the public server
API and main.c are unchanged. The previously vendored third-party
server is removed entirely.
Comes with a new 28-test transport + routing suite (tests/test_httpd.c)
covering parsing edge cases (strict CRLF, Content-Length limits, raw
path matching, percent-decode rules) and live-socket behavior (CORS
policy, RPC dispatch, receive deadline, clean shutdown). The security
audit scripts now check the new file layout and treat any network call
in vendored code as a failure.
mimalloc bumped to v3.3.2 (MI_MALLOC_VERSION 30302), which adds static-CRT malloc override on Windows. Makefile: define the real MI_MALLOC_OVERRIDE=1 ONLY on MinGW (enables the static-CRT override entry points — the generic Windows fix for #424); on Unix keep static-link-order override (defining MI_MALLOC_OVERRIDE there compiles alloc-override.c's forwarding defs and crashes macOS with 'mi_free: invalid pointer' since system libs keep the system allocator). The old -DMI_OVERRIDE=1 was never read by mimalloc source — kept as a prod marker. smoke-test.sh: hard-fail on any 'mimalloc: error' in stderr. Regenerated scripts/vendored-checksums.txt for the new tree (security-vendored.sh passes; no dangerous calls in vendored mimalloc).
The standard linux release binary dynamically links glibc 2.38+ and
GLIBCXX_3.4.32, so it fails to start on Debian 11, Ubuntu 20.04/22.04,
RHEL/Rocky 8/9, Amazon Linux 2, etc. — yet install.sh, the npm and PyPI
wrappers, and the binary's own self-update all fetched it by default,
contradicting the "single static binary" promise.
Point every linux install + self-update path at the fully-static
"-portable" asset (gcc -static), which has no glibc floor. macOS/Windows
are unaffected and unchanged.
- install.sh, pkg/npm/install.js, pkg/pypi _cli.py: select -portable on linux
- src/cli/cli.c: self-update download URL AND checksum archive name both use
-portable on linux (they must match or the update fails checksum verify)
- scripts/smoke-test.sh: assert linux self-update targets the -portable asset
- scripts/ci/check-glibc-compat.sh: new guard — runs the binary inside
debian:bullseye (glibc 2.31) and asserts it starts
- _smoke.yml: run the guard on the portable binary in smoke-linux-portable
Reproduced: standard binary -> "GLIBC_2.38 not found" on glibc 2.31;
portable binary runs cleanly.
The C++ heap-corruption crash only manifests in the production binary (MI_OVERRIDE=1), so the smoke job (which runs the prod binary) is where it reproduces — most reliably on Windows. A ~10k-line templated header exercises the parse-churn path; indexing it must succeed.
Phase 16's FIFO writer-close surfaced as POLLHUP rather than a clean POLLIN+EOF, which the server's poll()-based read loop does not treat as shutdown, so the probe left the process alive and the smoke job failed on every platform. Switch to a plain '< /dev/null' immediate EOF — the server exits reliably (~1s) via its normal getline-EOF path, and it is fully portable (POSIX + MSYS2 bash), so no OS gate is needed. Verified end-to-end: bash scripts/smoke-test.sh reaches 'ALL PASSED'.
Phase 16 of smoke-test.sh launches the stdio MCP server with its stdin held open (a running instance), closes stdin to trigger shutdown, and asserts the process terminates within ~6s — guarding the orphaned-background-process failure mode behind #406 from the prod binary on every platform. POSIX-gated (Windows uses job objects; mkfifo is unavailable on MSYS2).
Two reviewed external-fork contributions, re-implemented as clean maintainer code.
#406 / PR #407 (thanks @nvt-pankajsharma): a POSIX parent-death watchdog so the stdio MCP server exits when its launching parent dies instead of lingering on stdin. Refactors signal_handler into an idempotent request_shutdown(); a watchdog thread polls getppid() (500ms) and shuts down + exit(0) once the initial ppid (>1) changes; Windows unaffected (#ifndef _WIN32). The fork's getppid()<=1 startup early-exit was dropped (it could wrongly kill a legitimately-launched server during reparent races / in container launchers; the initial_ppid>1 guard already no-ops safely), and thread-create failure is now non-fatal (matches the watcher/HTTP background-thread policy). Adds tests/test_parent_watchdog.sh + scripts/test.sh wiring.
#413 / PR #414 (thanks @santanusinha): runtime log-level control via CBM_LOG_LEVEL (case-insensitive debug|info|warn|error|none, or numeric 0..4; unset/unknown leaves the level unchanged). getenv() before threads start (no race). Adds tests/test_log.c cases + README env-var row.
Four overload-scoring lookups (cbm_registry_lookup_method_by_types/_by_args, _symbol_by_types/_by_args) linear-scanned the entire project registry on every call site, ignoring the pre-built method_buckets/func_qn_buckets hash index that the by-name lookups already use. On a large C++ header (e.g. 10989 defs) that is O(calls x project_funcs) per file (~34s/file), so big codebases never finish indexing. Add a hashed fast path (O(overloads)) when the registry is finalized; preserve first-match semantics via lowest registration index (equivalence verified over 1606 lookups, 0 divergence); keep the linear fallback for un-finalized single-file registries so normal files are unaffected. Adds tests/test_cpp_index_hang.sh (scale-tier subprocess+timeout repro, opt-in via CBM_RUN_HANG_TEST).
The cypher unsupported-function check captured stdout, but the CLI prints the parse error to stderr (which cli() redirects into $CLI_STDERR) and exits non-zero, leaving stdout empty — so the assertion always failed once the run reached the smoke stage. Verify the exit status and the stderr channel where the binary actually reports the loud failure. Confirmed locally: full smoke-test.sh passes.
cyp_first_cell built the query_graph JSON by interpolation, so a query with a double-quoted string literal (e.g. replace(f.name, "a", "A")) produced invalid JSON and an empty result (FAIL: replace empty). Escape embedded double-quotes before building the JSON.
The update/star notice in src/mcp/mcp.c references the project's own GitHub URL, which scripts/security-audit.sh blocked as not allow-listed. Add it to scripts/security-allowlist.txt.
Add scripts/check-no-test-skips.sh (run from lint) which fails the lint phase on any plain SKIP() or direct tf_skip_count manipulation; only SKIP_PLATFORM() (for genuinely platform-specific tests) is tolerated. Add FAIL() and SKIP_PLATFORM() helpers to the test framework and convert the remaining SKIP()/perf-gated skips across the suite into pass-or-fail assertions, so a suite that cannot meet its preconditions reports a red failure instead of a silent skip.
#272: '&' is neutralised by the single-/double-quoting of the grep/Select-String/
git commands, so it no longer fails validation. A relaxed validate_search_path_arg
(cbm_validate_shell_arg minus '&') is used for search_code + detect_changes
root_path/file_pattern; all other shell metacharacters stay rejected and
base_branch keeps the strict check.
#282: search_code defaulted regex=false, so 'foo|bar' matched the literal pipe
and silently returned 0 results. The result now carries a warnings[] array that
flags a literal '|' under regex=false (with the regex=true hint), so the trap is
visible instead of looking like a legitimate no-match. (The related invalid-regex
silent-empty case was already fixed in #283.)
Also: every search_code result now reports elapsed_ms, and a perf warning fires
when a search exceeds 5s (also logged as search.slow) so slow calls are visible.
Tests for both issues + the multi-word/invalid-regex no-regression; generous
search_code smoke coverage (basic+elapsed_ms, literal-| warning, '&' acceptance).
Closes#272Closes#282
Wire the Leiden detector into get_architecture via a new 'clusters' aspect.
arch_clusters loads Function/Method/Class nodes + CALLS edges (capped for very
large graphs), runs cbm_leiden, and reports the top-N communities compactly —
each with a dominant-package label, member count, cohesion (internal vs boundary
edge ratio), representative top nodes (by degree), and the packages spanned.
Singletons are skipped to avoid noise. The mcp.c serializer already emitted the
clusters array; it is now populated.
Tests: arch_clusters_basic (two cliques + a bridge -> >=2 connected clusters).
Generous smoke coverage added for the full Cypher surface (labels/type/id/keys/
properties/size/reverse/replace/left/coalesce/substring, NOT EXISTS dead-code,
CASE, unsupported-function error) plus the get_architecture clusters aspect.
Add a single-hop, anchored existence predicate:
WHERE NOT EXISTS { (f)<-[:CALLS]-() } -- functions with no caller
WHERE EXISTS { (f)-[:CALLS]->() } -- functions that call something
Parsed via parse_exists_predicate (reusing parse_node/parse_rel) into a leaf
condition (op=EXISTS, anchor variable, edge type, direction); evaluated against
the bound node with cbm_store_find_edges_by_source_type / _by_target_type. This
is edge-type-specific, so it finds true orphans that in_degree/out_degree miss
(e.g. a node with only a DEFINES edge but no CALLS caller). Multi-hop / nested-
WHERE EXISTS is intentionally unsupported and errors clearly.
Tests cover the dead-code (NOT EXISTS) and has-outgoing (EXISTS) cases; smoke
coverage added. ASan-clean.
Refs: Cypher read suite (EXISTS predicate)
Add an args-list (cbm_func_arg_t) to return items and parse comma-separated
arguments (var.prop or string/number literals), enabling multi-argument scalar
functions in projections:
- coalesce(a, b, ...) first non-empty value
- substring(s, start[, len]) 0-indexed substring
- replace(s, from, to) replace all occurrences
- left(s, n) / right(s, n) leading/trailing n chars
Arguments are freed in free_return_clause (ASan-clean). Unit test
cypher_func_multiarg plus smoke coverage for substring + coalesce.
Refs: Cypher read suite (tier 2b)
Add single-argument string functions to projections — size, length, trim,
ltrim, rtrim, reverse — via the generic function recognizer.
More importantly, change the engine to FAIL LOUDLY on unsupported syntax instead
of silently projecting an empty column. An unknown function call (e.g.
split(...), coalesce(...)) or list indexing/slicing in RETURN/WITH now returns a
clear "unsupported function '<name>' (supported: ...)" error rather than a
valid-looking but blank result — the same silent-empty failure mode that hid the
labels() bug. This supersedes the #373 graceful-resync behaviour (its test is
flipped to assert the error; a new test covers an unknown function in RETURN).
Decision: full Cypher Tier 3 (lists/maps/paths/comprehensions/params — a value
data-model rewrite) is intentionally NOT implemented; clear errors on
unsupported features are the higher-value, lower-risk completeness win for the
read subset agents actually use.
Refs: Cypher read suite (tiers 1-2a + unsupported-syntax hardening)
Add single-argument functions to RETURN/WITH projections, dispatched via a
generic IDENT-call recognizer so new functions no longer need a dedicated lexer
keyword:
- labels(n) -> ["<label>"]
- type(r) -> relationship type
- id(n) -> node/edge identity
- keys(n) -> JSON list of the node's non-null property keys
- properties(n) -> the node/edge properties JSON object
- toInteger/toFloat/toBoolean(x) -> numeric/boolean casts (null on non-numeric)
Previously labels()/id()/keys()/etc. fell into the unknown-function path and
silently projected empty (labels(n) returned ""); they now evaluate correctly.
Regex =~ in WHERE was already supported. Unit tests for each function plus
smoke-test coverage via the live query_graph CLI.
Refs: full Cypher read suite (tier 1 of 3)
The Phase 8 agent-install E2E checked the pre-unification Antigravity paths
(~/.gemini/antigravity/), which broke after the config-path fix. Point the
stub setup at ~/.gemini/antigravity-cli/ and assert the MCP server lands in
the shared ~/.gemini/config/mcp_config.json with AGENTS.md under
~/.gemini/antigravity-cli/. Verified end-to-end (smoke ALL PASSED).
count(DISTINCT x) was a parse error ("expected token type 85, got 9").
parse_aggregate_item now consumes an optional DISTINCT inside the call and
records it on the return item. Both aggregation paths track the set of unique
values per group and emit its size for COUNT(DISTINCT): the RETURN path reuses
the existing collect_lists machinery; the WITH path gains distinct_lists/
distinct_n fields. Plain COUNT and the other aggregates are unchanged.
Unit test: COUNT(DISTINCT f.label)=1 (all same label), count(f.label)=4
(non-distinct), COUNT(DISTINCT f.name)=4 (unique names). Smoke suite adds a
count(DISTINCT) query_graph check.
openCypher node-label alternation was unsupported (parse error after the
first label). parse_node now consumes `:A|B|C` into a single "A|B|C" label
string (mirroring the existing relationship-type alternation). The seed path
unions per-label results (scan_alternation_labels), and target/hop label
filters use label_alt_matches, which ORs over the alternatives. Single-label
matching is unchanged.
Unit test (Function|Module → 5 rows; Function|Class → 4) plus a query_graph
alternation check in the smoke suite. Relationship-type alternation tests
remain green.
openCypher label predicates in WHERE clauses were a parse error
("expected token type 67, got 73"). Parse `var:Label` in the WHERE leaf as
a condition with op=HAS_LABEL/value=Label, and evaluate it against the bound
node label in eval_condition (honoring NOT). Composes with AND/OR and works
in count(...) queries.
Unit test covers true/false/negated label predicates; smoke suite adds a
query_graph WHERE-label check.
The DISTINCT keyword on a WITH clause was parsed (r->distinct) but never
applied, so WITH DISTINCT silently returned duplicate rows. Add a dedup pass
in execute_with_clause that drops projected rows whose full value tuple
duplicates an earlier one (first occurrence kept), gated on wc->distinct.
No-op for the aggregation path (which already collapses per group).
Unit test (4 same-label functions collapse to 1 row; control without DISTINCT
keeps 4) plus a query_graph WITH DISTINCT check added to the smoke suite.
Route Claude Code config paths (skills, .mcp.json, .claude.json,
settings.json, hook scripts) and agent detection through
CLAUDE_CONFIG_DIR-aware helpers, falling back to ~/.claude. Hook command
strings written to settings.json keep the legacy tilde form when the env
var is unset, so existing configs stay portable across HOME values. Prints
a one-line migration nudge when CLAUDE_CONFIG_DIR is set and a legacy
~/.claude tree still exists.
Adds cli_detect_agents_finds_claude_via_env and isolates CLAUDE_CONFIG_DIR
in the existing detection tests so the runner env can't leak in.
Distilled from #321 onto current main (adapts to the v0.7.0 non-blocking
augmenter hook signature, which the original branch predated). Closes#320.
cppcheck (warning+style, error-exitcode=1): guard cbm_fqn_compute against a NULL project/rel_path (ctunullpointer reachable from cbm_extract_file), and narrow prefix_len/suffix_len (path_alias) and ft_count (pass_githistory) to the scopes that use them.
security-audit.sh: the mcp.c file-read count grew to 13 (search/ADR/Windows-support reads — all path-contained or transport reads, audited, no new exfiltration surface); bump the reviewed maximum and document the update-check + request-body reads. Allow the diagnostics.c atomic metrics dump (.tmp+rename) in the file-write scan, and allow-list the sqlite WAL-checkpoint doc URL referenced in a store.c comment.
The previous PreToolUse hook gated Grep/Glob/Read/Search with 'exit 2'
on the first call per session, which broke Claude Code's
read-before-edit invariant (issue #362) and could deny tool calls under
upgrade/missing-binary failure modes.
Replace it with a structurally non-blocking augmenter:
- New 'codebase-memory-mcp hook-augment' subcommand reads the hook JSON
from stdin and, for Grep/Glob, queries search_graph (in-process, no
shell) and emits hookSpecificOutput.additionalContext. Every failure
path (no project, short token, missing binary, slow query, timeout)
exits 0 with no stdout — the hook physically cannot block a tool call.
- 300 ms SIGALRM/_exit(0) in-process deadline; 5 s settings.json timeout
backstop. Output is written exactly once at the very end, so a
mid-work timeout yields a clean no-op (never partial JSON).
- Matcher narrowed to 'Grep|Glob' (Read explicitly excluded) for Claude;
Gemini matcher narrowed to 'google_search|grep_search' (excludes
read_file) for the same reason.
- The installed shim is a thin wrapper that delegates to the binary;
legacy filename 'cbm-code-discovery-gate' is kept so existing
settings.json entries upgrade with zero migration. Installer refuses
to embed binary paths containing a double quote (shim injection
defense).
- Per-agent 'old matchers' lists let upsert/remove clean up historical
matcher strings during upgrade.
- Smoke tests (8d/8e/8l) updated to assert the new behavior and
regress-test against re-introducing Read in the matcher or 'exit 2'
in the shim.
- Session reminder text updated: 'always Read a file before editing it'
replaces the prior 'fall back to Read only for text content'.
(cherry picked from commit f72c8e68c4d91e52911a569a967ad782ce5472b2)
Flat BM25 queries of the form:
SELECT ... FROM nodes_fts JOIN nodes WHERE MATCH ? AND project=? ORDER BY bm25() LIMIT N
block FTS5 WAND/MaxScore early-exit — the outer JOIN+WHERE is invisible to
the FTS5 planner, so it scores every matching document before any filter fires.
On a large codebase with 100K+ matches this causes 2–16 minute queries.
Fix: two-step subquery. The inner FTS5-only query:
SELECT rowid, bm25(nodes_fts) FROM nodes_fts WHERE MATCH ? ORDER BY bm25() LIMIT 2000
can early-terminate because no outer predicate blocks it. The outer query
then joins and filters at most BM25_INNER_LIMIT (2000) candidates.
The count query uses the identical inner-limit subquery, so it benefits too.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Make project a required CLI argument instead of a hardcoded name,
and remove internal query strings used during development testing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three compounding bugs caused 1.5–8.5s latency on name_pattern= searches against
large projects (216K nodes), now reduced to ~0ms query time (cold-start dominates):
Fix 1 — regex compiled once per statement, not once per row
sqlite_regexp / sqlite_iregexp now use sqlite3_get_auxdata / sqlite3_set_auxdata
to cache the compiled cbm_regex_t for the lifetime of the statement. Previously
cbm_regcomp + cbm_regfree ran for every row scanned.
Fix 2 — LIKE pre-filter cuts rows reaching the regex
Wire cbm_extract_like_hints (already implemented but dead) into search_where_basic
via a new where_add_like_hints helper. For .*Controller.* this prepends
n.name LIKE '%Controller%', letting the idx_nodes_name index satisfy the LIKE
clause first and passing only matching rows to iregexp(). Added search_like_pool_t
to manage the malloc'd LIKE strings across both statement executions.
ST_SEARCH_MAX_BINDS raised 16 → 32 to accommodate extra bind slots.
Fix 3 — count query no longer runs per-row edge subqueries
The count SQL previously wrapped the full SELECT (which includes two correlated
subqueries for in_deg / out_deg) in SELECT COUNT(*) FROM (...), executing those
edge counts for every matching row even though the count needs none of that.
Non-degree-filter path now uses SELECT COUNT(*) FROM nodes n WHERE <same WHERE>,
which has no per-row subqueries. Degree-filter path retains the wrapped form
since it needs those columns for the filter.
Benchmark on home-ubuntu-dev-sis (216K nodes, 509MB DB):
Query BEFORE AFTER speedup
name_pattern=.*Controller.* 3099ms 508ms 6×
name_pattern=.*Service.* 2006ms 506ms 4×
name_pattern=.*Repository.* 2006ms 508ms 4×
name_pattern=specificFuncName 1506ms 507ms 3×
label=Method + name_pattern=.*get.* 8509ms 509ms 17×
name_pattern=.*Approve.* 1506ms 507ms 3×
name_pattern=.*authorize.* 1506ms 509ms 3×
The ~500ms floor is cold-start I/O (opening a 509MB file from disk). In the
long-running MCP server process the warm-cache query time is sub-millisecond.
All store search tests pass including pagination, degree filter, and extract_like_hints.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Martin Vogel