The *.sh eol=lf rule landed via #1314, but the git hooks
(scripts/git-hooks/commit-msg, scripts/hooks/pre-commit) are extensionless
and were still at the mercy of core.autocrlf. Add their explicit entries,
plus the contract test from #1272 wired as scripts/test.sh Step 0t: every
shell entrypoint (*.sh + both hook directories) must carry an eol=lf
attribute, with a matched-zero-files guard so a broken glob can never pass
vacuously.
Distilled from #1272: the diagnosis (CRLF checkouts breaking shebangs under
WSL/MSYS), the hook-file coverage, and the guard design are @xumian520's;
verified RED without the *.sh rule (105 uncovered entrypoints) and green
with it (107 files).
Co-Authored-By: xumian520 <126989134+xumian520@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Second mechanism of #1196, exposed by the reporter's v0.10.6 retest: every
relationship-expansion site capped its output buffer at bind_cap*10, so edges
past the cap were silently dropped BEFORE WHERE and aggregation ever ran.
count() then reported the scanned prefix as if it were a fact — 9,360 of
13,691 DEFINES field-measured at --max-rows 1000 — and a LABEL on the source
did not protect you (the label workaround only ever fixed source
enumeration, which #1323 already made exact). max_rows is an output-row
limit per the public header; projection already enforces it.
All five capped sites now share one growable append (geometric growth,
size_t sizing): the per-hop expansion, its fixed/variable-length helpers and
process_edges, the bound-terminal driver, and the cross-join outer buffer.
Only allocation failure stops materialisation; match_count stays truthful
either way, so the #627 OPTIONAL contracts hold (a saturated buffer can no
longer exist, and the fallback rows share the same append). The #601
deadline still bounds pathological time, and hop caps keep bounding depth —
this removes only the silent row-dropping.
Regression test: 2 labeled sources x 30 edges with max_rows=2 — the old cap
returned count=20; ground truth 60 now holds, and the list form returns
exactly max_rows rows. Proven RED before and RED again on revert; cypher
181/181 and mcp suites green, including every OPTIONAL/#627 semantics test.
Fixes#1196 (together with #1323, which fixed the unlabeled source scan).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The conflict's shared tail (#endif + closing brace) was emitted once for
two adjacent Windows tests; re-add the closer so both tests compile.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Trivial insertion-point collision in tests/test_mcp.c: both this branch and
main (#1704's UTF-8 pin) add a Windows search_code test at the same spot.
Both tests kept.
PowerShell 5.1 encodes stdout for a native-process pipe in the console
OEM codepage, so raw search content containing characters the inherited
CP cannot carry (Cyrillic under CP437/850, ...) reached
collect_grep_matches as '?' — and whether it degraded depended entirely
on which console the server happened to inherit. That surfaced as the
intermittent test_mcp raw-Русский mojibake on the windows CI leg and
means real Windows users in a default console get '?' for all
non-ASCII search_code raw content.
Every generated command now pins [Console]::OutputEncoding to UTF-8, so
the pipe is codepage-independent by construction. The read side needs
no pin: Select-String decodes BOM-less UTF-8 via .NET StreamReader
defaults. A Windows-side builder test asserts all five command variants
carry the prelude.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1181 gave list_projects pagination parameters, retiring the last
empty-properties schema — and the two #1359 regression tests leaned on
list_projects as their live zero-argument example, so main went red the
moment the merge train composed (the PR was green on its July base, which
predated these tests).
The gate's schema→decision core is split behind a CBM_CLI_ENABLE_TEST_API
seam, so the zero-argument branch stays pinned directly (empty properties,
absent properties, populated properties) regardless of what the registry
ships; list_projects now asserts its NEW truth (piped args accepted, TTY
still refused); and the schema↔gate parity sweep keeps running over every
tool without the impossible >=1 zero-argument floor. Production behavior
is unchanged — this is the tests catching up with an intended schema
change, plus a seam so they never again depend on a shipped example.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.6 release run failed deterministically at verify:
BLOCKED: expected scan object is missing:
objects/scan-3099e91c...--codebase-memory-mcp.exe
exclude-rescanned-selected-objects.sh (added after v0.10.5, first exercised
by this release) deliberately deletes the selected executables from the
surface-scan directory — their bytes were already scanned as candidates and
re-submitting identical bytes re-rolls a probabilistic classifier — and
writes binaries/virustotal-withheld.tsv. But check-virustotal.sh still
received the pre-withhold scan-set listing all sixteen objects and failed
closed on the first missing file. The rework's two halves never talked.
The gate now accepts an optional VT_WITHHELD manifest (strict parse: v1
marker, the stated reason required, sha256-keyed rows): an expected-set row
whose hash the manifest vouches for is exempt from the on-disk and
action-output contracts, while everything else keeps the strict path.
Fail-closed properties preserved and extended:
- no VT_WITHHELD -> byte-for-byte previous behavior (candidate
stage and dry-run call sites are unaffected;
verified against the original failure)
- withheld object present -> blocked (inconsistent staging)
- hash outside the set -> blocked (spurious withhold)
- everything withheld -> blocked (scan would cover nothing)
- mismatched object name -> blocked
vt-results.tsv keeps its exact shape (scanned objects only) — the release
notes table already uses the candidate results, and the withheld manifest is
now preserved with the rest of the evidence artifacts. release.yml passes
VT_WITHHELD only in the verify stage, right after the withhold step.
Verified offline with a fixture reproducing the release failure verbatim
plus the four negative cases above; the positive case passes staging and
association validation and proceeds to VT polling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Keep caller-requested discovery scope while rebuilding changed weaker-mode requests at the stronger stored coverage. Export persistent artifacts only after the replacement database generation is published.
Signed-off-by: astandrik <astandrik@yandex-team.ru>
Re-stamp the runtime DACL only when it is actually wrong, and repair cache
children left unusable by the pre-v0.10.3 DACL regime.
The unconditional per-start re-stamp rewrote an already-correct security
descriptor and propagated it to children (#1601 counted eleven no-op
"Security change" USN records against one _config.db in a day), and every
rewrite is a window in which a concurrent atomic publish can be refused
DELETE on the destination (#1620). The old regime's PROTECTED,
non-inheritable ACE also left every child born unusable — the 0-byte
worker-log class behind #1416's diagnosis — so the secured directory now
walks its regular children and repairs any with an empty DACL or a foreign
owner.
The fast path is gated on the ADOPTION-level predicate, not the general
secure() check: lock-directory adoption (private_win_owner_only_dacl)
demands the exact protected owner-only single-ACE descriptor the stamp
writes, while secure() also admits SYSTEM/Administrators ACEs. A fresh
directory with an inherited DACL passed secure(), skipped the stamp, and
stranded every subsequent lock adoption — 59/77 daemon-suite failures on
the real Windows VM. With the ported predicate (SE_DACL_PROTECTED,
single non-inherited owner ACE, FILE_ALL_ACCESS/GENERIC_ALL) the same VM
runs 77/77 and the full suite 7346/0.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Codex sanitizes stdio MCP subprocess environments to the names listed in
env_vars. Since #1645 CBM_RUNTIME_DIR relocates the daemon rendezvous, so a
Codex subprocess that does not receive it looks for the daemon in the DEFAULT
location and never finds it — the same silent client/daemon split
CBM_CACHE_DIR caused in #1562. Both names decide WHICH daemon a process talks
to and are now forwarded unconditionally (forward-if-present semantics);
behavioural knobs (log level, workers, budgets) deliberately stay
unforwarded — that broader list remains #1664's open enhancement question.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
gotspatel's live opencode.json stores our entry with backslashes
(`C:\...\codebase-memory-mcp.exe`) while the installer compares its own path
with forward slashes — the same file on disk, refused over the separator
spelling, so op=mcp_install failed on a correctly-installed machine (and on
Windows the dead-path probe rightly reported the binary PRESENT, which turned
the mismatch into a hard refusal).
Ownership comparison now treats `\` and `/` as equal everywhere and folds
case on Windows only, where the filesystem is case-insensitive; POSIX
byte-exactness otherwise holds. The annotated entry that names this binary is
recognised as already satisfied and preserved byte-for-byte.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
PowerShell 5.1's `Set-Content -Encoding UTF8` writes a BOM, so real
Windows-authored Hermes configs start with EF BB BF — and both edit ops
failed content-independently (the reporter's 26-byte reproduction is their
23-byte file plus exactly this BOM; reproduced RED on macOS with the same
bytes, so the platform was never the variable).
The document read now validates past a leading BOM and yaml_doc_init treats
it as a prologue: the first line's structure starts after it, the key lookup
still sees our own section when the BOM immediately precedes it (guarded by a
dedicated no-duplicate-section test), and every edit splices interior ranges,
so the BOM survives writes byte-for-byte. Non-document inputs — keys, entry
blocks, identity scalars — keep the strict no-BOM rule.
Also makes the moved-entry cli test fixture platform-correct: the Windows
dead-path probe can only prove a fixed-drive path absent, so the Windows
branch uses one; a POSIX-shaped path is refused there by design.
Fixes#1656.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two install failures with the same root theme — the entry we write is a
compatibility contract with the agent's parser, and both sides of that
contract needed repair:
goose (#1675): ExtensionConfig::Stdio declares `name` as a required serde
field with no default, and goose's loader silently drops entries that fail to
deserialize — install reported success and the extension was invisible. The
goose block now carries `name: codebase-memory-mcp`; the non-goose YAML
schema stays name-free. A CBM_CLI_ENABLE_TEST_API seam asserts the exact
block bytes per schema.
annotated MCP entries (#1630, the deferred field-merge): dbd20eaa recognised
an entry the client annotated ("enabled": true beside our command/type) but
could only leave it untouched, because replacing the whole entry would drop
the client's keys. config_json_like gains
cbm_json_like_replace_field_raw_if_unchanged — splice ONE member's value,
preserving every other byte (comments, ordering, client keys) — and the
upsert flow uses it on the two AUTHORIZED repair channels only:
- a relocating update (the entry names the previous managed binary), and
- the existing Windows dead-path probe, which previously fell back to a
wholesale rewrite and lost the annotations.
POSIX keeps its doctrine unchanged: a config-supplied path is never trusted,
so a moved-looking entry without that authority is preserved byte-for-byte
and install fails loudly (cli_editor_mcp_preserves_unrecorded_posix_absolute_
entries_without_probe holds). All repair/refusal paths are covered by tests
proven RED on the unfixed flow.
Fixes#1675.
Fixes#1630.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Four legal-YAML constructs from the reporters' real Hermes configs made
`install` fail permanently (any one of them aborted mcp_install and/or
pre_llm_hook_install):
- exact empty flow collections as values (`plugins: []`, `tool_choice: {}`)
— now validated key-only in both the mapping-body and sequence document
scans, mirroring the #1673 empty-mapping exception; non-empty flow
collections stay rejected.
- block sequences at the same indent as their mapping key (column-0 `- item`)
— item lines directly after a value-less key are structure, not malformed
keys, in the root walker, the key matcher, and the sequence mapping-range
walker.
- double-quoted scalars continued across lines with a trailing `\` — the doc
loader now precomputes per-line continuation flags; continuation lines are
value bytes every structural walker skips, and a document ending inside an
open continuation stays an error.
- mid-word quote characters in plain scalars (`LET'S`) — quotes are scalar
indicators only at a node start (range start, after `:`, after `-`),
exactly like the #1639 anchor/alias rule; real quoted values keep their
protection.
Byte-identity alone also froze users on canonicals older releases wrote:
galaxy's entry had `command:` unquoted, and the goose block gained `name:`
(#1675), so the existing entry was declared FOREIGN forever. An entry under
our key now repairs when it parses as a known prior shape (single command
line, or the pre-name goose block) with a codebase-memory-mcp[.exe] command
basename; anything else stays FOREIGN and the file untouched.
End-to-end: both reporters' full configs (iandol 15.6 KB, galaxy 15.2 KB) now
install with zero agent_config errors, every original line byte-preserved,
and the goose upgrade path rewrites the old block in place. Each construct
carries a distilled regression test proven RED on the unfixed editor.
Fixes#1631.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ts_eval_expr_type and ts_signature_for_call are mutually recursive: resolving
a call evaluates its argument expressions once per lookup path (method
dispatch + namespace fallback), and in tsc-compiled spread files the first
argument is itself the next nested Object.assign(...) call — the same subtree
re-evaluates once per enclosing level, 2^n total. The TS suite's
objectSpreadRepeatedComplexity.js (3.6 KB, 5 nodes) measured 20.4 s; with the
memo its eval cost is zero within measurement noise of a one-file control,
and the microsoft/TypeScript corpus drops 37.5 -> ~24 s warm (nodes
byte-identical, edges within the known scheduler jitter).
Expression types are position-pure within a file pass (one node = one scope
path; the per-file walk is single-threaded and deterministic), so one eval
per node is the correct semantics, not a cache trade-off. The memo is a
per-file, arena-backed, linear-probe table keyed on TSNode.id. Results
produced under a depth-cap or budget bail are never stored: both bail sites
bump a degradation counter, and a store only happens when the subtree
completed clean — a degraded UNKNOWN can therefore never shadow a later full
evaluation.
The regression guard asserts work, not wall-clock: the nested-Object.assign
shape must complete without exhausting the deterministic eval budget, read
back through a new CBM_ENABLE_TEST_SEAMS accessor pair. The seam lives in the
lsp_all unity object, so GRAMMAR_CFLAGS_TEST/TSAN now carry the seams define
(test artifacts always have seams; prod never does). Verified RED without the
memo (budget exhausted, suite 47.8 s) and green with it (3.2 s).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
cbm_arena_alloc(arena, 0) returns NULL, so the java and cs cross-registry
builders read a def_count of 0 as OOM and returned NULL — a corpus with no
files of that language silently lost its shared registry (and the seal tests
caught exactly that: cbm_cs_build_cross_registry(&arena, NULL, 0) == NULL).
Guard the partition alloc behind def_count > 0; the empty registry is still
built, finalized, and shared.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The no-skips lint gate (scripts/check-no-test-skips.sh) rightly rejected the
throughput-report test's two SKIP() calls:
- CBM_SKIP_PERF=1 is deliberate operator configuration, not a hidden
environment failure: reporting is off by request, so the test PASSes with a
stderr note instead of skipping.
- an uncreatable report dir IS an environment failure and now FAILs with the
remedy in the message (set CBM_COMPLEXITY_REPORT_DIR), per the policy text.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
ts_eval_expr_type was depth-capped but work-unbounded: crafted expressions
(the TS test suite's repeated object spreads) stay under the depth cap
while fanning out. Charge the same per-file budget the type-text parser
uses, at 16 units per entry (an eval entry does ~two orders of magnitude
more work than a text-parse unit), degrading to UNKNOWN on exhaustion.
Honest status: this hardens the documented budget design, but the known
3.6 KB spread-bomb baseline file still measures ~11 s in-corpus — its
entry path into the evaluator apparently runs unarmed and is recorded as
an open lead (zero budget warnings observed). Suites green (ts_lsp,
extraction, complexity).
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two measured changes on the TypeScript corpus (81,397 files; v0.9.0 17.9 s,
before 44.1 s, after 37.5 s):
- cov_rebuild_shadow_graph upserted every directory segment for every
failure row — 13,243 parse-partial baseline files under one tests/
subtree meant ~80k redundant node/edge round-trips, 9.1 s of a 9.2 s
coverage_replace. An in-rebuild path->id map creates each directory once;
identical graph (edges deduped by unique key before, absent now).
Coverage block 9,162 -> 2,920 ms. Sub-block timings
(publish.timing.coverage: del/rows/prune/meta/commit + row_count +
detail_bytes) are kept — the caller-level number could not name the
culprit.
- JS/TS export_statement is an import CONTEXT only in its re-export forms
(source field, or a bare specifier list without a declaration). The old
is_export_of_declaration blacklist missed TS-only forms
(ambient_declaration, function_signature, module_declaration), running
declare-heavy subtrees (.d.ts, export namespace) behind inside_import:
suppressed usages + per-identifier ancestor walks. Positive detection
replaces the blacklist; +3,954 restored usage edges on the corpus,
nodes identical. (Measured perf-neutral here — kept for correctness.)
Suites green incl. store_nodes/edges/search, mcp, extraction, ts_lsp,
complexity, and the 53-language calls-breadth contract.
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Four measured changes on the dotnet/runtime corpus (58,656 files; every
step's numbers below are from full-corpus runs on the same host, baselines
captured this session; v0.9.0 = 444.3 s):
1. Two-phase cs registry build (types -> finalize -> funcs), the same
pre-finalize linear-lookup disease fixed for Java: prepare 280 -> 140 s.
2. C# import-context carve-out: cs_import_types lists namespace_declaration
(for namespace-name mapping) and using_statement (C#'s RAII block — a
grammar-name collision), so EVERY namespaced C# file's whole body ran
with inside_import=true. That both suppressed ordinary usage extraction
under namespaces and sent every identifier through the ancestor-walking
import-binding check (tree-sitter's ts_node_parent re-descends from the
root, so wide files went quadratic: one 147 KB JIT torture file cost
490 s; 6.8 s after). Only using_directive / namespace_use_declaration
open an import scope now. Restores the suppressed usages:
edges 4,291,387 -> 5,869,093 (+37%), nodes identical.
3. The usages walker maintains call/import ancestry as enter/exit counters
on its explicit stack instead of per-node ancestor re-walks
(extract_usages.c had grown from 6 to 100 ts_node_parent calls since
v0.9.0; the two per-node gates are now O(1) with semantics preserved —
strict ancestors only, emit before self-count).
4. Registry short-name indexes replace the two remaining full scans:
cs_lookup_extension walked all 963k funcs per unresolved invocation
(now the existing free-func short-name iterator, first-match order
preserved via min-index selection), and cs_resolve_type_name's step-9
fallback scanned every type per unresolved name IN BOTH the builder and
per-file resolution (new type_short index in finalize, same
reverse-insertion ascending-order pattern, best-score ties keep the
first-in-registration-order winner). Builder 140,095 -> 500 ms; resolve
cross-LSP CPU 5.5M -> 317k ms (us_per_file_per_kdef 191 -> 11).
End state: 449.9 s wall (1.01x of v0.9.0) with +48.6% edges vs v0.9.0 —
per-edge cost 32% BETTER than v0.9.0. Extract's remaining 359 s is the
24 MB hugeexpr1.cs parse floor both versions pay.
Guarded by the complexity suite; cs_lsp/extraction/edge/lang-contract
suites green including the 53-language calls-breadth contract.
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Finding #1669 took an 11-corpus A/B across two release binaries. This
suite makes that bug class fail a unit test in seconds, on every
platform, from tiny corpora.
Method: build k and 2k REPLICATED module copies, run the full in-process
pipeline on both, assert counter RATIOS. Independent copies mean every
extensive quantity — nodes, edges, Σ per-file registry defs — must grow
linearly (ratio ~2). A files x corpus coupling makes per-file work itself
grow with k and lands at ratio ~4. Ratios expose the exponent regardless
of absolute scale, so 60-120 files suffice.
Verdicts are pure functions of (code, input): gates ride ONLY on
deterministic work counters and data-product counts, never on wall time.
Throughput (nodes/s, edges/s) is information-only, written to
private/benchmarks/complexity-<ts>.json (local, gitignored; skipped
under CBM_SKIP_PERF where rates are meaningless).
Two corpus shapes, both needed:
- Independent modules (java/py/go/ts templates): catches cross-module
contamination and dedup breakage. The #1669 bug is GREEN here — fully
closed modules filter perfectly, which is exactly why it survived.
- The growing shared package (bigpkg): one Java package whose file count
scales with k — the real-repo shape (files concentrate in large
packages). The JVM namespace filter branch makes per-file work track
package size, so this corpus is the honest #1669 reproducer:
ratio 4.00 RED on the pre-fix tree, 4.00 RED for a module-scoped
overlay, 2.00 GREEN for the own-file overlay. It discriminated the
correct fix design before the fix was written.
Both legs of every pair exceed MIN_FILES_FOR_PARALLEL(50): below it the
sequential path runs, which builds no shared registries and would be the
wrong code path to gate (its per-file cost is bounded by the 50-file
ceiling).
Recorded but deliberately NOT gated, with reasons at the case:
tail_candidates and fallback_rows are legitimately superlinear under
replication until those scans are bounded, and measured ~1 ns/unit.
Every ratio gate carries a non-vacuousness floor on the base counter so
broken counter wiring fails loudly instead of green-washing
(cbm_pxc_count_perfile_defs feeds the overlay path into the same
counter the fallback path already used; wired for Java, extend to the
TS overlay when touching ts_lsp).
Dynamic coverage: languages iterate CBM_LANG_COUNT; embedded templates
cover the LSP-hybrid languages, and tests/fixtures/complexity/<lang>/
dirs are auto-discovered so a new language joins the guard by dropping
fixtures. Uncovered languages are listed in the report with the reason.
The local report additionally carries per-language node/edge counts with
ratios and a per-pass elapsed_ms table per run (captured via a TEE log
sink during the in-process pipeline runs) — trend data for humans, still
never a gate.
Refs #1669.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel