scripts/test.sh labels its pre-build contracts as a running 0a..0u sequence.
The new blob-generator contract was slotted in as "0f2" to sit next to the
other build-tooling contract, which breaks the one convention the list has.
Append it as 0v instead; these steps all run before the build, so position
carries no meaning beyond the label.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
vendored/nomic/code_vectors_blob.S is a GENERATED file that was hand-edited.
1f674c80 added the ELF .note.GNU-stack section to it after finding that every
Linux binary we had ever shipped requested an executable stack: an object
carrying no such note tells GNU ld nothing about its stack requirement, and ld
then assumes the worst for the entire link. That fix went into the artifact
only. scripts/extract_nomic_vectors.py still emitted the Mach-O branch alone,
so the next regeneration would have overwritten the tracked .S, dropped both
the ELF and COFF branches, and put GNU_STACK RWE back into every release --
silently, because nothing compared the generator against its own output.
write_blob_s() now emits the tracked wrapper verbatim, and the new contract
test fails if the two ever drift again. The template is compared rather than
executed so the test does not need the torch/transformers import the extraction
script pulls in. The write is pinned to UTF-8 because the WHY comment carries
an em dash and the default encoding is locale-dependent.
check-binary-composition.sh already fails a release whose binary has an
executable stack; this closes the same hole at edit time, where the diff is
still small enough to read.
Reported and originally fixed by Anand Aiyer in #1151, which also carried the
COFF branch. That PR sat unreviewed for a month while the .S half was
rediscovered and landed independently without credit.
Co-authored-by: Anand Aiyer <anand@aiyer.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The *.sh eol=lf rule landed via #1314, but the git hooks
(scripts/git-hooks/commit-msg, scripts/hooks/pre-commit) are extensionless
and were still at the mercy of core.autocrlf. Add their explicit entries,
plus the contract test from #1272 wired as scripts/test.sh Step 0t: every
shell entrypoint (*.sh + both hook directories) must carry an eol=lf
attribute, with a matched-zero-files guard so a broken glob can never pass
vacuously.
Distilled from #1272: the diagnosis (CRLF checkouts breaking shebangs under
WSL/MSYS), the hook-file coverage, and the guard design are @xumian520's;
verified RED without the *.sh rule (105 uncovered entrypoints) and green
with it (107 files).
Co-Authored-By: xumian520 <126989134+xumian520@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.6 release run failed deterministically at verify:
BLOCKED: expected scan object is missing:
objects/scan-3099e91c...--codebase-memory-mcp.exe
exclude-rescanned-selected-objects.sh (added after v0.10.5, first exercised
by this release) deliberately deletes the selected executables from the
surface-scan directory — their bytes were already scanned as candidates and
re-submitting identical bytes re-rolls a probabilistic classifier — and
writes binaries/virustotal-withheld.tsv. But check-virustotal.sh still
received the pre-withhold scan-set listing all sixteen objects and failed
closed on the first missing file. The rework's two halves never talked.
The gate now accepts an optional VT_WITHHELD manifest (strict parse: v1
marker, the stated reason required, sha256-keyed rows): an expected-set row
whose hash the manifest vouches for is exempt from the on-disk and
action-output contracts, while everything else keeps the strict path.
Fail-closed properties preserved and extended:
- no VT_WITHHELD -> byte-for-byte previous behavior (candidate
stage and dry-run call sites are unaffected;
verified against the original failure)
- withheld object present -> blocked (inconsistent staging)
- hash outside the set -> blocked (spurious withhold)
- everything withheld -> blocked (scan would cover nothing)
- mismatched object name -> blocked
vt-results.tsv keeps its exact shape (scanned objects only) — the release
notes table already uses the candidate results, and the withheld manifest is
now preserved with the rest of the evidence artifacts. release.yml passes
VT_WITHHELD only in the verify stage, right after the withhold step.
Verified offline with a fixture reproducing the release failure verbatim
plus the four negative cases above; the positive case passes staging and
association validation and proceeds to VT polling.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The verify pass submitted every extracted object, selected executables included,
on the stated grounds that "VirusTotal is content-addressed, so identical bytes
return the analysis it already holds instead of re-running 70+ engines".
That is measurably false. On v0.10.5 all EIGHT re-submissions produced a NEW
analysis - same VirusTotal file-id, timestamp 47 minutes later:
candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z)
verify : file-id=2c00f485... ts=1786798758 (12:59:18Z)
Re-analysing identical bytes re-rolls a probabilistic classifier, and Microsoft's
ML engine answered differently within that hour, in BOTH directions:
82750cd1 (linux-amd64) microsoft-ml -> clean
6d3c5be6 (darwin-arm64) clean -> microsoft-ml
The published notes are generated from the candidate scan, so v0.10.5 shipped a
table calling linux-amd64 flagged when VirusTotal had it clean, and darwin-arm64
clean when VirusTotal was reporting Trojan:Script/Wacatac.B!ml. Every hash in
that table links to the page that contradicted it. Corrected in place after
publication; this removes the cause.
The second scan proved nothing the first did not. Identity is settled by hash
before this step runs: verify-release-selection.py reconciles every published
container to the selected bytes, and checksums.txt binds the same digests
publicly. A re-scan adds no assurance - only another roll.
What still gets scanned is exactly what the candidate pass never saw: install.sh,
install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the
unpacked UI assets. install.sh and install.ps1 are the highest-consequence
non-executable bytes we publish - users pipe them straight into a shell - and
that coverage is untouched. Measured on the v0.10.5 object set: 16 objects in,
8 withheld, 8 still scanned.
The withheld set is recorded as evidence (cbm-virustotal-withheld-v1) naming each
sha256 and pointing at virustotal-candidate-results.tsv, so the published
evidence still accounts for every shipped object.
Fails closed three ways, each with an actionable message: no object matches a
selected sha (the containers do not carry the recorded bytes), everything is
withheld (the surface scan would be a no-op), or the selection names no shas at
all. The zero-match grep is wrapped rather than left to pipefail, because a
guard that aborts silently is not a guard - found by testing the guards rather
than assuming them.
Also drops 8 VirusTotal submissions per release.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
audit-license-provenance.py ran `curl https://www.apache.org/licenses/LICENSE-2.0.txt`
every time the gate executed and byte-compared the result against
vendored/nomic/LICENSE, with capture_output and no error check. Any fetch
failure therefore produced an empty string, compared unequal, and reported:
vendored/nomic: DIFFERS [apache.org canonical LICENSE-2.0.txt]
PROVENANCE AUDIT FAILED: 1 unexplained verdict(s)
which is indistinguishable from a real licence discrepancy.
A gating verdict must be a pure function of the tree, not of whether a web
server answered. This one was neither reproducible nor attributable: it
reddened `security / license-gate` on PR #1337 - a branch touching cli.h,
hook_augment.c and test_cli.c, and no licence at all - and left that
contributor blocked for over two weeks on a signal that had nothing to do with
their change.
Verified before pinning: our vendored copy is byte-identical to the upstream
canonical text, 11358 bytes on both sides, diff clean. The Apache-2.0 text is
immutable and versioned, so a digest is the honest way to express "this is that
text". A mismatch now means our vendored copy changed, which is exactly - and
only - what this audit exists to detect.
The audit passes locally with no network access on the nomic entry.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The ui-* archives are byte-identical copies of the canonical ones, published
after verify so they inherit the hash-bound VirusTotal verdicts. They were
absent from checksums.txt, and publish-legacy-aliases.sh documented that as
intentional: "checksums.txt covers the canonical names current installers
request."
That reasoning has a hole. The aliases exist only for 0.9.x updaters (#1538),
and those verify the NAME they asked for. So the alias fixed the 404 and moved
the failure one step later - the updater downloads the archive, cannot find its
name in checksums.txt, and refuses:
warning: codebase-memory-mcp-ui-darwin-arm64.tar.gz not found in checksums.txt
error: refusing to install an unverified download
Reported by AmooAti in #1134. Confirmed on the live v0.10.4 release: eight ui-*
archives published, zero of them listed. Every pre-0.10 user who answered the
old variant chooser with "ui" is hard-blocked from updating by any path.
The same digest is now emitted under the legacy name before the attestation
step, so the attested artifact covers both names. No new bytes and no new scan
surface: an alias is a copy, so its sha256 is by construction the one already
computed.
The rule lives in scripts/ci/append-legacy-alias-checksums.sh rather than inline
in the workflow, because the venue-parity contract requires it: a venue may
provision, plumb artifacts, or call a canonical leg script, and text
transformation is none of those. Keeping it beside publish-legacy-aliases.sh
also puts the two halves of the alias rule in one place, which matters because
they must stay in step - .tar.gz and .zip only, never an already-ui-* name. It
fails closed when it matches nothing, since a name with no asset is as broken as
an asset with no name.
Validated against the real v0.10.4 checksums file: the generated set is exactly
the eight ui-* assets that release published - no phantom names, none missing -
and the empty case exits non-zero.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release notes now carry one row per product — the bytes actually published, with
their VirusTotal verdict — instead of a column per candidate.
Several candidates are scanned so the selector has an alternative when an opaque
classifier flags one, but a reader installing cbm cares about what they receive,
not about what we discarded. The rejected candidates' verdicts remain in the
published evidence TSVs for anyone auditing the selection, and they stay useful
to US as a development signal; in a changelog they are noise, and listing a
"microsoft-ml" verdict next to a binary nobody can download invites exactly the
wrong conclusion.
Also removes a claim that had become FALSE: the section asserted that every scan
had a minimum of 50 decisive engines. That floor was removed deliberately (it is
VirusTotal fleet availability, not a property of our binary) and the last release
observed 31. Published notes would have stated something untrue. The observed
range is still reported, now over the shipped binaries.
The disclosure of a tolerated Microsoft `!ml` is preserved and narrowed to the
binaries that ship. The contract test now pins both directions: a flagged binary
that SHIPS must be disclosed, and flagged candidates that were rejected must not
appear at all. Its fixture gained a target that draws the tolerated verdict on
all three candidates, so the disclosure branch is actually exercised — previously
every flagged candidate was rejected in favour of a clean sibling, leaving that
path untested.
README and SECURITY.md updated: they promised notes "link every candidate result".
All five release/VT contract tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The v0.10.4 release (run 31755436916) failed at the very last step of `verify`,
after all 69 other jobs had passed — full test matrix, all eight builds, smoke
and soak:
publish-vt-evidence: wrong evidence marker in binaries/vt-results.tsv
The results format was bumped to `cbm-virustotal-results-v2` in #1596, but
publish-vt-evidence.sh still demanded `-v1`. It was the only straggler: the
gate, the selector, the notes renderer and all three contract fixtures were
already on v2.
It survived because that PR had removed the script's only caller, leaving it
dead code that nothing exercised. Restoring the full-surface scan restored the
caller too, and the stale expectation surfaced in the worst possible place — at
the end of a real release rather than in a dry-run, since `verify` does not run
in dry-runs.
Also adds the guard that would have caught it: every marker publish-vt-evidence.sh
validates must be one the gate actually writes. Revert-checked — reintroducing
v1 fails with "publisher expects marker never written by the gate:
cbm-virustotal-results-v1".
The scan itself passed. This is purely the evidence-publishing step.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two things, both found by the dry-run and by sweeping what it exposed.
1. The three-candidate change was incomplete. The dry-run failed at
stage-release-candidates.py, which carries its OWN copy of the transform
validation that the previous commit only fixed in the selector:
stage-release-candidates: candidate linux-amd64/debug-stripped has
invalid transform: 'strip-debug'
Sweeping for that assumption found it in six places, not two:
stage-release-candidates.py, select-release-candidates.py,
verify-release-selection.py, append-vt-notes.sh and two contract-test
fixtures — as hardcoded 16s, `len(TARGETS) * 2`, two-entry VARIANTS tuples
and two-key truth tables. All are now derived from len(VARIANTS).
Field prefixes needed care: the variant NAME keeps its hyphen because it is
the on-disk directory, while the evidence columns use underscores, so
`debug-stripped` reads `debug_stripped_sha256`. Every lookup now goes through
an explicit FIELD_KEY map instead of interpolating the variant directly.
The selection contract test now covers the truth table EXHAUSTIVELY: three
variants x two tolerated classifications is exactly eight combinations, and
there are exactly eight targets, so every case is exercised once.
2. Full-surface VirusTotal scanning is restored. This PR had moved scanning
upstream to the candidates and deleted the post-package pass, which silently
narrowed coverage from everything we ship to executables only. The 42 runtime
files across the 14 containers — install.sh, install.ps1, LICENSE,
THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the unpacked UI assets —
were still extracted, structurally verified and strings-audited, but no
longer scanned at all. install.sh and install.ps1 are the highest-consequence
non-executable bytes we publish; users pipe them straight into a shell.
The verify job scans every extracted object again, under the same policy.
Re-submitting the selected executables alongside them is close to free
because VirusTotal is content-addressed and answers for identical bytes from
its own record — the same property that made the analysis-id equality check
untenable two commits ago.
The gate-chain contract asserted the opposite ("duplicate post-package
VirusTotal path remains"). That assertion is inverted: the pass is required,
and it is not a duplicate, since it covers a strictly larger set. README and
SECURITY.md updated from "archive containers are checksummed rather than
redundantly rescanned" to state the full covered surface.
All five release/VT contract tests pass.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Release evidence from run 31744302624 shows the tolerated Microsoft `!ml`
verdict is close to a coin flip per byte image rather than a property of the
code. Across the eight targets the stripped and unstripped candidates of the
SAME linker output disagreed on four, and in both directions:
linux-amd64 stripped microsoft-ml unstripped clean
darwin-arm64 stripped clean unstripped microsoft-ml
linux-arm64 stripped clean unstripped microsoft-ml
If the classifier were keying on something intrinsic to our code the siblings
would agree; they do not. So each variant is close to an independent draw, and
5 of 16 candidates drew the flag.
Two draws is not always enough. On that run linux-amd64-portable came back
microsoft-ml on BOTH candidates, leaving no clean binary to ship for that
target. A third independent draw at a ~31% observed per-candidate hit rate takes
the both/all-flagged case from roughly 1-in-10 per target to roughly 1-in-30.
The third candidate is `--strip-debug` (Apple: `-S`): debug information removed,
symbol table kept. Behaviourally identical to the other two — same linker
output, only metadata differs — but a distinct byte image, which is all
VirusTotal needs to scan it as its own file. Verified on the real v0.10.4
candidates: linux-amd64 gives three distinct hashes (294,634,656 /
294,623,208 / 293,746,096 bytes) and darwin-arm64 likewise, with the ad-hoc
signature verifying after strip.
Selection is unchanged in spirit and now ordered: smallest artifact first
(stripped, debug-stripped, unstripped), take the first CLEAN one, and only if
every candidate drew the tolerated verdict ship the smallest flagged one. A
hard verdict on any candidate still blocks the release before selection.
Cost is 24 objects per release instead of 16.
Derivation enforces that all three hashes differ — identical candidates would be
one draw wearing three hats, and the selector would believe it had alternatives
it does not have. Public claims in README, SECURITY.md and docs/index.html
updated from "both stripped and unstripped" to the three candidates.
All three release contract tests pass, including the native derivation test
which exercises the real strip and codesign path on this host.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The build+VT dry-run (run 31744302624) blocked all eight targets on:
BLOCKED: objects/3eb486a4... completed with only 48/75 decisive engines (< 50)
That object is the windows-arm64 STRIPPED candidate with ZERO detections. It
was classified "hard" — the same bucket as a real malware verdict — purely
because 48 engines answered instead of 50. Its own sibling scanned clean with
66.
How many engines return a decisive result is VirusTotal fleet availability on
the day, not a property of our binary. These artifacts are ~300 MB and many
engines skip or time out at that size, so the count sits near the floor and
moves run to run: on this run one object landed at 48 and the other fifteen
spread 59-68. A floor there makes shipping a lottery decided by someone else's
infrastructure, which is exactly what our CI-determinism rule forbids, and it is
not something we can influence by fixing anything on our side.
Classification now depends only on what engines FOUND:
hard - two or more engines, any non-Microsoft engine, any label that is
not `!ml`, or anything suspicious. Still blocks the release.
microsoft-ml- exactly one Microsoft `!ml`. Tolerated and disclosed.
clean - nothing flagged.
The decisive-engine count is still recorded per candidate in the results
manifest and printed as a NOTE when it is below the reference value, so the
evidence is unchanged; only its power to block is gone. The selector keeps
rejecting an incoherent response (total < completed).
Public claims updated to match in README.md, SECURITY.md and docs/index.html —
all three promised "at least 50 decisive engines" as a publication requirement,
which would no longer have been true.
Contract test updated: a clean verdict now stands regardless of engine count,
and the below-reference NOTE must still be reported. All three release contract
tests pass (vt-gate-policy, release-gate-chain, candidate-derivation).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The previous commit framed accepting VirusTotal's own analysis id as giving up
freshness. That was the wrong way round.
VirusTotal recognising our bytes is desirable: the evidence this pipeline
publishes is the hash-keyed file report — append-vt-notes.sh asserts the URL is
exactly .../gui/file/<sha256>/detection — so the report we gate on is the same
one a reader gets by looking that SHA-256 up themselves. Requiring a freshly
minted analysis id would have contradicted the evidence we publish next to it,
and made our verdict unreproducible by anyone outside the release job.
Comment-only; no behaviour change. The build+VT dry-run on 094929fd exercises
identical logic.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The dry-run for this branch (run 31729540899) failed in `select-package`, the
new job, after every build had completed and before smoke — the exact shape of
failure that would have taken a release down:
BLOCKED: VirusTotal response analysis id does not match the submitted
analysis: objects/70fbbc37...
That object is the linux-arm64 UNSTRIPPED candidate, 295,476,856 bytes.
VirusTotal is content-addressed, and for bytes it already holds it may answer
with its own canonical analysis rather than the one this upload created. A large
release binary is exactly the artifact most likely to be already on file, so
requiring the ids to be equal would have recurred on most releases rather than
being a one-off.
The equality check was guarding the right idea at the wrong level. What has to
hold is that a verdict describes THESE bytes, and parse_completed already
enforces that a few lines below against meta.file_info.sha256 and
file_info.size — a strictly stronger binding than the id. It is also what keeps
a tuple's two candidates apart: stripped and unstripped differ in hash by
construction (70fbbc37... vs 28b3890f... on linux-arm64), so neither can be read
as the other whatever ids VirusTotal hands out. The wrong-hash and wrong-size
contract cases pin that and still block.
The property deliberately given up is freshness: an older cached analysis of
identical bytes is now accepted. For identical bytes that is the same content
risk, and it buys a gate that is deterministic instead of one that fails
whenever VirusTotal recognises our own binary. The submitted id is still
recorded in the results manifest as evidence.
The selection policy is unchanged and needs no change: stripped clean ships
stripped; stripped microsoft-ml with a clean unstripped ships the clean one;
both microsoft-ml ships stripped; a hard verdict on either sibling still blocks
the release for analysis.
Contract test updated to assert the passing direction, and revert-checked: with
the fix reverted it fails with the production message, with it applied it passes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
On 0.9.0 -> 0.10.2, `update -y` auto-confirmed deleting other projects' indexes,
removed two .db files (~59 MB), and THEN aborted on an interactive prompt it
could not answer in a non-interactive shell. The run failed; the data was
already gone. Reported by @PsyTech-Rob.
The current code is already correct: both activation flows prompt early but
delete late, after the binary swap and agent configuration, and both return
before the deletion when configuration fails. #1538 also removed the specific
prompt that aborted. So this commit fixes nothing — it stops the fix from being
undone, which is the part that was missing. The reporter made exactly this
point: removing one bad prompt addressed the instance, not the rule.
Contract: in each flow that deletes indexes, cbm_remove_indexes() must appear
after the agent-configuration call. It keys on the CALL rather than a local
result variable, because the two flows spell that result differently, and it
fails loudly if a flow is renamed rather than passing vacuously.
Verified by reverting: moving the deletion ahead of configuration reproduces
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
#1558's shape and the contract goes red.
The v0.10.3 release failed its `verify` gate:
--- Auditing: scan-...--manifest.json ---
BLOCKED: Unauthorized URL in binary: https://github.com/DeusData
BLOCKED: Unauthorized URL in binary: https://deusdata.github.io/...
=== BINARY STRING AUDIT FAILED ===
The blocked URLs are OURS — the .mcpb bundle manifest's own homepage and
documentation fields.
security-strings.sh already knows that URL and dangerous-command auditing are
"designed for compiled artifacts" and skips them for text, keeping the
credential and base64 audits that are universally meaningful. Its detection
just had no case for JSON: `file` reports "JSON data", which matched none of
the shell-script/ASCII-text patterns, so a manifest was audited as if it were a
compiled binary. An MCP manifest is exactly the shape that misfires — it
carries project URLs by design, and a `command` field by specification.
This is the first release to ship .mcpb bundles, which is why it surfaced now:
it is the first time a manifest reached the scanned object set. The bundles
were noted as end-to-end unproven until a stable release; this is that proof
doing its job.
Two changes:
- Classify structured-text artifacts by extension FIRST (.json/.yaml/.toml/
.md/...), then by `file` type with JSON/XML added. The extension check also
covers hosts without file(1), where the existing `command -v` guard silently
left every artifact classified as binary.
- Allow-list our own org root and documentation site, so a compiled binary
embedding the docs link (the graph UI does) passes on its own merits rather
than relying on the text exemption.
Tests: the manifest case, plus a control proving the exemption is narrow — the
credential audit still runs on structured text. The existing negative control
(an unauthorized URL in a real binary is still BLOCKED) is unchanged and still
passes, so the binary audit is not weakened.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The local Windows leg caught this within hours of merging #1532: on a machine
that is not domain-joined, USERDOMAIN is the literal string WORKGROUP, and
`WORKGROUP\test` is not a resolvable principal —
WARN: build-dir DACL stamp (pre-wave) failed (user=WORKGROUP\test ...):
WORKGROUP\test: No mapping between account names and security IDs was done.
so the grant fails outright and the tree keeps its inherited
`Authenticated Users:(M)`. The guard then correctly reports that cross-account
mutation is still possible, and the whole leg stops. Since most Windows
machines are not domain-joined, that is the common case, not the exotic one.
#1532 was fixing a real bug in the other direction (a bare name resolves
against the machine first, so a host named like its user grants to an empty
principal). Both failures are name RESOLUTION failures, so this stops resolving
names: the account is identified by SID, which is unambiguous everywhere. The
SYSTEM and Administrators grants in these same commands have always used that
form — the current user was the odd one out. Name lookup survives only as a
fallback where PowerShell is unavailable.
Applied to all three scripts carrying the stamp (run-tests-parallel, soak-test,
memlab), since they share the failure and would otherwise drift apart.
Co-Authored-By: Kiborgik <Kiborgik@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
core.filemode=false hid the +x here, and release.yml invokes the
registry-entries generator directly — a 644 mode would fail the publish
job the first time it runs.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Every release now carries .mcpb one-click-install bundles alongside the
archives, and the MCP Registry entry lists them with per-file sha256:
- package-release.sh (canonical) builds codebase-memory-mcp-<target>.mcpb
for darwin/windows and the STATIC linux builds — manifest.json + the same
staged (stripped, gated) binary + LICENSE + THIRD_PARTY_NOTICES.md. The
glibc-dynamic linux targets stay archive-only: a dynamic binary defeats
the one-click promise.
- _build.yml / release-draft: bundles flow through provenance attestation,
checksums.txt, cosign signing and the release asset list; checksums.txt
is also preserved as a same-run artifact for the registry job.
- verify: the canonical scan matrix grows to 14 containers; MCPB manifests
are validated (parse, binary server, entry_point member, command binds
the entry point). Bundle binaries dedupe to the archive scan objects, so
the VT gate gains only the three distinct manifest.json files.
- publish-mcp-registry: gen-mcpb-registry-entries.sh appends one mcpb
package entry per bundle (release-asset URL + fileSha256 from the
attested checksums) to server.json before mcp-publisher runs.
Idempotent; a checksums file without bundles is a hard failure.
- contracts: Step 0o pins the bundle shape at its producer on every leg,
Step 0p pins the registry entries against the live server.json, and the
extractor contract covers the 14-container matrix incl. broken-manifest
fail-closed cases. The linux test image gains zip for the packager.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing:
gates that were right in principle refused real, ordinary setups, and then
failed to say why. Per the consolidated strictness decision, each gate keeps the
protection that matters and drops the part that was refusing legitimate users —
and every refusal now names what it refused and how to proceed.
**Daemon image gate: npx and every ephemeral install path (#1539, #1383).**
The admission check treated "the peer's image hashes differently" and "the
peer's image cannot be examined at all" as one failure. The second is what
`npx codebase-memory-mcp` always produces (ephemeral cache path,
unfingerprintable), so every npx-invoked client was rejected — and, because the
client never reported it, agents saw a transport that closed mid-handshake with
zero bytes on stdout. Reported by @wassolles with the admission path already
read and the fix space mapped.
An unverifiable image is now admitted: the rendezvous HELLO immediately above it
has already proven semantic version, build fingerprint, and protocol/store/
feature ABI, and the image check was trading that real proof for an unavailable
one. It logs daemon.client_image_unverifiable_admitted so the weaker check is
never invisible. A fingerprint MISMATCH — the tamper case the gate exists for —
still rejects hard. Separate test seams keep the two modes testable apart.
**Client bootstrap failures are no longer silent (#1539).**
An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout
naming the reason, plus the same text on stderr. Previously the reason sat in
bootstrap_result.message and the process exited having written nothing at all.
**POSIX activation: group-writable ancestors (#1535, discussion #1526).**
activation_directory_secure required no group or other write bit on the install
directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several
distro skeletons and any site using a shared primary group, so install.sh failed
for a large fraction of Linux users — reporting a policy refusal as "activation
transaction I/O failed", which sent reporters after disk errors and filesystem
types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and
@iandol confirmed independently.
World-writable ancestors are still refused (any local user could swap a path
component mid-transaction). Group-writable ancestors are now warned about and
admitted. The LEAF directory stays strictly owner-private — that is where the
binary is published, and group write there would let another account replace the
executable between validation and exec. Refusals now name the directory, its
mode, and which rule refused.
**The obsolete ui/standard chooser (#1538, from discussion #1526).**
v0.10.0 consolidated to one archive per platform with the UI always embedded,
but `update` still offered a variant choice: "ui" could only 404, and "standard"
quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The
chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along
with the CBM_VARIANT=ui remnant in the npm installer.
Already-released 0.9.x binaries cannot be fixed retroactively, so the release
workflow now publishes byte-identical ui-*-named alias assets — their updaters
work again with no user action. The aliases are uploaded AFTER the VirusTotal
gate: they are the same bytes as archives it already cleared, and uploading them
earlier would duplicate every object in the scan set and the provenance manifest.
**macOS install noise and attribution (#1537).**
install.sh silenced the "No such xattr: com.apple.quarantine" line, which is
what happens when a curl-downloaded archive carries no quarantine attribute —
harmless, and it became the title of a bug report about an unrelated failure.
The session-stop refusal now points at `daemon status` to list the client
processes actually holding the daemon, instead of asserting sessions exist and
leaving the reader to guess. Reported by @listepo.
**Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh
inside `python -m build` is what emitted Metadata-Version 2.5 and broke the
v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x.
Tests: separate seams for unverifiable vs mismatched peer images with a test per
outcome; activation refusal must name directory + mode + rule; a group-writable
ancestor must stage successfully. The update tests drop the flag that no longer
exists. Verified against each reporter's environment shape.
**Open security alerts (all three, OSSF Scorecard).**
- HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at
pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe
sibling are gitignored so `go build` in that directory cannot repeat it.
- HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom
generator is called with size 0. It reaches us transitively (postcss -> vite),
so it is pinned through the existing graph-ui overrides block rather than
promoted to a direct dependency; the lockfile resolves 3.3.18.
- MEDIUM, unpinned pip command: the publish step installed build/twine by
version only, leaving the whole transitive graph resolved at run time.
pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316
hashes), generated on a linux/amd64 python:3.12 image so the wheels match what
ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified
by installing from it in that same image.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The Windows DACL stamps in run-tests-parallel.sh, soak-test.sh and
memlab.sh pass the current user to icacls by bare name. Git Bash resolves
whoami to coreutils, which prints an unqualified name, and icacls resolves
an unqualified name against the machine before the user. On a host whose
name equals the user's (COMPUTERNAME=BUILD with user build) the grant
lands on an empty principal:
BUILD\:(OI)(CI)(F)
Combined with the /inheritance:r on the same invocation, the directory
ends up granting no usable account. run-tests-parallel.sh then cannot
write its own log directory and the wave dies with:
build/c/test-logs/results.txt: Permission denied
FAIL: test-runner --list-suites exited nonzero
Prefix the name with USERDOMAIN when it is set, which makes the account
unambiguous, and fall back to the bare name otherwise. SYSTEM and
Administrators in the same invocations already use SID form and are
unaffected.
Signed-off-by: PETRO YAKOVYSHYN <petroyakovyshyn@gmail.com>
Three regressions shipped in 0.10.0 share one failure shape: an empty result
with a success status, indistinguishable from "nothing found" for the LLM
clients that are cbm's primary consumers.
1) structuredContent {} on the whole tree-format surface (#1522 bug 1).
#1488 replaced the duplicated payload with an EMPTY structuredContent
object while every tool still declared a blanket permissive outputSchema.
Spec-honoring clients (Claude Code among them) treat structuredContent as
THE result when a schema is declared, so search_graph, trace_path,
query_graph, get_architecture, search_code, and detect_changes all rendered
as literally "{}" on their DEFAULT format, on every platform. The corrected
contract: no tool declares an outputSchema (tool output is
format-parameter-polymorphic — no static schema is truthful), JSON-object
payloads keep their parsed structuredContent, errors keep
structuredContent.error, and text-shaped payloads carry NO structuredContent
key at all — which also preserves #1375's no-duplication win.
2) Frontend queue overflow killed the session (found by the #1522 sweep).
Any 7+ requests pipelined in one stdin burst — an agent issuing parallel
tool calls does exactly this — overflowed the 8-frame frontend queue, which
failed the whole session: rc=1 with ZERO bytes of output, every buffered
response lost. A full queue is now backpressure: the stdin reader blocks
until the worker drains (bounded by the same stop/fail flags every teardown
path already sets); only a single frame larger than the entire 12 MiB byte
budget — which could never be admitted — remains a hard failure.
3) config get printed "" with exit 0 for every unset and every unknown key
(#1522 bug 2). list printed stored-or-DEFAULT while get printed
stored-or-EMPTY, and no subcommand validated key names, so a typo was
indistinguishable from a correctly-read setting. One config-key table now
drives help, list, get, set, and reset: get prints the stored value or the
key's real default (the same fallback the runtime readers use), and unknown
keys error with exit 1 on get, set, and reset alike.
Tests — each RED on the pre-fix tree and RED again on revert:
* test_mcp.c: text results carry no structuredContent key; tools/list
declares no outputSchema; the tool-table guard now binds all three
branches (absent / parsed-object / error) for every registered tool.
* test_daemon_frontend.c: the over-capacity contract flips from
"session fails" to "backpressure without loss" — the held first request
plus all 32 over-capacity frames are answered and the run closes cleanly.
* test_cli.c: the config command contract — defaults, round-trip, reset,
and unknown-key rejection on all three subcommands.
* smoke-test.sh: Phase 3z rewritten to the corrected structuredContent
contract, new Phase 3z1 (default-format replies usable in schema-honoring
clients, no outputSchema advertised), 3z2 (24 pipelined calls all
answered), 3z3 (config defaults + unknown-key rejection) — all asserted
against the SHIPPED artifact, where #1488's smoke phase previously pinned
the empty-object behavior as correct.
Verified end-to-end on the locally built production binary: all 15 tools
declare no schema; text tools return ABSENT structuredContent, object tools
populated, error envelopes intact; 7/24/64-deep pipelined bursts all answered
with rc=0; config get prints real defaults and exits 1 on unknown keys.
hook_augment (structuredContent.projects) and index_resilience
(structuredContent.status) consume object payloads and are unaffected.
Fixes#1522.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The layer varies argument values but always sends the expected type. These
nine send a value of the wrong JSON type for the request envelope fields
parsed before tool dispatch: tool name, arguments, params and method.
All nine pass against the current binary, so this is a regression guard for
a class the layer does not currently reach, not a bug report.
Signed-off-by: Mohak Gupta <mohakgupta0981@gmail.com>
Deleting this contract with the UI-pack architecture threw away coverage of
machinery that SURVIVED: the extractor still builds the scan-set and association
manifests that check-virustotal.sh reads. Without it the gate could report a
confident green over an incomplete set.
It immediately paid for itself. The extractor was BROKEN by my own edit:
NameError: name 'variant' is not defined (process_tar)
TypeError: add_association() missing 'variant'
Removing the variant from validate_namespace's return left the call sites still
threading it. python3 -m py_compile passes on that -- it is a runtime failure --
and the extractor runs ONLY in the release and dry-run VirusTotal jobs, so
nothing else in the tree or on the 3-OS ladder executes it. It would have
shipped and broken every release verification.
The contract is rewritten rather than adapted: the original was woven through
the pack format (16 archives, 88 associations, per-pack asset math), and a
half-adapted version that passes for the wrong reasons is worse than none. It
now asserts what still exists:
- the exact 8-archive matrix, 32 member associations, nothing dropped
- archive containers never enter the scan set as members
- deduplication is exact: 8 byte-identical LICENSE members collapse to ONE
scan object while 8 distinct binaries stay 8
- manifest metadata agrees with its own rows, and every staged object matches
its recorded SHA-256
- fail-closed on a surplus member, a missing member, and a short matrix
Registered as Step 0l so it runs in every venue, not just when someone
remembers.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Making REQUIRE_UI unconditional broke all three pr-smoke legs:
FAIL 15a: SMOKE_REQUIRE_UI=1 but this binary serves no embedded UI assets
The PR lane runs scripts/build.sh WITHOUT --with-ui on purpose, so an npm
frontend build does not land on every product PR. Demanding embedded assets
there asserts a property that lane deliberately does not produce.
The guard still exists where it means something: scripts/ci/smoke-artifact.sh
builds --with-ui, packages the real archive and smokes the EXTRACTED result, so
it exports SMOKE_REQUIRE_UI=1 and a frontend-less binary fails there. That is
the lane whose whole job is release fidelity.
Deliberately NOT fixed by adding --with-ui to pr.yml: that would put `npm ci &&
npm run build` on three runners for every product PR to re-prove something the
dry-run and release lanes already gate. PR CI cost is not free and this branch
should not quietly raise it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The arm64 W^X hole regressed silently once and section flags cannot catch it:
.rodata is correctly marked A (not AX), but the kernel applies SEGMENT
permissions, so a merged R E PT_LOAD maps it executable anyway. A Makefile flag
alone would regress the same way the next time a linker default differs.
A1b sums the executable PT_LOAD segments and fails when they cover most of the
file. Measured on aarch64 with a 1.2 MB const table:
without -z separate-code FAIL executable segments cover 96% of the file
with -z separate-code PASS executable segments cover 0%
Nearly shipped this vacuous: the first version summed with awk's strtonum(),
which is a gawk extension. CI's awk is mawk, where it is undefined -- the total
would have been 0 and the gate would have passed on exactly the artifacts it
exists to catch. It now emits the hex MemSiz fields and converts in the shell,
and the numbers above were produced with the container's mawk, not gawk.
Mach-O and PE report n/a; the real macOS artifact still passes all 13 assertions.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/embed-frontend.sh generates src/ui/embedded_assets.c into the SOURCE
tree (the linker needs it beside the other UI sources, and scripts/clean.sh
removes it). It was never gitignored, so a --with-ui build leaves a large
untracked generated file sitting in `git status` waiting to be committed by
accident. It predates this branch, but this change makes that build path the
default one, so close it here.
smoke-local.sh still documented a `ui` argument that no longer exists.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Caught by running the contracts rather than by inspection:
- package-release.sh built its archives from an ARCHIVE_MEMBERS array, which
only ever existed to append a UI pack conditionally. With a fixed four-file
inventory the array hides the member list from the very contract that is
supposed to pin it, so the members go back inline on the archive call.
- test_windows_bundle_contract.sh asserted the pypi wrapper exposes
_windows_binary_ready(). That function became _runtime_set_ready() when the
wrapper locks were reworked; the contract now asserts the current API, since
the property it guards is "resolves the single Windows binary", not a name.
- test_smoke_fixture_contract.sh required smoke-local.sh to emit both a
variant-suffixed archive and a standard alias. There is one archive name now,
so it asserts that instead -- and that ${SUFFIX} is gone entirely, so the
alias machinery cannot quietly return.
Verified on the built artifact: the binary runs, carries 9 shebangs, 1
node:child_process and 14 hook-augment references (the deliberately inverted
property), and contains zero references to cbm-integrations.json, CBMUIPK or
--verify-runtime-assets.
Contract status on macOS: build_dir_safety, makefile_ts_runtime_dependencies,
parallel_harness, security_fuzz_harness, security_strings_allowlist,
soak_daemon_recovery, spawn_no_window, ui_dev_proxy_security, venue_parity,
vm_worktree_manifest, windows_bundle, smoke_fixture, vt_gate_policy and
vt_release_notes all pass. hook_conflict_notice, parent_watchdog and
worker_error_response fail for the documented environmental reasons (TEST_SEAMS
build seam, and the developer's own installed daemon holding the version
cohort), not from this change.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Completes the collapse to a single shipped composition and replaces the
zero-tolerance VirusTotal gate with a narrow, disclosed policy.
Packaging and installers
- package-release.sh loses --variant; archives are codebase-memory-mcp-<os>-<arch>
with exactly four members. install.sh/install.ps1 lose --ui/--standard.
- The extractor drops CBMUIPK pack parsing and --archive-scope; its scan-set and
association manifests (which the gate depends on) are unchanged otherwise.
- npm/PyPI/Go wrappers: the runtime "set" is one file again. The Windows lock
and race fixes from #1495/#1496 are kept; only multi-file set membership goes.
This also fixes `pip install` on Windows, which rejected the fifth archive
member against a hardcoded four-name allowlist.
- The wrappers' post-download probe moves from --verify-runtime-assets (removed)
to --version, which proves the same thing: the binary executes.
VirusTotal gate
- Exactly ONE detection is tolerated, and only when the engine is Microsoft AND
the label ends in `!ml`. Two or more engines, any non-`!ml` label, any other
vendor, any suspicious verdict and every infrastructure error still block.
- A tolerated object prints TOLERATED:, never OK:, and its counts are recorded
in vt-results.tsv exactly as a blocked one would be.
- append-vt-notes.sh mirrors the policy. It previously hard-failed on any
malicious count, so loosening only the gate would have passed the scan and
then died at note publication. The notes now DISCLOSE a tolerated detection
and link to SECURITY.md rather than claiming "0 malicious" for everything.
Rationale for the tolerance is in the gate itself: the verdict is not a property
of our bytes. It inverts across architectures and link modes, moves between
sibling artifacts of one build, and lands in different variant buckets for the
same source. The same `!ml` family hits llama.cpp, GitHub's own `gh`, Microsoft's
own Go toolchain and Anthropic's Claude installer.
The zero-tolerance contract becomes test_vt_gate_policy_contract.sh, asserting
the full matrix: 1x Microsoft !ml passes and reports TOLERATED; a Microsoft
signature label, a non-Microsoft engine, two engines, a suspicious verdict and
every malformed-response case still block. Its tripwire is narrowed to the
reverted endpoint-verification mechanism rather than the words "false positive",
so it no longer fires on a deliberate in-gate policy branch.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.
Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.
So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.
Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.
Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.
Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
- cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
through the ANSI code page and mangles non-ASCII install paths.
- the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
`daemon start --open` still waits for a genuine CBM listener.
- X-Content-Type-Options: nosniff on served assets.
- the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.
Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Round 2's smoke fix added the archive member list 'cbm-integrations.json LICENSE
install.sh ...' to scripts/smoke-local.sh (mirroring package-release.sh), which
reintroduced the exact ScanCode false positive that commit 83df500 quieted for
package-release.sh: the '.json LICENSE' token adjacency reads as the SPDX 'JSON'
license. Same class, same fix — smoke-local.sh is a first-party MIT build
script listing its own archive members, added to the license policy's
ignored_paths with the shared justification.
Swept every ScanCode-scanned file (src/pkg/scripts, the gate's own file set) for
the adjacency: the only three carrying it — package-release.sh, smoke-local.sh,
and license-policy.json itself — are all now in ignored_paths, so this closes it
rather than trading one flagged file for another.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
pr-smoke failed on all three platforms: the binary's install/uninstall render
templates from the integration asset, and every place that stages the binary
without the asset made those operations fail closed with "integration assets
missing". The asset must travel with the binary in EVERY layout, not just the
release archive. Five staging paths were missing it:
- scripts/build.sh — stage cbm-integrations.json next to the freshly built
binary, so `build/c/codebase-memory-mcp install` works straight out of a
build tree (dev, and the base for the smoke fixture).
- scripts/smoke-local.sh — include the asset in the fixture tarball and its
required-sidecar check. Member set and ORDER now mirror package-release.sh
exactly (binary, cbm-integrations.json, LICENSE, install.sh, notices); the
fixture was smoking an archive layout we never actually ship.
- install.sh / install.ps1 — after installing the binary, copy the asset beside
it in the install dir. `install` already publishes a verified copy to
~/.cbm/assets/<version>/, but a later install/uninstall run from the install
dir resolves the asset NEXT TO THE BINARY first, so without the adjacent copy
that lookup misses and a re-install or uninstall fails on a machine that just
installed successfully. Best-effort atomic rename, same shape as the existing
install.sh/ps1 sidecar copy. install.ps1 stays pure ASCII.
- scripts/smoke-test.sh Phase 14 — this phase hand-stages the binary into a
fresh HOME without going through install, so nothing populates ~/.cbm/assets;
stage the asset next to each staged copy so the uninstall it drives resolves.
Verified: scripts/smoke-local.sh on the standard binary now runs clean through
all 16 phases — zero "integration assets missing" / hook_script_uninstall /
agent-cleanup-failed lines (was 5). windows-bundle, smoke-fixture, exec-bit and
no-embedded-scripts contracts pass; install.ps1 has zero non-ASCII bytes.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Two CI failures on the integration-asset change, both build/gate plumbing, not
the architecture:
1. scripts/gen-integrations-hash.sh was committed 100644 while Makefile.cbm
invoked it directly, so a fresh CI checkout died with "Permission denied"
building the hash header — which failed EVERY test leg at step 0, since the
contract step builds first. It passed locally only because the generated
header was already cached, so make never re-ran the generator. Fixed both
ways: the recipe now runs it via `sh` (mode-independent, cannot regress from
a checkout mode), and the file is committed 100755 to match its siblings.
Note: test_script_exec_bit_contract.sh scans shell call sites, not Makefile
recipes, so it did not catch this — the `sh` prefix is the durable guard.
2. The license gate (ScanCode) flagged scripts/package-release.sh with the SPDX
'JSON' license. The archive member lists place 'cbm-integrations.json' and
'LICENSE' adjacently, and ScanCode reads the '.json LICENSE' token adjacency
as a JSON-license reference. The order is not free to change — the Windows
single-binary contract locks the exact member sequence — so this is a genuine
false positive on a first-party MIT build script. Added it to the policy's
ignored_paths with the same justification the six existing first-party
entries carry (the gate scripts, provenance auditor, and discover.c all name
licenses for legitimate reasons). No allow-listed SPDX id was added; the
JSON license is NOT now permitted anywhere else.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The product binary embedded nine complete shebang'd shell scripts, their
PowerShell/.cmd twins, and two node:child_process client-adapter modules as C
string literals, and wrote them into agent config dirs at 0755 on install. A
block of script-shaped bytes inside a large unsigned executable is exactly the
surface Microsoft Defender's ML scored Trojan:Script/Wacatac.B!ml — the
detection that has recurred across the last several release attempts, on the
Linux ELF and darwin-arm64 standard binaries while every UI and Windows variant
stayed clean.
This moves every integration template out of the binary into a single shipped
data file, assets/cbm-integrations.json (compact: one asset, because each
shipped file is itself scanned). The binary now carries only the templates'
identity — one embedded SHA-256, generated at build time from the JSON so it
can never drift — plus the code to load, verify, and render them.
Why a hash and not just a loose file: content compiled into the binary was
tamper-evident with the binary. Moving it to disk must not turn it into
unauthenticated code the installer blindly executes. install verifies the asset
against the embedded SHA-256 and fails closed on mismatch ("integration assets
missing or modified - reinstall from the release archive") before writing
anything. Verified end to end: a one-byte edit to the asset makes install
refuse.
Ownership, for safe upgrade/removal: the verified copy is published to
~/.cbm/assets/<version>/ — a sibling of the DB cache, never inside it, so
clearing the cache cannot strip a user's hooks. A deployed integration file is
ours to remove only if it matches a template rendered from that stored copy;
foreign files and user-edited files at the reserved paths are preserved. The
historical template bodies travel in the JSON's released[] arrays so files
written by older versions are still recognised and cleaned up on uninstall.
Removed from src/: cmm_gate/session/subagent script-prefix and suffix
constants, cbm_build_released_gate_script, the shell/PowerShell/.cmd bodies in
cli.c, and the JS/TS generators in client_adapter.c. Confirmed on the built
standard binary: 0 occurrences of '#!/usr/bin/env bash', '#!/bin/sh',
'node:child_process', and 'ExecutionPolicy' (was non-zero for each).
test_no_embedded_scripts_contract.sh makes the removal a build property, not a
reachability assumption: no production C source may contain a shebang or
node:child_process literal, and the built binary is scanned directly. It strips
comments first (a contract a comment can satisfy is a false guard), and is
revert-checked — reintroducing one shebang literal makes it fail naming the
file:line. check-binary-composition.sh (A6) enforces the same needles on every
release artifact post-strip.
This does not, by itself, prove the detection is cured: the clean UI binary is a
strict superset of the flagged standard one, so no single passage is a
deterministic trigger. It removes the most coherent malware-shaped composite
from every shipped binary, aligns with how gh/starship/zoxide externalise
integration material, and makes the templates maintainable as data rather than
escaped C literals. Causal confirmation still needs a VirusTotal ablation.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The gate reported "1 malicious, 0 suspicious / 59 engines" and stopped there —
withholding the one fact a detection is actionable on. A false-positive
submission goes to a SPECIFIC vendor, and triage begins with "known-noisy engine
or a new family?". Neither question is answerable from a count.
It cost exactly that in the 2026-08-07 dry run: three artifacts flagged
(linux-amd64, linux-amd64-portable, darwin-arm64), one engine each, and the logs
could not say by what. The vendor had to be recovered by hand from the web UI
afterwards — and the WDSI developer false-positive path is a web form with no
public API, so guessing the wrong vendor wastes a filing round trip.
The analysis response already carries per-engine results; the gate simply threw
them away. It now prints, under the BLOCKED line, each engine that returned
malicious or suspicious with its detection label and definition date:
BLOCKED: codebase-memory-mcp-linux-amd64 flagged (1 malicious, ...)
detected by: Microsoft = Trojan:Win32/Wacatac.B!ml (engine ..., defs ...)
Engines that did not flag are not listed, or the actionable line drowns in
sixty clean rows.
Deliberately unchanged: the gate's strictness. Zero tolerance still means any
detection on any artifact on any version blocks, and an analysis WITHOUT
per-engine results still blocks on the counts alone — the contract pins both, so
naming engines cannot become a route to tolerating one. A missing engine list
prints an explicit "<engine names unavailable>" rather than nothing, so the gap
is visible instead of reading as a detection-free block.
Verified by revert: against the old gate the new assertions fail with the
counts-only output quoted above.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Preparation for the next dry-run cycle. The release gate scans every shipped
artifact with VirusTotal and blocks on any detection; the dry run did not, so
the one question most likely to stop a release was the one a dry run could not
answer. v0.9.1-rc found that out the expensive way.
The scan mirrors the release job. The only difference is where the archives come
from: the release pulls them off a published release, a dry run has none, so
they come from the build job's artifacts — the same files, produced by the same
canonical package-release.sh, extracted the same way and scanned as the same
bytes a user would download and run. Same zero-tolerance gate
(scripts/ci/check-virustotal.sh), so a dry run now fails where the release would.
It runs LAST, after smoke and soak. It is the slowest job — polling can take two
hours — and its verdict does not depend on our code at all: the same unchanged
bytes can come back clean one day and flagged the next. That must not delay the
feedback that IS about our code.
Two independent skips, because they answer different questions:
skip_builds no artifacts at all "do lint and tests pass?"
skip_virustotal build + smoke + soak, no scan "do the archives pass CI?"
The second is the one asked for: during a debug cycle, when the question is
whether the archives survive smoke and soak, a two-hour scan burning API quota
is pure noise. skip_builds implies no scan, since there is nothing to scan.
Extraction lives in scripts/ci/extract-release-archives.sh rather than inline in
the YAML because the venue-parity contract requires it — a venue may provision,
plumb artifacts, or call a canonical leg script, and this is leg logic. The
contract caught the inline version.
Two defects found by testing the script instead of trusting it:
* The empty-input guard did not guard. It counted files in the output
directory, but the install scripts are copied in unconditionally, so the
count was never zero and an input with no archives reported success — a
clean VirusTotal run over nothing, the exact false green the check exists to
prevent. It now counts binaries recovered from archives, and exits 1 on
zero.
* The script was committed 100644 while the workflow invokes it directly as a
command, which is the "Permission denied" failure
test_script_exec_bit_contract.sh was written for. That contract scans
scripts/ but not .github/workflows/, so it did not catch it here.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Martin Vogel