16 Commits

Author SHA1 Message Date
cyphercodes ee6abbef37 fix(installer): accept legacy release archive layout
Signed-off-by: cyphercodes <cyphercodes@users.noreply.github.com>
2026-08-11 18:01:23 +03:00
Martin Vogel b377c62a4e Merge pull request #1543 from DeusData/fix/batch1-strictness-and-installers
fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters (+3 security alerts)
2026-08-11 15:11:19 +02:00
Martin Vogel 8eabe191d2 fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing:
gates that were right in principle refused real, ordinary setups, and then
failed to say why. Per the consolidated strictness decision, each gate keeps the
protection that matters and drops the part that was refusing legitimate users —
and every refusal now names what it refused and how to proceed.

**Daemon image gate: npx and every ephemeral install path (#1539, #1383).**
The admission check treated "the peer's image hashes differently" and "the
peer's image cannot be examined at all" as one failure. The second is what
`npx codebase-memory-mcp` always produces (ephemeral cache path,
unfingerprintable), so every npx-invoked client was rejected — and, because the
client never reported it, agents saw a transport that closed mid-handshake with
zero bytes on stdout. Reported by @wassolles with the admission path already
read and the fix space mapped.

An unverifiable image is now admitted: the rendezvous HELLO immediately above it
has already proven semantic version, build fingerprint, and protocol/store/
feature ABI, and the image check was trading that real proof for an unavailable
one. It logs daemon.client_image_unverifiable_admitted so the weaker check is
never invisible. A fingerprint MISMATCH — the tamper case the gate exists for —
still rejects hard. Separate test seams keep the two modes testable apart.

**Client bootstrap failures are no longer silent (#1539).**
An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout
naming the reason, plus the same text on stderr. Previously the reason sat in
bootstrap_result.message and the process exited having written nothing at all.

**POSIX activation: group-writable ancestors (#1535, discussion #1526).**
activation_directory_secure required no group or other write bit on the install
directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several
distro skeletons and any site using a shared primary group, so install.sh failed
for a large fraction of Linux users — reporting a policy refusal as "activation
transaction I/O failed", which sent reporters after disk errors and filesystem
types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and
@iandol confirmed independently.

World-writable ancestors are still refused (any local user could swap a path
component mid-transaction). Group-writable ancestors are now warned about and
admitted. The LEAF directory stays strictly owner-private — that is where the
binary is published, and group write there would let another account replace the
executable between validation and exec. Refusals now name the directory, its
mode, and which rule refused.

**The obsolete ui/standard chooser (#1538, from discussion #1526).**
v0.10.0 consolidated to one archive per platform with the UI always embedded,
but `update` still offered a variant choice: "ui" could only 404, and "standard"
quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The
chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along
with the CBM_VARIANT=ui remnant in the npm installer.

Already-released 0.9.x binaries cannot be fixed retroactively, so the release
workflow now publishes byte-identical ui-*-named alias assets — their updaters
work again with no user action. The aliases are uploaded AFTER the VirusTotal
gate: they are the same bytes as archives it already cleared, and uploading them
earlier would duplicate every object in the scan set and the provenance manifest.

**macOS install noise and attribution (#1537).**
install.sh silenced the "No such xattr: com.apple.quarantine" line, which is
what happens when a curl-downloaded archive carries no quarantine attribute —
harmless, and it became the title of a bug report about an unrelated failure.
The session-stop refusal now points at `daemon status` to list the client
processes actually holding the daemon, instead of asserting sessions exist and
leaving the reader to guess. Reported by @listepo.

**Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh
inside `python -m build` is what emitted Metadata-Version 2.5 and broke the
v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x.

Tests: separate seams for unverifiable vs mismatched peer images with a test per
outcome; activation refusal must name directory + mode + rule; a group-writable
ancestor must stage successfully. The update tests drop the flag that no longer
exists. Verified against each reporter's environment shape.

**Open security alerts (all three, OSSF Scorecard).**
- HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at
  pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe
  sibling are gitignored so `go build` in that directory cannot repeat it.
- HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom
  generator is called with size 0. It reaches us transitively (postcss -> vite),
  so it is pinned through the existing graph-ui overrides block rather than
  promoted to a direct dependency; the lockfile resolves 3.3.18.
- MEDIUM, unpinned pip command: the publish step installed build/twine by
  version only, leaving the whole transitive graph resolved at run time.
  pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316
  hashes), generated on a linux/amd64 python:3.12 image so the wheels match what
  ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified
  by installing from it in that same image.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-11 13:37:32 +02:00
xz-dev 6184bfe3e0 fix(installer): ignore archive ownership on extraction
Signed-off-by: xz-dev <xiangzhedev@gmail.com>
2026-08-11 13:15:21 +08:00
Martin Vogel d58afe562d revert(release): re-embed runtime assets into the single shipped binary
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.

Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.

So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.

Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.

Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.

Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
  - cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
    through the ANSI code page and mangles non-ASCII install paths.
  - the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
    `daemon start --open` still waits for a genuine CBM listener.
  - X-Content-Type-Options: nosniff on served assets.
  - the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.

Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-09 13:06:42 +02:00
Martin Vogel 8018561cfe fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-08 17:35:05 +02:00
Martin Vogel 365841304c fix(install,smoke): carry cbm-integrations.json wherever the binary goes
pr-smoke failed on all three platforms: the binary's install/uninstall render
templates from the integration asset, and every place that stages the binary
without the asset made those operations fail closed with "integration assets
missing". The asset must travel with the binary in EVERY layout, not just the
release archive. Five staging paths were missing it:

- scripts/build.sh — stage cbm-integrations.json next to the freshly built
  binary, so `build/c/codebase-memory-mcp install` works straight out of a
  build tree (dev, and the base for the smoke fixture).
- scripts/smoke-local.sh — include the asset in the fixture tarball and its
  required-sidecar check. Member set and ORDER now mirror package-release.sh
  exactly (binary, cbm-integrations.json, LICENSE, install.sh, notices); the
  fixture was smoking an archive layout we never actually ship.
- install.sh / install.ps1 — after installing the binary, copy the asset beside
  it in the install dir. `install` already publishes a verified copy to
  ~/.cbm/assets/<version>/, but a later install/uninstall run from the install
  dir resolves the asset NEXT TO THE BINARY first, so without the adjacent copy
  that lookup misses and a re-install or uninstall fails on a machine that just
  installed successfully. Best-effort atomic rename, same shape as the existing
  install.sh/ps1 sidecar copy. install.ps1 stays pure ASCII.
- scripts/smoke-test.sh Phase 14 — this phase hand-stages the binary into a
  fresh HOME without going through install, so nothing populates ~/.cbm/assets;
  stage the asset next to each staged copy so the uninstall it drives resolves.

Verified: scripts/smoke-local.sh on the standard binary now runs clean through
all 16 phases — zero "integration assets missing" / hook_script_uninstall /
agent-cleanup-failed lines (was 5). windows-bundle, smoke-fixture, exec-bit and
no-embedded-scripts contracts pass; install.ps1 has zero non-ASCII bytes.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-08 05:08:25 +02:00
Martin Vogel afd67c51b0 refactor(cli): install scripts own updating; the binary carries no URL
Completes the move started on Windows. `update` prints the install script's
command and exits 0 on every platform; the download/extract/chmod/exec sequence
is compiled out of release builds entirely, and the MCP background thread that
ran cbm_popen("curl ... api.github.com ...") on first tool call is gone.

The installers now place themselves beside the binary, which closes the last
gap: `update` referenced a raw.githubusercontent URL purely because an
install.sh-installed user had no local copy to point at. Two details make that
safe.

The source is the install script from the archive we JUST checksum-verified,
never "$0" -- which does not exist under `curl | bash`, and would pin the user
to the OLD installer forever. Since the script ships inside the verified
archive, it inherits that verification instead of needing its own.

And it is published by atomic rename, never written over the live path. Bash
reads a script incrementally by byte offset, so overwriting the file it is
executing continues reading NEW bytes at the OLD offset: silent, bizarre
corruption. cp to a temp name then mv -f swaps the directory entry while the
running shell keeps its original inode. Windows follows the same rule via
Copy-Item + Move-Item even though PowerShell parses up front.

Deliberately NOT done: spawning the new installer to delete and replace its
predecessor. Fetch remote code -> drop to disk -> spawn -> self-delete is a
textbook stager, and self-deletion is among the most heavily weighted
heuristics there is. It also gains nothing -- `"$DLBIN" install` already runs
the NEW release's install logic, because the binary owns the install.

Net: zero install or download URLs remain in the shipped binary, and the
allow-list entry is retired with it. Both scripts scan CLEAN (0/61) with the
self-placement code in them.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-29 17:04:50 +02:00
Martin Vogel 0e00ef5702 feat: coordinate concurrent CBM sessions
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-16 19:20:46 +02:00
Martin Vogel 69c16cada4 fix: deliver the static -portable linux binary on all install/update paths
The standard linux release binary dynamically links glibc 2.38+ and
GLIBCXX_3.4.32, so it fails to start on Debian 11, Ubuntu 20.04/22.04,
RHEL/Rocky 8/9, Amazon Linux 2, etc. — yet install.sh, the npm and PyPI
wrappers, and the binary's own self-update all fetched it by default,
contradicting the "single static binary" promise.

Point every linux install + self-update path at the fully-static
"-portable" asset (gcc -static), which has no glibc floor. macOS/Windows
are unaffected and unchanged.

- install.sh, pkg/npm/install.js, pkg/pypi _cli.py: select -portable on linux
- src/cli/cli.c: self-update download URL AND checksum archive name both use
  -portable on linux (they must match or the update fails checksum verify)
- scripts/smoke-test.sh: assert linux self-update targets the -portable asset
- scripts/ci/check-glibc-compat.sh: new guard — runs the binary inside
  debian:bullseye (glibc 2.31) and asserts it starts
- _smoke.yml: run the guard on the portable binary in smoke-linux-portable

Reproduced: standard binary -> "GLIBC_2.38 not found" on glibc 2.31;
portable binary runs cleanly.
2026-06-09 11:36:00 +02:00
Martin Vogel c1eb3dd2d1 Comprehensive security hardening across all installers
install.sh:
- Wrap entire body in main() to prevent partial execution from curl|bash
  (interrupted download would execute truncated script)

install.ps1:
- Enforce TLS 1.2+ (older PowerShell defaults to TLS 1.0)

PyPI (_cli.py):
- Add SHA256 checksum verification against checksums.txt (was the only
  installer without checksums)

npm (install.js):
- Add SHA256 checksum verification against checksums.txt
- Validate HTTPS on every redirect hop (max 5 redirects)
- Replace execSync string interpolation with execFileSync array args
  (eliminates shell injection vector in tar/PowerShell calls)
- Add path traversal check on extracted binary

npm (bin.js):
- Auto-download binary if missing (handles --ignore-scripts / pnpm)

Go wrapper (main.go):
- Custom HTTP client with CheckRedirect that rejects non-HTTPS redirects
  (Go's default http.Get follows redirects without scheme validation)
- Fix variable name bug in error message (url → rawURL)
2026-04-14 21:00:42 +02:00
Martin Vogel a89bf96382 Harden installers: remove Unblock-File, add HTTPS-only URL validation
install.ps1:
- Remove both Unblock-File calls (Defender ClickFix.R!ml signal)
- Remove ExecutionPolicy ByPass from usage comment (scanned by Defender)
- Add HTTPS-only URL scheme check on CBM_DOWNLOAD_URL

install.sh:
- Add HTTPS-only URL scheme check on CBM_DOWNLOAD_URL

pkg/go wrapper:
- Add validateURLScheme() to httpGet and fetchChecksums (defense-in-depth,
  matching Python wrapper's _validate_url_scheme from PR #248)

All installers now allow localhost/127.0.0.1 for testing but reject
http://, ftp://, file:// and other schemes.
2026-04-14 20:44:17 +02:00
Martin Vogel 0c35f152a0 Add --skip-config flag to install.sh (#145)
Allows skipping automatic agent configuration after binary install.
Useful for custom setups or CI environments.

Usage: curl ... | bash -s -- --skip-config
2026-03-25 22:24:02 +01:00
Martin Vogel 006e6dbf20 Add install scripts, CI pre-signing, download E2E smoke tests
install.sh: one-liner for macOS/Linux — detects OS/arch (Rosetta-
aware), downloads release, verifies checksum, extracts, signs on
macOS, runs install -y for all 10 agents. Supports --ui flag and
CBM_DOWNLOAD_URL env var for testing.

install.ps1: one-liner for Windows — Invoke-WebRequest + Expand-
Archive + Unblock-File (strips MOTW), installs to %LOCALAPPDATA%,
adds to user PATH via [Environment]::SetEnvironmentVariable.

CI pre-signing: add codesign --sign - step for macOS builds in
both dry-run.yml and release.yml, before archiving. Release
binaries now ship pre-signed.

Phase 12 smoke tests: real HTTP download via local artifact server,
checksum verification, archive extraction, binary verification.
Runs only when SMOKE_DOWNLOAD_URL is set (CI provides it).

Phase 13 smoke tests: install.sh E2E — runs full script with local
URL + isolated HOME, verifies binary placed, signed, runs, and
agent configs created.

CI HTTP server: smoke jobs start python3 HTTP server serving the
built binary as a tar.gz/zip archive + checksums.txt. Enables
Phases 12-13 in CI on all platforms.

Update security allowlist: remove system() entry (eliminated),
add cbm_popen for pgrep. Update README with one-liner Quick Start.
2026-03-24 23:14:33 +01:00
Martin Vogel 93cd5cf0df Revert "Add one-line installer script with macOS signing fix"
This reverts commit 45eb6d412a.
2026-03-24 11:02:58 +01:00
Martin Vogel 45eb6d412a Add one-line installer script with macOS signing fix
install.sh detects OS/arch, downloads latest release, fixes macOS
code signing (removes quarantine/provenance xattrs, re-signs locally),
installs to ~/.local/bin, and configures all detected coding agents.

Supports --ui flag for graph visualization variant and --dir for
custom install path. Handles both curl and wget.

Update README Quick Start with one-liner curl | bash usage.
2026-03-24 00:06:38 +01:00