main
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Martin Vogel
|
8eabe191d2 |
fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing: gates that were right in principle refused real, ordinary setups, and then failed to say why. Per the consolidated strictness decision, each gate keeps the protection that matters and drops the part that was refusing legitimate users — and every refusal now names what it refused and how to proceed. **Daemon image gate: npx and every ephemeral install path (#1539, #1383).** The admission check treated "the peer's image hashes differently" and "the peer's image cannot be examined at all" as one failure. The second is what `npx codebase-memory-mcp` always produces (ephemeral cache path, unfingerprintable), so every npx-invoked client was rejected — and, because the client never reported it, agents saw a transport that closed mid-handshake with zero bytes on stdout. Reported by @wassolles with the admission path already read and the fix space mapped. An unverifiable image is now admitted: the rendezvous HELLO immediately above it has already proven semantic version, build fingerprint, and protocol/store/ feature ABI, and the image check was trading that real proof for an unavailable one. It logs daemon.client_image_unverifiable_admitted so the weaker check is never invisible. A fingerprint MISMATCH — the tamper case the gate exists for — still rejects hard. Separate test seams keep the two modes testable apart. **Client bootstrap failures are no longer silent (#1539).** An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout naming the reason, plus the same text on stderr. Previously the reason sat in bootstrap_result.message and the process exited having written nothing at all. **POSIX activation: group-writable ancestors (#1535, discussion #1526).** activation_directory_secure required no group or other write bit on the install directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several distro skeletons and any site using a shared primary group, so install.sh failed for a large fraction of Linux users — reporting a policy refusal as "activation transaction I/O failed", which sent reporters after disk errors and filesystem types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and @iandol confirmed independently. World-writable ancestors are still refused (any local user could swap a path component mid-transaction). Group-writable ancestors are now warned about and admitted. The LEAF directory stays strictly owner-private — that is where the binary is published, and group write there would let another account replace the executable between validation and exec. Refusals now name the directory, its mode, and which rule refused. **The obsolete ui/standard chooser (#1538, from discussion #1526).** v0.10.0 consolidated to one archive per platform with the UI always embedded, but `update` still offered a variant choice: "ui" could only 404, and "standard" quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along with the CBM_VARIANT=ui remnant in the npm installer. Already-released 0.9.x binaries cannot be fixed retroactively, so the release workflow now publishes byte-identical ui-*-named alias assets — their updaters work again with no user action. The aliases are uploaded AFTER the VirusTotal gate: they are the same bytes as archives it already cleared, and uploading them earlier would duplicate every object in the scan set and the provenance manifest. **macOS install noise and attribution (#1537).** install.sh silenced the "No such xattr: com.apple.quarantine" line, which is what happens when a curl-downloaded archive carries no quarantine attribute — harmless, and it became the title of a bug report about an unrelated failure. The session-stop refusal now points at `daemon status` to list the client processes actually holding the daemon, instead of asserting sessions exist and leaving the reader to guess. Reported by @listepo. **Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh inside `python -m build` is what emitted Metadata-Version 2.5 and broke the v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x. Tests: separate seams for unverifiable vs mismatched peer images with a test per outcome; activation refusal must name directory + mode + rule; a group-writable ancestor must stage successfully. The update tests drop the flag that no longer exists. Verified against each reporter's environment shape. **Open security alerts (all three, OSSF Scorecard).** - HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe sibling are gitignored so `go build` in that directory cannot repeat it. - HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom generator is called with size 0. It reaches us transitively (postcss -> vite), so it is pinned through the existing graph-ui overrides block rather than promoted to a direct dependency; the lockfile resolves 3.3.18. - MEDIUM, unpinned pip command: the publish step installed build/twine by version only, leaving the whole transitive graph resolved at run time. pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316 hashes), generated on a linux/amd64 python:3.12 image so the wheels match what ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified by installing from it in that same image. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
8018561cfe |
fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
2dd296bab4 |
fix(daemon): answer the client when its binary image is rejected (#1383)
An image-verification rejection logged daemon.client_image_rejected and finished the worker WITHOUT sending a hello response - the only admission-failure path that never answered the peer. The client reported status "pending" indefinitely, indistinguishable from a slow cold start, with the reason visible only in the daemon log. The rejection now sends a CONNECT_REJECTED hello response carrying the reason (fingerprint_mismatch / image_unverifiable) and an actionable message, matching the version-conflict and capacity paths. The version-conflict path already responds to unverified peers, so this discloses nothing new to a same-uid local peer; admission stays rejected either way. Adds a CBM_ENABLE_TEST_SEAMS-only seam forcing peer-image verification to fail, since the in-process harness's peer pid is OS-authenticated socket credentials and always verifies against the service's own active image. Test: daemon_runtime_image_rejection_reaches_client_issue1383 - the client receives CONNECT_REJECTED with the reason in the message. RED without the response block (client times out), GREEN with it. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
db91b88cc4 |
fix(test): stop asserting a scheduling race in the forged-identity test
daemon_runtime_rejects_forged_identity_extension failed on the macos-15-intel CI leg. The suite is untouched by this branch and no daemon production code changed here; the leg flakes on main too, so this is a pre-existing defect the lane surfaced rather than a regression. The test asserted that transmitting the forged frame SUCCEEDS. It cannot be relied on to. The forged HELLO is 149 bytes against the 137-byte first-frame envelope cap, so the worker rejects it from the header and closes without ever reading the payload -- deliberately, so that no attacker-controlled bytes are read. send_frame writes the header and the payload as two separate writes, so whether the payload write lands before that close is pure scheduling: it wins on an idle host and loses on a loaded 4-vCPU runner. Both outcomes ARE the rejection, so the transmit result is now recorded and not asserted. Anti-vacuousness is preserved rather than dropped: the test now asserts it connected at all, and the drain check waits for the state it asserts (wait_for_clients 0) instead of sampling the count once, so the bound is a liveness backstop and never the verdict. Verified by mutation, not just by passing: with the envelope cap raised to 512 and the exact-length check removed -- a daemon that accepts the forged identity -- the repaired test goes RED on ASSERT(rejected). The mutation was reverted and the suite is 43/43 green. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
ade9f09a4f |
test(daemon-runtime): spawn the blocked executable without forking
runtime_test_spawn_blocked_executable() used fork() + execl(). fork() duplicates the parent's address space, and under a sanitizer that address space includes a very large shadow mapping. On macOS the duplicate trips the per-process memory limit and jetsam SIGKILLs the child before exec ever gets to replace the image, so the child is already dead by the time the parent checks it. The visible effect was daemon_runtime_process_fingerprint_never_hashes_ replacement_path failing at ASSERT(setup) under TSan while passing in every non-sanitized and ASan build. The diagnosis is direct rather than inferred: waitpid() reported the child as WIFSIGNALED with WTERMSIG 9, and DYLD_INSERT_LIBRARIES was unset, ruling out library-validation refusal of the copied system binary. posix_spawn() never copies the parent address space, so the child is never charged for the shadow mapping. The file actions reproduce the child-side setup exactly (close the ready-read and input-write ends, dup2 the input-read end onto stdin, close the original), and exec failure is now reported by posix_spawn itself rather than over the ready pipe. The FD_CLOEXEC-on-successful-exec EOF that the parent uses to detect a live child is unchanged. This keeps the test running under TSan on macOS instead of dropping the suite from the lane. Verified: daemon_runtime 43 passed under TSan and under the ordinary build, and the full macOS TSan set is 940 passed / 3 skipped / 0 races. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
b5b96e7006 |
fix(tests): leak-free teardown for the disconnect-cancel runtime test
LSan (Linux amd64 leg): 200 bytes — the runtime client (192) plus its IPC connection (8) — leaked whenever an earlier stage of daemon_runtime_disconnect_cancels_blocked_non_index_child... failed: both close blocks were gated on the request thread having completed, so a failed run skipped the close entirely. Teardown now closes unconditionally: close_begin interrupts the transport (forcing the blocked call to complete), the join becomes safe, and close_finish frees the client on every path. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0baddf7887 |
daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions
Daemon lifecycle and Windows correctness, verified on a real Windows 11 ARM64 VM through the maintained test-infrastructure/vm drivers, plus the macOS and Linux arm64 suites and the container lint gate. Daemon lifecycle: - daemon start/stop/status subcommands. `daemon start` launches a PERMANENT daemon (spawn shape is byte-exact argv; survives idle periods and session ends) and reports an already-active daemon instead of failing. Permanence is honored at every stop latch: last-committed-client disconnect, host initial-client window, coordinator release, and application final-session close — a permanent daemon also keeps admitting new sessions after its last one closes. - daemon stop refuses while sessions are active and lists the blocking peers (pid/role) that must finish first; an idle daemon drains through the activation-shutdown machinery with the ACK ordered after connection interrupts. A second stop is idempotent. The wire ops are no-cohort first-frame requests with peer fingerprint authentication, so stop/status never conflict with an exact-build admission gate. - One-shot CLI commands now execute through the daemon (index workers keep their local supervised path). A cold CLI run that had to spawn a temporary daemon prints a hint that `daemon start` removes the per-command startup tax; a warm daemon is recycled silently. - Hooks are connect-only fail-open: with no daemon present the hook emits a visible, rate-limited notice (Claude-dialect systemMessage plus stderr for other dialects) and always exits 0 — augmentation is never allowed to block the caller's tool use. - Version skew: a newer-build client automatically drains an older-build permanent daemon (strict semantic-version triples only; dev builds never auto-drain) and the build-conflict message names `cbm daemon stop` as the manual escape hatch. Windows IPC/runtime (real-VM verified): - ipc(win): persistent pending overlapped ConnectNamedPipe. The accept path used to destroy its listening pipe instance on every 20 ms poll timeout; a client attaching in the teardown window was severed or left on an orphaned pipe object whose HELLO no server handle could ever read, absorbing the connect until the client's own timeout expired. The pending connect now survives poll timeouts and nothing is destroyed while a client could be attaching. - ipc(win): drain-before-close for final responses. Closing a named-pipe server handle can discard a just-sent response before the peer reads it (POSIX stream sockets never lose buffered data on close). A bounded cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX, immediate on interrupted connections) now precedes close in runtime_worker_finish and runtime_reject_inline, so hello-conflict, capacity and disconnect acknowledgements reliably reach the peer. - runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no transport half-close, so close_begin now announces departure with an explicit frame (ordered after APPLICATION_CANCEL, before the local interrupt); the server releases the client's admission on receipt instead of waiting for the handle to close. Admission-drop timing is now identical to POSIX shutdown() semantics on every platform. - runtime(win): client close cancellation. close_begin serializes with request publication under the send lock, best-effort sends the active token's APPLICATION_CANCEL frame, then interrupts local I/O; the server cancels MCP/subprocess work promptly. Contract tests accept both correct outcomes (interrupted transport or decoded CANCELLED). - runtime: activation acknowledgement ordering. The activation ACK is the requester's license to act on "snapshotted and draining", so every connection interrupt is now initiated before the ACK is sent; a session could previously get one more request serviced after the requester observed the ACK. - service(win): deadline-bounded private-file prepare. The conflict-log prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms timer granularity) burned ~1.6 s against permanently obstructed paths, stalling hello rejections past the client's timeout. The retry budget is now a 250 ms deadline; transient share collisions still retry. - subprocess(win): cmd.exe /C payload encoder quotes metacharacters correctly (root cause of the git-on-Windows failure cluster). - watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native Windows stop/unwatch cancellation test with exact-image verification. - httpd: send_all writes in bounded 64 KiB slices. A single giant nonblocking send() on Windows is absorbed wholesale into AFD kernel buffering regardless of SO_SNDBUF, so send deadlines and interrupts could never engage against a slow peer (and the full payload was pinned in nonpaged pool). Slicing restores a deterministic backpressure point; a test hook pins SO_SNDBUF for the deadline and interrupt tests. - ui/http: shutdown lifecycle — interrupt checks, response-wide send deadline, explicit connection states, refusal to free a server while a listener-owned connection is active. Windows long-path support: - Central path-aware wide conversion (canonicalize via GetFullPathNameW and prepend the extended-length prefix for absolute paths >=240) at the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite store opens, and the daemon build-fingerprint/log paths. Deep managed installs (a 64-hex generation directory routinely exceeds MAX_PATH) now index, stage and activate correctly. - activation transaction: its own file APIs and the component-walking ancestry validators now operate in the extended-length namespace; the launcher path is canonicalized (and prefixed when deep) once at entry so every downstream exact-string comparison stays form-consistent. - Executable self-resolution uses the wide APIs (GetModuleFileNameW, GetFileAttributesW) so non-ASCII install paths survive argv[0] resolution. Windows launcher install/uninstall transaction: - FileRenameInfoEx names are NUL-terminated in an over-allocated buffer. FileNameLength governs per the contract, but filter drivers read FileName as NUL-terminated and appended adjacent heap bytes to created names — a flaky, garbage-suffixed rename target. Both the CLI and the launcher rename helpers are fixed. - Uninstall retires state via rename-aside (.cbm -> .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex chars so the bare rename target stays under the FileRenameInfoEx NT-conversion ceiling at guard depths; 64 bits still uniquely identify the generation. - When the running launcher's mapped generation backings pin .cbm against rename, the backings are relocated to activation-<pid>-N .retired tombstones beside the install (a mapped image may be renamed, never deleted; the launcher's liveness-guarded sweep reclaims stale tombstones). Every relocation is recorded, and a FAILED uninstall reverses the moves after restoring .cbm — via MoveFileExW with extended-length paths on both arguments, since the deep generation target is beyond the handle-based rename's bare-path reach — so a restored install keeps its generation backings and stays runnable. - After a committed uninstall the retired tree's backings are relocated out so the tree is shallow enough for the detached cleanup's rd, and the cleanup's working directory strips the extended-length prefix (CreateProcessW lpCurrentDirectory silently ignores prefixed paths). - Files created under Administrators-default-owner directories (CopyFileW destinations, CREATE_NEW tombstones, probe directories) are explicitly owner-stamped so the exact-owner validators hold on runner images; guard fixtures stamp hand-built trees the same way. Diagnostics, tests and infra: - diagnostics: discovery is now an always-delivered JSON control record (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and paths containing spaces; placement honors $TMPDIR with /tmp fallback via a diagnostics-local helper; the soak parser reads the JSON record; documented in docs/CONFIGURATION.md. Red-first coverage for suppressed log levels, TMPDIR-with-spaces, and native Windows output-contract assertions. - tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint parents with production-shaped ancestry (LocalAppData on Windows, via th_secure_runtime_parent_new) — the runtime ancestry validation correctly refuses temp roots whose ancestors grant mutation rights to Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and drive the documented startup-owner publication flow before reading generation-bound endpoint addresses. This turns the 26 Windows failures previously visible in CI's full-test job green without weakening any validation. - tests(win): the launcher guard covers the full permanent-launcher contract including failed-uninstall restore and immediate reinstall after uninstall; new daemon lifecycle and reworked hook-augment guards run the start/recycle/stop flow end to end. - tests: CBM_SKIP_PERF is now actually consumed by the test runner (it was set by CI but never read, so perf suites ran everywhere); four throughput/bench suites are classified as perf, the heavy store_arch suite moved to the slow-timeout tier, and two wall-clock-sensitive assertions were rewritten as invariant checks with coarse hang-detector backstops. - build/test infra: build-dir safety contract, UI dev-proxy security contract, soak daemon-recovery contract, path-safety helper, the Windows VM worktree-sync contract wired into scripts/test.sh, and vm/win.sh guards building its clean embedded-UI product in an isolated BUILD_DIR so it cannot clobber the incremental test build. provision-windows.sh now installs Node.js for the guards UI build. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
f35b10b79b |
fix: harden cross-platform daemon startup smoke
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
89c7c98f00 |
fix: support standard Windows profile ACLs
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
ad58e71bea |
fix: stabilize cross-platform daemon coordination
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
83c137d2a5 |
feat: complete shared daemon lifecycle
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0e00ef5702 |
feat: coordinate concurrent CBM sessions
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |