github-featured/deusdata--codebase-memory-mcp
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
2,440 Commits 300 Branches 52 Tags
fix/java-enum-methods
Commit Graph

6 Commits

Author SHA1 Message Date
Martin Vogel 8eabe191d2 fix(install,daemon): unbreak npx clients, group-writable homes, and legacy updaters 
Five field reports in the 24 hours after v0.10.0 all pointed at the same thing:
gates that were right in principle refused real, ordinary setups, and then
failed to say why. Per the consolidated strictness decision, each gate keeps the
protection that matters and drops the part that was refusing legitimate users —
and every refusal now names what it refused and how to proceed.

**Daemon image gate: npx and every ephemeral install path (#1539, #1383).**
The admission check treated "the peer's image hashes differently" and "the
peer's image cannot be examined at all" as one failure. The second is what
`npx codebase-memory-mcp` always produces (ephemeral cache path,
unfingerprintable), so every npx-invoked client was rejected — and, because the
client never reported it, agents saw a transport that closed mid-handshake with
zero bytes on stdout. Reported by @wassolles with the admission path already
read and the fix space mapped.

An unverifiable image is now admitted: the rendezvous HELLO immediately above it
has already proven semantic version, build fingerprint, and protocol/store/
feature ABI, and the image check was trading that real proof for an unavailable
one. It logs daemon.client_image_unverifiable_admitted so the weaker check is
never invisible. A fingerprint MISMATCH — the tamper case the gate exists for —
still rejects hard. Separate test seams keep the two modes testable apart.

**Client bootstrap failures are no longer silent (#1539).**
An MCP client that cannot reach the daemon now emits a JSON-RPC error on stdout
naming the reason, plus the same text on stderr. Previously the reason sat in
bootstrap_result.message and the process exited having written nothing at all.

**POSIX activation: group-writable ancestors (#1535, discussion #1526).**
activation_directory_secure required no group or other write bit on the install
directory AND every ancestor. WSL2 ships ~ and ~/.local at 0775, as do several
distro skeletons and any site using a shared primary group, so install.sh failed
for a large fraction of Linux users — reporting a policy refusal as "activation
transaction I/O failed", which sent reporters after disk errors and filesystem
types. Root-caused by @AmirF194 in a clean ubuntu container; @shochdoerfer and
@iandol confirmed independently.

World-writable ancestors are still refused (any local user could swap a path
component mid-transaction). Group-writable ancestors are now warned about and
admitted. The LEAF directory stays strictly owner-private — that is where the
binary is published, and group write there would let another account replace the
executable between validation and exec. Refusals now name the directory, its
mode, and which rule refused.

**The obsolete ui/standard chooser (#1538, from discussion #1526).**
v0.10.0 consolidated to one archive per platform with the UI always embedded,
but `update` still offered a variant choice: "ui" could only 404, and "standard"
quietly WAS the UI build. Reported by @iandol upgrading 0.9.0 -> 0.10.0. The
chooser, its --standard/--ui flags, and the ui- URL plumbing are removed, along
with the CBM_VARIANT=ui remnant in the npm installer.

Already-released 0.9.x binaries cannot be fixed retroactively, so the release
workflow now publishes byte-identical ui-*-named alias assets — their updaters
work again with no user action. The aliases are uploaded AFTER the VirusTotal
gate: they are the same bytes as archives it already cleared, and uploading them
earlier would duplicate every object in the scan set and the provenance manifest.

**macOS install noise and attribution (#1537).**
install.sh silenced the "No such xattr: com.apple.quarantine" line, which is
what happens when a curl-downloaded archive carries no quarantine attribute —
harmless, and it became the title of a bug report about an unrelated failure.
The session-stop refusal now points at `daemon status` to list the client
processes actually holding the daemon, instead of asserting sessions exist and
leaving the reader to guess. Reported by @listepo.

**Riders.** hatchling is pinned in pkg/pypi (an unpinned backend resolved fresh
inside `python -m build` is what emitted Metadata-Version 2.5 and broke the
v0.10.1 publish); SECURITY.md's supported-versions table moves to 0.10.x.

Tests: separate seams for unverifiable vs mismatched peer images with a test per
outcome; activation refusal must name directory + mode + rule; a group-writable
ancestor must stage successfully. The update tests drop the flag that no longer
exists. Verified against each reporter's environment shape.

**Open security alerts (all three, OSSF Scorecard).**
- HIGH, binary artifact: an 8.8 MB compiled Go ELF wrapper had been committed at
  pkg/go/codebase-memory-mcp by accident. Removed, and both it and its .exe
  sibling are gitignored so `go build` in that directory cannot repeat it.
- HIGH, GHSA-2v37-7h3g-55p8: nanoid < 3.3.17 loops forever when a custom
  generator is called with size 0. It reaches us transitively (postcss -> vite),
  so it is pinned through the existing graph-ui overrides block rather than
  promoted to a direct dependency; the lockfile resolves 3.3.18.
- MEDIUM, unpinned pip command: the publish step installed build/twine by
  version only, leaving the whole transitive graph resolved at run time.
  pkg/pypi/requirements-publish.txt now hash-pins the complete toolchain (316
  hashes), generated on a linux/amd64 python:3.12 image so the wheels match what
  ubuntu-latest resolves, and the step runs pip with --require-hashes. Verified
  by installing from it in that same image.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-08-11 13:37:32 +02:00
Martin Vogel 0a22fb14df fix(windows): absorb transient file locks when retiring a running image
DCO / dco (push) Has been cancelled
Release run 30385988475 lost three Windows smoke jobs to one phenomenon on two
different surfaces. A Windows file that was just written or just executed can
refuse deletion and rename for a moment while a scanner reads it or while the OS
finishes reaping a child. Both conditions clear on their own within moments.

It reads as nondeterminism because it is timing: windows-2025 passed both amd64
variants while windows-latest -- the SAME image, windows-2025-vs2026 -- failed
both, and windows-11-arm failed too.

PRODUCT: uninstall abandoned a live installation.

With one binary per platform, uninstall must remove the very image it is
executing, and on Windows a running image can only be renamed, never
overwritten. That rename got a single attempt, so a transient sharing violation
surfaced to the user as

    error: failed to remove ...; completed configuration/index cleanup may remain

with the install left half-torn-down. install.ps1 already retries this exact
operation ten times for this exact reason; the C path now does the same, and
only for sharing/access/lock violations, so a file that is genuinely held still
fails closed instead of spinning.

Proven with a failure-injection seam rather than by racing a real scanner:
3 injected failures must still succeed, 64 must still fail AND leave the target
in place. Deterministic, so it does not need CI to learn whether it works.

HARNESS: the smoke killed itself silently.

Phases that install a binary then `rm -rf` the directory hit the same lock. Under
`set -euo pipefail` that returns non-zero and ends the run WITHOUT printing
anything -- the logs jump straight from "OK 13h" to job cleanup with no FAIL
line, which is what made this expensive to read.

None of those 16 removals is an assertion; they are fixture cleanup, and
_smoke.yml already states the principle ("a temp dir on an ephemeral runner must
never fail the job") that smoke-test.sh did not honour. They now go through
smoke_rmtree, which retries so the disk is actually reclaimed -- these fixtures
hold ~300 MB binaries and runners are disk-tight -- then warns and continues.

Verified: Windows VM full smoke ALL PASSED (previously died after 13h), macOS
full smoke ALL PASSED, activation_transaction + cli 259 passed on Windows, all
10 shell contracts pass, lint-ci clean.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-28 22:49:22 +02:00
Martin Vogel f51745533c fix(win): reconcile the C suite with the managed-launcher contract 
The first full Windows x64 CI run of this branch (earlier contract-step
walls had always aborted the leg) surfaced 40 failures: the managed
launcher's fail-closed install/update/uninstall gates and the suite's
POSIX-modeled activation tests were both new in this branch and had
never been reconciled, and they hid three real production bugs.

Production fixes:

- ui: config publishes with POSIX rename semantics. MoveFileExW's legacy
  replace cannot supersede a destination a reader holds open (the name
  lingers until the last handle closes, even with FILE_SHARE_DELETE), so
  a config save racing an open reader failed. The temp file is opened
  with DELETE access and renamed in-hand (bare-path target, NUL-padded
  rename record) before close, falling back to MoveFileExW on any
  rename-by-handle failure so non-NTFS config homes keep the previous
  behavior.
- subprocess(win): the child's redirected log handle now shares write
  access. POSIX places no lock on the redirected log, so a second
  producer may append while the child runs; the missing FILE_SHARE_WRITE
  mandatory-locked every other writer out, which the index-supervisor
  callback-isolation test legitimately exercises.
- cli(win): the portable uninstall path composes the managed binary name
  with its .exe suffix; it could never find (or remove) the installed
  binary before.
- activation: security-predicate refusals are now diagnosable. The
  transaction's predicates refuse without a usable OS last-error, which
  collapsed five distinct refusal classes into one blind "status -3,
  os 0" at every caller. Each refusing predicate records what refused —
  predicate, untrusted SID, and object path — first-wins so the deepest
  cause survives, and the staging error line appends it. This is what
  identified every Windows refusal in this batch, including the real
  install-failure class on drive-root-rooted payload directories
  (Authenticated Users holds inherited Modify there; profile-rooted
  directories do not).

Windows test dispatch (release binaries unchanged):

- Test builds (CBM_CLI_ENABLE_TEST_API, the existing test-object macro)
  compile the portable activation flow on Windows, reachable solely
  through the installed activation test seam and announced loudly on
  stderr; release binaries compile exclusively the managed transaction,
  so the dispatch polarity fails closed. The suite's install/update/
  uninstall calls route through a seam-if-absent wrapper, keeping the
  shared activation and agent-config semantics unit-covered on Windows;
  POSIX arms keep exercising the real activation machinery.
- Two new release-gate tests assert the fail-closed contract with the
  seam OFF: a portable payload may not self-update/self-uninstall, and
  install refuses to adopt or replace a foreign file at the canonical
  launcher path even under --force.
- Test-bug fixes the first valid Windows runs exposed: the version-
  cohort crash test reused one conflict record across the probe and the
  turnover acquire (whose entry memset erased the probe's detail before
  the assertions read it); the activation staged-path assertion now
  checks containment rather than a separator (the \\?\ namespace
  performs no forward-slash translation); the ui atomic test captures,
  cleans up, then asserts so a failure cannot poison later tests; the
  PATH-visibility guard is POSIX-only (Windows configures PATH through
  the user registry, not a shell rc).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-23 00:53:41 +02:00
Martin Vogel 0baddf7887 daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions 
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.

Daemon lifecycle:

- daemon start/stop/status subcommands. `daemon start` launches a
  PERMANENT daemon (spawn shape is byte-exact argv; survives idle
  periods and session ends) and reports an already-active daemon
  instead of failing. Permanence is honored at every stop latch:
  last-committed-client disconnect, host initial-client window,
  coordinator release, and application final-session close — a
  permanent daemon also keeps admitting new sessions after its last
  one closes.
- daemon stop refuses while sessions are active and lists the blocking
  peers (pid/role) that must finish first; an idle daemon drains
  through the activation-shutdown machinery with the ACK ordered after
  connection interrupts. A second stop is idempotent. The wire ops are
  no-cohort first-frame requests with peer fingerprint authentication,
  so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
  keep their local supervised path). A cold CLI run that had to spawn a
  temporary daemon prints a hint that `daemon start` removes the
  per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
  emits a visible, rate-limited notice (Claude-dialect systemMessage
  plus stderr for other dialects) and always exits 0 — augmentation is
  never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
  older-build permanent daemon (strict semantic-version triples only;
  dev builds never auto-drain) and the build-conflict message names
  `cbm daemon stop` as the manual escape hatch.

Windows IPC/runtime (real-VM verified):

- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
  path used to destroy its listening pipe instance on every 20 ms poll
  timeout; a client attaching in the teardown window was severed or left
  on an orphaned pipe object whose HELLO no server handle could ever
  read, absorbing the connect until the client's own timeout expired.
  The pending connect now survives poll timeouts and nothing is
  destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
  server handle can discard a just-sent response before the peer reads
  it (POSIX stream sockets never lose buffered data on close). A bounded
  cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
  immediate on interrupted connections) now precedes close in
  runtime_worker_finish and runtime_reject_inline, so hello-conflict,
  capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
  transport half-close, so close_begin now announces departure with an
  explicit frame (ordered after APPLICATION_CANCEL, before the local
  interrupt); the server releases the client's admission on receipt
  instead of waiting for the handle to close. Admission-drop timing is
  now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
  request publication under the send lock, best-effort sends the active
  token's APPLICATION_CANCEL frame, then interrupts local I/O; the
  server cancels MCP/subprocess work promptly. Contract tests accept
  both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
  the requester's license to act on "snapshotted and draining", so every
  connection interrupt is now initiated before the ACK is sent; a
  session could previously get one more request serviced after the
  requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
  prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
  timer granularity) burned ~1.6 s against permanently obstructed paths,
  stalling hello rejections past the client's timeout. The retry budget
  is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
  correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
  Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
  nonblocking send() on Windows is absorbed wholesale into AFD kernel
  buffering regardless of SO_SNDBUF, so send deadlines and interrupts
  could never engage against a slow peer (and the full payload was
  pinned in nonpaged pool). Slicing restores a deterministic
  backpressure point; a test hook pins SO_SNDBUF for the deadline and
  interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
  deadline, explicit connection states, refusal to free a server while
  a listener-owned connection is active.

Windows long-path support:

- Central path-aware wide conversion (canonicalize via GetFullPathNameW
  and prepend the extended-length prefix for absolute paths >=240) at
  the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
  store opens, and the daemon build-fingerprint/log paths. Deep managed
  installs (a 64-hex generation directory routinely exceeds MAX_PATH)
  now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
  ancestry validators now operate in the extended-length namespace;
  the launcher path is canonicalized (and prefixed when deep) once at
  entry so every downstream exact-string comparison stays
  form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
  GetFileAttributesW) so non-ASCII install paths survive argv[0]
  resolution.

Windows launcher install/uninstall transaction:

- FileRenameInfoEx names are NUL-terminated in an over-allocated
  buffer. FileNameLength governs per the contract, but filter drivers
  read FileName as NUL-terminated and appended adjacent heap bytes to
  created names — a flaky, garbage-suffixed rename target. Both the CLI
  and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
  .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
  chars so the bare rename target stays under the FileRenameInfoEx
  NT-conversion ceiling at guard depths; 64 bits still uniquely
  identify the generation.
- When the running launcher's mapped generation backings pin .cbm
  against rename, the backings are relocated to activation-<pid>-N
  .retired tombstones beside the install (a mapped image may be renamed,
  never deleted; the launcher's liveness-guarded sweep reclaims stale
  tombstones). Every relocation is recorded, and a FAILED uninstall
  reverses the moves after restoring .cbm — via MoveFileExW with
  extended-length paths on both arguments, since the deep generation
  target is beyond the handle-based rename's bare-path reach — so a
  restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
  out so the tree is shallow enough for the detached cleanup's rd, and
  the cleanup's working directory strips the extended-length prefix
  (CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
  (CopyFileW destinations, CREATE_NEW tombstones, probe directories)
  are explicitly owner-stamped so the exact-owner validators hold on
  runner images; guard fixtures stamp hand-built trees the same way.

Diagnostics, tests and infra:

- diagnostics: discovery is now an always-delivered JSON control record
  (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
  paths containing spaces; placement honors $TMPDIR with /tmp fallback
  via a diagnostics-local helper; the soak parser reads the JSON record;
  documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
  log levels, TMPDIR-with-spaces, and native Windows output-contract
  assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
  parents with production-shaped ancestry (LocalAppData on Windows, via
  th_secure_runtime_parent_new) — the runtime ancestry validation
  correctly refuses temp roots whose ancestors grant mutation rights to
  Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
  drive the documented startup-owner publication flow before reading
  generation-bound endpoint addresses. This turns the 26 Windows
  failures previously visible in CI's full-test job green without
  weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
  contract including failed-uninstall restore and immediate reinstall
  after uninstall; new daemon lifecycle and reworked hook-augment
  guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
  (it was set by CI but never read, so perf suites ran everywhere);
  four throughput/bench suites are classified as perf, the heavy
  store_arch suite moved to the slow-timeout tier, and two
  wall-clock-sensitive assertions were rewritten as invariant checks
  with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
  contract, soak daemon-recovery contract, path-safety helper, the
  Windows VM worktree-sync contract wired into scripts/test.sh, and
  vm/win.sh guards building its clean embedded-UI product in an
  isolated BUILD_DIR so it cannot clobber the incremental test build.
  provision-windows.sh now installs Node.js for the guards UI build.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-21 15:32:15 +02:00
Martin Vogel 83c137d2a5 feat: complete shared daemon lifecycle 
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-18 01:26:08 +02:00
Martin Vogel 0e00ef5702 feat: coordinate concurrent CBM sessions 
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
 2026-07-16 19:20:46 +02:00