56 Commits

Author SHA1 Message Date
Martin Vogel b6a5d2c35b feat(release): ship MCPB bundles and publish them to the MCP Registry (#1246)
Every release now carries .mcpb one-click-install bundles alongside the
archives, and the MCP Registry entry lists them with per-file sha256:

- package-release.sh (canonical) builds codebase-memory-mcp-<target>.mcpb
  for darwin/windows and the STATIC linux builds — manifest.json + the same
  staged (stripped, gated) binary + LICENSE + THIRD_PARTY_NOTICES.md. The
  glibc-dynamic linux targets stay archive-only: a dynamic binary defeats
  the one-click promise.
- _build.yml / release-draft: bundles flow through provenance attestation,
  checksums.txt, cosign signing and the release asset list; checksums.txt
  is also preserved as a same-run artifact for the registry job.
- verify: the canonical scan matrix grows to 14 containers; MCPB manifests
  are validated (parse, binary server, entry_point member, command binds
  the entry point). Bundle binaries dedupe to the archive scan objects, so
  the VT gate gains only the three distinct manifest.json files.
- publish-mcp-registry: gen-mcpb-registry-entries.sh appends one mcpb
  package entry per bundle (release-asset URL + fileSha256 from the
  attested checksums) to server.json before mcp-publisher runs.
  Idempotent; a checksums file without bundles is a hard failure.
- contracts: Step 0o pins the bundle shape at its producer on every leg,
  Step 0p pins the registry entries against the live server.json, and the
  extractor contract covers the 14-container matrix incl. broken-manifest
  fail-closed cases. The linux test image gains zip for the packager.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-11 17:01:24 +02:00
Martin Vogel 39e02544cc fix(smoke): UI presence is the caller's claim, not every lane's assumption
Making REQUIRE_UI unconditional broke all three pr-smoke legs:

  FAIL 15a: SMOKE_REQUIRE_UI=1 but this binary serves no embedded UI assets

The PR lane runs scripts/build.sh WITHOUT --with-ui on purpose, so an npm
frontend build does not land on every product PR. Demanding embedded assets
there asserts a property that lane deliberately does not produce.

The guard still exists where it means something: scripts/ci/smoke-artifact.sh
builds --with-ui, packages the real archive and smokes the EXTRACTED result, so
it exports SMOKE_REQUIRE_UI=1 and a frontend-less binary fails there. That is
the lane whose whole job is release fidelity.

Deliberately NOT fixed by adding --with-ui to pr.yml: that would put `npm ci &&
npm run build` on three runners for every product PR to re-prove something the
dry-run and release lanes already gate. PR CI cost is not free and this branch
should not quietly raise it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-09 18:17:42 +02:00
Martin Vogel a4336dc40a feat(release): ship one archive set and tolerate a single Microsoft !ml verdict
Completes the collapse to a single shipped composition and replaces the
zero-tolerance VirusTotal gate with a narrow, disclosed policy.

Packaging and installers
  - package-release.sh loses --variant; archives are codebase-memory-mcp-<os>-<arch>
    with exactly four members. install.sh/install.ps1 lose --ui/--standard.
  - The extractor drops CBMUIPK pack parsing and --archive-scope; its scan-set and
    association manifests (which the gate depends on) are unchanged otherwise.
  - npm/PyPI/Go wrappers: the runtime "set" is one file again. The Windows lock
    and race fixes from #1495/#1496 are kept; only multi-file set membership goes.
    This also fixes `pip install` on Windows, which rejected the fifth archive
    member against a hardcoded four-name allowlist.
  - The wrappers' post-download probe moves from --verify-runtime-assets (removed)
    to --version, which proves the same thing: the binary executes.

VirusTotal gate
  - Exactly ONE detection is tolerated, and only when the engine is Microsoft AND
    the label ends in `!ml`. Two or more engines, any non-`!ml` label, any other
    vendor, any suspicious verdict and every infrastructure error still block.
  - A tolerated object prints TOLERATED:, never OK:, and its counts are recorded
    in vt-results.tsv exactly as a blocked one would be.
  - append-vt-notes.sh mirrors the policy. It previously hard-failed on any
    malicious count, so loosening only the gate would have passed the scan and
    then died at note publication. The notes now DISCLOSE a tolerated detection
    and link to SECURITY.md rather than claiming "0 malicious" for everything.

Rationale for the tolerance is in the gate itself: the verdict is not a property
of our bytes. It inverts across architectures and link modes, moves between
sibling artifacts of one build, and lands in different variant buckets for the
same source. The same `!ml` family hits llama.cpp, GitHub's own `gh`, Microsoft's
own Go toolchain and Anthropic's Claude installer.

The zero-tolerance contract becomes test_vt_gate_policy_contract.sh, asserting
the full matrix: 1x Microsoft !ml passes and reports TOLERATED; a Microsoft
signature label, a non-Microsoft engine, two engines, a suspicious verdict and
every malformed-response case still block. Its tripwire is narrowed to the
reverted endpoint-verification mechanism rather than the words "false positive",
so it no longer fires on a deliberate in-gate policy branch.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-09 16:16:08 +02:00
Martin Vogel d58afe562d revert(release): re-embed runtime assets into the single shipped binary
Externalizing the integration templates (#1492/#1493) and the UI bundle
(#1501/#1503) was done to reduce the Microsoft `Wacatac.B!ml` surface. It did
not work: across dry runs the flagged artifact count stayed at ~3 and the
detections merely moved between artifacts.

Dissection of run 31286803592 shows there is no structural cause to fix. The
verdicts split across every axis at once — linux-amd64 (dynamic) flagged while
linux-amd64-portable (static) is clean, but linux-arm64 (dynamic) clean while
linux-arm64-portable (static) is flagged. The two macOS binaries have identical
segment structure and split clean/flagged. Siblings from one build landed in
different variant buckets (.B vs .C). Entropy is low everywhere
(code_vectors.bin 4.166, grammar tables 3.464 bits/byte, against 7.5-8.0 for
packed payloads), so the packed-payload hypothesis is excluded too.

So the complexity bought nothing, and installation goes back to being
self-contained: one binary that carries its own UI and agent integration
templates, with no adjacent data file that has to resolve before `install`
works. Only the UI-capable composition ships from now on, under the historical
unsuffixed archive name.

Removed: src/ui/asset_pack.{c,h}, asset_pack_stub.c, asset_manifest_stub.c,
scripts/pack-ui-assets.mjs, src/cli/integration_assets.{c,h},
assets/cbm-integrations.json, scripts/gen-integrations-hash.sh, the
--verify-runtime-assets probe (nothing adjacent left to verify), and the
composition gates A6/A7 whose property is now deliberately inverted.

Restored: scripts/embed-frontend.sh, src/ui/embedded_{assets.h,stub.c}, the
compiled-in hook/adapter template bodies, and the embed/EMBED_OBJS build path.

Kept from the reverted commits, re-applied by hand where a wholesale file
restore would have dropped them:
  - cbm_module_path_utf8() in both self-path sites. GetModuleFileNameA renders
    through the ANSI code page and mangles non-ASCII install paths.
  - the /__cbm/ui-readiness HMAC proof, secure_random and cbm_hmac_sha256, so
    `daemon start --open` still waits for a genuine CBM listener.
  - X-Content-Type-Options: nosniff on served assets.
  - the MinGW noexecstack gate, -lbcrypt, and the cppcheck/zip CI fixes.

Archives are now codebase-memory-mcp-<os>-<arch>[-portable] with exactly four
members (binary, LICENSE, installer, THIRD_PARTY_NOTICES.md). That restores the
names every static package manifest already points at — aur, chocolatey,
homebrew, scoop, winget and glama were all broken by the -ui- rename.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-09 13:06:42 +02:00
Martin Vogel 8018561cfe fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-08 17:35:05 +02:00
Martin Vogel 1c16786b4b fix(smoke): stage cbm-integrations.json in the Windows smoke path too
The Windows pr-smoke uses test-infrastructure/vm/vm-smoke.sh, the Windows analog
of smoke-local.sh, which had the same three staging gaps the Unix fix closed:
its required-sidecar check, fixture copy, and zip member list all omitted
cbm-integrations.json, so the served archive lacked the asset and install/
uninstall failed closed with "integration assets missing".

Plus one Windows-specific spot: vm-smoke.sh copies the smoke binary into
$SMOKE_DIR and runs smoke-test.sh against it there, but the asset only reached
$FIXTURE_DIR — so Phase 14 (which hand-stages the binary and drives an
uninstall, resolving the asset next to the binary) missed it. Now staged next to
the binary in $SMOKE_DIR as well. build.sh already places it next to
build/c/*.exe, so the non-artifact source is $(dirname BINARY_SRC); artifact
mode takes the release copy.

Member set + order in the zip mirror scripts/package-release.sh (Windows).
vm-smoke.sh lives under test-infrastructure/, outside the license gate's scan
set, so the cbm-integrations.json/LICENSE adjacency raises no ScanCode finding
there. Syntax-checked; vm-worktree-manifest, smoke-fixture and windows-bundle
contracts pass. The Unix smoke already runs clean through all 16 phases; this
mirrors that staging onto the one path I cannot exercise locally.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-08 05:31:53 +02:00
Martin Vogel ba5724319c fix(ci): accept the documented test-amd64 leg spelling
run.sh --help advertises 'amd64|test-amd64|tsan-amd64', but only 'amd64'
was dispatched, so the documented spelling failed with
"unknown leg 'test-amd64'". Accept both.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-06 02:54:23 +02:00
Martin Vogel 3e06362160 fix(ci): isolated VM checkout inherits the base's real origin URL
`git clone --local` points origin at the base PATH, so a later
`git fetch origin <branch>` inside the isolated tree would resolve against
whatever the shared base happens to hold - i.e. another session's state, the
exact coupling the per-run checkout exists to remove. Inherit the base's
origin URL at clone time so the isolated tree fetches from GitHub itself.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-06 02:12:01 +02:00
Martin Vogel cad2b02cee fix(ci): give the Windows VM leg a per-run checkout and a HEAD assertion
The VM's last piece of shared state was the working tree itself. /c/cbm is
ONE checkout and update/sync REPLACE it (git reset --hard + clean -fdx), so a
second session syncing its branch swaps the code under a running leg.

Observed 2026-08-06: a gate run on main compiled tests/test_workspace.c and
src/foundation/workspace.c - files that do not exist in main - and died on
'sys/wait.h' not found. The VM was sitting at another session's commit
7fa5b07. That is worse than a red: a confident verdict about the wrong tree.

- CBM_CI_RUN_ID gives the run its own checkout, /c/cbm-run-<id>, created with
  a LOCAL clone so .git/objects are hardlinked (seconds, near-zero disk).
  Unset keeps the historical single-tree behaviour byte-for-byte.
- CBM_VM_EXPECT_HEAD=<sha> asserts the tree is the intended one BEFORE and
  AFTER the suites, because the swap can land mid-flight.
- win.sh drop-run-checkout removes THIS run's tree (refuses in shared mode -
  a run may only delete what it created; another session's tree is never ours
  to remove).

Verified: bash -n and shellcheck -S error clean.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-06 02:09:21 +02:00
Martin Vogel 5444da4166 fix(ci): isolate local-CI runs per run id, and tidy what each run creates
Concurrent local-CI legs clobbered each other. Container names were already
unique per `compose run --rm`, but the mutable state was not:

- Docker: `cbm-build` is ONE named volume shared by every service and every
  concurrent run, so two legs wrote the same /src/build - objects and
  test-logs included. The parallel scheduler then dies reading a suite log
  another run replaced ('cannot read suite log .../extraction.log').
- Windows VM: one checkout (/c/cbm) plus a FIXED log path (/tmp/win-test.log)
  and a shared build dir; -PruneStale could also delete a live run's temp root.

Each run now carries a unique id (pid+epoch, overridable via CBM_CI_RUN_ID):

- docker-compose: the build volume takes its name from CBM_CI_BUILD_VOLUME,
  defaulting to today's `cbm-build` so the single-run path is unchanged.
- run.sh: derives the id, points the build volume at it, and removes that
  volume on success. CBM_CI_SHARED_BUILD=1 opts back into the shared volume.
- vm-run-tests.sh: per-run log; and when a CALLER sets CBM_CI_RUN_ID (i.e.
  declares concurrency) a per-run BUILD_DIR too, removed on success.

ccache and the fixture cache stay SHARED deliberately - ccache is
concurrency-safe and content-verified, and sharing them is what keeps an
isolated run fast instead of cold.

Cleanup follows one rule: a run tidies what it created, EXCEPT when it failed -
then the artifacts are the post-mortem and the run prints how to inspect and
drop them. CBM_CI_KEEP=1 keeps them regardless.

Verified: bash -n on both scripts; `docker compose config` valid, and with
CBM_CI_BUILD_VOLUME set the volume resolves to the per-run name.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-06 01:45:56 +02:00
Martin Vogel 0fe0f453be build(test): close the gaps in the sanitizer matrix
Every lane here was either asserting coverage it did not have, or was
reachable only by bypassing the entry point that is supposed to define
the ladder.

TSan: no exclusions any more. The three suites the Makefile documented
as excluded are back. daemon_ipc and daemon_frontend no longer reproduce
the harness race and thread leaks they were excluded for. daemon_runtime
did not deadlock as the comment claimed -- it reported a real production
data race on the log sink, fixed separately. Excluding a suite from a
sanitizer lane hides exactly the class of bug the lane exists to find,
so the comment block now records what was actually true rather than what
was assumed.

TSAN_OPTIONS gains report_thread_leaks=0. This disables the thread-
HYGIENE check only; race detection is untouched. Several daemon fixtures
fork after the process has gone multi-threaded, and in the forked child
TSan sees the parent's already-finished threads as never-joined even
where the fixture joins them. It fires on macOS and not Linux, i.e. it
tracks fork semantics rather than anything about this code. The
alternative was dropping whole suites, which costs real race coverage;
this costs none.

UBSan: tre.o no longer builds with -fno-sanitize=alignment. Alignment
was switched off for a vendored regex engine that ships in the product,
which is where the check is least redundant, not most.

LSan on macOS: new test-lsan target and test-lsan-macos CI leg. LSan is
on by default under ASan on Linux, so the Linux legs have always had
leak coverage. On macOS it is off by default and Apple's clang refuses
to enable it outright, so that platform had none at all. Apple's refusal
is not a darwin limitation -- upstream LLVM supports LSan on darwin/
arm64. The lane is the ordinary ASan suite built with Homebrew LLVM and
run with detect_leaks=1; it runs the full suite clean and was checked to
still catch a deliberately leaked allocation.

MSan: reachable from the local ladder. The image and compose service
existed but run.sh had no leg, so the only way in was to drive docker
compose by hand -- which means it was not part of the ladder in any
meaningful sense. The image also moves to clang 22, matching the diag
and analyzer lanes instead of sitting four majors behind on noble's
default. The leg documents the aarch64 shadow-mapping failure so a local
arm64 stack overflow in the grammar suites is not mistaken for a code
defect; the GitHub leg runs x86-64, which is the mapping that matters.

Off-by-default ASan checks: the diagnostic lane, and its CI twin, now run
detect_stack_use_after_return, detect_stack_use_after_scope,
detect_invalid_pointer_pairs (with the -fsanitize=pointer-compare,
pointer-subtract instrumentation it requires) and strict_string_checks.
Running ASan is not the same as running all of it, and these four cover
bug classes nothing in the matrix was looking for. They stay on the
diagnostic lane rather than the gating ones until they have a clean
history there; promoting them is a separate deliberate step.

Verified: macOS TSan 940 passed / 3 skipped / 0 races over the full
suite set; the macOS leak lane 7375 passed / 4 skipped / 0 leaks, with
LeakSanitizer confirmed armed under that exact toolchain and option set
by checking it still reports a deliberately leaked allocation.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-03 15:37:33 +02:00
Martin Vogel c9886d4f25 feat(ci): MemorySanitizer lane — instrumented libc++/zlib image, full C++ coverage
Stage 2 of the memory-diagnostics program (user decision: go directly to the
instrumented image rather than a C-only probe). MSan detects uninitialized
READS, the one memory-error class no other lane covers dynamically, and it
requires every linked library to be instrumented -- vendored C deps compile
in-tree and instrument for free; the two external links do not:

- test-infrastructure/Dockerfile.msan: pinned-base image building
  libc++/libc++abi/libunwind (llvmorg-18.1.8, LLVM_USE_SANITIZER=
  MemoryWithOrigins) and static zlib v1.3.1 into /opt/msan, with the
  symbolizer and MSan runtime in a separate last layer so tool additions
  never invalidate the ~30-min libc++ build.
- scripts/msan.sh: the canonical lane entry. ALWAYS clean-builds its
  BUILD_DIR: make does not encode flags into dependencies, and a stage-1
  probe's libstdc++ objects surviving into the libc++ lane produced a
  convincing-looking uninitialized-value report at preprocessor.cpp:168 --
  the uninstrumented .so string constructor wrote the temporary, the
  instrumented move constructor read it. The clean rebuild proved it an
  artifact: extraction (incl. the C++ preprocessing path) runs 272/272 with
  zero reports.
- Makefile.cbm: CXX_STDLIB / CXX_STDLIB_FLAGS hooks so the lane can swap
  libstdc++ for the instrumented libc++ (defaults identical; the shipping
  build is byte-for-byte unaffected).
- docker-compose test-msan service: same aarch64 seccomp/setarch remedy as
  the TSan service (MSan's shadow layout hits the same personality() block).
- CI test-msan job (_test.yml): buildx local-cache via the repo's existing
  pinned actions/cache -- no new third-party action pins; a warm run skips
  the libc++ build entirely.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-03 15:37:33 +02:00
Martin Vogel 98a2856a90 fix(security): patch postcss and pin the glibc-floor image
Two open security alerts.

Dependabot #13 (GHSA-r28c-9q8g-f849, high): postcss path traversal via
sourceMappingURL auto-loading. Transitive dev dependency of vite, so it never
ships in the binary, but the fix is a clean lockfile bump -- vite requires
^8.5.3 and the patch floor is 8.5.18, so 8.5.24 satisfies it with no dependency
graph change. Verified: npm ci resolves and the UI still builds.

Scorecard #76 (PinnedDependenciesID): Dockerfile.glibc22 used a floating
`ubuntu:22.04`. Every other venue image is digest-pinned; this one was missed.
It matters more here than the checkbox suggests -- that image IS the glibc-floor
assertion, the oldest userland we claim the portable binary runs on. A floating
tag means the floor silently drifts to whatever 22.04 points at, and a floor
that moves is not a floor.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-29 17:04:13 +02:00
Martin Vogel a54ea95719 fix(windows): ship one binary — remove the launcher stub flagged as a dropper
DCO / dco (push) Has been cancelled
Windows shipped a PAIR: a small permanent launcher (codebase-memory-mcp.exe)
plus the real product binary (codebase-memory-mcp.payload.exe). The launcher
existed for exactly one reason — a running .exe cannot replace its own image
on Windows, so an in-process self-update needs a second resident binary to do
the swap.

That stub is statically indistinguishable from a dropper: a small, unsigned,
zero-prevalence PE whose whole job is verify-and-execute another binary.
Defender's ML scored it Trojan:Win32/Wacatac.B!ml and blocked the v0.9.1-rc.1
release at the VirusTotal gate. It is not fixable in our code on x64 —
bcrypt-free, stripped, VERSIONINFO'd, minimal-resource and even
resource-FREE builds on CI's own MSYS2 CLANG64 toolchain were all flagged,
while the product binary scans clean on every platform.

So remove the stub and move self-update OUT of the process into install.ps1,
which runs while cbm is NOT running: Windows' image lock only blocks a
process from replacing ITSELF.  now prints the exact PowerShell
command (with the Unblock-File hint for Mark-of-the-Web); install.ps1 is
idempotent, so re-running it IS the update — it stops the daemon, renames the
running binary aside (the one mutation Windows permits on a running image),
publishes the new one, and sweeps retired copies.

Windows now matches Linux and macOS: ONE binary per platform.

  * packaging, install.ps1, npm and PyPI wrappers all carry a single binary
  * the launcher/payload ABI contract and ~2500 lines of stub state machinery
    are deleted
  * every daemon start, CLI call and hook fire loses a process spawn, a named
    pipe handshake and an stdio relay
  * test_windows_bundle_contract.sh is rewritten as an INVERTED contract: it
    now asserts no shipped surface can reintroduce a launcher/payload pair,
    and that install.ps1 retires the running binary before publishing

Verified: VirusTotal 0/67 on the packaged binary and 0/58 on install.ps1 (no
certificate involved); macOS and Linux full suites green; Windows guards all
green including the new update-handoff contract; npm 10/10; PyPI 3/3.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-28 17:03:01 +02:00
Martin Vogel 0c0111eb33 fix(smoke): header-only fixture-server readiness polls
Hosted Windows runners reset the readiness poll's full-archive GET
mid-body (WinError 10054 on the server, ~99 aborted transfers per
job) while the identical stack passes on the VM; every windows smoke
job died at 'fixture server did not serve' with a healthy server. A
HEAD proves the artifact routes without transferring the body per
poll; the download phases still own full-body transport and report
phase-precise if that is ever broken.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-27 04:45:13 +02:00
Martin Vogel 6655b858ea ci(local): curl in the Alpine smoke image
DCO / dco (push) Has been cancelled
The unified smoke-local.sh readiness-polls the fixture server with
curl; the Alpine image never had it, so every poll iteration failed
command-not-found and the portable smoke leg reported 'fixture server
did not serve' with a perfectly healthy server. Local-only image (CI
portable smokes run on ubuntu runners, curl preinstalled).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-27 03:32:25 +02:00
Martin Vogel c400dc1665 ci: hosted runners cannot enable Defender RTP — revert runner-side gating
Dry run 30225685743 failed at the ensure-defender step on every
Windows job: the hosted images policy-lock real-time protection OFF
(WinDefend starts, Set-MpPreference accepts, RTP stays off —
untamperable from the job).

Revert to the pre-gate runner behavior: no ensure-defender step in
runner jobs, the release Defender scan is on-demand and fail-soft
again. The local VM preflight KEEPS the fail-closed Defender-ON gate
(same canonical script) — AV-interaction coverage is a deliberate
local superset where the platform allows it.

The venue-parity contract now enforces the INVERSE for runners
(ensure-defender must not gate hosted jobs) and still requires the VM
preflight gate; VM README documents the verified posture delta.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-27 01:56:33 +02:00
Martin Vogel 0a40fe8bea feat(ci): local-remote parity lanes — artifact-flow smoke, glibc floor, Defender-ON everywhere
Closes the closable local-vs-remote venue gaps so that, outside arch
legs / YAML glue / release plumbing, every local red is a remote red
and vice versa on shared legs:

- scripts/package-release.sh: THE canonical archive step (names +
  five-file Windows bundle layout defined once); _build.yml's eight
  inline archive blocks become calls to it.
- scripts/ci/smoke-artifact.sh: the artifact-flow smoke lane — build,
  package, extract, then the canonical wrapper in artifact mode; wired
  as run.sh smoke-artifact (compose service), win.sh smoke-artifact,
  and directly runnable on macOS. Archive-layout bugs now surface
  locally instead of in a release dry run.
- glibc-floor leg (Dockerfile.glibc22 + compose + run.sh): portable
  binary smokes on ubuntu-22.04/glibc 2.35; the dynamic binary must
  refuse there (2.38+ floor by design).
- Defender-ON parity (user directive): scripts/ci/ensure-defender.ps1
  enables + VERIFIES real-time protection, fail-closed; runs in every
  Windows CI job (_test x2, _soak x3, _smoke, pr.yml) AND in the VM
  preflight; _smoke.yml's scan engine-failure soft-skip becomes a red
  gate. Expected cost: slower Windows jobs (AV scanning during
  install/build/test I/O); the next dry run proves the runner side.
- Contracts: launcher-bundle five-file check retargeted onto
  package-release.sh + per-archive canonical-call association;
  venue-parity contract requires the new lanes, counts one
  ensure-defender step per Windows job, adds --help probes for the new
  entries. Extended contract fails on the pre-change tree (verified:
  20 violations on HEAD).
- VM README: ephemerality/Defender posture documented — utmctl has no
  snapshot verb, so per-run revert stays a manual qcow2 option; the
  sweep preflight remains the standing mechanism.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-26 22:37:48 +02:00
Martin Vogel 3fc93d12da ci: unify smoke/soak/test venues onto canonical leg entries
The staged venue-unification bundle: _smoke.yml onto the shared
wrappers with an extracted-artifact input (CBM_SMOKE_ARTIFACT_DIR),
soak-legs.sh as the one canonical soak entry (quick + query-leak legs
across all venues), protected per-user TEMP roots shared across
venues, clean-disk preflights before every VM/Docker run, win.sh and
run.sh routed through the canonical scripts, soak.yml retired, and
the venue-parity contract (whitelist walker: workflows may provision
or call canonical entries, nothing else).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-26 22:23:05 +02:00
Martin Vogel 636dd3f92c fix(test-infra): repair the native Windows soak transport and release checks
The native Windows soak could never start its server, so that gate had
never actually measured anything. The root cause was the harness, not the
daemon: an MSYS filesystem FIFO feeding a native Windows process delivers
empty stdin (reproduced 5/5), while a Bash anonymous coprocess carries the
full JSON-RPC initialize exchange to the same protected payload, and the
failing daemon log showed an orderly client disconnect rather than a
daemon-side eviction. The soak now uses a named coprocess on native
Windows only, duplicating its endpoints to stable fd3/fd4 and closing the
originals so closing fd3 still delivers EOF; POSIX keeps the FIFO path.
The coproc syntax sits inside eval because macOS system Bash 3.2 must
still parse this file even though only MSYS2 Bash 5 executes that branch.

With transport repaired the next exact failure surfaced: native Windows
Python cannot open an MSYS /c/... diagnostics path. Both consumers now
pipe the file through stdin, the pattern already established elsewhere in
the repo, and the soak recovery contract forbids reintroducing a direct
native-Python open.

Also on the release path:

- The smoke fixture server no longer fsyncs before atomically publishing
  its port. This is ephemeral readiness signalling, not crash-durable
  state, and the macOS Intel runner failed inside that durability sync.
  When the contract does fail it now reports the observable state --
  waited, exit status, port file, staged temp files, interpreter, startup
  log -- because the previous verdict named nothing on the one runner we
  cannot reproduce locally.
- The POSIX publication test hook no longer compiles into Windows builds,
  fixing an exact -Werror unused-variable failure. The setter keeps a
  parameter-consuming Windows stub because it is public API.
- The Wine leg assembles the real release layout (payload plus canonical
  launcher) and version-checks both, running the launcher through cmd so
  it has a Windows-visible parent. The unsupported Wine soak is removed:
  Wine stays a fast compile/package/version check, and native Windows is
  authoritative for daemon, locking, ACL and process-lifetime semantics.
- run.sh soak-windows routes to the native Windows VM, which validates the
  payload, builds the protected per-user temp root, stamps ACLs, and
  refuses success without a completion summary.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-25 00:37:05 +02:00
Martin Vogel 62821375c4 fix: harden daemon release paths and verification
DCO / dco (push) Has been cancelled
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-24 16:08:42 +02:00
Martin Vogel 1f50c9d46c build(test-infra): pin container base images and package downloads by hash
Resolves the two open Pinned-Dependencies code-scanning alerts (and one
sibling the scanner had not flagged yet):

- ubuntu:noble in Dockerfile and Dockerfile.lint is pinned to its
  multi-arch manifest-list digest (verified against the registry and the
  scanner's remediation digest, which match).
- mstorsjo/llvm-mingw drops the floating :latest and pins the current
  digest the same way.
- The MSYS2 zlib sysroot package in Dockerfile.mingw is verified against
  a pinned sha256 before extraction instead of piping the download
  straight into tar.

Dockerfile.alpine was already pinned; both rebuilt images (test, lint)
verified building with the new digests.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-23 17:49:41 +02:00
Martin Vogel 568e613141 test-infra: overlap the local ladder, open the serial tail, keep container I/O off virtiofs
Local runs left most of the machine idle: the serial tail ran sixteen
suites one at a time on otherwise-idle cores, the three platform legs
were launched by hand (usually sequentially), and every container-leg
object write crossed the virtiofs bind mount.

- run-tests-parallel.sh tail scheduling in two phases: the FLEX suites
  (timing-shaped but free of the shared per-account daemon runtime
  namespace) run CBM_TAIL_JOBS-wide (default 2), then the EXCL group —
  daemon-family plus the suites that drive daemon one-shots or
  supervisor rendezvous — runs strictly sequentially on a machine
  exactly as quiet as the old fully-serial tail gave it. The wave was
  already fed longest-first by the shard dealing order, so the drain-out
  no longer ends on a heavy straggler.
- ladder.sh: one maintained entry point for the full local push gate
  with the legs overlapped — lint, the Linux container suite, and the
  Windows VM suite in the background, the macOS suite in the
  foreground, one verdict per leg, logs kept per leg. A missing
  prerequisite fails its leg loudly instead of silently skipping.
- win.sh test-par now runs through vm-run-tests.sh (--par mode): the
  full parallel harness under the CI-shaped protected temp root with
  complete output. It previously ran under the MSYS-shared /tmp and
  piped through `tail -25` — the same truncated-blindness class that
  hid 40 Windows failures from the `test` command.
- docker-compose: build artifacts and the incremental fixture cache
  move to named volumes on the container VM's native filesystem. Object
  writes over the virtiofs bind mount are the container legs' largest
  avoidable I/O cost, and the fixture cache now survives across
  container runs.
- test_mem(win): the first valid full-parallel VM run proved working-set
  trimming beats the re-touch mitigation (19 MB resident of a 256 MB
  double-touch at 18 parallel suites). The RSS probe now VirtualLocks a
  64 MB span — locked pages are exempt from trimming, making the
  measurement pressure-immune — with bounded touch-and-sample retries
  when the lock is unavailable. Red-to-green under the same 18-job load.
- cli: the portable install's staging error now appends the activation
  refusal note (predicate, SID, object) like the managed path already
  does — a bare "activation transaction I/O failed" on a CI-only
  failure is undiagnosable without it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-23 01:33:58 +02:00
Martin Vogel 664058e86b test-infra(win): CI-shaped VM harness, managed-install smoke gate, calibrated suite tier
The 40 hidden Windows failures fixed in the previous commit reached CI
unseen because the local Windows leg never validly ran the C suite:
MSYS2's shared /tmp fails the secure-ancestry validation (refusals, not
signal) and the ssh driver piped everything through `tail -40`. This
closes those holes and hardens the gates that let an install failure
scroll past.

- vm-run-tests.sh (new, used by win.sh test / ubsan-test /
  trap-ubsan-test): gives the suites CI's exact protected per-user temp
  root (owner-stamped, protected current-SID DACL, mirroring the
  workflow's pwsh step), shapes the runner's build directory like a real
  user checkout (drive-root trees inherit an Authenticated-Users Modify
  ACE that profile-rooted checkouts do not have, which the activation
  source-directory policy correctly refuses; DACL re-rooted two-step —
  directory protect then child /reset, since inheritance flags are
  directory-only and a /T re-root leaves files with empty deny-all
  DACLs), streams the FULL output, and refuses to report success unless
  the runner printed its completion summary.
- smoke: Phase 8 no longer swallows the install exit code. On Windows it
  now requires the managed install to SUCCEED and to publish an exact
  two-link canonical launcher — a staging refusal used to scroll past as
  tolerated noise while the downstream config assertions kept passing.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-23 00:53:51 +02:00
Martin Vogel ece838c525 test: widen ThreadSanitizer coverage and add native ARM64 Windows UBSan
Close the cross-platform sanitizer gaps that were leaving real
concurrency and undefined-behavior bugs uncaught, and fix a data race
the first widened run surfaced.

- tsan: the data-race gate ran three suites (mem, slab_alloc, parallel)
  over no real threaded production code. It now covers every threaded
  surface that runs clean and stable under TSan: the parallel-extraction
  worker pool (parallel, worker_pool, pipeline), the filesystem watcher,
  the embedded HTTP server (httpd), diagnostics sampling, the MCP server
  and mutation guard, subprocess supervision, and the runnable
  daemon-coordination paths (daemon, daemon_application). daemon_runtime
  (deadlocks under TSan+fork), daemon_ipc and daemon_frontend
  (test-harness synchronization, not production) are excluded with the
  reasons recorded in the Makefile.
- tsan: fixed a genuine data race the widened gate immediately found —
  cbm_lsp_max_walk_depth's lazy cache was read and written by parallel
  LSP-extraction workers without synchronization. A data race is
  undefined behavior even when every worker computes the same value, so
  the cache slot is now a relaxed atomic: a plain load on the hot path,
  and a first-touch double-compute simply stores the same value.
- tsan(ci + local): the test-tsan job now runs on Linux amd64, Linux
  arm64, AND native ARM64 macOS (the threading code is shared, so a race
  is usually caught on all three, but scheduler differences let each
  surface one the others miss). The local ladder gained `run.sh tsan`
  and `tsan-amd64` plus the matching compose services. TSan's shadow
  memory aborts under modern high-entropy ASLR, so the containers run
  under `setarch -R` with an unconfined seccomp profile (the personality
  syscall is otherwise blocked) and the CI Linux legs lower
  vm.mmap_rnd_bits first; amd64 TSan cannot run under x86_64-on-ARM
  translation and is a real-hardware/CI gate only (documented in
  run.sh).
- ubsan(win/arm64): native ARM64 Windows had no sanitizer at all —
  AddressSanitizer ships no aarch64-w64-windows-gnu runtime. UBSan in
  trap mode (-fsanitize-trap=undefined) needs no runtime library, so it
  instruments natively and turns undefined behavior into an
  illegal-instruction trap; -fstack-protector-strong adds stack-smash
  coverage the heap tools miss. The GitHub windows-11-arm leg switches
  from unsanitized to this, and vm/win.sh gains trap-ubsan-build /
  trap-ubsan-test for local iteration (reproduce under the emulated
  x86_64 UBSan to see which check fired). The whole codebase builds and
  runs clean under it.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-22 17:48:54 +02:00
Martin Vogel 30550fbf75 test-infra(vm): restore the executable bit on win.sh lost in a merge resolution
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-21 15:32:42 +02:00
Martin Vogel 0baddf7887 daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions
Daemon lifecycle and Windows correctness, verified on a real Windows 11
ARM64 VM through the maintained test-infrastructure/vm drivers, plus the
macOS and Linux arm64 suites and the container lint gate.

Daemon lifecycle:

- daemon start/stop/status subcommands. `daemon start` launches a
  PERMANENT daemon (spawn shape is byte-exact argv; survives idle
  periods and session ends) and reports an already-active daemon
  instead of failing. Permanence is honored at every stop latch:
  last-committed-client disconnect, host initial-client window,
  coordinator release, and application final-session close — a
  permanent daemon also keeps admitting new sessions after its last
  one closes.
- daemon stop refuses while sessions are active and lists the blocking
  peers (pid/role) that must finish first; an idle daemon drains
  through the activation-shutdown machinery with the ACK ordered after
  connection interrupts. A second stop is idempotent. The wire ops are
  no-cohort first-frame requests with peer fingerprint authentication,
  so stop/status never conflict with an exact-build admission gate.
- One-shot CLI commands now execute through the daemon (index workers
  keep their local supervised path). A cold CLI run that had to spawn a
  temporary daemon prints a hint that `daemon start` removes the
  per-command startup tax; a warm daemon is recycled silently.
- Hooks are connect-only fail-open: with no daemon present the hook
  emits a visible, rate-limited notice (Claude-dialect systemMessage
  plus stderr for other dialects) and always exits 0 — augmentation is
  never allowed to block the caller's tool use.
- Version skew: a newer-build client automatically drains an
  older-build permanent daemon (strict semantic-version triples only;
  dev builds never auto-drain) and the build-conflict message names
  `cbm daemon stop` as the manual escape hatch.

Windows IPC/runtime (real-VM verified):

- ipc(win): persistent pending overlapped ConnectNamedPipe. The accept
  path used to destroy its listening pipe instance on every 20 ms poll
  timeout; a client attaching in the teardown window was severed or left
  on an orphaned pipe object whose HELLO no server handle could ever
  read, absorbing the connect until the client's own timeout expired.
  The pending connect now survives poll timeouts and nothing is
  destroyed while a client could be attaching.
- ipc(win): drain-before-close for final responses. Closing a named-pipe
  server handle can discard a just-sent response before the peer reads
  it (POSIX stream sockets never lose buffered data on close). A bounded
  cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX,
  immediate on interrupted connections) now precedes close in
  runtime_worker_finish and runtime_reject_inline, so hello-conflict,
  capacity and disconnect acknowledgements reliably reach the peer.
- runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no
  transport half-close, so close_begin now announces departure with an
  explicit frame (ordered after APPLICATION_CANCEL, before the local
  interrupt); the server releases the client's admission on receipt
  instead of waiting for the handle to close. Admission-drop timing is
  now identical to POSIX shutdown() semantics on every platform.
- runtime(win): client close cancellation. close_begin serializes with
  request publication under the send lock, best-effort sends the active
  token's APPLICATION_CANCEL frame, then interrupts local I/O; the
  server cancels MCP/subprocess work promptly. Contract tests accept
  both correct outcomes (interrupted transport or decoded CANCELLED).
- runtime: activation acknowledgement ordering. The activation ACK is
  the requester's license to act on "snapshotted and draining", so every
  connection interrupt is now initiated before the ACK is sent; a
  session could previously get one more request serviced after the
  requester observed the ACK.
- service(win): deadline-bounded private-file prepare. The conflict-log
  prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms
  timer granularity) burned ~1.6 s against permanently obstructed paths,
  stalling hello rejections past the client's timeout. The retry budget
  is now a 250 ms deadline; transient share collisions still retry.
- subprocess(win): cmd.exe /C payload encoder quotes metacharacters
  correctly (root cause of the git-on-Windows failure cluster).
- watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native
  Windows stop/unwatch cancellation test with exact-image verification.
- httpd: send_all writes in bounded 64 KiB slices. A single giant
  nonblocking send() on Windows is absorbed wholesale into AFD kernel
  buffering regardless of SO_SNDBUF, so send deadlines and interrupts
  could never engage against a slow peer (and the full payload was
  pinned in nonpaged pool). Slicing restores a deterministic
  backpressure point; a test hook pins SO_SNDBUF for the deadline and
  interrupt tests.
- ui/http: shutdown lifecycle — interrupt checks, response-wide send
  deadline, explicit connection states, refusal to free a server while
  a listener-owned connection is active.

Windows long-path support:

- Central path-aware wide conversion (canonicalize via GetFullPathNameW
  and prepend the extended-length prefix for absolute paths >=240) at
  the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite
  store opens, and the daemon build-fingerprint/log paths. Deep managed
  installs (a 64-hex generation directory routinely exceeds MAX_PATH)
  now index, stage and activate correctly.
- activation transaction: its own file APIs and the component-walking
  ancestry validators now operate in the extended-length namespace;
  the launcher path is canonicalized (and prefixed when deep) once at
  entry so every downstream exact-string comparison stays
  form-consistent.
- Executable self-resolution uses the wide APIs (GetModuleFileNameW,
  GetFileAttributesW) so non-ASCII install paths survive argv[0]
  resolution.

Windows launcher install/uninstall transaction:

- FileRenameInfoEx names are NUL-terminated in an over-allocated
  buffer. FileNameLength governs per the contract, but filter drivers
  read FileName as NUL-terminated and appended adjacent heap bytes to
  created names — a flaky, garbage-suffixed rename target. Both the CLI
  and the launcher rename helpers are fixed.
- Uninstall retires state via rename-aside (.cbm ->
  .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex
  chars so the bare rename target stays under the FileRenameInfoEx
  NT-conversion ceiling at guard depths; 64 bits still uniquely
  identify the generation.
- When the running launcher's mapped generation backings pin .cbm
  against rename, the backings are relocated to activation-<pid>-N
  .retired tombstones beside the install (a mapped image may be renamed,
  never deleted; the launcher's liveness-guarded sweep reclaims stale
  tombstones). Every relocation is recorded, and a FAILED uninstall
  reverses the moves after restoring .cbm — via MoveFileExW with
  extended-length paths on both arguments, since the deep generation
  target is beyond the handle-based rename's bare-path reach — so a
  restored install keeps its generation backings and stays runnable.
- After a committed uninstall the retired tree's backings are relocated
  out so the tree is shallow enough for the detached cleanup's rd, and
  the cleanup's working directory strips the extended-length prefix
  (CreateProcessW lpCurrentDirectory silently ignores prefixed paths).
- Files created under Administrators-default-owner directories
  (CopyFileW destinations, CREATE_NEW tombstones, probe directories)
  are explicitly owner-stamped so the exact-owner validators hold on
  runner images; guard fixtures stamp hand-built trees the same way.

Diagnostics, tests and infra:

- diagnostics: discovery is now an always-delivered JSON control record
  (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and
  paths containing spaces; placement honors $TMPDIR with /tmp fallback
  via a diagnostics-local helper; the soak parser reads the JSON record;
  documented in docs/CONFIGURATION.md. Red-first coverage for suppressed
  log levels, TMPDIR-with-spaces, and native Windows output-contract
  assertions.
- tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint
  parents with production-shaped ancestry (LocalAppData on Windows, via
  th_secure_runtime_parent_new) — the runtime ancestry validation
  correctly refuses temp roots whose ancestors grant mutation rights to
  Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and
  drive the documented startup-owner publication flow before reading
  generation-bound endpoint addresses. This turns the 26 Windows
  failures previously visible in CI's full-test job green without
  weakening any validation.
- tests(win): the launcher guard covers the full permanent-launcher
  contract including failed-uninstall restore and immediate reinstall
  after uninstall; new daemon lifecycle and reworked hook-augment
  guards run the start/recycle/stop flow end to end.
- tests: CBM_SKIP_PERF is now actually consumed by the test runner
  (it was set by CI but never read, so perf suites ran everywhere);
  four throughput/bench suites are classified as perf, the heavy
  store_arch suite moved to the slow-timeout tier, and two
  wall-clock-sensitive assertions were rewritten as invariant checks
  with coarse hang-detector backstops.
- build/test infra: build-dir safety contract, UI dev-proxy security
  contract, soak daemon-recovery contract, path-safety helper, the
  Windows VM worktree-sync contract wired into scripts/test.sh, and
  vm/win.sh guards building its clean embedded-UI product in an
  isolated BUILD_DIR so it cannot clobber the incremental test build.
  provision-windows.sh now installs Node.js for the guards UI build.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-21 15:32:15 +02:00
Martin Vogel d826374577 test-infra(vm): remove embedded SSH key, pin host identity and installer
The Windows-VM bootstrap embedded a personal SSH public key and the
drivers connected with StrictHostKeyChecking=no. Key material and host
identity are now strictly operator-supplied and live outside the
repository (~/.claude/cbm-vm/, never committed):

- windows-bootstrap.ps1 takes the operator's public key as a mandatory
  -SshPublicKeyPath argument (validated single Ed25519 line) and prints
  the VM host-key fingerprint to pin. Generating your own key is
  documented in the script and README:
  ssh-keygen -t ed25519 -N "" -C <label> -f ~/.claude/cbm-vm/id_ed25519
- win.sh and provision-windows.sh verify the pinned
  CBM_VM_HOST_KEY_SHA256 fingerprint before every connection
  (StrictHostKeyChecking=yes against a keyscanned known_hosts, Ed25519
  only, IdentitiesOnly), validate branch names before remote
  interpolation, and fail remote pipelines loudly (set -e -o pipefail).
- The MSYS2 installer download is pinned to a versioned release URL with
  SHA-256 verification instead of an unpinned latest.
- win.sh gains a sync subcommand mirroring the uncommitted worktree onto
  the VM; its manifest logic is guarded by
  tests/test_vm_worktree_manifest.sh, wired into scripts/test.sh.
  guards now runs the maintained PowerShell driver against a clean
  product build; smoke-install drives vm-smoke.sh.
- run.sh drops the stale duplicate windows-vm/mac-vm legs (the former
  carried its own unpinned ssh path) in favor of the vm/ drivers.

The previously committed public key has been rotated off the VM and is
no longer authorized anywhere.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 23:21:53 +02:00
Martin Vogel 08b7ff54d2 test-infra(vm): CI-identical smoke plus ubsan/pageheap/parallel drivers
Add vm-smoke.sh, which stages the release pair (launcher + payload) in a
profile-rooted temp dir and runs scripts/smoke-test.sh exactly as the CI
windows pr-smoke step does, so the real-Windows leg exercises the same
managed-install path. Extend win.sh with ubsan-build/ubsan-test (UBSan works
under x64-on-ARM emulation; ASan does not), a PageHeap toggle for OS-level
heap checking of the native runner, and a full-suite parallel run. Document
the honest sanitizer limits and the daily loop in the README.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 16:56:02 +02:00
Martin Vogel 593d126813 test-infra: give each leg its own build dir so parallel legs never collide
scripts/build.sh, test.sh, and clean.sh hardcoded build/c, and clean.sh
rm -rf'd it unconditionally. Running the container legs concurrently with a
native macOS build then had them delete and relink each other's objects
mid-build (ld "file cannot be opened"). Thread a BUILD_DIR through the three
scripts (default build/c) and point every compose service at its own dir
(build/linux-arm64, build/linux-amd64, build/win-cross, build/alpine); native
macOS keeps build/c exclusively. Also run the lint service in CI-parity mode
(--ci: cppcheck + clang-format only), matching CI, since clang-tidy is in
neither the lint image nor the CI job.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 16:56:02 +02:00
Martin Vogel aa90491792 fix(cli): stage Windows install transactions under a per-user root
The activation transaction and the release-descriptor probe validate the
full ancestor chain of everything they stage: every path component must
have a trusted owner and grant mutation rights only to trusted identities.
That walk is deliberate TOCTOU defense and stays strict. The staging
directory, however, came from the process temp root, which routinely lives
in a tree that legitimately fails the walk through no fault of ours: under
msys2 TMP resolves inside the msys install tree (C:\msys64\tmp), and on
GitHub runners inside the runner work directory, both of which carry broad
inherited grants such as Authenticated Users modify on an upper component.
The staged pair then failed validation ("launcher descriptor candidate is
unsafe" / the smoke Phase 8 --version probe refusal) even though the staged
files themselves were private.

Add cli_windows_staging_root(): stage install and update transactions under
%LOCALAPPDATA%\codebase-memory-mcp, whose ancestor chain is user-owned end
to end, so the strict walk passes by construction instead of by environment
luck. Falls back to USERPROFILE\AppData\Local and finally the process temp
root when the environment is degraded.

Also make the staging observable where it previously failed blind:
cli_windows_stage_private_file reports which transaction step failed with
the OS error code, and cbm_mkdtemp warns once if the private security
descriptor cannot be applied and it falls back to default directory
security. A blind boolean here cost several blind CI round-trips.

win.sh: scp targets need native C:/ paths on Windows OpenSSH.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 14:32:05 +02:00
Martin Vogel 343582aa93 test-infra(vm): script-driven Windows VM leg for the local 3-OS gate
The Windows VM becomes a first-class, reproducible local-CI leg driven
entirely over the local ssh link by maintained scripts:

- provision-windows.sh: idempotent host-side provisioner — msys2 base,
  CLANGARM64 (native, fast) + CLANG64 (CI arch) toolchains, repo
  clone/update, full build, smoke check. One command recovers a fresh VM
  or a lost disk.
- win.sh: daily driver — status/update/build/test/guards/smoke-install/
  sh/push-file/asan-build. smoke-install runs the managed-install E2E
  with stderr visible, the class the CI smoke probe structurally hides.
- windows-bootstrap.ps1: reduced to the one-time in-VM door-opener
  (OpenSSH + host key + GitHub-runner default-owner policy mirror), with
  the ISO delivery trick documented; toolchain work moved to the
  host-side provisioner. The previous winget path does not work over ssh
  and clipboard/shared folders do not exist on Windows-ARM guests.
- README.md: full lifecycle — VM creation, clone-before-drivers rollback
  point, SPICE guest tools, bootstrap, provisioning, daily loop, and the
  honest limits (no ASan on aarch64-windows; ASan also faults under
  x64-on-ARM emulation, so Windows sanitizer coverage stays on CI).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 14:14:11 +02:00
Martin Vogel 53d867dfc1 test-infra: per-suite timeout + arm64-safe mingw image (reliability)
Two reliability defects that bit this session: a wedged suite could
hang the runner (and the single local build slot) forever, and the
mingw image failed to build on an arm64 Docker host because its i386
Wine packages are unsatisfiable.

- run-tests-parallel.sh now caps each suite with `timeout --kill-after`
  (900s default, 3600s for the legitimately-slow incremental suite),
  where `timeout` exists; a killed suite fails loudly instead of
  blocking. Hosts without `timeout` run uncapped as before.
- Dockerfile.mingw makes Wine best-effort: the cross-COMPILE toolchain
  builds on any arch, and only the .exe-execution step needs Wine, so a
  Wine-unavailable host still gets a working compile check.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-19 00:42:19 +02:00
Martin Vogel a6a17e3a24 test-infra: fetch the Windows ISO from Microsoft's official page
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 20:57:43 +02:00
Martin Vogel 43b3ae722d test-infra: Windows VM gets all cores - no artificial limits
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:53:57 +02:00
Martin Vogel d09867d096 test-infra: size the Windows VM generously - vCPUs are scheduled, not pinned
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:47:58 +02:00
Martin Vogel 37956940da feat(test-infra): opt-in windows-vm and mac-vm legs
Two failure classes stay invisible to the container legs: kernel-real
Windows ACL/token/owner semantics (Wine only compile-checks) and the
GitHub-runner macOS environment (a developer Mac differs in temp
ancestry, ACLs, and policies - runner-only failures never reproduce
locally). Add real-VM legs for both, strictly opt-in: run.sh detects
the tooling and the named VM and prints setup guidance with a distinct
exit code when absent - explicitly requested legs are never silently
skipped.

windows-vm: UTM + CrystalFetch (both free), driven over ssh; the
in-VM bootstrap (vm/windows-bootstrap.ps1) installs the CLANG64
toolchain, enables sshd, and mirrors the runner's
NoDefaultAdminOwner=0 policy so freshly created objects are owned by
BUILTIN\Administrators - the exact class the daemon's owner gates must
survive.

mac-vm: Cirrus Labs' runner-equivalent image via tart (free OSS),
booted headless, worktree synced over key-based ssh, scripts/test.sh
executed in the runner-class environment.

vm/README.md documents both setups end to end.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:46:02 +02:00
Martin Vogel 886825d9f5 test-infra: clarify vCPU semantics - shared scheduling, no core seizure
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:31:02 +02:00
Martin Vogel 15320024ea test-infra: all-cores VM sizing default + live-monitoring guidance
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:29:55 +02:00
Martin Vogel 7a49df448f test-infra: preflight daemon/compose checks + Colima runtime docs
The infra previously assumed Docker Desktop: an implicitly running
daemon and bundled CLI plugins. On the Colima runtime (free OSS,
CLI-first) neither is implicit, and a missing daemon surfaced as
cryptic compose errors. run.sh now fails fast with the exact setup
commands (colima start with --vz-rosetta for fast amd64 legs, plugin
symlinks for the brew docker client) and documents the runtime in the
header.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:27:54 +02:00
Martin Vogel 1445dd4dc8 test-infra: align run.sh docs with the speed-first default
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:23:41 +02:00
Martin Vogel 9450a51f77 test-infra: default to full cores; CBM_LOCAL_CI_CPUS=4 is the CI-fidelity mode
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:23:23 +02:00
Martin Vogel f71b34fd28 feat(test-infra): CI-fidelity CPU caps + verified ccache for the local legs
The local containers ran with every host core, which masks the exact
failure class the 4-core GitHub runners catch: deadline/starvation
misses in worker-readiness waits and watcher/subprocess deadlines. The
test services now default to a 4-CPU cap (CBM_LOCAL_CI_CPUS overrides,
0 = unconstrained), so a local run exercises the same scheduling
pressure as CI before CI ever sees the push.

Compilation is cached through ccache in persistent named volumes with
CCACHE_COMPILERCHECK=content: every hit is keyed on the compiler-binary
content plus the fully preprocessed input, so a stale or foreign entry
can only miss, never return wrong output.

Also documents the Wine leg's honest limit: it compile-checks Windows
but cannot reproduce real ACL/token/owner semantics.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-18 18:13:23 +02:00
Martin Vogel 50392a4188 fix(scripts): use /usr/bin/env bash shebangs (NixOS has no /bin/bash)
On NixOS (and other non-FHS systems) /bin/bash does not exist, so
scripts with an absolute shebang fail to run. Switch the remaining
holdouts to /usr/bin/env bash: eleven scripts/*.sh,
test-infrastructure/run.sh, and the three Claude Code hook scripts
emitted by src/cli/cli.c (gate, session reminder, subagent reminder).

Distilled from PR #674, with parser-test coverage preserved: the
infra_parse_shell* fixtures in tests/test_pipeline.c intentionally keep
#!/bin/bash so absolute-path shebang extraction stays covered, and
tests/repro fixtures are untouched.

Also replace the GitHub-PAT-shaped fixture string flagged in the #674
thread with an obviously fake placeholder (ghp_FAKE...) that still
matches the ghp_ + 36-alnum secret detector.

Co-authored-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-07-03 20:11:49 +02:00
Martin Vogel 43cde32d3d Fix CodeQL alerts: pin Alpine image digest, scope release permissions
- Pin Dockerfile.alpine to SHA digest (PinnedDependenciesID)
- Narrow workflow-level permissions to contents:read (TokenPermissionsID)
- Elevated permissions only on release-draft and verify jobs
2026-04-02 22:42:44 +02:00
Martin Vogel 9662063090 Add Alpine portable as 4th local test platform, fix musl compat
Local test infra now covers 4 platforms:
- Linux arm64 (Ubuntu, ASan+LeakSan)
- Linux portable (Alpine musl, static binary)
- Windows (mingw cross-compile)
- macOS (native)

Fixes:
- Remove sys/unistd.h and sys/poll.h (glibc-only, musl lacks them)
- Add STATIC=1 support to Makefile (appends -static to LDFLAGS)
- CI portable build uses docker run alpine instead of container directive
  (GitHub Actions JS actions break in Alpine containers on ARM64)
2026-04-02 20:28:20 +02:00
Martin Vogel 1b84943000 Separate perf tests from CI, fix cross-platform build issues
- Add CBM_SKIP_PERF=1 env var to skip incremental/perf test suite
- CI and Docker test targets skip perf by default (run.sh perf for manual)
- Convert all perf assertions to warnings (log timing, never block)
- Fix store.h anonymous enum in struct (GCC rejects, clang accepts)
- Fix test_store_search.c mkstemp on non-template path
- Add ca-certificates to Docker test image for git HTTPS
- Add cbm_gmtime_r shim in compat.h (Windows gmtime_s wrapper)
- Fix compat.c missing constants.h include (Windows build)
- Fix platform.c _environ redeclaration on mingw
- Rename trace_call_path -> trace_path in smoke/soak/fuzz scripts
2026-04-02 14:52:14 +02:00
Martin Vogel 988d975813 Fix uninstall .exe path on Windows + add Windows smoke/soak to local test infra 2026-03-26 00:41:40 +01:00
Martin Vogel e6e9c58e80 Fix all memory leaks + Docker test infra gaps
Leaks fixed (5506 bytes → 0):
- sqlite3_open_v2 allocates handle even on CANTOPEN — must call
  sqlite3_close before freeing struct (4x 792 bytes, 32 indirect)
- graph_buffer node_by_id: reuse existing heap key on hash table
  replace to avoid leak (2 bytes)

Docker test infrastructure:
- python3 (full) instead of python3-minimal (missing json module)
- git installed (19 tests were silently skipped)
- sqlite3_close_v2 + sqlite3_shutdown at test exit
2026-03-25 01:59:36 +01:00
Martin Vogel 20814e8b27 Add smoke tests to Docker test infra + CI pipeline
Expand smoke-test.sh with Phase 5 (MCP stdio transport), Phase 6
(CLI subcommands: install/uninstall/update --dry-run), and Phase 7
(MCP advanced tool calls: search_code v2, get_code_snippet).

Add smoke/smoke-amd64 services to Docker compose that build then
run all 7 smoke test phases. Include in run.sh full/all flows.

Add python3-minimal to Dockerfile for smoke test JSON parsing.
Fix Phase 4a shutdown test to use portable background+kill pattern
instead of `timeout` (not available on macOS).

Add --dry-run, --standard, --ui flags to update command. Fix
clang-tidy readability-implicit-bool-conversion in dry_run ternary.
2026-03-23 21:56:33 +01:00