fix/java-enum-methods
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Martin Vogel
|
8018561cfe |
fix(release): externalize runtime assets and harden VT verification
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
5efa57492a |
fix(daemon): answer an oversized MCP reply with an error, not process death (#1375)
A single query_graph reply larger than one 10 MiB frame killed the MCP server:
exit code 1, no JSON-RPC error, nothing on stderr, no crash report. Under a host
that does not respawn the server, every tool from it was gone for the rest of
the session — the reporter lost their tooling mid-investigation to one dead-code
query and finished by driving the binary over stdio by hand.
Root cause is a conflation, not the limit itself. application_mcp_request
checked only response_length > UINT32_MAX and passed anything smaller on, so an
oversized reply reached the transport, which rejected the frame. The frontend
worker cannot tell that rejection from a dead socket, and for a dead socket it
deliberately _Exit()s the process — closing the kernel IPC handle is the only
portable way to cancel daemon session ownership from a thread blocked in stdio.
Correct for a broken transport; a fatal over-reaction to a large answer.
The substitution therefore has to happen in application_mcp_request, which is
the last layer still holding the request id. Replies that fit are returned
byte-identical; a reply that cannot be framed becomes a JSON-RPC -32603 naming
both numbers and what to do ("narrow the projection, add LIMIT, or paginate"),
because the failure it replaces was a silent exit and an opaque "internal error"
would be no improvement.
Reproduced first, on a locally generated 20k-node fixture rather than the
reporter's repo (external code stays unverified until reviewed):
before LIMIT 10000 -> ok, 8,529,990 bytes
LIMIT 20000 -> SERVER DIED exit=1 stderr=''
after LIMIT 20000 -> JSON-RPC -32603, and a follow-up query on the SAME
session succeeds — which is the property that actually matters
The decision and the substitution live in one function on purpose. The first
version of the test built the error object directly and PASSED with the size
check removed: it proved the error was well-formed while the server still died.
Testing application_framable_response instead binds the branch — verified by
removing the guard, which reddens it at ASSERT(replaced != big).
Not addressed here, deliberately: every reply still carries its payload twice
(content[0].text and structuredContent.text are verbatim duplicates, ~2.05x
measured), which is why the ceiling is reached at ~4 MB of real content. That is
a wire-format question affecting every tool and every caller's token bill, so it
belongs in its own change with a decision behind it, not smuggled into a crash
fix.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
ae52db3147 |
test: make three Windows nondeterminisms deterministic
All three cost real release cycles, and none is fixed by widening a budget. daemon_application_cancels_physical_job_only_after_final_session waited for the SUBSCRIBER COUNT to reach 2, then cancelled both sessions and asserted the physical job had started exactly once. The job starts asynchronously after subscription, so both cancels could land first and leave starts == 0 -- arguably the correct outcome. It now waits for the state the assertions actually require. Verified 47/47 on Windows, the only platform it ever failed. The parallel harness refused outright when the suite leader had already exited, because taskkill /T cannot walk a tree from a dead PID. But the leader can exit between the timeout decision and that call, so the harness itself lost a race: a natural exit at the wrong moment failed the whole wave. It now proves cleanup the only way still available -- nothing parented to that PID -- and its contract asserts the PROPERTY rather than the phrase "tree cleanup" it used to grep for. That string pin is what broke when the guard was reworded while behaving correctly; the contract now checks rc==2 AND that the descendant really did survive, which would also catch a guard that claims to fail closed while leaking. extract_wide_flat_file_is_linear took ONE sample per size, so the ratio carried the noise of both. On a loaded Windows VM linear code measured 51x against a 40x bound (184ms -> 9387ms). Best-of-N instead: timing noise only ever adds time, so the minimum is the cheapest good estimate of the noise-free cost. The bound is deliberately unchanged -- it sits where linear (~20x) and quadratic (~128x) are each >=2x away, so raising it would move the test toward the very signal it exists to catch. Now measures 19.1x on Windows, 21.4x on macOS. Also: the smoke's `cli` helper redirected stderr to a file and discarded it, so any of the 10 bare `VAR=$(cli ...)` assignments could kill the run under `set -euo pipefail` printing NOTHING. One such abort cost a full Windows cycle just to locate and still could not be attributed. It now surfaces the command and its stderr. Neutral wording on purpose: one call site expects a non-zero exit and must not read as a failure. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
46b77c3e3b |
test(daemon-app): wait for the stable terminal state, not the draining window
DCO / dco (push) Has been cancelled
The fresh-request guard test waited for 'job terminal AND >= N/2 subscribers still attached'. Publish flips terminal and releases the blocked prior requests in the same locked step, so that conjunction is a transient window whose width is thread-wakeup scheduling: the busy-spin caught it by luck, a 1ms-yield poll (#1307) sampled past it on fast x64 runners, and no budget can pin it (release runs 30305464193 and 30309182389 failed it from both directions). The production guard never consults subscriber counts — application_find_active_job_locked skips any terminal job — and the test's downstream assertions (starts==2, destroys==2, stale/fresh response separation) catch a terminal-job reuse in every interleaving. So wait only for the stable end-state (active jobs == 0) and drop the racy helper. 47/47 locally. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
8268af4188 |
test(daemon-app): yield in the terminal-job wait like every sibling helper
DCO / dco (push) Has been cancelled
app_wait_for_terminal_job_with_subscribers was the only app_wait_for_* helper polling without cbm_usleep(1000): a sleepless spin pins a core for up to 10s and can starve the daemon threads it is polling on scarce-CPU runners. Release run 30305464193 hit exactly this on windows-11-arm (test_daemon_application.c:2671 ASSERT(terminal_with_prior_subscribers)); the suite passes 47/47 locally with the yield in place. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
87f7c0d784 |
daemon: deterministic job interleavings and queued explicit index admission
Two changes to the index coordinator, developed together and validated as one state: Determinism seams. A cancel racing a worker start was only observable on a loaded runner; now the interleaving is forced by construction: a test hold parks every job thread before its first pre-start cancel check, and a background-initialize completion counter turns sleep-then-assert negatives into positive-signal waits. The cancel-before-worker-start case is pinned by a dedicated test; three auto-index admission tests now wait for the admission tail instead of sleeping through it. Queued admission. An explicit index_repository request that hits the physical job limit used to surface a raw busy error. The request thread already blocks for the whole index, so it now queues: a cancellable, shutdown-aware wait-and-resubscribe loop replaces the error path, with a wait counter as the parked-state observable. The programmatic coordinator API keeps its non-blocking busy answer for callers with their own retry policy (watcher, auto-index). The job-limit test asserts the new contract: parked without a second worker start, admitted and completed once the occupying job finishes. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
2af28c9d48 |
test: size the daemon-application wait budget for TSan's slowdown tail
The suite's observation waits shared a 2000 ms budget that is a hang detector, not a latency assertion — every waiter returns the moment its condition holds, so green runs never pay it. Under ThreadSanitizer's 5-20x slowdown on a loaded 3-core macOS CI runner the cancel-delivery wait lost the tail of that distribution once in seven otherwise-green TSan rounds (the cancel itself had succeeded; only the fake worker's counter observation timed out). The budget is now 10000 ms. Verified: nine consecutive daemon_application runs under native macOS TSan with the widened budget, all green, zero ThreadSanitizer warnings. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0baddf7887 |
daemon: permanent lifecycle, daemon-backed CLI/hooks, real-Windows hardening, long-path launcher transactions
Daemon lifecycle and Windows correctness, verified on a real Windows 11 ARM64 VM through the maintained test-infrastructure/vm drivers, plus the macOS and Linux arm64 suites and the container lint gate. Daemon lifecycle: - daemon start/stop/status subcommands. `daemon start` launches a PERMANENT daemon (spawn shape is byte-exact argv; survives idle periods and session ends) and reports an already-active daemon instead of failing. Permanence is honored at every stop latch: last-committed-client disconnect, host initial-client window, coordinator release, and application final-session close — a permanent daemon also keeps admitting new sessions after its last one closes. - daemon stop refuses while sessions are active and lists the blocking peers (pid/role) that must finish first; an idle daemon drains through the activation-shutdown machinery with the ACK ordered after connection interrupts. A second stop is idempotent. The wire ops are no-cohort first-frame requests with peer fingerprint authentication, so stop/status never conflict with an exact-build admission gate. - One-shot CLI commands now execute through the daemon (index workers keep their local supervised path). A cold CLI run that had to spawn a temporary daemon prints a hint that `daemon start` removes the per-command startup tax; a warm daemon is recycled silently. - Hooks are connect-only fail-open: with no daemon present the hook emits a visible, rate-limited notice (Claude-dialect systemMessage plus stderr for other dialects) and always exits 0 — augmentation is never allowed to block the caller's tool use. - Version skew: a newer-build client automatically drains an older-build permanent daemon (strict semantic-version triples only; dev builds never auto-drain) and the build-conflict message names `cbm daemon stop` as the manual escape hatch. Windows IPC/runtime (real-VM verified): - ipc(win): persistent pending overlapped ConnectNamedPipe. The accept path used to destroy its listening pipe instance on every 20 ms poll timeout; a client attaching in the teardown window was severed or left on an orphaned pipe object whose HELLO no server handle could ever read, absorbing the connect until the client's own timeout expired. The pending connect now survives poll timeouts and nothing is destroyed while a client could be attaching. - ipc(win): drain-before-close for final responses. Closing a named-pipe server handle can discard a just-sent response before the peer reads it (POSIX stream sockets never lose buffered data on close). A bounded cbm_daemon_ipc_connection_drain (read-until-peer-EOF; no-op on POSIX, immediate on interrupted connections) now precedes close in runtime_worker_finish and runtime_reject_inline, so hello-conflict, capacity and disconnect acknowledgements reliably reach the peer. - runtime: CLOSE_INTENT wire frame. A Windows named-pipe client has no transport half-close, so close_begin now announces departure with an explicit frame (ordered after APPLICATION_CANCEL, before the local interrupt); the server releases the client's admission on receipt instead of waiting for the handle to close. Admission-drop timing is now identical to POSIX shutdown() semantics on every platform. - runtime(win): client close cancellation. close_begin serializes with request publication under the send lock, best-effort sends the active token's APPLICATION_CANCEL frame, then interrupts local I/O; the server cancels MCP/subprocess work promptly. Contract tests accept both correct outcomes (interrupted transport or decoded CANCELLED). - runtime: activation acknowledgement ordering. The activation ACK is the requester's license to act on "snapshotted and draining", so every connection interrupt is now initiated before the ACK is sent; a session could previously get one more request serviced after the requester observed the ACK. - service(win): deadline-bounded private-file prepare. The conflict-log prepare retry loop (100 x Sleep(2), which rounds up to the ~16 ms timer granularity) burned ~1.6 s against permanently obstructed paths, stalling hello rejections past the client's timeout. The retry budget is now a 250 ms deadline; transient share collisions still retry. - subprocess(win): cmd.exe /C payload encoder quotes metacharacters correctly (root cause of the git-on-Windows failure cluster). - watcher: SHA-256 buffer sizing (CBM_SZ_64 -> CBM_SZ_128) and a native Windows stop/unwatch cancellation test with exact-image verification. - httpd: send_all writes in bounded 64 KiB slices. A single giant nonblocking send() on Windows is absorbed wholesale into AFD kernel buffering regardless of SO_SNDBUF, so send deadlines and interrupts could never engage against a slow peer (and the full payload was pinned in nonpaged pool). Slicing restores a deterministic backpressure point; a test hook pins SO_SNDBUF for the deadline and interrupt tests. - ui/http: shutdown lifecycle — interrupt checks, response-wide send deadline, explicit connection states, refusal to free a server while a listener-owned connection is active. Windows long-path support: - Central path-aware wide conversion (canonicalize via GetFullPathNameW and prepend the extended-length prefix for absolute paths >=240) at the compat chokepoints (cbm_fopen/compat_fs/mkstemp/mkdtemp), sqlite store opens, and the daemon build-fingerprint/log paths. Deep managed installs (a 64-hex generation directory routinely exceeds MAX_PATH) now index, stage and activate correctly. - activation transaction: its own file APIs and the component-walking ancestry validators now operate in the extended-length namespace; the launcher path is canonicalized (and prefixed when deep) once at entry so every downstream exact-string comparison stays form-consistent. - Executable self-resolution uses the wide APIs (GetModuleFileNameW, GetFileAttributesW) so non-ASCII install paths survive argv[0] resolution. Windows launcher install/uninstall transaction: - FileRenameInfoEx names are NUL-terminated in an over-allocated buffer. FileNameLength governs per the contract, but filter drivers read FileName as NUL-terminated and appended adjacent heap bytes to created names — a flaky, garbage-suffixed rename target. Both the CLI and the launcher rename helpers are fixed. - Uninstall retires state via rename-aside (.cbm -> .cbm-retired-v1-<tag>-<pid>) with the retired tag shortened to 16 hex chars so the bare rename target stays under the FileRenameInfoEx NT-conversion ceiling at guard depths; 64 bits still uniquely identify the generation. - When the running launcher's mapped generation backings pin .cbm against rename, the backings are relocated to activation-<pid>-N .retired tombstones beside the install (a mapped image may be renamed, never deleted; the launcher's liveness-guarded sweep reclaims stale tombstones). Every relocation is recorded, and a FAILED uninstall reverses the moves after restoring .cbm — via MoveFileExW with extended-length paths on both arguments, since the deep generation target is beyond the handle-based rename's bare-path reach — so a restored install keeps its generation backings and stays runnable. - After a committed uninstall the retired tree's backings are relocated out so the tree is shallow enough for the detached cleanup's rd, and the cleanup's working directory strips the extended-length prefix (CreateProcessW lpCurrentDirectory silently ignores prefixed paths). - Files created under Administrators-default-owner directories (CopyFileW destinations, CREATE_NEW tombstones, probe directories) are explicitly owner-stamped so the exact-owner validators hold on runner images; guard fixtures stamp hand-built trees the same way. Diagnostics, tests and infra: - diagnostics: discovery is now an always-delivered JSON control record (new cbm_log_control) that survives CBM_LOG_LEVEL suppression and paths containing spaces; placement honors $TMPDIR with /tmp fallback via a diagnostics-local helper; the soak parser reads the JSON record; documented in docs/CONFIGURATION.md. Red-first coverage for suppressed log levels, TMPDIR-with-spaces, and native Windows output-contract assertions. - tests(win): daemon_ipc/daemon_frontend fixtures now build endpoint parents with production-shaped ancestry (LocalAppData on Windows, via th_secure_runtime_parent_new) — the runtime ancestry validation correctly refuses temp roots whose ancestors grant mutation rights to Authenticated Users (C:/msys64/tmp, GitHub-runner work dirs) — and drive the documented startup-owner publication flow before reading generation-bound endpoint addresses. This turns the 26 Windows failures previously visible in CI's full-test job green without weakening any validation. - tests(win): the launcher guard covers the full permanent-launcher contract including failed-uninstall restore and immediate reinstall after uninstall; new daemon lifecycle and reworked hook-augment guards run the start/recycle/stop flow end to end. - tests: CBM_SKIP_PERF is now actually consumed by the test runner (it was set by CI but never read, so perf suites ran everywhere); four throughput/bench suites are classified as perf, the heavy store_arch suite moved to the slow-timeout tier, and two wall-clock-sensitive assertions were rewritten as invariant checks with coarse hang-detector backstops. - build/test infra: build-dir safety contract, UI dev-proxy security contract, soak daemon-recovery contract, path-safety helper, the Windows VM worktree-sync contract wired into scripts/test.sh, and vm/win.sh guards building its clean embedded-UI product in an isolated BUILD_DIR so it cannot clobber the incremental test build. provision-windows.sh now installs Node.js for the guards UI build. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
ad58e71bea |
fix: stabilize cross-platform daemon coordination
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
83c137d2a5 |
feat: complete shared daemon lifecycle
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0e00ef5702 |
feat: coordinate concurrent CBM sessions
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |