Martin Vogel
5a479facf7
Completes the previous commit, which carried only the two mode changes because
the call-site edits were not staged when it landed (--amend is denied in this
repo, so this is additive rather than a rewrite).
Call sites now go through `bash`, which is what most of this repo already does and
which cannot break if a mode bit is lost to a patch application or a non-POSIX
checkout:
scripts/package-release.sh -> bash scripts/ci/check-binary-composition.sh
.github/workflows/release.yml -> bash scripts/ci/append-vt-notes.sh
With the 100755 modes from the previous commit, both sides are now correct, and
either alone would have been sufficient.
tests/test_script_exec_bit_contract.sh pins the class so it cannot recur: any
tracked .sh whose COMMITTED mode is non-executable must not appear as the first
word of a command in workflows, scripts, test-infrastructure or the Makefiles.
The committed mode is the thing that matters and the thing no local run can
check -- the working copy having the bit is exactly why this shipped.
Verified in both directions: passes on this tree, and fails on the exact defect
when the mode and the call site are reverted together. It joins backslash
continuations before analysing, because its own first draft reported a false
positive on
... && bash \
test-infrastructure/vm/vm-run-tests.sh --soak
where the interpreter sits on the preceding line. A contract that cries wolf
teaches people to ignore contracts, so that had to be right before it could be
useful.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/ — the canonical leg entries
The doctrine (enforced, not advisory): a venue may provision a machine;
only a canonical leg script may exercise the product. Local CI, PR CI, the
dry run and the release all call the SAME files below — venues differ only in
host specs, architecture and inputs, never in what a leg does. Platform
differences (arm64 sanitizer flags, the Windows launcher, the linux portable
binary) live INSIDE these scripts, once, never per-venue.
tests/test_venue_parity_contract.sh (run as Step 0j of every test leg) turns
any violation — inline harness logic in a workflow, a bypassed wrapper, a
missing --help — into a red build.
Every entry answers --help (authoritative, agent-oriented: modes, env, exit
codes) and rejects unknown flags with exit 2 + Please consult --help.
The legs
| leg | entry | what a run gives you |
|---|---|---|
| test | test.sh |
DEFAULT = the venue leg: static contracts (Step 0a–0j) + CLEAN ASan+UBSan build + all suites via the parallel harness + prod-binary guards. --suites a,b = iteration mode (incremental rebuild, subset, seconds). --tsan = the ThreadSanitizer leg. CLANGARM64 gets CI's trap-UBSan flags automatically. |
| build | build.sh |
CLEAN production build (+ launcher on Windows). ccache via env.sh makes repeats fast; CCACHE_COMPILERCHECK=content guarantees a hit is byte-identical to a cold compile — never stale. --with-ui, --version, STATIC=1, BUILD_DIR=. |
| lint | lint.sh |
clang-tidy + cppcheck + clang-format (+ no-skips policy). --ci = the CI gate set (no clang-tidy). Drives the same make targets as make lint/lint-ci. |
| smoke (unix) | smoke-local.sh |
Stages a full release fixture, serves it on a kernel-assigned port, runs smoke-test.sh (ALL phases incl. download/install/update E2E) inside a disposable HOME/XDG/TMP sandbox. ui variant makes a missing embedded UI a FAILURE. CBM_SMOKE_ARTIFACT_DIR = smoke an extracted release artifact verbatim (release mode). |
| smoke (windows) | ../test-infrastructure/vm/vm-smoke.sh |
Same contract for the launcher+payload pair, plus the user-PATH registry guard (prepare/verify/cleanup). |
| smoke-invariants | smoke-invariants.sh |
The "shipped binary does not fail" battery (MCP handshake, all tools invocable, malformed-input resilience) — no fixture server, no install E2E. smoke.yml runs it on the WIDEST build-from-source matrix (incl. older-glibc legs the release artifacts cannot cover). |
| soak | soak-legs.sh |
The release-gating soak SEQUENCE: quick then query-leak (the #581 detector — never reindexes, so RSS growth = query-path leak), each guarded by a completion-summary check. --legs quick for the ASan single-leg variant. Duration is per leg. |
Internal harnesses — never called directly by a venue (the contract forbids
it): smoke-test.sh (phases; wrappers provide fixture server + sandbox),
soak-test.sh (one soak run; soak-legs.sh provides the sequence + guards),
run-tests-parallel.sh (reached through test.sh).
Conventions
- Exit codes: 0 = pass · 2 = usage error · 90 = guard (a run died without its completion summary — never counts as green) · anything else = the leg's real failure.
- Iteration is a flag, not a side-tool: the fast paths (
--suites,--legs) are modes of the SAME entry the gates run, so a dev loop can never drift from the venue behaviour. - Env sandboxing: the smoke wrappers neutralize every agent-config destination override; a smoke can never scribble on your real config.
Recommended workflows
- Iterating on a change:
scripts/test.sh --suites <suite>(seconds, incremental, same ASan+UBSan flags as the gate). List suites:build/c/test-runner --list-suites. Debugging a Windows-on-ARM trap: re-run withSANITIZE=for a plain build, or use the emulatedwin.sh ubsan-*pair for full diagnostics. - Before any push (the 3-OS ladder):
scripts/test.sh(macOS, full) →./test-infrastructure/run.sh full(Linux + TSan + smoke) →test-infrastructure/vm/win.sh test-par+guards+smoke-install(+soakwhen the change touches memory/daemon paths). - Concurrency-touching change: add
scripts/test.sh --tsanearly — the same leg CI gates on. - Release-shaped verification:
CBM_SMOKE_ARTIFACT_DIR=<extracted artifact> scripts/smoke-local.sh <binary> [ui]smokes exactly what would ship. - A leg is red in CI but green locally: first suspect environment shape,
not code — the preflights (
win.shautomatic;scripts/ci/preflight-docker.sh) andtest-infrastructure/README.md's residuals list cover the knowable differences.
See scripts/ci/README.md for the CI plumbing and
test-infrastructure/README.md for the venue map.