Martin Vogel
bdb99d7750
The verify pass submitted every extracted object, selected executables included,
on the stated grounds that "VirusTotal is content-addressed, so identical bytes
return the analysis it already holds instead of re-running 70+ engines".
That is measurably false. On v0.10.5 all EIGHT re-submissions produced a NEW
analysis - same VirusTotal file-id, timestamp 47 minutes later:
candidate: file-id=2c00f485... ts=1786795957 (12:12:37Z)
verify : file-id=2c00f485... ts=1786798758 (12:59:18Z)
Re-analysing identical bytes re-rolls a probabilistic classifier, and Microsoft's
ML engine answered differently within that hour, in BOTH directions:
82750cd1 (linux-amd64) microsoft-ml -> clean
6d3c5be6 (darwin-arm64) clean -> microsoft-ml
The published notes are generated from the candidate scan, so v0.10.5 shipped a
table calling linux-amd64 flagged when VirusTotal had it clean, and darwin-arm64
clean when VirusTotal was reporting Trojan:Script/Wacatac.B!ml. Every hash in
that table links to the page that contradicted it. Corrected in place after
publication; this removes the cause.
The second scan proved nothing the first did not. Identity is settled by hash
before this step runs: verify-release-selection.py reconciles every published
container to the selected bytes, and checksums.txt binds the same digests
publicly. A re-scan adds no assurance - only another roll.
What still gets scanned is exactly what the candidate pass never saw: install.sh,
install.ps1, LICENSE, THIRD_PARTY_NOTICES.md, the MCPB manifest.json and the
unpacked UI assets. install.sh and install.ps1 are the highest-consequence
non-executable bytes we publish - users pipe them straight into a shell - and
that coverage is untouched. Measured on the v0.10.5 object set: 16 objects in,
8 withheld, 8 still scanned.
The withheld set is recorded as evidence (cbm-virustotal-withheld-v1) naming each
sha256 and pointing at virustotal-candidate-results.tsv, so the published
evidence still accounts for every shipped object.
Fails closed three ways, each with an actionable message: no object matches a
selected sha (the containers do not carry the recorded bytes), everything is
withheld (the surface scan would be a no-op), or the selection names no shas at
all. The zero-match grep is wrapped rather than left to pipefail, because a
guard that aborts silently is not a guard - found by testing the guards rather
than assuming them.
Also drops 8 VirusTotal submissions per release.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
scripts/ci/ — venue plumbing (single implementations)
Support scripts that keep the venues in the SAME shape. Each exists because
the logic used to live inline in workflow YAML or was hand-duplicated between
CI and the local infrastructure — both of which the venue-parity contract
(tests/test_venue_parity_contract.sh) now forbids. Everything here answers
--help (PowerShell: comment-based help, Get-Help <script>).
| script | job | called by |
|---|---|---|
new-protected-temp-root.ps1 |
Create the owner-stamped, inheritance-protected per-user TEMP root the daemon/install suites require (shared /tmp and default runner TEMP grant Authenticated-Users mutation rights, which the trust policy correctly refuses — running there produces security refusals, not signal). -ProtectDir stamps build dirs the same way. |
_test.yml, _soak.yml, vm-run-tests.sh |
clean-test-residue.ps1 |
Sweep cbm-* residue from the Windows VM and assert runner-like free disk (default 14 GB — the GitHub runner's SSD). Long-path \\?\ fallback for the guard suites' adversarial trees; every removal VERIFIED (an earlier version counted attempts and reported 86 swept while 11 GB remained). BLOCKS below the floor: a disk that fills mid-run reads as a product bug. |
win.sh before every build/run |
preflight-docker.sh |
Same idea for Colima/docker: prune runner-unlike residue, assert free space on the filesystem backing the docker data root (not the VM's /). Build cache + named volumes KEPT (the local analogue of actions/cache); --deep drops them. |
test-infrastructure/run.sh |
check-glibc-compat.sh |
Run a linux binary in debian:bullseye (glibc 2.31) — the portable binary must start on old glibc. | _smoke.yml portable legs |
generate-sbom.py |
The release SPDX SBOM (vendored versions reviewable here, diffable by vendoring PRs — was inline YAML). | release.yml |
require-all-green.sh |
The aggregate gate: fail unless every needed job succeeded or legitimately skipped (was inline YAML). | pr.yml ci-ok |
verify-shard-union.sh |
Prove sharded test legs lost nothing: shard count agreement, indices 1..n, identical suite lists, union of slices == full list (was inline YAML). | _test.yml shard-completeness |
prepare-release-candidates.sh |
Copy one linker output into stripped/unstripped candidates, finalize signatures, composition-check them without execution, and record their hashes. | _build.yml, local artifact smoke |
stage-release-candidates.py |
Admit exactly eight candidate artifacts / sixteen byte-distinct binaries into the content-addressed VirusTotal scan set. | _build.yml |
select-release-candidates.py |
Apply the reviewed tuple-local VT truth table, or the explicit dry-run stripped default, and atomically copy one content-bound binary per target. | _build.yml |
verify-release-selection.py |
Recompute the selection policy and prove every executable member in all 14 public containers equals its selected SHA-256. | _build.yml, release.yml final draft verification |
check-virustotal.sh |
Poll and validate the exact candidate scan set, enforce engine coverage and the narrow documented Microsoft !ml policy, and emit content-bound results evidence. |
_build.yml |