ci(dry-run): scan the archive artifacts with VirusTotal as the final step

Preparation for the next dry-run cycle. The release gate scans every shipped
artifact with VirusTotal and blocks on any detection; the dry run did not, so
the one question most likely to stop a release was the one a dry run could not
answer. v0.9.1-rc found that out the expensive way.

The scan mirrors the release job. The only difference is where the archives come
from: the release pulls them off a published release, a dry run has none, so
they come from the build job's artifacts — the same files, produced by the same
canonical package-release.sh, extracted the same way and scanned as the same
bytes a user would download and run. Same zero-tolerance gate
(scripts/ci/check-virustotal.sh), so a dry run now fails where the release would.

It runs LAST, after smoke and soak. It is the slowest job — polling can take two
hours — and its verdict does not depend on our code at all: the same unchanged
bytes can come back clean one day and flagged the next. That must not delay the
feedback that IS about our code.

Two independent skips, because they answer different questions:

  skip_builds      no artifacts at all          "do lint and tests pass?"
  skip_virustotal  build + smoke + soak, no scan "do the archives pass CI?"

The second is the one asked for: during a debug cycle, when the question is
whether the archives survive smoke and soak, a two-hour scan burning API quota
is pure noise. skip_builds implies no scan, since there is nothing to scan.

Extraction lives in scripts/ci/extract-release-archives.sh rather than inline in
the YAML because the venue-parity contract requires it — a venue may provision,
plumb artifacts, or call a canonical leg script, and this is leg logic. The
contract caught the inline version.

Two defects found by testing the script instead of trusting it:

  * The empty-input guard did not guard. It counted files in the output
    directory, but the install scripts are copied in unconditionally, so the
    count was never zero and an input with no archives reported success — a
    clean VirusTotal run over nothing, the exact false green the check exists to
    prevent. It now counts binaries recovered from archives, and exits 1 on
    zero.
  * The script was committed 100644 while the workflow invokes it directly as a
    command, which is the "Permission denied" failure
    test_script_exec_bit_contract.sh was written for. That contract scans
    scripts/ but not .github/workflows/, so it did not catch it here.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This commit is contained in:
Martin Vogel
2026-08-07 17:59:50 +02:00
parent 17c2a513a6
commit b67401a03b
2 changed files with 136 additions and 3 deletions
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Extract the release archives into a flat directory of scannable files.
#
# Turns the per-platform artifacts (codebase-memory-mcp-<goos>-<goarch>.tar.gz /
# .zip, produced by the canonical package-release.sh) into one directory holding
# the actual executables, each renamed to its platform so a scan report names
# something a human can act on. The install scripts are copied in beside them:
# they ship in the release too, and a compromised installer matters as much as a
# compromised binary.
#
# Usage: extract-release-archives.sh <archive-dir> <output-dir>
#
# Lives in scripts/ rather than inline in the workflow because the venue-parity
# contract requires it: a venue may provision, plumb artifacts, or call a
# canonical leg script, and this is leg logic. It also means the dry run and the
# release can extract identically instead of drifting apart in two YAML copies.
set -euo pipefail
ARCHIVE_DIR="${1:?extract-release-archives: missing <archive-dir>}"
OUT_DIR="${2:?extract-release-archives: missing <output-dir>}"
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
mkdir -p "$OUT_DIR"
shopt -s nullglob
# Counts BINARIES recovered from archives, deliberately not "files in OUT_DIR".
# The install scripts below are copied in unconditionally, so a whole-directory
# count is never zero and would have reported success on an empty input — a
# clean VirusTotal run over nothing, which is the precise false green this
# script exists to prevent. Caught by running it against an empty directory.
extracted=0
for archive in "$ARCHIVE_DIR"/*.tar.gz; do
name="$(basename "$archive" .tar.gz)"
tar -xzf "$archive" -C "$OUT_DIR" 2>/dev/null || true
if [ -f "$OUT_DIR/codebase-memory-mcp" ]; then
mv "$OUT_DIR/codebase-memory-mcp" "$OUT_DIR/${name}"
extracted=$((extracted + 1))
fi
done
for archive in "$ARCHIVE_DIR"/*.zip; do
name="$(basename "$archive" .zip)"
unzip -o "$archive" -d "$OUT_DIR" >/dev/null 2>&1 || true
if [ -f "$OUT_DIR/codebase-memory-mcp.exe" ]; then
mv "$OUT_DIR/codebase-memory-mcp.exe" "$OUT_DIR/${name}.exe"
extracted=$((extracted + 1))
fi
done
if [ "$extracted" -eq 0 ]; then
echo "FAIL: no binaries extracted from '$ARCHIVE_DIR' — a scan here would have"
echo " reported clean without examining a single byte"
ls -la "$ARCHIVE_DIR" 2>/dev/null || true
exit 1
fi
# Shipped alongside the binaries, so scanned alongside them: a compromised
# installer matters as much as a compromised executable.
cp "$ROOT/install.sh" "$OUT_DIR/" 2>/dev/null || true
cp "$ROOT/install.ps1" "$OUT_DIR/" 2>/dev/null || true
ls -la "$OUT_DIR"
total="$(find "$OUT_DIR" -maxdepth 1 -type f | wc -l | tr -d ' ')"
echo "extracted $extracted binarie(s); $total file(s) queued for scanning"