Files
Martin Vogel 98d7dbab01 feat(ci): add a third release candidate (debug-stripped) as an extra VT draw
Release evidence from run 31744302624 shows the tolerated Microsoft `!ml`
verdict is close to a coin flip per byte image rather than a property of the
code. Across the eight targets the stripped and unstripped candidates of the
SAME linker output disagreed on four, and in both directions:

  linux-amd64    stripped microsoft-ml   unstripped clean
  darwin-arm64   stripped clean          unstripped microsoft-ml
  linux-arm64    stripped clean          unstripped microsoft-ml

If the classifier were keying on something intrinsic to our code the siblings
would agree; they do not. So each variant is close to an independent draw, and
5 of 16 candidates drew the flag.

Two draws is not always enough. On that run linux-amd64-portable came back
microsoft-ml on BOTH candidates, leaving no clean binary to ship for that
target. A third independent draw at a ~31% observed per-candidate hit rate takes
the both/all-flagged case from roughly 1-in-10 per target to roughly 1-in-30.

The third candidate is `--strip-debug` (Apple: `-S`): debug information removed,
symbol table kept. Behaviourally identical to the other two — same linker
output, only metadata differs — but a distinct byte image, which is all
VirusTotal needs to scan it as its own file. Verified on the real v0.10.4
candidates: linux-amd64 gives three distinct hashes (294,634,656 /
294,623,208 / 293,746,096 bytes) and darwin-arm64 likewise, with the ad-hoc
signature verifying after strip.

Selection is unchanged in spirit and now ordered: smallest artifact first
(stripped, debug-stripped, unstripped), take the first CLEAN one, and only if
every candidate drew the tolerated verdict ship the smallest flagged one. A
hard verdict on any candidate still blocks the release before selection.

Cost is 24 objects per release instead of 16.

Derivation enforces that all three hashes differ — identical candidates would be
one draw wearing three hats, and the selector would believe it had alternatives
it does not have. Public claims in README, SECURITY.md and docs/index.html
updated from "both stripped and unstripped" to the three candidates.

All three release contract tests pass, including the native derivation test
which exercises the real strip and codesign path on this host.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-13 23:52:21 +02:00

508 lines
20 KiB
YAML

# Reusable: build release binaries on all supported platforms
name: Build
on:
workflow_call:
inputs:
version:
description: 'Version string (e.g. v0.8.0)'
type: string
default: ''
attest:
description: 'Generate build provenance attestations for release artifacts'
type: boolean
default: false
scan_candidates:
description: 'VirusTotal-scan both candidates before tuple-local selection (release must keep true)'
type: boolean
default: true
secrets:
VIRUS_TOTAL_SCANNER_API_KEY:
description: 'VirusTotal API key used for candidate selection'
required: false
permissions:
contents: read
id-token: write
attestations: write
jobs:
build-unix:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
goos: linux
goarch: amd64
cc: gcc
cxx: g++
- os: ubuntu-24.04-arm
goos: linux
goarch: arm64
cc: gcc
cxx: g++
- os: macos-14
goos: darwin
goarch: arm64
cc: cc
cxx: c++
- os: macos-15-intel
goos: darwin
goarch: amd64
cc: cc
cxx: c++
runs-on: ${{ matrix.os }}
# Every leg is BLOCKING — no continue-on-error. The darwin-amd64 binary
# built on macos-15-intel must ship with every release; if that runner is
# unavailable the build fails loudly rather than silently publishing a
# release with no Intel macOS binary (a user-reported gap). macos-15-intel
# is GitHub's supported Intel image through Aug 2027 (the last x86_64 macOS
# runner); revisit the Intel leg before that retirement.
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install deps (Ubuntu)
if: startsWith(matrix.os, 'ubuntu')
run: sudo apt-get update && sudo apt-get install -y zlib1g-dev
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
env:
VERSION: ${{ inputs.version }}
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" "CC=$CC" "CXX=$CXX"
else
scripts/build.sh --with-ui "CC=$CC" "CXX=$CXX"
fi
- name: Frontend integrity scan
if: matrix.goos == 'linux' && matrix.goarch == 'amd64'
run: scripts/security-ui.sh
- name: Derive immutable stripped and unstripped release candidates
run: |
scripts/ci/prepare-release-candidates.sh \
"${{ matrix.goos }}" "${{ matrix.goarch }}" \
--binary build/c/codebase-memory-mcp \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
"release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/THIRD_PARTY_NOTICES.md"
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: |
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/candidate-provenance.tsv
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/stripped/*
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/unstripped/*
release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-${{ matrix.goos }}-${{ matrix.goarch }}
path: release-candidates/${{ matrix.goos }}-${{ matrix.goarch }}
if-no-files-found: error
build-windows:
runs-on: windows-latest
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
with:
msystem: CLANG64
path-type: inherit
install: >-
mingw-w64-clang-x86_64-clang
mingw-w64-clang-x86_64-zlib
make
python
unzip
zip
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
shell: msys2 {0}
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=clang CXX=clang++
else
scripts/build.sh --with-ui CC=clang CXX=clang++
fi
- name: Derive immutable stripped and unstripped release candidates
shell: msys2 {0}
run: |
scripts/ci/prepare-release-candidates.sh windows amd64 \
--binary build/c/codebase-memory-mcp.exe \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
release-candidates/windows-amd64/THIRD_PARTY_NOTICES.md
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: |
release-candidates/windows-amd64/candidate-provenance.tsv
release-candidates/windows-amd64/stripped/*
release-candidates/windows-amd64/unstripped/*
release-candidates/windows-amd64/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-windows-amd64
path: release-candidates/windows-amd64
if-no-files-found: error
build-windows-arm64:
# Native ARM64 Windows binary via the CLANGARM64 toolchain on the
# windows-11-arm runner (the same toolchain the test suite already
# exercises). Without it, ARM-Windows users fall back to the x86_64
# binary under emulation, and npm/pip on native-ARM Node/Python 404.
runs-on: windows-11-arm
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
with:
msystem: CLANGARM64
path-type: inherit
install: >-
mingw-w64-clang-aarch64-clang
mingw-w64-clang-aarch64-zlib
make
python
unzip
zip
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI
shell: msys2 {0}
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=clang CXX=clang++
else
scripts/build.sh --with-ui CC=clang CXX=clang++
fi
- name: Derive immutable stripped and unstripped release candidates
shell: msys2 {0}
run: |
scripts/ci/prepare-release-candidates.sh windows arm64 \
--binary build/c/codebase-memory-mcp.exe \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
release-candidates/windows-arm64/THIRD_PARTY_NOTICES.md
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: |
release-candidates/windows-arm64/candidate-provenance.tsv
release-candidates/windows-arm64/stripped/*
release-candidates/windows-arm64/unstripped/*
release-candidates/windows-arm64/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-windows-arm64
path: release-candidates/windows-arm64
if-no-files-found: error
build-linux-portable:
# Fully static Linux binaries (gcc -static on Ubuntu).
# Runs on any Linux distro without shared library dependencies.
strategy:
fail-fast: false
matrix:
include:
- arch: amd64
runner: ubuntu-latest
- arch: arm64
runner: ubuntu-24.04-arm
runs-on: ${{ matrix.runner }}
timeout-minutes: 240
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Install deps
run: sudo apt-get update && sudo apt-get install -y zlib1g-dev
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: "22"
- name: Build release binary with UI (static)
env:
VERSION: ${{ inputs.version }}
run: |
if [ -n "$VERSION" ]; then
scripts/build.sh --with-ui --version "$VERSION" CC=gcc CXX=g++ STATIC=1
else
scripts/build.sh --with-ui CC=gcc CXX=g++ STATIC=1
fi
- name: Verify release static linking
run: |
file build/c/codebase-memory-mcp
ldd build/c/codebase-memory-mcp 2>&1 | grep -q "not a dynamic executable" || ldd build/c/codebase-memory-mcp 2>&1 | grep -q "statically linked"
- name: Derive immutable stripped and unstripped release candidates
run: |
scripts/ci/prepare-release-candidates.sh \
linux "${{ matrix.arch }}-portable" \
--binary build/c/codebase-memory-mcp \
--out-dir release-candidates
scripts/gen-third-party-notices.sh \
"release-candidates/linux-${{ matrix.arch }}-portable/THIRD_PARTY_NOTICES.md"
- name: Attest release candidate provenance
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: |
release-candidates/linux-${{ matrix.arch }}-portable/candidate-provenance.tsv
release-candidates/linux-${{ matrix.arch }}-portable/stripped/*
release-candidates/linux-${{ matrix.arch }}-portable/unstripped/*
release-candidates/linux-${{ matrix.arch }}-portable/THIRD_PARTY_NOTICES.md
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-candidates-linux-${{ matrix.arch }}-portable
path: release-candidates/linux-${{ matrix.arch }}-portable
if-no-files-found: error
# One blocking join owns the security-sensitive transition from candidates
# to public artifacts. Each target tuple contributes exactly three behavior-
# equivalent candidates (unstripped/debug-stripped/stripped); the selector may
# only choose within that tuple. A hard result on ANY sibling blocks
# the all-target release rather than selecting around a real detection.
select-package:
needs: [build-unix, build-windows, build-windows-arm64, build-linux-portable]
runs-on: ubuntu-latest
timeout-minutes: 330
permissions:
contents: read
id-token: write
attestations: write
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Download all native candidate pairs
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
pattern: release-candidates-*
path: ${{ runner.temp }}/release-candidate-artifacts
- name: Stage the exact candidate scan set
run: |
python3 scripts/ci/stage-release-candidates.py \
"$RUNNER_TEMP/release-candidate-artifacts" \
release-candidate-scan \
--expect-targets 8 \
--expect-candidates 24
# Staging copied and re-hashed every byte into the immutable object
# set. Drop the redundant artifact-download tree before the VT wait.
test -n "$RUNNER_TEMP"
rm -rf -- "$RUNNER_TEMP/release-candidate-artifacts"
test ! -e "$RUNNER_TEMP/release-candidate-artifacts"
- name: Submit all 24 candidates to VirusTotal
if: ${{ inputs.scan_candidates }}
uses: crazy-max/ghaction-virustotal@936d8c5c00afe97d3d9a1af26d017cfdf26800a2 # v5.0.0
id: virustotal-candidates
with:
vt_api_key: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
files: release-candidate-scan/objects/*
request_rate: 4
# The established gate tolerates only one Microsoft malicious label
# ending !ml and rejects every other malicious/suspicious combination.
# Every candidate still must complete with the minimum engine coverage.
- name: Wait for complete VirusTotal candidate results
if: ${{ inputs.scan_candidates }}
env:
VT_API_KEY: ${{ secrets.VIRUS_TOTAL_SCANNER_API_KEY }}
VT_ANALYSIS: ${{ steps.virustotal-candidates.outputs.analysis }}
VT_EXPECTED_SCAN_SET: release-candidate-scan/scan-set.tsv
VT_ASSOCIATIONS: release-candidate-scan/associations.tsv
VT_RESULTS_PATH: release-candidate-scan/vt-results.tsv
VT_POLL_TIMEOUT_SECONDS: 14400
MIN_ENGINES: 50
run: scripts/ci/check-virustotal.sh
- name: Select one scanned candidate inside every target tuple
if: ${{ inputs.scan_candidates }}
run: |
python3 scripts/ci/select-release-candidates.py \
--candidates release-candidate-scan/candidates.tsv \
--objects-dir release-candidate-scan/objects \
--results release-candidate-scan/vt-results.tsv \
--out-dir release-selection
# This is intentionally reachable only from dry-run.yml's explicit
# skip_virustotal input. release.yml pins scan_candidates=true.
- name: Mark unscanned dry run and default to stripped candidates
if: ${{ !inputs.scan_candidates }}
run: |
echo '::warning::VirusTotal candidate scan explicitly skipped; selecting stripped candidates as unscanned-dry-run.'
python3 scripts/ci/select-release-candidates.py \
--candidates release-candidate-scan/candidates.tsv \
--objects-dir release-candidate-scan/objects \
--default-stripped \
--out-dir release-selection
- name: Reclaim rejected and redundant candidate bytes
run: |
# release-selection owns fresh, hash-verified copies of the eight
# chosen executables. No later release step may consume the 24-object
# scan pool, so discard it before packaging to bound runner disk use.
rm -rf -- release-candidate-scan/objects
test ! -e release-candidate-scan/objects
- name: Package the selected exact bytes
env:
VERSION: ${{ inputs.version }}
run: |
for target in \
linux-amd64 linux-arm64 \
linux-amd64-portable linux-arm64-portable \
darwin-amd64 darwin-arm64 \
windows-amd64 windows-arm64; do
goos="${target%%-*}"
goarch="${target#*-}"
selected_name=codebase-memory-mcp
test "$goos" != windows || selected_name=codebase-memory-mcp.exe
selected="release-selection/selected/$target/$selected_name"
notices="release-candidate-scan/notices/$target/THIRD_PARTY_NOTICES.md"
test -f "$selected"
test -f "$notices"
# The expected hash comes from the selector's provenance manifest,
# not from the file being checked. package-release.sh independently
# re-hashes the file and therefore exercises manifest -> file ->
# archive/MCPB binding instead of comparing a file to itself.
selected_sha="$(awk -F '\t' -v wanted="$target" '!header { for (i = 1; i <= NF; i++) { if ($i == "target") target_col = i; if ($i == "selected_sha256") sha_col = i } if (target_col && sha_col) header = 1; next } $target_col == wanted { count++; selected = $sha_col } END { if (count != 1 || length(selected) != 64 || selected ~ /[^0-9a-f]/) exit 3; print selected }' release-selection/release-selection.tsv)"
scripts/package-release.sh "$goos" "$goarch" \
--selected-binary "$selected" \
--expected-sha256 "$selected_sha" \
--third-party-notices "$notices" \
--out-dir "release-containers/$target"
done
- name: Verify every packaged executable matches its selected hash
run: |
python3 scripts/ci/verify-release-selection.py \
--selection release-selection/release-selection.tsv \
--archive-dir release-containers
- name: Prepare durable candidate-selection evidence
if: ${{ always() }}
run: |
mkdir -p selection-evidence
test ! -f release-candidate-scan/candidates.tsv || \
cp release-candidate-scan/candidates.tsv selection-evidence/release-candidates.tsv
test ! -f release-candidate-scan/vt-results.tsv || \
cp release-candidate-scan/vt-results.tsv selection-evidence/virustotal-candidate-results.tsv
test ! -f release-selection/release-selection.tsv || \
cp release-selection/release-selection.tsv selection-evidence/release-selection.tsv
- name: Attest selected containers and decision evidence
if: ${{ inputs.attest }}
uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1
with:
subject-path: |
release-containers/**/*.tar.gz
release-containers/**/*.zip
release-containers/**/*.mcpb
selection-evidence/*.tsv
- name: Preserve candidate selection evidence
if: ${{ always() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-selection-evidence
path: selection-evidence
if-no-files-found: error
# Keep the established artifact names so the reusable smoke workflow and
# all callers consume only the selected, byte-verified containers.
- name: Upload linux-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-amd64
path: release-containers/linux-amd64/*
if-no-files-found: error
- name: Upload linux-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-arm64
path: release-containers/linux-arm64/*
if-no-files-found: error
- name: Upload linux-amd64-portable containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-amd64-portable
path: release-containers/linux-amd64-portable/*
if-no-files-found: error
- name: Upload linux-arm64-portable containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-linux-arm64-portable
path: release-containers/linux-arm64-portable/*
if-no-files-found: error
- name: Upload darwin-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-darwin-amd64
path: release-containers/darwin-amd64/*
if-no-files-found: error
- name: Upload darwin-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-darwin-arm64
path: release-containers/darwin-arm64/*
if-no-files-found: error
- name: Upload windows-amd64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-windows-amd64
path: release-containers/windows-amd64/*
if-no-files-found: error
- name: Upload windows-arm64 containers
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: binaries-windows-arm64
path: release-containers/windows-arm64/*
if-no-files-found: error