Martin Vogel
38148b87a4
The first full dry run after #1508 failed in all three smoke-linux-portable legs: _smoke.yml still expanded a variant matrix and extracted codebase-memory-mcp-ui-<os>-<arch>.tar.gz — a name the build no longer produces. PR CI never sees this job (pr.yml calls the smoke wrappers directly), so the miss only surfaced in the dry-run/release path this workflow serves. The matrix loses its variant dimension, all three legs extract the unsuffixed archive, and the positional/SMOKE_VARIANT plumbing is replaced by SMOKE_REQUIRE_UI=1: these legs smoke the SHIPPED artifact, so a binary serving no embedded UI is a defect here, exactly like scripts/ci/smoke-artifact.sh. Verified locally: venue-parity and smoke-fixture contracts pass; YAML parses. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
348 lines
17 KiB
YAML
348 lines
17 KiB
YAML
# Reusable: smoke test shipped release binaries on all platforms
|
|
name: Smoke
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
broad_platforms:
|
|
description: 'Smoke the shipped binaries on the broad platform matrix (extra OS versions) instead of the core set'
|
|
type: boolean
|
|
default: false
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
# Emit the platform matrices as JSON. The CORE set is the default (fast,
|
|
# unchanged); the BROAD set adds extra free runners (additional OS versions)
|
|
# that download the SAME shipped artifact for their goos/goarch and verify it
|
|
# runs on a wider range of OS versions. No new artifacts are built — broad
|
|
# legs reuse the exact binaries produced by _build.yml.
|
|
setup-matrix:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
outputs:
|
|
unix: ${{ steps.set.outputs.unix }}
|
|
windows: ${{ steps.set.outputs.windows }}
|
|
portable: ${{ steps.set.outputs.portable }}
|
|
steps:
|
|
- name: Compute matrices
|
|
id: set
|
|
env:
|
|
BROAD: ${{ inputs.broad_platforms }}
|
|
run: |
|
|
CORE_UNIX='[
|
|
{"os":"ubuntu-latest","goos":"linux","goarch":"amd64"},
|
|
{"os":"ubuntu-24.04-arm","goos":"linux","goarch":"arm64"},
|
|
{"os":"macos-14","goos":"darwin","goarch":"arm64"},
|
|
{"os":"macos-15-intel","goos":"darwin","goarch":"amd64"}
|
|
]'
|
|
# Broad legs reuse existing goos/goarch artifacts on additional OS
|
|
# versions to widen the run-anywhere signal without building new targets.
|
|
# Broad legs are REQUIRED gates (no optional / continue-on-error):
|
|
# every smoke leg must pass. No `optional` flags anywhere.
|
|
#
|
|
# NOTE: the *dynamic* linux binary links glibc 2.38+ and cannot run on
|
|
# older distros by design — older-glibc coverage is the -portable (static)
|
|
# binary's job, exercised green by the smoke-linux-portable broad legs
|
|
# (ubuntu-22.04 / 22.04-arm). So the dynamic broad legs stay on
|
|
# forward-compatible OSes only (macOS); running the dynamic binary on
|
|
# ubuntu-22.04 would fail Phase 1 (glibc too old), not a real regression.
|
|
BROAD_UNIX='[
|
|
{"os":"macos-15","goos":"darwin","goarch":"arm64"}
|
|
]'
|
|
CORE_WIN='[{"os":"windows-latest","arch":"amd64"}]'
|
|
# windows-11-arm now runs the NATIVE arm64 binary (build-windows-arm64),
|
|
# not the x86_64 binary under emulation — we smoke the artifact we ship.
|
|
BROAD_WIN='[{"os":"windows-2025","arch":"amd64"},{"os":"windows-11-arm","arch":"arm64"}]'
|
|
CORE_PORTABLE='[
|
|
{"arch":"amd64","runner":"ubuntu-latest"},
|
|
{"arch":"arm64","runner":"ubuntu-24.04-arm"}
|
|
]'
|
|
BROAD_PORTABLE='[
|
|
{"arch":"amd64","runner":"ubuntu-22.04"},
|
|
{"arch":"arm64","runner":"ubuntu-22.04-arm"}
|
|
]'
|
|
if [ "$BROAD" = "true" ]; then
|
|
UNIX=$(jq -cn --argjson a "$CORE_UNIX" --argjson b "$BROAD_UNIX" '$a + $b')
|
|
WIN=$(jq -cn --argjson a "$CORE_WIN" --argjson b "$BROAD_WIN" '$a + $b')
|
|
PORTABLE=$(jq -cn --argjson a "$CORE_PORTABLE" --argjson b "$BROAD_PORTABLE" '$a + $b')
|
|
else
|
|
UNIX=$(jq -cn --argjson a "$CORE_UNIX" '$a')
|
|
WIN=$(jq -cn --argjson a "$CORE_WIN" '$a')
|
|
PORTABLE=$(jq -cn --argjson a "$CORE_PORTABLE" '$a')
|
|
fi
|
|
# One composition ships; the wrappers assert the embedded UI themselves
|
|
# (SMOKE_REQUIRE_UI in scripts/ci/smoke-artifact.sh and below).
|
|
UNIX_M=$(jq -cn --argjson a "$UNIX" '{include:$a}')
|
|
WIN_M=$(jq -cn --argjson a "$WIN" '{include:$a}')
|
|
PORTABLE_M=$(jq -cn --argjson a "$PORTABLE" '{include:$a}')
|
|
echo "unix=$UNIX_M" >> "$GITHUB_OUTPUT"
|
|
echo "windows=$WIN_M" >> "$GITHUB_OUTPUT"
|
|
echo "portable=$PORTABLE_M" >> "$GITHUB_OUTPUT"
|
|
|
|
smoke-unix:
|
|
needs: setup-matrix
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJSON(needs.setup-matrix.outputs.unix) }}
|
|
runs-on: ${{ matrix.os }}
|
|
# 30, not 15: the unified smoke legitimately runs MORE (artifact mode,
|
|
# real download/update phases, a Phase 15 that actually executes), and
|
|
# v4's PASSING legs already took 12-14 min — v5 lost 13 green-path jobs
|
|
# to the 15-min wall (GitHub reports job timeouts as 'cancelled').
|
|
# Budget doctrine: the timeout sits above the worst case, twice over.
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
name: binaries-${{ matrix.goos }}-${{ matrix.goarch }}
|
|
|
|
- name: Extract release artifact
|
|
run: |
|
|
mkdir -p "$RUNNER_TEMP/cbm-artifact"
|
|
tar -xzf codebase-memory-mcp-${{ matrix.goos }}-${{ matrix.goarch }}.tar.gz \
|
|
-C "$RUNNER_TEMP/cbm-artifact"
|
|
chmod +x "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
|
|
# ONE harness for every venue. This is the same wrapper local CI and PR CI
|
|
# run, handed the DOWNLOADED artifact instead of a local build: it stages
|
|
# the release fixture, serves it on a kernel-assigned port, and runs
|
|
# smoke-test.sh under the isolated profile/TEMP the other venues use.
|
|
# It replaces an inline copy of that staging logic which had drifted from
|
|
# the wrapper (no env isolation, no user-PATH guard, fixed port).
|
|
- name: Smoke test (shared wrapper)
|
|
run: |
|
|
scripts/smoke-local.sh "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
env:
|
|
CBM_SMOKE_ARTIFACT_DIR: ${{ runner.temp }}/cbm-artifact
|
|
SMOKE_REQUIRE_UI: "1"
|
|
|
|
- name: Security audits
|
|
run: |
|
|
BIN="$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
scripts/security-strings.sh "$BIN"
|
|
scripts/security-install.sh "$BIN"
|
|
scripts/security-network.sh "$BIN"
|
|
|
|
- name: MCP robustness test (selected linux-amd64 runtime)
|
|
if: matrix.goos == 'linux' && matrix.goarch == 'amd64'
|
|
run: |
|
|
BIN="$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
scripts/security-fuzz.sh "$BIN"
|
|
scripts/security-fuzz-random.sh "$BIN" 60
|
|
|
|
- name: ClamAV scan (Linux)
|
|
if: startsWith(matrix.os, 'ubuntu')
|
|
run: |
|
|
sudo apt-get update -qq && sudo apt-get install -y -qq clamav > /dev/null 2>&1
|
|
# apt auto-starts the clamav-freshclam daemon, which holds a lock on
|
|
# freshclam's log/db; stop it so the manual freshclam below can run
|
|
# (else: "Failed to lock the log file ... Resource temporarily unavailable").
|
|
sudo systemctl stop clamav-freshclam 2>/dev/null || true
|
|
sudo sed -i 's/^Example/#Example/' /etc/clamav/freshclam.conf 2>/dev/null || true
|
|
grep -q "DatabaseMirror" /etc/clamav/freshclam.conf 2>/dev/null || \
|
|
echo "DatabaseMirror database.clamav.net" | sudo tee -a /etc/clamav/freshclam.conf > /dev/null
|
|
sudo freshclam --quiet
|
|
clamscan --no-summary "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
|
|
- name: ClamAV scan (macOS)
|
|
if: startsWith(matrix.os, 'macos')
|
|
run: |
|
|
brew install clamav > /dev/null 2>&1
|
|
CLAMAV_ETC=$(brew --prefix)/etc/clamav
|
|
if [ ! -f "$CLAMAV_ETC/freshclam.conf" ]; then
|
|
cp "$CLAMAV_ETC/freshclam.conf.sample" "$CLAMAV_ETC/freshclam.conf" 2>/dev/null || true
|
|
sed -i '' 's/^Example/#Example/' "$CLAMAV_ETC/freshclam.conf" 2>/dev/null || true
|
|
echo "DatabaseMirror database.clamav.net" >> "$CLAMAV_ETC/freshclam.conf"
|
|
fi
|
|
freshclam --quiet --no-warnings 2>/dev/null || freshclam --quiet 2>/dev/null || echo "WARNING: freshclam update failed"
|
|
clamscan --no-summary "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
|
|
smoke-windows:
|
|
needs: setup-matrix
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJSON(needs.setup-matrix.outputs.windows) }}
|
|
runs-on: ${{ matrix.os }}
|
|
# 30, not 15: the unified smoke legitimately runs MORE (artifact mode,
|
|
# real download/update phases, a Phase 15 that actually executes), and
|
|
# v4's PASSING legs already took 12-14 min — v5 lost 13 green-path jobs
|
|
# to the 15-min wall (GitHub reports job timeouts as 'cancelled').
|
|
# Budget doctrine: the timeout sits above the worst case, twice over.
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: msys2/setup-msys2@66cd2cce69caa17b53920067426061ca1de3a884 # v2
|
|
with:
|
|
msystem: ${{ matrix.arch == 'arm64' && 'CLANGARM64' || 'CLANG64' }}
|
|
path-type: inherit
|
|
install: >-
|
|
mingw-w64-clang-${{ matrix.arch == 'arm64' && 'aarch64' || 'x86_64' }}-python3
|
|
mingw-w64-clang-${{ matrix.arch == 'arm64' && 'aarch64' || 'x86_64' }}-curl
|
|
unzip
|
|
zip
|
|
coreutils
|
|
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
name: binaries-windows-${{ matrix.arch }}
|
|
|
|
- name: Extract release artifact
|
|
shell: msys2 {0}
|
|
run: |
|
|
ARCH=${{ matrix.arch }}
|
|
ARTIFACT_DIR="$(cygpath -u "$RUNNER_TEMP")/cbm-artifact"
|
|
mkdir -p "$ARTIFACT_DIR"
|
|
unzip -o "codebase-memory-mcp-windows-${ARCH}.zip" -d "$ARTIFACT_DIR"
|
|
test -s "$ARTIFACT_DIR/codebase-memory-mcp.exe"
|
|
# ONE executable per runtime set: a payload sibling would mean the
|
|
# removed launcher split came back.
|
|
test ! -e "$ARTIFACT_DIR/codebase-memory-mcp.payload.exe"
|
|
# Pre-flight the image from a profile-rooted directory: the shared
|
|
# runner workspace is deliberately not a valid install root.
|
|
PROFILE_ROOT="$(cygpath -u "$USERPROFILE")"
|
|
LAUNCH_DIR="$(mktemp -d "$PROFILE_ROOT/cbm-release-version.XXXXXX")"
|
|
# Best-effort cleanup with one retry: Windows holds the just-exited
|
|
# image's file busy for ~100ms (section teardown / first-touch AV),
|
|
# and a temp dir on an ephemeral runner must never fail the job.
|
|
trap 'rm -rf "$LAUNCH_DIR" 2>/dev/null || { sleep 2; rm -rf "$LAUNCH_DIR" 2>/dev/null || true; }' EXIT
|
|
cp "$ARTIFACT_DIR/codebase-memory-mcp.exe" "$LAUNCH_DIR/"
|
|
"$LAUNCH_DIR/codebase-memory-mcp.exe" --version
|
|
|
|
# ONE harness for every venue: the same script win.sh smoke-install and
|
|
# pr.yml run, handed the DOWNLOADED artifact instead of a local build. It
|
|
# stages the release fixture under a disposable profile root, serves it on
|
|
# a kernel-assigned port, guards AND verifies the user-PATH registry
|
|
# mutation, and neutralizes every agent-config destination before running
|
|
# smoke-test.sh. It replaces an inline copy of that staging which had none
|
|
# of the isolation and never verified the PATH mutation it performed.
|
|
- name: Smoke test (shared wrapper)
|
|
shell: msys2 {0}
|
|
env:
|
|
SMOKE_ARCH: ${{ matrix.arch }}
|
|
SMOKE_REQUIRE_UI: "1"
|
|
run: |
|
|
export CBM_SMOKE_ARTIFACT_DIR="$(cygpath -u "$RUNNER_TEMP")/cbm-artifact"
|
|
bash test-infrastructure/vm/vm-smoke.sh
|
|
|
|
- name: Security audits
|
|
shell: msys2 {0}
|
|
run: |
|
|
ARTIFACT_DIR="$(cygpath -u "$RUNNER_TEMP")/cbm-artifact"
|
|
scripts/security-strings.sh "$ARTIFACT_DIR/codebase-memory-mcp.exe"
|
|
PROFILE_ROOT="$(cygpath -u "$USERPROFILE")"
|
|
SECURITY_DIR="$(mktemp -d "$PROFILE_ROOT/cbm-release-security.XXXXXX")"
|
|
# Best-effort cleanup with one retry (see the version preflight).
|
|
trap 'rm -rf "$SECURITY_DIR" 2>/dev/null || { sleep 2; rm -rf "$SECURITY_DIR" 2>/dev/null || true; }' EXIT
|
|
cp "$ARTIFACT_DIR/codebase-memory-mcp.exe" "$SECURITY_DIR/"
|
|
TMPDIR="$SECURITY_DIR" \
|
|
scripts/security-install.sh "$SECURITY_DIR/codebase-memory-mcp.exe"
|
|
|
|
- name: Windows Defender scan
|
|
shell: pwsh
|
|
run: |
|
|
& "C:\Program Files\Windows Defender\MpCmdRun.exe" -SignatureUpdate 2>$null
|
|
$artifactDir = Join-Path $env:RUNNER_TEMP "cbm-artifact"
|
|
foreach ($binary in @("codebase-memory-mcp.exe")) {
|
|
$result = & "C:\Program Files\Windows Defender\MpCmdRun.exe" -Scan -ScanType 3 -File (Join-Path $artifactDir $binary) -DisableRemediation
|
|
$code = $LASTEXITCODE
|
|
Write-Host $result
|
|
# MpCmdRun -Scan exit codes: 0 = clean, 2 = threat found. Any OTHER non-zero
|
|
# means the scan engine could not run at all (e.g. hr=0x800106ba: the Defender
|
|
# antimalware service is unavailable on the runner) — that is NOT a detection.
|
|
# Fail soft on an engine failure so a transient runner-side AV outage can't
|
|
# false-fail a release; only a real detection (exit 2) hard-blocks.
|
|
if ($code -eq 2) {
|
|
Write-Host "BLOCKED: Windows Defender flagged $binary!"; exit 1
|
|
} elseif ($code -ne 0) {
|
|
Write-Host "::warning::Windows Defender scan could not run for $binary (exit $code) - skipping AV gate on this runner"
|
|
} else {
|
|
Write-Host "=== Windows Defender: $binary clean ==="
|
|
}
|
|
}
|
|
|
|
smoke-linux-portable:
|
|
needs: setup-matrix
|
|
strategy:
|
|
fail-fast: false
|
|
matrix: ${{ fromJSON(needs.setup-matrix.outputs.portable) }}
|
|
runs-on: ${{ matrix.runner }}
|
|
# 30, not 15: the unified smoke legitimately runs MORE (artifact mode,
|
|
# real download/update phases, a Phase 15 that actually executes), and
|
|
# v4's PASSING legs already took 12-14 min — v5 lost 13 green-path jobs
|
|
# to the 15-min wall (GitHub reports job timeouts as 'cancelled').
|
|
# Budget doctrine: the timeout sits above the worst case, twice over.
|
|
timeout-minutes: 30
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
|
|
with:
|
|
name: binaries-linux-${{ matrix.arch }}-portable
|
|
|
|
- name: Extract release artifact
|
|
run: |
|
|
mkdir -p "$RUNNER_TEMP/cbm-artifact"
|
|
tar -xzf codebase-memory-mcp-linux-${{ matrix.arch }}-portable.tar.gz \
|
|
-C "$RUNNER_TEMP/cbm-artifact"
|
|
chmod +x "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
|
|
# Same wrapper as every other venue. This leg previously called
|
|
# smoke-test.sh with no fixture server, which silently SKIPPED the
|
|
# download/checksum/install-script phases (12-13) — the portable binary is
|
|
# what every linux install and update actually delivers, so those are
|
|
# exactly the phases it most needs to run.
|
|
- name: Smoke test (shared wrapper)
|
|
run: |
|
|
scripts/smoke-local.sh "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
env:
|
|
CBM_SMOKE_ARTIFACT_DIR: ${{ runner.temp }}/cbm-artifact
|
|
SMOKE_REQUIRE_UI: "1"
|
|
|
|
# The -portable binary is what all linux install/update paths now deliver;
|
|
# it MUST start on old glibc (Debian 11 / RHEL 8 / Ubuntu 20.04). Runs it
|
|
# in debian:bullseye (glibc 2.31) — the standard dynamic binary would fail
|
|
# here with `GLIBC_2.38 not found`.
|
|
- name: Old-glibc compatibility
|
|
run: scripts/ci/check-glibc-compat.sh "$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
|
|
- name: Security audits
|
|
run: |
|
|
BIN="$RUNNER_TEMP/cbm-artifact/codebase-memory-mcp"
|
|
scripts/security-strings.sh "$BIN"
|
|
scripts/security-install.sh "$BIN"
|
|
scripts/security-network.sh "$BIN"
|
|
|
|
# ── Packaging check (non-gating) ──────────────────────────────────
|
|
# Builds pkg/glama/Dockerfile — the image Glama builds to score the
|
|
# server — and runs an MCP initialize + tools/list handshake to confirm the
|
|
# containerized stdio server still starts and introspects. Guards the Glama
|
|
# listing integration against drift (Dockerfile breakage, release-asset
|
|
# renames, introspection regressions).
|
|
#
|
|
# Also tests the brew tap/install flow.
|
|
#
|
|
# continue-on-error: a broken *directory* image must never block a release —
|
|
# the shipped product binaries are unaffected. The red X is the signal to fix
|
|
# the integration, not a release gate.
|
|
smoke-packages:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 10
|
|
continue-on-error: true
|
|
steps:
|
|
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
- name: Build Glama image and verify MCP introspection
|
|
run: bash pkg/glama/verify.sh
|
|
- name: Test homebrew installation
|
|
env:
|
|
HOMEBREW_NO_AUTO_UPDATE: 1
|
|
run: |
|
|
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
|
|
brew tap deusdata/codebase-memory-mcp "$GITHUB_WORKSPACE"
|
|
brew trust deusdata/codebase-memory-mcp
|
|
brew install codebase-memory-mcp
|
|
codebase-memory-mcp --version
|