4f064a3c25
* fix: handle architecture-specific license differences The licenses script now: - Generates separate license reports per GOOS/GOARCH combination - Groups identical reports together (comma-separated arch names) - Adds a Table of Contents at the top of each platform file - Handles cases where different architectures have different dependencies (e.g., x/sys/unix vs x/sys/windows, mousetrap on Windows only) This addresses the issue discovered in cli/cli where some deps changed which changed the mod graph for different GOARCH and affected the exported licenses because go-licenses tries to find common ancestors. * fix: make license script portable and deterministic Address review feedback: - Remove bash 4.0+ associative array requirement for macOS compatibility - Add cross-platform hash function (md5sum on Linux, md5 on macOS) - Ensure deterministic iteration order using sorted groups file - Add better error handling for failed go-licenses commands - Fix grammar: 'architecture(s)' -> 'architectures' - Add documentation for third-party/ being a union of all architectures - Use file-based state instead of associative arrays for portability * fix: update licenses-check to use new architecture-aware format - Check now regenerates using ./script/licenses and compares - Add GOROOT/PATH setup in CI to fix go-licenses module info errors - Check both license files AND third-party directory for changes - See: https://github.com/google/go-licenses/issues/244 * fix: use LC_ALL=C for consistent sorting across systems The sort command uses locale-specific ordering which can differ between systems. Use LC_ALL=C to ensure consistent ordering in CI and locally. * feat: auto-fix license files on PRs and improve CI reliability Changes: - Pin go-licenses version in CI for reproducibility (commit 5348b744) - Add GOROOT/PATH setup for 'Package does not have module info' fix - Update license-check.yml to auto-fix and push to PR branches - Add CI=true env var to use pinned go-licenses version - Add dependabot exclusion from auto-fix workflow - Add code-scanning exclusion for third-party files * feat: auto-close PRs that only needed license updates After the bot pushes license fixes, check if the PR now only contains license file changes. If so, close it automatically with a comment explaining that the license updates are complete. This prevents stale PRs from accumulating when someone creates a PR just to fix licenses, or when all other changes were already merged to the base branch. * feat: auto-create/manage license fix PRs for failing PRs Creates stacked PRs to fix license issues: - Detects when a PR needs license updates - Creates child PR: main <- PR:feature <- PR:license-fix - Tracks PRs with metadata and hash of license changes - Auto-closes if user fixes licenses manually - Auto-closes and recreates if dependencies change - Prevents multiple fix PRs for same base PR Rules: - Only targets PRs against main (not stacked PRs) - Only runs on ready-for-review PRs (not drafts) - Skips bots and forks - Hash-based detection avoids unnecessary work * fix: allow auto-fix workflow to run on dependabot PRs Dependabot PRs frequently need license updates and can't be merged until fixed. The auto-fix workflow helps by creating a child PR with the license changes, making it easy to merge both together. * fix: address Copilot review comments - Remove dependabot exclusion (we want to support dependabot PRs) - Comment indentation already fixed - CI env var already set for reproducibility * refactor: move base branch filter to on: block Moved the 'targets main' check from job if: to workflow on.pull_request.branches. This prevents the workflow from even triggering for PRs targeting other branches, saving CI resources. Draft check is implicit in the types list (opened + ready_for_review). Fork check must stay in if: condition (can't be filtered in on: block). * refactor: merge auto-fix into license-check workflow Combines both workflows into one with two jobs: 1. license-check: Checks licenses, fails if needed, sets outputs 2. auto-create-fix-pr: Creates child PR if needed (only for non-forks) Benefits: - Single workflow file, easier to maintain - Check fails (blocks merge) while still creating helpful fix PR - Fork detection in first job, second job skips for forks - Hash-based tracking prevents duplicate PRs * refactor: simplify license-check to auto-commit approach Much simpler workflow: 1. Always try to auto-commit fix directly to PR branch 2. If push fails (fork without permissions), comment once with instructions 3. Don't create child PRs - just fix in place or give instructions 4. Only comment if not already commented (prevent spam) 5. Always fail check if licenses need updating Benefits: - Much simpler - single job - No child PR management complexity - Clear UX: either fixed or instructed - Works for all PRs (internal/fork/dependabot)
177 lines
6.8 KiB
Bash
Executable File
177 lines
6.8 KiB
Bash
Executable File
#!/bin/bash
|
|
#
|
|
# Generate license files for all platform/arch combinations.
|
|
# This script handles architecture-specific dependency differences by:
|
|
# 1. Generating separate license reports per GOOS/GOARCH combination
|
|
# 2. Grouping identical reports together (comma-separated arch names)
|
|
# 3. Creating an index at the top of each platform file
|
|
# 4. Copying all license files to third-party/
|
|
#
|
|
# Note: third-party/ is a union of all license files across all architectures.
|
|
# This means that license files for dependencies present in only some architectures
|
|
# may still appear in third-party/. This is intentional and ensures compliance.
|
|
#
|
|
# Note: we ignore warnings because we want the command to succeed, however the output should be checked
|
|
# for any new warnings, and potentially we may need to add license information.
|
|
#
|
|
# Normally these warnings are packages containing non go code, which may or may not require explicit attribution,
|
|
# depending on the license.
|
|
set -e
|
|
|
|
# Pinned version for CI reproducibility, latest for local development
|
|
# See: https://github.com/cli/cli/pull/11161
|
|
if [ "$CI" = "true" ]; then
|
|
go install github.com/google/go-licenses@5348b744d0983d85713295ea08a20cca1654a45e # v2.0.1
|
|
else
|
|
go install github.com/google/go-licenses@latest
|
|
fi
|
|
|
|
# actions/setup-go does not setup the installed toolchain to be preferred over the system install,
|
|
# which causes go-licenses to raise "Package ... does not have module info" errors in CI.
|
|
# For more information, https://github.com/google/go-licenses/issues/244#issuecomment-1885098633
|
|
if [ "$CI" = "true" ]; then
|
|
export GOROOT=$(go env GOROOT)
|
|
export PATH=${GOROOT}/bin:$PATH
|
|
fi
|
|
|
|
# actions/setup-go does not setup the installed toolchain to be preferred over the system install,
|
|
# which causes go-licenses to raise "Package ... does not have module info" errors in CI.
|
|
# For more information, https://github.com/google/go-licenses/issues/244#issuecomment-1885098633
|
|
if [ "$CI" = "true" ]; then
|
|
export GOROOT=$(go env GOROOT)
|
|
export PATH=${GOROOT}/bin:$PATH
|
|
fi
|
|
|
|
rm -rf third-party
|
|
mkdir -p third-party
|
|
export TEMPDIR="$(mktemp -d)"
|
|
|
|
trap "rm -fr ${TEMPDIR}" EXIT
|
|
|
|
# Cross-platform hash function (works on both Linux and macOS)
|
|
compute_hash() {
|
|
if command -v md5sum >/dev/null 2>&1; then
|
|
md5sum | cut -d' ' -f1
|
|
elif command -v md5 >/dev/null 2>&1; then
|
|
md5 -q
|
|
else
|
|
# Fallback to cksum if neither is available
|
|
cksum | cut -d' ' -f1
|
|
fi
|
|
}
|
|
|
|
# Function to get architectures for a given OS
|
|
get_archs() {
|
|
case "$1" in
|
|
linux) echo "386 amd64 arm64" ;;
|
|
darwin) echo "amd64 arm64" ;;
|
|
windows) echo "386 amd64 arm64" ;;
|
|
esac
|
|
}
|
|
|
|
# Generate reports for each platform/arch combination
|
|
for goos in darwin linux windows; do
|
|
echo "Processing ${goos}..."
|
|
|
|
archs=$(get_archs "$goos")
|
|
|
|
for goarch in $archs; do
|
|
echo " Generating for ${goos}/${goarch}..."
|
|
|
|
# Generate the license report for this arch
|
|
report_file="${TEMPDIR}/${goos}_${goarch}_report.md"
|
|
GOOS="${goos}" GOARCH="${goarch}" GOFLAGS=-mod=mod go-licenses report ./... --template .github/licenses.tmpl > "${report_file}" 2>/dev/null || echo " (warnings ignored for ${goos}/${goarch})"
|
|
|
|
# Save licenses to temp directory
|
|
GOOS="${goos}" GOARCH="${goarch}" GOFLAGS=-mod=mod go-licenses save ./... --save_path="${TEMPDIR}/${goos}_${goarch}" --force 2>/dev/null || echo " (warnings ignored for ${goos}/${goarch})"
|
|
|
|
# Copy to third-party (accumulate all - union of all architectures for compliance)
|
|
if [ -d "${TEMPDIR}/${goos}_${goarch}" ]; then
|
|
cp -fR "${TEMPDIR}/${goos}_${goarch}"/* third-party/ 2>/dev/null || true
|
|
fi
|
|
|
|
# Extract just the package list (skip header), sort it, and hash it
|
|
# Use LC_ALL=C for consistent sorting across different systems
|
|
packages_file="${TEMPDIR}/${goos}_${goarch}_packages.txt"
|
|
if [ -s "${report_file}" ] && grep -qE '^ - \[' "${report_file}" 2>/dev/null; then
|
|
grep -E '^ - \[' "${report_file}" | LC_ALL=C sort > "${packages_file}"
|
|
hash=$(cat "${packages_file}" | compute_hash)
|
|
else
|
|
echo "(FAILED TO GENERATE LICENSE REPORT FOR ${goos}/${goarch})" > "${packages_file}"
|
|
hash="FAILED_${goos}_${goarch}"
|
|
fi
|
|
|
|
# Store hash for grouping
|
|
echo "${hash}" > "${TEMPDIR}/${goos}_${goarch}_hash.txt"
|
|
done
|
|
|
|
# Group architectures with identical reports (deterministic order)
|
|
# Create groups file: hash -> comma-separated archs
|
|
groups_file="${TEMPDIR}/${goos}_groups.txt"
|
|
rm -f "${groups_file}"
|
|
|
|
# Process architectures in order to build groups
|
|
for goarch in $archs; do
|
|
hash=$(cat "${TEMPDIR}/${goos}_${goarch}_hash.txt")
|
|
# Check if we've seen this hash before
|
|
if grep -q "^${hash}:" "${groups_file}" 2>/dev/null; then
|
|
# Append to existing group
|
|
existing=$(grep "^${hash}:" "${groups_file}" | cut -d: -f2)
|
|
sed -i.bak "s/^${hash}:.*/${hash}:${existing}, ${goarch}/" "${groups_file}"
|
|
rm -f "${groups_file}.bak"
|
|
else
|
|
# New group
|
|
echo "${hash}:${goarch}" >> "${groups_file}"
|
|
fi
|
|
done
|
|
|
|
# Generate the combined report for this platform
|
|
output_file="third-party-licenses.${goos}.md"
|
|
|
|
cat > "${output_file}" << 'EOF'
|
|
# GitHub MCP Server dependencies
|
|
|
|
The following open source dependencies are used to build the [github/github-mcp-server][] GitHub Model Context Protocol Server.
|
|
|
|
## Table of Contents
|
|
|
|
EOF
|
|
|
|
# Build table of contents (sorted for determinism)
|
|
# Use LC_ALL=C for consistent sorting across different systems
|
|
LC_ALL=C sort "${groups_file}" | while IFS=: read -r hash group_archs; do
|
|
# Create anchor-friendly name
|
|
anchor=$(echo "${group_archs}" | tr ', ' '-' | tr -s '-')
|
|
echo "- [${group_archs}](#${anchor})" >> "${output_file}"
|
|
done
|
|
|
|
echo "" >> "${output_file}"
|
|
echo "---" >> "${output_file}"
|
|
echo "" >> "${output_file}"
|
|
|
|
# Add each unique report section (sorted for determinism)
|
|
# Use LC_ALL=C for consistent sorting across different systems
|
|
LC_ALL=C sort "${groups_file}" | while IFS=: read -r hash group_archs; do
|
|
# Get the packages from the first arch in this group
|
|
first_arch=$(echo "${group_archs}" | cut -d',' -f1 | tr -d ' ')
|
|
packages=$(cat "${TEMPDIR}/${goos}_${first_arch}_packages.txt")
|
|
|
|
cat >> "${output_file}" << EOF
|
|
## ${group_archs}
|
|
|
|
The following packages are included for the ${group_archs} architectures.
|
|
|
|
${packages}
|
|
|
|
EOF
|
|
done
|
|
|
|
# Add footer
|
|
echo "[github/github-mcp-server]: https://github.com/github/github-mcp-server" >> "${output_file}"
|
|
|
|
echo "Generated ${output_file}"
|
|
done
|
|
|
|
echo "Done! License files generated."
|
|
|