github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.70-5718 d3e3f0c0a1
    比较

    [OPIK-6028] [CI] perf: build Docker images once via GHCR, parallelize E2E tests (#7117)

    frostbyte_neo 发布于 2026-06-18 11:18:21 +00:00

    • [OPIK-6028] [CI] perf: build Docker images once via GHCR, parallelize E2E tests

    Build E2E Docker images once per workflow run and push to GHCR with
    ci-e2e- tags (ci-e2e- for non-PR). Test matrix runners pull
    pre-built images instead of each building their own.

    • Add reusable build_e2e_docker.yaml workflow (build + push to GHCR)
    • Update all 6 E2E workflows to pull from GHCR instead of building
    • Make backend/guardrails pull_policy configurable in docker-compose
    • Remove --build flag from opik.sh invocations in CI

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • feat(ci): add GHCR cleanup for ci-e2e-* image tags

    Deletes ci-e2e- tags on PR close (immediate) and sweeps
    stale images every Saturday at 03:00 UTC (safety net).

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(ci): implement age-based cleanup for SHA-tagged CI images

    SHA-based ci-e2e- tags (from workflow_dispatch, schedule, main
    pushes) were skipped by the cleanup sweep. Now deletes them when
    older than 7 days.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • feat(ci): expose max_age_days as workflow_dispatch input

    Allows setting 0 to delete all SHA-based CI tags on manual dispatch.
    Defaults to 7 days for scheduled runs.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(cleanup): use process substitution to fix subshell counter bug

    The echo | jq | while read pipeline ran the loop body in a subshell,
    so DELETED/SKIPPED/FAILED counter increments were lost — summary always
    showed 0/0/0. Switched to while read ... done < <(...) process
    substitution to keep counters in the parent shell.

    Also improved diagnostics: log version counts per image, print the API
    error on failure instead of silently continuing, and log skip reasons.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(cleanup): use GITHUB_TOKEN instead of PAT for package API

    The GH_PAT_TO_ACCESS_GITHUB_API secret lacks read:packages scope,
    causing 403 on the org package versions endpoint. Since the workflow
    already declares packages: write permission, GITHUB_TOKEN has the
    needed access.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • refactor(cleanup): filter CI versions server-side with --jq

    Instead of fetching all 14K+ versions into memory and filtering
    client-side, use --jq with --paginate to filter for ci-e2e-* tagged
    versions during pagination. Only matching versions are kept in the
    shell variable, reducing memory and processing time.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(cleanup): improve naming, diagnostics, and date parse safety
    • Rename jobs: "merged PR" → "closed PR", "stale CI" → "old CI"
    • Remove all 2>/dev/null from DELETE calls so errors are visible
    • Skip images with unparseable timestamps instead of silently deleting

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(cleanup): increase cleanup-pr timeout to 15 minutes

    Each tag lookup paginates through ~14K versions (~80s per tag).
    With 6 lookups (3 images × 2 tags), the 5-minute timeout was too tight —
    the job was cancelled after deleting only 3 of 6 images.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • perf(ci): parallelize docker push to GHCR

    The build job's push step pushes images sequentially (~92s total: backend
    20s + python-backend 66s + frontend 6s observed on PR #6763). Switch to
    background pushes with wait + accumulated failure check.

    Expected: wall time = max(push_time_per_image) instead of sum. On the
    observed numbers that's ~66s instead of ~92s, saving ~26s per CI run
    across all 6 consumer workflows + the sharded TS E2E suite.

    Tag step kept sequential (metadata op, no measurable benefit from
    parallelizing).

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • fix(ci): restore pre-existing comments removed during GHCR migration

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(cleanup): add untagged version cleanup to weekly sweep

    Re-pushing a tag leaves the old manifest untagged in GHCR. The existing
    cleanup only matched versions with ci-e2e-* tags, so these accumulated.
    Add a second pass that deletes untagged versions older than max_age_days.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(ci): quote shell variables to satisfy SC2086 actionlint check

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • feat(ci): use immutable SHA-based image tags instead of mutable PR tags

    Tags are now ci-e2e-pr- for PR pushes and ci-e2e- for
    non-PR triggers. Each push produces a unique tag, eliminating:

    • Cross-workflow race conditions on the same tag
    • Orphaned untagged manifests from tag overwrites

    Cleanup simplified: PR-close deletes all ci-e2e-pr-* versions,
    weekly sweep is pure age-based. Untagged cleanup step removed.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • fix(ci): use type: number for max_age_days workflow_dispatch input

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    • refactor(ci): unify cleanup-pr and cleanup-stale into single job

    The delete logic was duplicated across two jobs. Now a single job
    varies the jq filter and age cutoff by event type, with one shared
    scan/delete loop.

    Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

    Co-authored-by: Claude Opus 4.6 noreply@anthropic.com

    下载附件