-
feat(eval): auto-ingest fixed CI misses (#1012)
发布于
2026-09-08 03:09:58 +00:00 - feat(eval): auto-ingest fixed CI findings as review false negatives
A real code finding CI surfaces (category ci-check or ci-review-bot, not
ci-transient/provider-infra) that is confirmed and fixed in a run is by
definition a Review false negative: Review passed green and missed it. Ingest
those into the eval corpus automatically.The CI-findings model already persists its structured findings on each round
(FindingsJSON) with the IDs selected for repair (SelectedFindingIDs), so
CIFalseNegativesFromRun reads them back: a finding counts as confirmed and
fixed when it was selected for a fix and the CI step completed genuinely green
(run completed, CI step completed, no approval override). ci-transient and
provider/infra failures, merge conflicts, and dismissed (unselected) findings
are excluded. It reuses IngestPostPRMiss to attach false-negative gold onto the
run's green review case, carrying only the file/line/description the structured
finding recorded - no log enrichment, no head/commit or cross-run judgement.The daemon runs it beside auto-capture when a run finishes, under the same
eval.auto_capture/capture_provenance settings. Re-ingesting a run is a no-op.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01CQMRBne3PAKZT5TyXA2MHH-
no-mistakes(review): Require landed repairs and green checks for ingestion
-
no-mistakes(review): Persist published repair evidence independently of summaries
-
no-mistakes(review): Ingest CI false negatives after recovered runs
-
no-mistakes(document): Document automatic CI false-negative ingestion
-
no-mistakes(ci): Added explicit
eval.auto_ingest_ci_missesconsent, defaulting to false, and gated CI false-negative ingestion on it. Updated configuration tests and documentation. Verified with lint, targeted race tests, and build -
no-mistakes(ci): Removed the separate default-off eval.auto_ingest_ci_misses gate. Fixed CI findings now auto-ingest whenever the existing auto_capture and capture_provenance settings are enabled. Updated configuration tests and documentation. Verified with make lint and race-enabled tests for internal/config, internal/eval, and internal/daemon
Co-authored-by: Claude Opus 4.8 noreply@anthropic.com
下载附件