feat(eval): score replay findings against human gold (#726)

* feat(eval): score findings as true/false positives, not park/pass

Park/pass treated skip and approve as a pass and asked the wrong question.
Capture now writes finding-level gold from recorded Fix and add-finding
evidence, and replay/report score TP/FN with unmatched findings left pending.

* no-mistakes(review): Capture user-added eval gold independently

* no-mistakes(review): Remove duplicate eval guidance

* no-mistakes(review): Persist eval decision provenance atomically

* no-mistakes(review): Make eval finding scoring evidence-safe

* no-mistakes(review): Make eval scoring evidence-safe and relabel recall range

* no-mistakes(review): Keep eval matching and recall evidence-safe

* no-mistakes(test): Match eval findings by finding ID

* no-mistakes(document): Document finding-level eval scoring

AGENTS.md

@@ -198,6 +198,7 @@ Safest local verification sequence after non-trivial changes:
 - Collection is automatic and default-on through `eval.capture_provenance` / `eval.auto_capture` / `eval.max_cases` in `config.yaml`, never an environment variable: the daemon's launchd/systemd unit is re-rendered on install and update and preserves only proxy variables (`internal/daemon/service.go` `proxyEnvKeys`), so an env-gated corpus silently stops collecting after an update. The keys are global-only - `Merge` copies them straight from `GlobalConfig`, and an `eval` block in a repo's `.no-mistakes.yaml` is ignored.
 - Provenance is unrecoverable: `executor.go` writes it with the review round or never. A round recorded with `capture_provenance` off can never be captured, so the rejection names the setting rather than the round's age.
 - The trigger is `RunManager.autoCaptureEvalCase`, called last in the run goroutine after the outcome is already reported: it recovers its own panic (the enclosing recover would otherwise mark a finished run failed), bounds itself with `evalAutoCaptureTimeout` off the run context, serializes runs on `evalCaptureMu` (shared pool + registry), and logs rather than propagates. `ErrNoCapturableReview` separates "nothing to freeze" (DEBUG) from a real fault (WARN). Automatic and manual capture call the same `eval.Capture`.
+- The unit of truth is finding-level gold, not park/pass: a user-selected Fix is true-positive gold, a human-added finding is false-negative gold, skip/approve stay unlabeled / pending, and unmatched candidate findings stay queued - never inferred as false positives. Owner: `internal/eval` (`goldFromRound`, `ScoreCandidate`); user-facing language is `docs/src/content/docs/reference/eval.md`. Regressions: `TestCaptureDoesNotLabelSkipOrApproveAsPass`, `TestCaptureWritesFalseNegativeGoldForUserAddedFinding`, `TestCaptureAndReport*`, CLI `TestEvalCaptureAndSetsSpeakInFindingGoldTerms`.
 - A case stores no Git bundle. Bundles were a full history copy per review pass (~8 MB each here) and cannot be trimmed, because a bundle built with negative refs records prerequisites an empty restore gate lacks. Cases of one repository instead share `<NM_HOME>/eval/pools/<fingerprint>.git`, pinned by `refs/no-mistakes/eval/<caseID>/{head,source-head,base,trusted-config}`; the marginal case costs ~8 KB. `Store.Prune` applies `max_cases` oldest-first but protects active replay reservations and cases with recorded evaluations, so the cap is a retention target rather than a hard bound.
 - Capture stays read-only against the gate, so objects reach the pool through a throwaway bare clone plus a refspec fetch - never a bare-object-id fetch, whose want policy is off by default and version-dependent.
 - Regressions: `TestCaptureDoesNotCopyRepositoryHistoryPerCase`, `TestPruneBoundsTheCorpusOldestFirstAndKeepsEvaluatedCases`, `TestDropCaseObjectsReleasesOnlyItsOwnPins`, `TestAutoCaptureEvalCase*` (`internal/daemon`), `TestEvalDefaultsCollectWithoutSetup`, `TestRepoConfigCannotChangeEvalCollection`, e2e `TestEvalAutoCaptureJourney`.

docs/src/content/docs/reference/cli.md

@@ -371,7 +371,7 @@ Shows runs newest-first with branch, status (styled), short SHA, timestamp, and
 
 ## no-mistakes eval
 
-Inspect the locally collected review-case corpus before spending tokens, replay an explicit agent and model in isolation, and report verdict accuracy, token cost, wall time, and the accuracy-versus-cost frontier. Eligible cases are collected automatically as runs finish; `eval capture <run-id>` collects one on demand.
+Inspect the locally collected review-case corpus before spending tokens, replay an explicit agent and model in isolation, and report finding-level true-positive / false-negative scores, token cost, wall time, and the recall-versus-cost frontier. Eligible cases are collected automatically as runs finish; `eval capture <run-id>` collects one on demand.
 
 See [Evaluation toolkit](/no-mistakes/reference/eval/) for the local-only boundary, collection and retention, command flags, label policy, and reporting semantics.
 

docs/src/content/docs/reference/environment.md

@@ -21,7 +21,7 @@ When set, everything else moves under this root:
 - Database: `$NM_HOME/state.sqlite`
 - Socket / PID / singleton lock: `$NM_HOME/socket`, `$NM_HOME/daemon.pid`, and `$NM_HOME/daemon.lock`
 - Managed agent server PID records: `$NM_HOME/servers/`
-- Opt-in evaluation cases and registry: `$NM_HOME/eval/` (created only by an explicit `no-mistakes eval` command)
+- Local evaluation cases and registry: `$NM_HOME/eval/` (created by automatic collection or an explicit `no-mistakes eval` command)
 - Managed service names get a short stable suffix derived from `$NM_HOME` so multiple installs don't collide.
 
 ## `NM_DAEMON_CONNECT_TIMEOUT`

docs/src/content/docs/reference/eval.md

@@ -35,10 +35,24 @@ A case includes:
 - agent-neutral global configuration and the effective repository configuration frozen at capture
 - the original run, step, review-round, decision, and local invocation-metric records
 - a manifest with commit pins, changed-file counts, build identity, and a hash of the redacted remote URL
-- a local `labels.json` file that can grow in later evaluation phases
+- a local `labels.json` file that stores finding-level gold and queued unmatched candidate findings
 
 The manifest never stores a remote URL. Capture is read-only against the existing local database and gate. It does not fetch from the network.
 
+## Finding-level gold
+
+The unit of truth is whether a review **finding** was a real issue, scored with scientific terms, not whether the run parked or passed.
+
+Capture writes gold only from recorded human gate evidence. It does not invent labels the human did not give:
+
+- A finding the human selected for Fix (`selected_finding_ids` with a user source) is **true-positive** gold: that finding is a true issue.
+- A finding the human added (`user_findings_json`, source `user`) is **false-negative** gold: the original review missed a real issue.
+- Skip, and approve-with-findings, are **ambiguous**. They do not become invalid, pass, or true-negative gold. The case stays unlabeled / pending until later adjudication.
+- A later replay that raises a new issue absent from the gold set is queued as an unmatched candidate finding. It is never auto-scored as a false positive.
+- A merged pull request is not ground truth.
+
+A case with no finding-level gold is unlabeled / pending, never a pass. True-negative also stays unlabeled because the current capture evidence cannot establish that a finding is invalid.
+
 ## Disk use and retention
 
 Cases from the same repository share one local Git object pool under `<NM_HOME>/eval/pools/`. The first case from a repository stores its history once; every later case adds only the objects its own commits introduced, which is normally a few kilobytes.
@@ -47,7 +61,7 @@ Cases from the same repository share one local Git object pool under `<NM_HOME>/
 
 Because the objects live in the pool rather than inside each case, a case directory is not a portable archive: copying it elsewhere does not carry the code it replays.
 
-Cases captured by releases using manifest version 1 are not compatible with the shared-pool format. If an eval command reports an unsupported case manifest version, remove `<NM_HOME>/eval/` to start a fresh corpus; automatic collection will refill it from later runs.
+Finding-level gold uses `labels.json` schema version 2. There is no migration from labels that store a park/pass verdict, and manifest version 1 cases are also incompatible. If an eval command reports an unsupported case or labels version, remove `<NM_HOME>/eval/` to start a fresh corpus; automatic collection will refill it from later runs.
 
 ## Inspect case sets before spending tokens
 
@@ -55,13 +69,13 @@ Cases captured by releases using manifest version 1 are not compatible with the
 no-mistakes eval sets
 ```
 
-The command shows counts, verdict-label coverage, queued candidate findings, and composition by repository fingerprint, dominant language, change-size bucket, and source severity.
+The command shows counts, finding-level gold coverage, unlabeled / pending cases, queued candidate findings, and composition by repository fingerprint, dominant language, change-size bucket, and source severity.
 
 Three logical sets are available to replay:
 
 - `all` - every captured review pass
-- `labeled` - only cases with a verdict label derived from a recorded human gate decision
-- `diversified` - a deterministic representative, retaining one earliest case per repository, language, size, and expected-verdict bucket
+- `labeled` - only cases with at least one finding-level gold label
+- `diversified` - a deterministic representative, retaining one earliest case per repository, language, size, and gold-status bucket
 
 ## Replay a candidate
 
@@ -72,15 +86,16 @@ no-mistakes eval run \
   --repeats 3
 ```
 
-A candidate is always explicit: `agent+model`. The replay restores each case into a fresh temporary bare gate and worktree, then invokes only the existing Review step. Push, PR, CI, test, lint, document, and fix loops are outside this MVP.
+A candidate is always explicit: `agent+model`. The replay restores each case into a fresh temporary bare gate and worktree, then invokes only the existing Review step. Push, PR, CI, test, lint, document, and fix loops are outside this subject under test.
 
-The captured human gate evidence supplies the verdict policy:
+Replay scores each candidate finding against that gold:
 
-- a user selection recorded for a fix means the candidate should park
-- a skipped Review gate, or a completed gate whose recorded findings required a user decision, means the candidate should pass
-- clean completions, approvals that cannot be established from the persisted round, and other incomplete or ambiguous historical decisions remain unlabeled and are excluded from verdict scoring
+- **true-positive**: the candidate raises the same underlying issue as a human-accepted finding, or finds a human-added miss
+- **false-negative**: the candidate misses a human-accepted finding or a human-added miss
+- **false-positive**: only when a candidate finding is explicitly labeled invalid. Unmatched candidate findings are never treated as false positives
+- **pending / unlabeled**: unmatched candidate findings, and cases with no finding-level gold yet
 
-If a candidate parks on a human-pass case, the finding is queued locally for later adjudication. It is not automatically called wrong. This protects potentially good, unexpected findings until finding-level labeling exists.
+Matching is conservative: findings match by the same finding ID, or by the same file and description after whitespace and case normalization. A candidate that does not raise explicitly invalid gold would be a true-negative, but that outcome remains unlabeled on the current surface.
 
 `--repeats` defaults to `3` and must be at least `1`. Candidates must use an agent that can enforce an explicit model; ACP targets such as `cursor` and `acp:<target>` are rejected. Replays are intentionally isolated from the production `NM_HOME`; they do not contact the shared no-mistakes daemon. The selected agent still communicates with its configured model provider in the normal way.
 
@@ -92,15 +107,17 @@ no-mistakes eval report
 
 The report groups local replays by candidate and cohort. A cohort pins the selected case IDs and repeat count, so frontier comparisons only compare candidates run over the same corpus and repeat plan. It shows:
 
-- confirmed verdict agreement and its conservative lower bound
-- queued unexpected parks and failed candidate invocations
+- finding-level true-positive, false-negative, false-positive, and pending counts
+- recall over gold issues, or unlabeled / pending when a case has no finding-level gold
+- queued unmatched candidate findings, which are not scored as false positives
+- failed candidate invocations
 - reported fresh-input plus output token cost
 - average wall time
-- a 95% Wilson score confidence interval over cases, with repeats averaged inside each case
-- whether a candidate lies on the observed accuracy-versus-token-cost frontier
+- a finite-sample case-level recall range, with repeats averaged inside each case
+- whether a candidate lies on the observed recall-versus-token-cost frontier
 
 The report is deliberately cautious. It never treats an unadjudicated candidate finding as a false positive, excludes candidates with failed replays from the frontier, and distinguishes missing token instrumentation from a real zero.
 
-## MVP boundary
+## Current boundary
 
-The MVP measures verdict-level agreement only. Finding-level valid/invalid labels, an adjudication CLI, matching candidate findings to labels, PR-comment miss scanning, precision/recall/F1, holdouts, sharing, sync, and full-pipeline replay are not part of this command surface.
+Finding-level gold is derived from recorded Fix and add-finding evidence. An adjudication CLI, explicit invalid labels, PR-comment miss scanning, holdouts, sharing, sync, and full-pipeline replay are not part of this command surface.

internal/cli/eval.go

@@ -60,7 +60,7 @@ func newEvalRunCmd() *cobra.Command {
 	var repeats int
 	cmd := &cobra.Command{
 		Use:   "run --cases <all|labeled|diversified> --candidate <agent+model>",
-		Short: "Replay captured review passes in an isolated local sandbox",
+		Short: "Replay captured review passes and score findings against gold",
 		Args:  cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			candidate, err := eval.ParseCandidate(candidateRaw)
@@ -84,7 +84,7 @@ func newEvalRunCmd() *cobra.Command {
 			return nil
 		},
 	}
-	cmd.Flags().StringVar(&cases, "cases", "", "case set: all, labeled, or diversified")
+	cmd.Flags().StringVar(&cases, "cases", "", "case set: all, labeled (finding-level gold), or diversified")
 	cmd.Flags().StringVar(&candidateRaw, "candidate", "", "candidate as agent+model (for example codex+gpt-5.4)")
 	cmd.Flags().IntVar(&repeats, "repeats", 3, "replays per case (minimum 1)")
 	_ = cmd.MarkFlagRequired("cases")
@@ -95,7 +95,7 @@ func newEvalRunCmd() *cobra.Command {
 func newEvalSetsCmd() *cobra.Command {
 	return &cobra.Command{
 		Use:   "sets",
-		Short: "Inspect local case-set size, labels, and diversified composition",
+		Short: "Inspect local case-set size, finding-level gold, and composition",
 		Args:  cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			p, err := paths.New()
@@ -120,7 +120,7 @@ func newEvalSetsCmd() *cobra.Command {
 func newEvalReportCmd() *cobra.Command {
 	return &cobra.Command{
 		Use:   "report",
-		Short: "Report local verdict accuracy, tokens, time, and cost frontier",
+		Short: "Report local true-positive / false-negative scores, tokens, and cost",
 		Args:  cobra.NoArgs,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			p, err := paths.New()

internal/cli/eval_test.go

@@ -1,10 +1,17 @@
 package cli
 
 import (
+	"context"
+	"os"
+	"path/filepath"
 	"strings"
 	"testing"
 
+	"github.com/kunchenguid/no-mistakes/internal/db"
+	"github.com/kunchenguid/no-mistakes/internal/git"
+	"github.com/kunchenguid/no-mistakes/internal/paths"
 	"github.com/kunchenguid/no-mistakes/internal/telemetry"
+	"github.com/kunchenguid/no-mistakes/internal/types"
 )
 
 func TestEvalSetsIsLocalOnlyAndEmitsNoTelemetry(t *testing.T) {
@@ -22,7 +29,110 @@ func TestEvalSetsIsLocalOnlyAndEmitsNoTelemetry(t *testing.T) {
 	if !strings.Contains(out, "LOCAL-ONLY EVAL CASE SETS") {
 		t.Fatalf("output = %q", out)
 	}
+	if strings.Contains(out, "verdict") || strings.Contains(out, "park") || strings.Contains(out, ", pass ") {
+		t.Fatalf("eval sets still uses park/pass accuracy language: %q", out)
+	}
 	if recorder.count("command") != 0 || recorder.count("pageview") != 0 {
 		t.Fatalf("eval emitted remote telemetry: %#v", recorder.events)
 	}
 }
+
+func TestEvalCaptureAndSetsSpeakInFindingGoldTerms(t *testing.T) {
+	ctx := context.Background()
+	root := t.TempDir()
+	t.Setenv("NM_HOME", root)
+	chdir(t, t.TempDir())
+
+	p := paths.WithRoot(root)
+	if err := p.EnsureDirs(); err != nil {
+		t.Fatal(err)
+	}
+	database, err := db.Open(p.DB())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer database.Close()
+
+	gateDir := p.RepoDir("eval-repo")
+	if err := git.InitBare(ctx, gateDir); err != nil {
+		t.Fatal(err)
+	}
+	workDir := filepath.Join(root, "source")
+	mustCLIGit(t, ctx, root, "clone", gateDir, workDir)
+	mustCLIGit(t, ctx, workDir, "config", "user.email", "eval@example.test")
+	mustCLIGit(t, ctx, workDir, "config", "user.name", "Eval Test")
+	if err := os.WriteFile(filepath.Join(workDir, "main.go"), []byte("package sample\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	mustCLIGit(t, ctx, workDir, "add", ".")
+	mustCLIGit(t, ctx, workDir, "commit", "-m", "base")
+	mustCLIGit(t, ctx, workDir, "branch", "-M", "main")
+	mustCLIGit(t, ctx, workDir, "push", "origin", "main")
+	baseSHA := mustCLIGit(t, ctx, workDir, "rev-parse", "HEAD")
+	mustCLIGit(t, ctx, workDir, "checkout", "-b", "feature/eval")
+	if err := os.WriteFile(filepath.Join(workDir, "main.go"), []byte("package sample\n\nfunc Changed() {}\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	mustCLIGit(t, ctx, workDir, "add", "main.go")
+	mustCLIGit(t, ctx, workDir, "commit", "-m", "change")
+	mustCLIGit(t, ctx, workDir, "push", "origin", "feature/eval")
+	headSHA := mustCLIGit(t, ctx, workDir, "rev-parse", "HEAD")
+
+	repo, err := database.InsertRepoWithID("eval-repo", workDir, "https://example.test/org/repo", "main")
+	if err != nil {
+		t.Fatal(err)
+	}
+	run, err := database.InsertRun(repo.ID, "feature/eval", headSHA, baseSHA)
+	if err != nil {
+		t.Fatal(err)
+	}
+	step, err := database.InsertStepResult(run.ID, types.StepReview)
+	if err != nil {
+		t.Fatal(err)
+	}
+	findings := `{"findings":[{"id":"real-bug","severity":"error","file":"main.go","line":3,"description":"bug","action":"ask-user","review_scope":"source"}],"risk_level":"high","risk_rationale":"bug","risk_scope":"source-or-external"}`
+	round, err := database.InsertReviewStepRoundWithProvenance(step.ID, 1, "initial", &findings, nil, headSHA, headSHA, baseSHA, []byte("{}\n"), []byte("{}\n"), 50)
+	if err != nil {
+		t.Fatal(err)
+	}
+	selected := `["real-bug"]`
+	if err := database.SetStepRoundSelection(round.ID, &selected, db.RoundSelectionSourceUser); err != nil {
+		t.Fatal(err)
+	}
+
+	out, err := executeCmd("eval", "capture", run.ID)
+	if err != nil {
+		t.Fatalf("eval capture: %v\n%s", err, out)
+	}
+	if !strings.Contains(out, "captured 1 local review case") {
+		t.Fatalf("capture output = %q", out)
+	}
+
+	out, err = executeCmd("eval", "sets")
+	if err != nil {
+		t.Fatalf("eval sets: %v\n%s", err, out)
+	}
+	if !strings.Contains(out, "1 with finding-level gold (true-positive 1, false-negative 0)") || !strings.Contains(out, "0 unlabeled / pending") {
+		t.Fatalf("sets output = %q, want finding-level gold, not park/pass", out)
+	}
+	if strings.Contains(out, "verdict") || strings.Contains(out, "park") || strings.Contains(out, ", pass ") {
+		t.Fatalf("sets output still uses park/pass accuracy language: %q", out)
+	}
+
+	out, err = executeCmd("eval", "report")
+	if err != nil {
+		t.Fatalf("eval report: %v\n%s", err, out)
+	}
+	if !strings.Contains(out, "LOCAL-ONLY EVAL REPORT") || !strings.Contains(out, "no candidate replays recorded yet") {
+		t.Fatalf("report output = %q", out)
+	}
+}
+
+func mustCLIGit(t *testing.T, ctx context.Context, dir string, args ...string) string {
+	t.Helper()
+	out, err := git.Run(ctx, dir, args...)
+	if err != nil {
+		t.Fatalf("git %v: %v", args, err)
+	}
+	return out
+}

internal/db/round.go

@@ -190,6 +190,20 @@ func (d *DB) SetStepRoundSelection(id string, selectedFindingIDs *string, source
 	return nil
 }
 
+func (d *DB) SetStepRoundUserDecision(id string, selectedFindingIDs *string, source string, userFindingsJSON *string) error {
+	var selectionSource *string
+	if selectedFindingIDs != nil && *selectedFindingIDs != "" && source != "" {
+		selectionSource = &source
+	}
+	if _, err := d.sql.Exec(
+		`UPDATE step_rounds SET selected_finding_ids = ?, selection_source = ?, user_findings_json = ? WHERE id = ?`,
+		selectedFindingIDs, selectionSource, userFindingsJSON, id,
+	); err != nil {
+		return fmt.Errorf("set step round user decision: %w", err)
+	}
+	return nil
+}
+
 // SetStepRoundSelectedFindingIDs preserves the old API for callers that do not
 // need to distinguish how the selection was made.
 func (d *DB) SetStepRoundSelectedFindingIDs(id string, selectedFindingIDs *string) error {

internal/db/round_test.go

@@ -262,7 +262,7 @@ func TestStepRoundCascadeDelete(t *testing.T) {
 	}
 }
 
-func TestSetStepRoundSelectedFindingIDs(t *testing.T) {
+func TestSetStepRoundUserDecision(t *testing.T) {
 	d := openTestDB(t)
 	repo, _ := d.InsertRepo("/home/user/project", "git@github.com:user/project.git", "main")
 	run, _ := d.InsertRun(repo.ID, "feature", "abc", "def")
@@ -275,8 +275,9 @@ func TestSetStepRoundSelectedFindingIDs(t *testing.T) {
 	}
 
 	selected := `["review-1"]`
-	if err := d.SetStepRoundSelection(r.ID, &selected, RoundSelectionSourceUser); err != nil {
-		t.Fatalf("set selected: %v", err)
+	userFindings := `{"findings":[{"id":"user-1","source":"user","description":"missing check"}]}`
+	if err := d.SetStepRoundUserDecision(r.ID, &selected, RoundSelectionSourceUser, &userFindings); err != nil {
+		t.Fatalf("set user decision: %v", err)
 	}
 
 	rounds, err := d.GetRoundsByStep(step.ID)
@@ -292,10 +293,12 @@ func TestSetStepRoundSelectedFindingIDs(t *testing.T) {
 	if rounds[0].SelectionSource == nil || *rounds[0].SelectionSource != RoundSelectionSourceUser {
 		t.Errorf("selection_source = %v, want %q", rounds[0].SelectionSource, RoundSelectionSourceUser)
 	}
+	if rounds[0].UserFindingsJSON == nil || *rounds[0].UserFindingsJSON != userFindings {
+		t.Errorf("user_findings_json = %v, want %q", rounds[0].UserFindingsJSON, userFindings)
+	}
 
-	// Clearing the selection resets the column to NULL.
-	if err := d.SetStepRoundSelection(r.ID, nil, RoundSelectionSourceUser); err != nil {
-		t.Fatalf("clear selected: %v", err)
+	if err := d.SetStepRoundUserDecision(r.ID, nil, RoundSelectionSourceUser, nil); err != nil {
+		t.Fatalf("clear user decision: %v", err)
 	}
 	rounds, err = d.GetRoundsByStep(step.ID)
 	if err != nil {
@@ -307,4 +310,7 @@ func TestSetStepRoundSelectedFindingIDs(t *testing.T) {
 	if rounds[0].SelectionSource != nil {
 		t.Errorf("expected nil selection_source after clear, got %v", rounds[0].SelectionSource)
 	}
+	if rounds[0].UserFindingsJSON != nil {
+		t.Errorf("expected nil user_findings_json after clear, got %v", rounds[0].UserFindingsJSON)
+	}
 }

internal/e2e/eval_test.go

@@ -98,7 +98,7 @@ func TestEvalJourney(t *testing.T) {
 	if err != nil {
 		t.Fatalf("eval report: %v\n%s", err, out)
 	}
-	if !strings.Contains(out, "LOCAL-ONLY EVAL REPORT") || !strings.Contains(out, "claude+claude-opus-4-7") || !strings.Contains(out, "queued unexpected parks: 1") {
+	if !strings.Contains(out, "LOCAL-ONLY EVAL REPORT") || !strings.Contains(out, "claude+claude-opus-4-7") || !strings.Contains(out, "unlabeled / pending") || !strings.Contains(out, "queued unmatched candidate findings: 1") {
 		t.Fatalf("report output = %q", out)
 	}
 	t.Logf("eval report output:\n%s", out)

internal/eval/capture.go

@@ -177,8 +177,8 @@ func Capture(ctx context.Context, store *Store, p *paths.Paths, database *db.DB,
 			return nil, fmt.Errorf("%w: review round %q has no recorded findings", ErrNoCapturableReview, round.ID)
 		}
 		decision := decisionForRound(round, reviewStep)
-		labels := Labels{Version: labelsVersion, Verdict: verdictFromDecision(round, decision)}
-		if !labels.Verdict.Known && (reviewStep.Status == types.StepStatusAwaitingApproval || reviewStep.Status == types.StepStatusFixReview) {
+		labels := goldFromRound(round, decision)
+		if !labels.HasGold() && (reviewStep.Status == types.StepStatusAwaitingApproval || reviewStep.Status == types.StepStatusFixReview) {
 			return nil, fmt.Errorf("%w: review round %q has no recorded gate decision", ErrNoCapturableReview, round.ID)
 		}
 		reviewedSHA := run.HeadSHA
@@ -487,14 +487,89 @@ func decisionForRound(round *db.StepRound, step *db.StepResult) Decision {
 	return decision
 }
 
-func verdictFromDecision(round *db.StepRound, decision Decision) VerdictLabel {
-	if decision.SelectionSource == db.RoundSelectionSourceUser && (len(decision.SelectedFindingIDs) > 0 || decision.HasUserFindings) {
-		return VerdictLabel{Known: true, ShouldPark: true, Source: "recorded-user-fix"}
+// goldFromRound writes only labels the recorded human evidence supports.
+// A user-selected agent finding is true-positive gold. A human-added finding
+// is false-negative gold. Skip and approve-with-findings stay unlabeled.
+func goldFromRound(round *db.StepRound, decision Decision) Labels {
+	labels := Labels{Version: labelsVersion}
+	byID := findingIndex(round)
+	seen := map[string]bool{}
+	if decision.SelectionSource == db.RoundSelectionSourceUser {
+		for _, id := range decision.SelectedFindingIDs {
+			id = strings.TrimSpace(id)
+			if id == "" || seen[id] {
+				continue
+			}
+			finding, ok := byID[id]
+			if !ok {
+				continue
+			}
+			seen[id] = true
+			labels.Findings = append(labels.Findings, goldForRecordedFinding(finding, true))
+		}
 	}
-	if decision.Action == "approve" || decision.Action == "skip" {
-		return VerdictLabel{Known: true, ShouldPark: false, Source: "recorded-human-pass"}
+	if round.UserFindingsJSON == nil || strings.TrimSpace(*round.UserFindingsJSON) == "" {
+		return labels
 	}
-	return VerdictLabel{Source: "unlabeled"}
+	for _, finding := range parseFindingItems(*round.UserFindingsJSON) {
+		if finding.Source != types.FindingSourceUser {
+			continue
+		}
+		id := strings.TrimSpace(finding.ID)
+		if id != "" && seen[id] {
+			continue
+		}
+		if id != "" {
+			seen[id] = true
+		}
+		labels.Findings = append(labels.Findings, goldForRecordedFinding(finding, false))
+	}
+	return labels
+}
+
+func goldForRecordedFinding(finding types.Finding, selected bool) FindingGold {
+	gold := FindingGold{
+		ID:          finding.ID,
+		File:        finding.File,
+		Line:        finding.Line,
+		Description: finding.Description,
+		Severity:    finding.Severity,
+	}
+	if finding.Source == types.FindingSourceUser {
+		gold.Kind = GoldFalseNegative
+		gold.Source = goldSourceUserAdded
+		return gold
+	}
+	if selected {
+		gold.Kind = GoldTruePositive
+		gold.Source = goldSourceUserFix
+		return gold
+	}
+	gold.Kind = GoldTruePositive
+	gold.Source = goldSourceUserFix
+	return gold
+}
+
+func findingIndex(round *db.StepRound) map[string]types.Finding {
+	index := map[string]types.Finding{}
+	if round == nil {
+		return index
+	}
+	if round.FindingsJSON != nil {
+		for _, finding := range parseFindingItems(*round.FindingsJSON) {
+			if id := strings.TrimSpace(finding.ID); id != "" {
+				index[id] = finding
+			}
+		}
+	}
+	if round.UserFindingsJSON != nil {
+		for _, finding := range parseFindingItems(*round.UserFindingsJSON) {
+			if id := strings.TrimSpace(finding.ID); id != "" {
+				index[id] = finding
+			}
+		}
+	}
+	return index
 }
 
 func fingerprint(rawURL string) string {

internal/eval/eval_test.go

@@ -39,8 +39,12 @@ func TestCaptureCreatesPortableReviewCaseWithoutRecordingRemoteURL(t *testing.T)
 	if captured.SourceRunID != run.ID || captured.SourceRoundID != reviewRound.ID {
 		t.Fatalf("capture provenance = %#v, want run %q round %q", captured, run.ID, reviewRound.ID)
 	}
-	if !captured.Labels.Verdict.Known || !captured.Labels.Verdict.ShouldPark {
-		t.Fatalf("verdict label = %#v, want recorded user-fix park label", captured.Labels.Verdict)
+	if !captured.Labels.HasGold() || len(captured.Labels.Findings) != 1 {
+		t.Fatalf("gold labels = %#v, want one recorded user-fix finding", captured.Labels)
+	}
+	gold := captured.Labels.Findings[0]
+	if gold.Kind != GoldTruePositive || gold.Source != goldSourceUserFix || gold.ID != "real-bug" || gold.Description != "bug" {
+		t.Fatalf("true-positive gold = %#v, want recorded user-fix for real-bug", gold)
 	}
 	restored := filepath.Join(t.TempDir(), "restore.git")
 	if err := git.InitBare(ctx, restored); err != nil {
@@ -241,8 +245,8 @@ func TestReplayRestoresCaseIntoAnIsolatedWorktree(t *testing.T) {
 		t.Fatalf("replay = session %#v evaluations %#v", session, evaluations)
 	}
 	got := evaluations[0]
-	if got.Status != "completed" || got.CandidateParked {
-		t.Fatalf("replay outcome = %#v, want completed non-parked review", got)
+	if got.Status != "completed" || got.GoldCount != 1 || got.TruePositive != 0 || got.FalseNegative != 1 || got.Pending != 0 {
+		t.Fatalf("replay outcome = %#v, want a completed miss of the true-positive gold", got)
 	}
 	if !got.TokensReported || got.FreshInputTokens != 12 || got.OutputTokens != 3 {
 		t.Fatalf("replay metrics = %#v", got)
@@ -280,32 +284,77 @@ func TestBaselineForRoundIncludesOnlyCompleteReviewInvocationMetrics(t *testing.
 	}
 }
 
-func TestReportQueuesUnexpectedParksInsteadOfScoringThemWrong(t *testing.T) {
-	summary := SummarizeEvaluations([]Evaluation{
-		{CaseID: "must-park", Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true},
-		{CaseID: "must-park", Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: false},
-		{CaseID: "human-passed", Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(false), CandidateParked: true},
-		{CaseID: "human-passed", Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(false), CandidateParked: false},
-	})
+func TestScoreCandidateMatchesSameFindingID(t *testing.T) {
+	labels := Labels{Findings: []FindingGold{{
+		ID:          "error-handling",
+		Kind:        GoldTruePositive,
+		File:        "old.go",
+		Description: "drops an HTTP error",
+	}}}
+	candidate := `{"findings":[{"id":"error-handling","file":"new.go","description":"drops a database error"}]}`
 
-	if summary.Conclusive != 3 || summary.Correct != 2 || summary.UnexpectedParks != 1 {
-		t.Fatalf("summary = %#v, want 3 conclusive, 2 correct, 1 queued unexpected park", summary)
-	}
-	if got := summary.ConfirmedAccuracy(); got != 2.0/3.0 {
-		t.Fatalf("confirmed accuracy = %v, want %v", got, 2.0/3.0)
-	}
-	if got := summary.LowerBoundAccuracy(); got != 0.5 {
-		t.Fatalf("lower-bound accuracy = %v, want 0.5", got)
+	score := ScoreCandidate(labels, candidate)
+	if score.TruePositive != 1 || score.FalseNegative != 0 || score.Pending != 0 {
+		t.Fatalf("score = %#v, want same finding ID matched", score)
 	}
 }
 
-func TestFailedLabeledReplayCountsAgainstAccuracyAndFrontier(t *testing.T) {
+func TestScoreCandidateMatchesNormalizedFileAndDescription(t *testing.T) {
+	labels := Labels{Findings: []FindingGold{{ID: "review-1", Kind: GoldTruePositive, File: " internal/eval/score.go ", Description: "Drops   an HTTP Error"}}}
+	candidate := `{"findings":[{"id":"different","file":"internal/eval/score.go","description":"drops an http error"}]}`
+
+	score := ScoreCandidate(labels, candidate)
+	if score.TruePositive != 1 || score.FalseNegative != 0 || score.Pending != 0 {
+		t.Fatalf("score = %#v, want conservative file-and-description match", score)
+	}
+}
+
+func TestScoreCandidateDoesNotMatchFindingsWithoutFiles(t *testing.T) {
+	labels := Labels{Findings: []FindingGold{{ID: "gold", Kind: GoldTruePositive, Description: "drops an HTTP error"}}}
+	candidate := `{"findings":[{"id":"candidate","description":"drops an http error"}]}`
+
+	score := ScoreCandidate(labels, candidate)
+	if score.TruePositive != 0 || score.FalseNegative != 1 || score.Pending != 1 {
+		t.Fatalf("score = %#v, want file-less findings left unmatched", score)
+	}
+}
+
+func TestSummarizeEvaluationsScoresFindingGoldAndLeavesUnmatchedPending(t *testing.T) {
 	summary := SummarizeEvaluations([]Evaluation{
-		{Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true},
-		{Candidate: "claude+test", Status: "failed", ExpectedPark: boolPtr(true)},
+		{CaseID: "fix-gold", Candidate: "claude+test", Status: "completed", HasFindingGold: true, GoldCount: 1, TruePositive: 1},
+		{CaseID: "fix-gold", Candidate: "claude+test", Status: "completed", HasFindingGold: true, GoldCount: 1, FalseNegative: 1},
+		{CaseID: "approve-unlabeled", Candidate: "claude+test", Status: "completed", Pending: 2},
+		{CaseID: "approve-unlabeled", Candidate: "claude+test", Status: "completed"},
 	})
-	if summary.Labeled != 2 || summary.Conclusive != 2 || summary.Correct != 1 || summary.Misses != 1 || summary.ConfirmedAccuracy() != 0.5 {
-		t.Fatalf("summary = %#v, want failed labeled replay scored conservatively", summary)
+
+	if summary.Labeled != 2 || summary.TruePositive != 1 || summary.FalseNegative != 1 || summary.FalsePositive != 0 || summary.Pending != 2 {
+		t.Fatalf("summary = %#v, want TP/FN gold plus queued unmatched findings", summary)
+	}
+	if got := summary.Recall(); got != 0.5 {
+		t.Fatalf("recall = %v, want 0.5", got)
+	}
+}
+
+func TestSummarizeEvaluationsKeepsExplicitInvalidOnlyScoresLabeled(t *testing.T) {
+	summary := SummarizeEvaluations([]Evaluation{{
+		Candidate:      "claude+test",
+		Status:         "completed",
+		HasFindingGold: true,
+		FalsePositive:  1,
+	}})
+
+	if summary.Labeled != 1 || summary.FalsePositive != 1 {
+		t.Fatalf("summary = %#v, want explicit-invalid-only evaluation retained", summary)
+	}
+}
+
+func TestFailedLabeledReplayCountsAsFalseNegativeAndBlocksFrontier(t *testing.T) {
+	summary := SummarizeEvaluations([]Evaluation{
+		{Candidate: "claude+test", Status: "completed", GoldCount: 1, TruePositive: 1},
+		{Candidate: "claude+test", Status: "failed", GoldCount: 1, FalseNegative: 1},
+	})
+	if summary.Labeled != 2 || summary.TruePositive != 1 || summary.FalseNegative != 1 || summary.Recall() != 0.5 {
+		t.Fatalf("summary = %#v, want failed labeled replay counted as a false-negative", summary)
 	}
 	cost := 10.0
 	reports := []CandidateReport{{Cohort: "same", Summary: summary, AverageTokens: &cost}}
@@ -326,7 +375,7 @@ func TestPersistEvaluationQueuesEveryUnexpectedCandidateFinding(t *testing.T) {
 	if err := os.MkdirAll(caseDir, 0o755); err != nil {
 		t.Fatal(err)
 	}
-	labels := Labels{Version: 1, Verdict: VerdictLabel{Known: true}}
+	labels := Labels{Version: labelsVersion}
 	if err := writeJSON(filepath.Join(caseDir, "labels.json"), labels); err != nil {
 		t.Fatal(err)
 	}
@@ -335,15 +384,14 @@ func TestPersistEvaluationQueuesEveryUnexpectedCandidateFinding(t *testing.T) {
 		t.Fatal(err)
 	}
 	if err := store.persistEvaluation(c, Evaluation{
-		ID:              "evaluation",
-		SessionID:       "session",
-		CaseID:          c.ID,
-		Candidate:       "claude+test",
-		Repeat:          1,
-		Status:          "completed",
-		ExpectedPark:    boolPtr(false),
-		CandidateParked: true,
-		FindingCount:    3,
+		ID:           "evaluation",
+		SessionID:    "session",
+		CaseID:       c.ID,
+		Candidate:    "claude+test",
+		Repeat:       1,
+		Status:       "completed",
+		Pending:      3,
+		FindingCount: 3,
 	}); err != nil {
 		t.Fatal(err)
 	}
@@ -378,7 +426,7 @@ func TestBaselineForRoundDerivesFreshTokensFromPerRoundDeltas(t *testing.T) {
 }
 
 func TestConfidenceIntervalRequiresMultipleIndependentCases(t *testing.T) {
-	rows := []Evaluation{{CaseID: "only", Candidate: "claude+test", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true}}
+	rows := []Evaluation{{CaseID: "only", Candidate: "claude+test", Status: "completed", GoldCount: 1, TruePositive: 1}}
 	if got := confidenceInterval("claude+test", rows); got != nil {
 		t.Fatalf("single-case confidence interval = %#v, want unavailable", got)
 	}
@@ -386,8 +434,8 @@ func TestConfidenceIntervalRequiresMultipleIndependentCases(t *testing.T) {
 
 func TestConfidenceIntervalRepresentsUniformSampleUncertainty(t *testing.T) {
 	rows := []Evaluation{
-		{CaseID: "one", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true},
-		{CaseID: "two", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true},
+		{CaseID: "one", Status: "completed", GoldCount: 1, TruePositive: 1},
+		{CaseID: "two", Status: "completed", GoldCount: 1, TruePositive: 1},
 	}
 	got := confidenceInterval("claude+test", rows)
 	if got == nil || got.Lower <= 0 || got.Lower >= 1 || got.Upper < 0.999 {
@@ -397,8 +445,8 @@ func TestConfidenceIntervalRepresentsUniformSampleUncertainty(t *testing.T) {
 
 func TestConfidenceIntervalIncludesFailedLabeledReplays(t *testing.T) {
 	rows := []Evaluation{
-		{CaseID: "passed", Status: "completed", ExpectedPark: boolPtr(true), CandidateParked: true},
-		{CaseID: "failed", Status: "failed", ExpectedPark: boolPtr(true)},
+		{CaseID: "passed", Status: "completed", GoldCount: 1, TruePositive: 1},
+		{CaseID: "failed", Status: "failed", GoldCount: 1, FalseNegative: 1},
 	}
 	got := confidenceInterval("claude+test", rows)
 	if got == nil || got.Cases != 2 || got.Lower >= 0.5 || got.Upper <= 0.5 {
@@ -406,16 +454,31 @@ func TestConfidenceIntervalIncludesFailedLabeledReplays(t *testing.T) {
 	}
 }
 
-func TestRenderReportNamesWilsonScoreInterval(t *testing.T) {
+func TestRenderReportNamesCaseLevelRecallRange(t *testing.T) {
 	output := RenderReport([]CandidateReport{
 		{
 			Cohort:     "cohort",
-			Summary:    EvaluationSummary{Candidate: "claude+test", Total: 2, Labeled: 2, Conclusive: 2, Correct: 2},
+			Summary:    EvaluationSummary{Candidate: "claude+test", Total: 2, Labeled: 2, TruePositive: 2},
 			Confidence: &Interval{Lower: 0.34, Upper: 1, Cases: 2},
 		},
 	})
-	if !strings.Contains(output, "95% Wilson score CI: 34.0%-100.0% over 2 case(s)") {
-		t.Fatalf("report confidence interval = %q", output)
+	if !strings.Contains(output, "case-level recall range: 34.0%-100.0% over 2 case(s)") {
+		t.Fatalf("report recall range = %q", output)
+	}
+}
+
+func TestRenderReportKeepsInvalidOnlyScoreWithoutClaimingRecall(t *testing.T) {
+	cost := 10.0
+	output := RenderReport([]CandidateReport{{
+		Cohort:        "cohort",
+		Summary:       EvaluationSummary{Candidate: "claude+test", Total: 1, Labeled: 1, FalsePositive: 1},
+		AverageTokens: &cost,
+	}})
+	if !strings.Contains(output, "false-positive 1") || !strings.Contains(output, "recall: unavailable (no true-issue gold)") {
+		t.Fatalf("invalid-only report = %q, want FP score with unavailable recall", output)
+	}
+	if strings.Contains(output, "0/0 gold issues") || strings.Contains(output, "recall-vs-cost frontier: true") {
+		t.Fatalf("invalid-only report claims recall evidence: %q", output)
 	}
 }
 
@@ -433,8 +496,8 @@ func TestFrontierDoesNotCompareDifferentCohorts(t *testing.T) {
 	cheap := 10.0
 	expensive := 100.0
 	reports := []CandidateReport{
-		{Cohort: "a", Summary: EvaluationSummary{Labeled: 1, Correct: 1}, AverageTokens: &expensive},
-		{Cohort: "b", Summary: EvaluationSummary{Labeled: 1, Correct: 1}, AverageTokens: &cheap},
+		{Cohort: "a", Summary: EvaluationSummary{Labeled: 1, TruePositive: 1}, AverageTokens: &expensive},
+		{Cohort: "b", Summary: EvaluationSummary{Labeled: 1, TruePositive: 1}, AverageTokens: &cheap},
 	}
 	markFrontier(reports)
 	if !reports[0].OnFrontier || !reports[1].OnFrontier {
@@ -442,6 +505,218 @@ func TestFrontierDoesNotCompareDifferentCohorts(t *testing.T) {
 	}
 }
 
+func TestCaptureDoesNotLabelSkipOrApproveAsPass(t *testing.T) {
+	for _, tc := range []struct {
+		name   string
+		status types.StepStatus
+	}{
+		{name: "approve-with-findings", status: types.StepStatusCompleted},
+		{name: "skip", status: types.StepStatusSkipped},
+	} {
+		t.Run(tc.name, func(t *testing.T) {
+			ctx := context.Background()
+			p, sourceDB, run, _, reviewRound := setupCapturedRun(t, ctx)
+			defer sourceDB.Close()
+			if err := sourceDB.SetStepRoundSelection(reviewRound.ID, nil, ""); err != nil {
+				t.Fatal(err)
+			}
+			steps, err := sourceDB.GetStepsByRun(run.ID)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if err := sourceDB.UpdateStepStatus(steps[0].ID, tc.status); err != nil {
+				t.Fatal(err)
+			}
+			store, err := Open(p.EvalDir())
+			if err != nil {
+				t.Fatal(err)
+			}
+			defer store.Close()
+			cases, err := Capture(ctx, store, p, sourceDB, run.ID)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if len(cases) != 1 || cases[0].Labels.HasGold() {
+				t.Fatalf("captured labels = %#v, want unlabeled pending gold", cases)
+			}
+			output := RenderSets(mustInspectSets(t, store))
+			if !strings.Contains(output, "0 with finding-level gold") || !strings.Contains(output, "1 unlabeled / pending") {
+				t.Fatalf("sets output = %q, want unlabeled / pending, not a pass", output)
+			}
+			if strings.Contains(output, "park") || strings.Contains(output, "verdict") || strings.Contains(output, ", pass ") {
+				t.Fatalf("sets output still uses park/pass accuracy language: %q", output)
+			}
+		})
+	}
+}
+
+func TestCaptureWritesFalseNegativeGoldForUserAddedFinding(t *testing.T) {
+	ctx := context.Background()
+	p, sourceDB, run, _, reviewRound := setupCapturedRun(t, ctx)
+	defer sourceDB.Close()
+	userFindings := `{"findings":[{"id":"real-bug","severity":"error","file":"main.go","line":3,"description":"bug","action":"ask-user","review_scope":"source"},{"id":"user-1","severity":"warning","file":"main.go","line":1,"description":"missing audit","action":"auto-fix","source":"user"}],"risk_level":"high","risk_rationale":"bug","risk_scope":"source-or-external"}`
+	if err := sourceDB.SetStepRoundUserFindings(reviewRound.ID, &userFindings); err != nil {
+		t.Fatal(err)
+	}
+	selected := `["real-bug","user-1"]`
+	if err := sourceDB.SetStepRoundSelection(reviewRound.ID, &selected, db.RoundSelectionSourceUser); err != nil {
+		t.Fatal(err)
+	}
+
+	store, err := Open(p.EvalDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer store.Close()
+	cases, err := Capture(ctx, store, p, sourceDB, run.ID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(cases) != 1 || len(cases[0].Labels.Findings) != 2 {
+		t.Fatalf("captured gold = %#v, want accepted finding plus human-added miss", cases)
+	}
+	byID := map[string]FindingGold{}
+	for _, gold := range cases[0].Labels.Findings {
+		byID[gold.ID] = gold
+	}
+	if got := byID["real-bug"]; got.Kind != GoldTruePositive || got.Source != goldSourceUserFix {
+		t.Fatalf("selected agent finding gold = %#v, want true-positive", got)
+	}
+	if got := byID["user-1"]; got.Kind != GoldFalseNegative || got.Source != goldSourceUserAdded || got.Description != "missing audit" {
+		t.Fatalf("user-added gold = %#v, want false-negative miss", got)
+	}
+}
+
+func TestCaptureWritesUserAddedGoldWithoutSelectionSource(t *testing.T) {
+	ctx := context.Background()
+	p, sourceDB, run, _, reviewRound := setupCapturedRun(t, ctx)
+	defer sourceDB.Close()
+	userFindings := `{"findings":[{"id":"user-1","severity":"warning","file":"main.go","line":1,"description":"missing audit","action":"auto-fix","source":"user"}],"risk_level":"high","risk_rationale":"bug","risk_scope":"source-or-external"}`
+	if err := sourceDB.SetStepRoundUserFindings(reviewRound.ID, &userFindings); err != nil {
+		t.Fatal(err)
+	}
+	if err := sourceDB.SetStepRoundSelection(reviewRound.ID, nil, ""); err != nil {
+		t.Fatal(err)
+	}
+
+	store, err := Open(p.EvalDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer store.Close()
+	cases, err := Capture(ctx, store, p, sourceDB, run.ID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(cases) != 1 || len(cases[0].Labels.Findings) != 1 {
+		t.Fatalf("captured gold = %#v, want independent human-added miss", cases)
+	}
+	if got := cases[0].Labels.Findings[0]; got.ID != "user-1" || got.Kind != GoldFalseNegative || got.Source != goldSourceUserAdded {
+		t.Fatalf("user-added gold = %#v, want false-negative without selection evidence", got)
+	}
+}
+
+func TestCaptureLeavesUnknownSelectedFindingUnlabeled(t *testing.T) {
+	ctx := context.Background()
+	p, sourceDB, run, _, reviewRound := setupCapturedRun(t, ctx)
+	defer sourceDB.Close()
+	selected := `["user-added-write-was-lost"]`
+	if err := sourceDB.SetStepRoundSelection(reviewRound.ID, &selected, db.RoundSelectionSourceUser); err != nil {
+		t.Fatal(err)
+	}
+
+	store, err := Open(p.EvalDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer store.Close()
+	cases, err := Capture(ctx, store, p, sourceDB, run.ID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(cases) != 1 || cases[0].Labels.HasGold() {
+		t.Fatalf("captured labels = %#v, want unknown selection left unlabeled", cases)
+	}
+}
+
+func TestCaptureAndReportScoresMatchingCandidateAsTruePositive(t *testing.T) {
+	ctx := context.Background()
+	p, sourceDB, run, _, _ := setupCapturedRun(t, ctx)
+	defer sourceDB.Close()
+	installFakeReviewAgent(t, p, `{"findings":[{"id":"other","severity":"error","file":"main.go","line":3,"description":"bug","action":"ask-user","review_scope":"source"}],"risk_level":"high","risk_rationale":"bug","risk_scope":"source-or-external"}`)
+
+	store, err := Open(p.EvalDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer store.Close()
+	if _, err := Capture(ctx, store, p, sourceDB, run.ID); err != nil {
+		t.Fatal(err)
+	}
+	if _, evaluations, err := Replay(ctx, store, ReplayOptions{Set: "labeled", Candidate: Candidate{Agent: types.AgentClaude, Model: "test"}, Repeats: 1}); err != nil {
+		t.Fatal(err)
+	} else if len(evaluations) != 1 || evaluations[0].TruePositive != 1 || evaluations[0].FalseNegative != 0 || evaluations[0].Pending != 0 {
+		t.Fatalf("replay scores = %#v, want true-positive match on the same issue", evaluations)
+	}
+	reports, err := Report(store)
+	if err != nil {
+		t.Fatal(err)
+	}
+	output := RenderReport(reports)
+	if !strings.Contains(output, "true-positive 1, false-negative 0, false-positive 0, pending 0") || !strings.Contains(output, "recall: 100.0%") {
+		t.Fatalf("report = %q, want true-positive recall", output)
+	}
+	if strings.Contains(output, "park") || strings.Contains(output, "verdict") || strings.Contains(output, "agreement") {
+		t.Fatalf("report still uses park/pass accuracy language: %q", output)
+	}
+}
+
+func TestCaptureAndReportLeavesUnmatchedCandidateFindingsPending(t *testing.T) {
+	ctx := context.Background()
+	p, sourceDB, run, _, reviewRound := setupCapturedRun(t, ctx)
+	defer sourceDB.Close()
+	if err := sourceDB.SetStepRoundSelection(reviewRound.ID, nil, ""); err != nil {
+		t.Fatal(err)
+	}
+	steps, err := sourceDB.GetStepsByRun(run.ID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := sourceDB.UpdateStepStatus(steps[0].ID, types.StepStatusCompleted); err != nil {
+		t.Fatal(err)
+	}
+	installFakeReviewAgent(t, p, `{"findings":[{"id":"new-issue","severity":"error","file":"main.go","line":3,"description":"unexpected later issue","action":"ask-user","review_scope":"source"}],"risk_level":"high","risk_rationale":"new","risk_scope":"source-or-external"}`)
+
+	store, err := Open(p.EvalDir())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer store.Close()
+	cases, err := Capture(ctx, store, p, sourceDB, run.ID)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if len(cases) != 1 || cases[0].Labels.HasGold() {
+		t.Fatalf("approve capture = %#v, want unlabeled gold", cases)
+	}
+	if _, evaluations, err := Replay(ctx, store, ReplayOptions{Set: "all", Candidate: Candidate{Agent: types.AgentClaude, Model: "test"}, Repeats: 1}); err != nil {
+		t.Fatal(err)
+	} else if len(evaluations) != 1 || evaluations[0].FalsePositive != 0 || evaluations[0].Pending != 1 {
+		t.Fatalf("replay scores = %#v, want unmatched finding queued, not a false-positive", evaluations)
+	}
+	reports, err := Report(store)
+	if err != nil {
+		t.Fatal(err)
+	}
+	output := RenderReport(reports)
+	if !strings.Contains(output, "unlabeled / pending") || !strings.Contains(output, "queued unmatched candidate findings: 1") {
+		t.Fatalf("report = %q, want unlabeled pending, not a pass or false-positive", output)
+	}
+	if strings.Contains(output, "false-positive 1") || strings.Contains(output, "park") || strings.Contains(output, "verdict") {
+		t.Fatalf("report punished or passed an unlabeled approve case: %q", output)
+	}
+}
+
 func TestParseCandidateRequiresAgentAndModel(t *testing.T) {
 	for _, input := range []string{"claude", "+model", "claude+", "claude+model+extra", "cursor+model", "acp:custom+model"} {
 		if _, err := ParseCandidate(input); err == nil {
@@ -544,4 +819,31 @@ func mustGit(t *testing.T, ctx context.Context, dir string, args ...string) stri
 	return out
 }
 
-func boolPtr(v bool) *bool { return &v }
+func mustInspectSets(t *testing.T, store *Store) []SetSummary {
+	t.Helper()
+	summaries, err := InspectSets(store)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return summaries
+}
+
+func installFakeReviewAgent(t *testing.T, p *paths.Paths, findingsJSON string) {
+	t.Helper()
+	fakeDir := t.TempDir()
+	fake := filepath.Join(fakeDir, "claude")
+	reply := `{"type":"assistant","message":{"usage":{"input_tokens":12,"output_tokens":3},"content":[{"type":"text","text":"review"}]}}
+{"type":"result","subtype":"success","is_error":false,"structured_output":` + findingsJSON + `,"usage":{"input_tokens":12,"output_tokens":3}}
+`
+	var script string
+	if runtime.GOOS == "windows" {
+		fake += ".cmd"
+		script = "@echo off\r\nmore >nul\r\necho " + strings.ReplaceAll(strings.TrimSpace(reply), "\n", "\r\necho ") + "\r\n"
+	} else {
+		script = "#!/bin/sh\n[ \"$NM_HOME\" = \"" + p.Root() + "\" ] && touch \"" + p.Root() + "/shared-home-used\"\ncat >/dev/null\ncat <<'EOF'\n" + reply + "EOF\n"
+	}
+	if err := os.WriteFile(fake, []byte(script), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("PATH", filepath.Dir(fake)+string(os.PathListSeparator)+os.Getenv("PATH"))
+}

internal/eval/model.go

@@ -22,7 +22,24 @@ const (
 	// Store.poolDir). A version-1 case on disk points at a bundle this code no
 	// longer reads, so it is rejected on load rather than half-restored.
 	manifestVersion = 2
-	labelsVersion   = 1
+	// labelsVersion is 2 because the unit of truth is a finding-level
+	// confusion-matrix gold label, not a case-level park/pass verdict.
+	// Version-1 labels.json files are rejected on load.
+	labelsVersion = 2
+)
+
+// Gold kinds are the scientific labels written from recorded human evidence.
+// Capture only writes true-positive and false-negative gold. False-positive
+// gold requires an explicit invalid label and is never inferred.
+const (
+	GoldTruePositive  = "true-positive"
+	GoldFalseNegative = "false-negative"
+	GoldFalsePositive = "false-positive"
+)
+
+const (
+	goldSourceUserFix   = "recorded-user-fix"
+	goldSourceUserAdded = "recorded-user-added"
 )
 
 // Candidate identifies one agent and model combination under evaluation. The
@@ -92,20 +109,44 @@ type Decision struct {
 	HasUserFindings    bool     `json:"has_user_findings"`
 }
 
-// VerdictLabel is intentionally only verdict-level in the MVP. Finding-level
-// valid/invalid labels and their adjudication UI belong to phase 1.
-type VerdictLabel struct {
-	Known      bool   `json:"known"`
-	ShouldPark bool   `json:"should_park"`
-	Source     string `json:"source,omitempty"`
+// FindingGold is one human-given label for an underlying issue. Capture writes
+// only what the recorded gate evidence supports: a user-selected Fix is a
+// true-positive gold issue, and a user-added finding is a false-negative gold
+// miss. Skip, approve-with-findings, and unmatched later candidate findings
+// stay unlabeled.
+type FindingGold struct {
+	ID          string `json:"id"`
+	Kind        string `json:"kind"`
+	Source      string `json:"source,omitempty"`
+	File        string `json:"file,omitempty"`
+	Line        int    `json:"line,omitempty"`
+	Description string `json:"description,omitempty"`
+	Severity    string `json:"severity,omitempty"`
 }
 
-// Labels is a local, growing label file. Queued candidate findings are kept as
-// evidence for a future adjudication pass, never scored as false positives.
+// Labels is a local, growing label file. Finding-level gold is the unit of
+// truth. Queued candidate findings are kept as evidence for later
+// adjudication and are never scored as false positives.
 type Labels struct {
-	Version                 int          `json:"version"`
-	Verdict                 VerdictLabel `json:"verdict"`
-	QueuedCandidateFindings int          `json:"queued_candidate_findings"`
+	Version                 int           `json:"version"`
+	Findings                []FindingGold `json:"findings,omitempty"`
+	QueuedCandidateFindings int           `json:"queued_candidate_findings"`
+}
+
+func (l Labels) HasGold() bool { return len(l.Findings) > 0 }
+
+func (l Labels) TrueIssueCount() int {
+	n := 0
+	for _, finding := range l.Findings {
+		if isTrueIssueGold(finding.Kind) {
+			n++
+		}
+	}
+	return n
+}
+
+func isTrueIssueGold(kind string) bool {
+	return kind == GoldTruePositive || kind == GoldFalseNegative
 }
 
 // BaselineMetrics is the recorded source-review performance baseline. A false
@@ -131,6 +172,8 @@ type Case struct {
 
 // Evaluation is one candidate replay over one case. Status is "completed" or
 // "failed"; failures remain visible in reports and are not silently scored.
+// Confusion-matrix fields are finding-level: unmatched candidate findings stay
+// in Pending and are never treated as false positives.
 type Evaluation struct {
 	ID               string `json:"id"`
 	SessionID        string `json:"session_id"`
@@ -142,8 +185,12 @@ type Evaluation struct {
 	CompletedAt      int64  `json:"completed_at"`
 	Status           string `json:"status"`
 	Error            string `json:"error,omitempty"`
-	ExpectedPark     *bool  `json:"expected_park,omitempty"`
-	CandidateParked  bool   `json:"candidate_parked"`
+	HasFindingGold   bool   `json:"has_finding_gold"`
+	GoldCount        int    `json:"gold_count"`
+	TruePositive     int    `json:"true_positive"`
+	FalseNegative    int    `json:"false_negative"`
+	FalsePositive    int    `json:"false_positive"`
+	Pending          int    `json:"pending"`
 	FindingsJSON     string `json:"findings_json,omitempty"`
 	FindingCount     int    `json:"finding_count"`
 	InputTokens      int64  `json:"input_tokens"`
@@ -155,17 +202,17 @@ type Evaluation struct {
 	Model            string `json:"model,omitempty"`
 }
 
-// EvaluationSummary is deliberately three-valued for a human-pass label:
-// an unexpected candidate park is queued rather than declared wrong before
-// finding-level adjudication exists.
+// EvaluationSummary aggregates finding-level scores. A case with no gold is
+// unlabeled / pending, never a pass. Unmatched candidate findings stay in
+// Pending and do not become false positives.
 type EvaluationSummary struct {
 	Candidate        string
 	Total            int
 	Labeled          int
-	Conclusive       int
-	Correct          int
-	Misses           int
-	UnexpectedParks  int
+	TruePositive     int
+	FalseNegative    int
+	FalsePositive    int
+	Pending          int
 	Failures         int
 	InputTokens      int64
 	OutputTokens     int64
@@ -174,25 +221,16 @@ type EvaluationSummary struct {
 	DurationMS       int64
 }
 
-func (s EvaluationSummary) ConfirmedAccuracy() float64 {
-	if s.Conclusive == 0 {
+func (s EvaluationSummary) Recall() float64 {
+	denom := s.TruePositive + s.FalseNegative
+	if denom == 0 {
 		return 0
 	}
-	return float64(s.Correct) / float64(s.Conclusive)
+	return float64(s.TruePositive) / float64(denom)
 }
 
-// LowerBoundAccuracy counts a queued unexpected park in the denominator but
-// not the numerator. It is the conservative number suitable for comparing
-// candidates before phase-1 finding adjudication is available.
-func (s EvaluationSummary) LowerBoundAccuracy() float64 {
-	if s.Labeled == 0 {
-		return 0
-	}
-	return float64(s.Correct) / float64(s.Labeled)
-}
-
-// SummarizeEvaluations applies the MVP verdict policy without inferring that a
-// new finding is invalid merely because the original human passed the run.
+// SummarizeEvaluations scores finding-level gold only. Unmatched candidate
+// findings stay pending. A replay with no gold is unlabeled, not a pass.
 func SummarizeEvaluations(evaluations []Evaluation) EvaluationSummary {
 	var summary EvaluationSummary
 	for _, evaluation := range evaluations {
@@ -207,32 +245,24 @@ func SummarizeEvaluations(evaluations []Evaluation) EvaluationSummary {
 		if evaluation.TokensReported {
 			summary.TokensReported++
 		}
+		hasFindingGold := evaluation.HasFindingGold || evaluation.GoldCount > 0
 		if evaluation.Status != "completed" {
 			summary.Failures++
-			if evaluation.ExpectedPark != nil {
+			if hasFindingGold {
 				summary.Labeled++
-				summary.Conclusive++
-				summary.Misses++
+				summary.FalseNegative += evaluation.GoldCount
 			}
+			summary.Pending += evaluation.Pending
 			continue
 		}
-		if evaluation.ExpectedPark == nil {
+		summary.Pending += evaluation.Pending
+		if !hasFindingGold {
 			continue
 		}
 		summary.Labeled++
-		switch {
-		case *evaluation.ExpectedPark && evaluation.CandidateParked:
-			summary.Conclusive++
-			summary.Correct++
-		case *evaluation.ExpectedPark && !evaluation.CandidateParked:
-			summary.Conclusive++
-			summary.Misses++
-		case !*evaluation.ExpectedPark && !evaluation.CandidateParked:
-			summary.Conclusive++
-			summary.Correct++
-		case !*evaluation.ExpectedPark && evaluation.CandidateParked:
-			summary.UnexpectedParks++
-		}
+		summary.TruePositive += evaluation.TruePositive
+		summary.FalseNegative += evaluation.FalseNegative
+		summary.FalsePositive += evaluation.FalsePositive
 	}
 	return summary
 }

internal/eval/replay.go

@@ -172,9 +172,9 @@ func (s *Store) releaseReplayReservation(sessionID string) {
 	_, _ = s.db.Exec(`DELETE FROM replay_case_reservations WHERE session_id = ?`, sessionID)
 }
 
-func replayOne(ctx context.Context, store *Store, c Case, session Session, candidate Candidate, repeat int) Evaluation {
+func replayOne(ctx context.Context, store *Store, c Case, session Session, candidate Candidate, repeat int) (evaluation Evaluation) {
 	started := time.Now()
-	evaluation := Evaluation{
+	evaluation = Evaluation{
 		ID:        newSessionID(),
 		SessionID: session.ID,
 		CaseID:    c.ID,
@@ -184,10 +184,16 @@ func replayOne(ctx context.Context, store *Store, c Case, session Session, candi
 		StartedAt: started.Unix(),
 		Status:    "failed",
 	}
-	if c.Labels.Verdict.Known {
-		expected := c.Labels.Verdict.ShouldPark
-		evaluation.ExpectedPark = &expected
-	}
+	evaluation.HasFindingGold = c.Labels.HasGold()
+	evaluation.GoldCount = c.Labels.TrueIssueCount()
+	defer func() {
+		if evaluation.Status != "completed" {
+			evaluation.FalseNegative = evaluation.GoldCount
+			if evaluation.CompletedAt == 0 {
+				evaluation.CompletedAt = time.Now().Unix()
+			}
+		}
+	}()
 
 	root, err := os.MkdirTemp("", "nm-eval-replay-")
 	if err != nil {
@@ -329,9 +335,13 @@ func replayOne(ctx context.Context, store *Store, c Case, session Session, candi
 		return evaluation
 	}
 	evaluation.Status = "completed"
-	evaluation.CandidateParked = outcome.NeedsApproval || hasAskUserFindings(outcome.Findings)
 	evaluation.FindingsJSON = outcome.Findings
 	evaluation.FindingCount = findingCount(outcome.Findings)
+	score := ScoreCandidate(c.Labels, outcome.Findings)
+	evaluation.TruePositive = score.TruePositive
+	evaluation.FalseNegative = score.FalseNegative
+	evaluation.FalsePositive = score.FalsePositive
+	evaluation.Pending = score.Pending
 	return evaluation
 }
 
@@ -496,11 +506,6 @@ func (a *observedAgent) Run(ctx context.Context, opts agent.RunOpts) (*agent.Res
 	return result, err
 }
 
-func hasAskUserFindings(raw string) bool {
-	findings, err := types.ParseFindingsJSON(raw)
-	return err == nil && types.HasAskUserFindings(findings)
-}
-
 func findingCount(raw string) int {
 	findings, err := types.ParseFindingsJSON(raw)
 	if err != nil {
@@ -522,33 +527,22 @@ func (s *Store) persistEvaluation(c Case, evaluation Evaluation) error {
 	if err := writeJSON(path, evaluation); err != nil {
 		return fmt.Errorf("write eval result: %w", err)
 	}
-	var expected any
-	if evaluation.ExpectedPark != nil {
-		if *evaluation.ExpectedPark {
-			expected = 1
-		} else {
-			expected = 0
-		}
-	}
-	parked := 0
-	if evaluation.CandidateParked {
-		parked = 1
-	}
 	reported := 0
 	if evaluation.TokensReported {
 		reported = 1
 	}
 	_, err := s.db.Exec(`INSERT INTO evaluations
-(id, session_id, case_id, candidate, repeat_number, started_at, completed_at, status, expected_park, candidate_parked, tokens_reported, input_tokens, output_tokens, fresh_input_tokens, duration_ms, path)
-VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+(id, session_id, case_id, candidate, repeat_number, started_at, completed_at, status, gold_count, true_positive, false_negative, false_positive, pending, tokens_reported, input_tokens, output_tokens, fresh_input_tokens, duration_ms, path)
+VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
 		evaluation.ID, evaluation.SessionID, evaluation.CaseID, evaluation.Candidate, evaluation.Repeat,
-		evaluation.StartedAt, evaluation.CompletedAt, evaluation.Status, expected, parked, reported,
+		evaluation.StartedAt, evaluation.CompletedAt, evaluation.Status, evaluation.GoldCount,
+		evaluation.TruePositive, evaluation.FalseNegative, evaluation.FalsePositive, evaluation.Pending, reported,
 		evaluation.InputTokens, evaluation.OutputTokens, evaluation.FreshInputTokens, evaluation.DurationMS, path)
 	if err != nil {
 		return fmt.Errorf("record eval result: %w", err)
 	}
-	if evaluation.Status == "completed" && evaluation.ExpectedPark != nil && !*evaluation.ExpectedPark && evaluation.CandidateParked {
-		if err := incrementQueuedFindings(c.Dir, evaluation.FindingCount); err != nil {
+	if evaluation.Status == "completed" && evaluation.Pending > 0 {
+		if err := incrementQueuedFindings(c.Dir, evaluation.Pending); err != nil {
 			return err
 		}
 	}

internal/eval/report.go

@@ -7,9 +7,8 @@ import (
 	"strings"
 )
 
-// Interval is a two-sided 95% Wilson score interval over cases. Cases are the
-// independent unit; repeats are averaged inside each case so a noisy provider
-// does not inflate apparent sample size.
+// Interval is a finite-sample recall range over cases. Repeats are averaged
+// inside each case so a noisy provider does not inflate apparent sample size.
 type Interval struct {
 	Lower float64
 	Upper float64
@@ -131,33 +130,22 @@ func averageTokens(rows []Evaluation) (float64, bool) {
 }
 
 func confidenceInterval(_ string, rows []Evaluation) *Interval {
-	// First turn each case into a mean over CONCLUSIVE repeats. A pending
-	// unexpected park stays out of this conditional interval and is surfaced
-	// separately in every report as a queue count and a lower-bound accuracy.
+	// Each case becomes a mean recall over labeled repeats. Unlabeled
+	// replays stay out of this interval and are reported as pending.
 	perCase := map[string][]float64{}
 	for _, row := range rows {
-		if row.ExpectedPark == nil {
+		if row.GoldCount == 0 {
 			continue
 		}
 		if row.Status != "completed" {
 			perCase[row.CaseID] = append(perCase[row.CaseID], 0)
 			continue
 		}
-		var score *float64
-		switch {
-		case *row.ExpectedPark && row.CandidateParked:
-			v := 1.0
-			score = &v
-		case *row.ExpectedPark && !row.CandidateParked:
-			v := 0.0
-			score = &v
-		case !*row.ExpectedPark && !row.CandidateParked:
-			v := 1.0
-			score = &v
-		}
-		if score != nil {
-			perCase[row.CaseID] = append(perCase[row.CaseID], *score)
+		denom := row.TruePositive + row.FalseNegative
+		if denom == 0 {
+			continue
 		}
+		perCase[row.CaseID] = append(perCase[row.CaseID], float64(row.TruePositive)/float64(denom))
 	}
 	values := make([]float64, 0, len(perCase))
 	for _, scores := range perCase {
@@ -167,9 +155,6 @@ func confidenceInterval(_ string, rows []Evaluation) *Interval {
 		}
 		values = append(values, total/float64(len(scores)))
 	}
-	if len(values) == 0 {
-		return nil
-	}
 	if len(values) < 2 {
 		return nil
 	}
@@ -197,10 +182,10 @@ func markFrontier(reports []CandidateReport) {
 			if i == j || reports[i].Cohort != reports[j].Cohort || reports[j].AverageTokens == nil || reports[j].Summary.Labeled == 0 || reports[j].Summary.Failures > 0 {
 				continue
 			}
-			betterAccuracy := reports[j].Summary.LowerBoundAccuracy() >= reports[i].Summary.LowerBoundAccuracy()
+			betterRecall := reports[j].Summary.Recall() >= reports[i].Summary.Recall()
 			cheaper := *reports[j].AverageTokens <= *reports[i].AverageTokens
-			strict := reports[j].Summary.LowerBoundAccuracy() > reports[i].Summary.LowerBoundAccuracy() || *reports[j].AverageTokens < *reports[i].AverageTokens
-			if betterAccuracy && cheaper && strict {
+			strict := reports[j].Summary.Recall() > reports[i].Summary.Recall() || *reports[j].AverageTokens < *reports[i].AverageTokens
+			if betterRecall && cheaper && strict {
 				dominated = true
 				break
 			}
@@ -213,14 +198,15 @@ func markFrontier(reports []CandidateReport) {
 type SetSummary struct {
 	Name           string
 	Cases          int
-	VerdictLabels  int
-	ShouldPark     int
-	ShouldPass     int
+	GoldCases      int
+	TruePositive   int
+	FalseNegative  int
+	Unlabeled      int
 	QueuedFindings int
 	Composition    map[string]int
 }
 
-// InspectSets summarizes all logical MVP sets and their diversified mix.
+// InspectSets summarizes all logical sets and their diversified mix.
 func InspectSets(store *Store) ([]SetSummary, error) {
 	sets := []string{"all", "labeled", "diversified"}
 	result := make([]SetSummary, 0, len(sets))
@@ -231,13 +217,18 @@ func InspectSets(store *Store) ([]SetSummary, error) {
 		}
 		summary := SetSummary{Name: name, Cases: len(cases), Composition: map[string]int{}}
 		for _, c := range cases {
-			if c.Labels.Verdict.Known {
-				summary.VerdictLabels++
-				if c.Labels.Verdict.ShouldPark {
-					summary.ShouldPark++
-				} else {
-					summary.ShouldPass++
+			if c.Labels.HasGold() {
+				summary.GoldCases++
+				for _, finding := range c.Labels.Findings {
+					switch finding.Kind {
+					case GoldTruePositive:
+						summary.TruePositive++
+					case GoldFalseNegative:
+						summary.FalseNegative++
+					}
 				}
+			} else {
+				summary.Unlabeled++
 			}
 			summary.QueuedFindings += c.Labels.QueuedCandidateFindings
 			language, size, severity := caseComposition(c)
@@ -261,7 +252,8 @@ func RenderSets(summaries []SetSummary) string {
 	var b strings.Builder
 	b.WriteString("LOCAL-ONLY EVAL CASE SETS\n")
 	for _, summary := range summaries {
-		fmt.Fprintf(&b, "\n%s: %d cases, %d verdict labels (park %d, pass %d), %d candidate findings queued\n", summary.Name, summary.Cases, summary.VerdictLabels, summary.ShouldPark, summary.ShouldPass, summary.QueuedFindings)
+		fmt.Fprintf(&b, "\n%s: %d cases, %d with finding-level gold (true-positive %d, false-negative %d), %d unlabeled / pending, %d candidate findings queued\n",
+			summary.Name, summary.Cases, summary.GoldCases, summary.TruePositive, summary.FalseNegative, summary.Unlabeled, summary.QueuedFindings)
 		keys := make([]string, 0, len(summary.Composition))
 		for key := range summary.Composition {
 			keys = append(keys, key)
@@ -274,9 +266,9 @@ func RenderSets(summaries []SetSummary) string {
 	return b.String()
 }
 
-// RenderReport is a stable human-readable local comparison. Confidence
-// intervals are conditional on conclusive verdicts; the lower-bound metric
-// includes queued unexpected parks so the uncertainty is explicit.
+// RenderReport is a stable human-readable local comparison. Scores are
+// finding-level. Unmatched candidate findings stay pending and are never
+// called false positives. A replay with no gold is unlabeled, not a pass.
 func RenderReport(reports []CandidateReport) string {
 	if len(reports) == 0 {
 		return "LOCAL-ONLY EVAL REPORT\nno candidate replays recorded yet\n"
@@ -288,16 +280,21 @@ func RenderReport(reports []CandidateReport) string {
 		fmt.Fprintf(&b, "\n%s (cohort %s)\n", s.Candidate, report.Cohort)
 		fmt.Fprintf(&b, "  replays: %d across %d repeat(s); labeled: %d; failures: %d\n", s.Total, report.RepeatCount, s.Labeled, s.Failures)
 		if s.Labeled == 0 {
-			b.WriteString("  verdict agreement: no human-confirmed verdict labels yet\n")
+			b.WriteString("  finding scores: unlabeled / pending (no finding-level gold yet)\n")
 		} else {
-			fmt.Fprintf(&b, "  confirmed verdict agreement: %.1f%% (%d/%d); conservative lower bound: %.1f%%\n", 100*s.ConfirmedAccuracy(), s.Correct, s.Conclusive, 100*s.LowerBoundAccuracy())
-			if report.Confidence != nil {
-				fmt.Fprintf(&b, "  95%% Wilson score CI: %.1f%%-%.1f%% over %d case(s)\n", 100*report.Confidence.Lower, 100*report.Confidence.Upper, report.Confidence.Cases)
-			}
-			if s.UnexpectedParks > 0 {
-				fmt.Fprintf(&b, "  queued unexpected parks: %d (not scored wrong pending finding-level adjudication)\n", s.UnexpectedParks)
+			fmt.Fprintf(&b, "  finding scores: true-positive %d, false-negative %d, false-positive %d, pending %d\n", s.TruePositive, s.FalseNegative, s.FalsePositive, s.Pending)
+			if s.TruePositive+s.FalseNegative == 0 {
+				b.WriteString("  recall: unavailable (no true-issue gold)\n")
+			} else {
+				fmt.Fprintf(&b, "  recall: %.1f%% (%d/%d gold issues)\n", 100*s.Recall(), s.TruePositive, s.TruePositive+s.FalseNegative)
+				if report.Confidence != nil {
+					fmt.Fprintf(&b, "  case-level recall range: %.1f%%-%.1f%% over %d case(s)\n", 100*report.Confidence.Lower, 100*report.Confidence.Upper, report.Confidence.Cases)
+				}
 			}
 		}
+		if s.Pending > 0 {
+			fmt.Fprintf(&b, "  queued unmatched candidate findings: %d (not scored as false-positive)\n", s.Pending)
+		}
 		if report.AverageTokens == nil {
 			b.WriteString("  token cost: unknown (token usage was not reported for every replay)\n")
 		} else {
@@ -305,7 +302,11 @@ func RenderReport(reports []CandidateReport) string {
 		}
 		fmt.Fprintf(&b, "  wall time: %.1fs average\n", report.AverageWallMS/1000)
 		if report.AverageTokens != nil {
-			fmt.Fprintf(&b, "  accuracy-vs-cost frontier: %t\n", report.OnFrontier)
+			if s.TruePositive+s.FalseNegative == 0 {
+				b.WriteString("  recall-vs-cost frontier: unavailable (no true-issue gold)\n")
+			} else {
+				fmt.Fprintf(&b, "  recall-vs-cost frontier: %t\n", report.OnFrontier)
+			}
 		}
 	}
 	return b.String()

internal/eval/score.go

@@ -0,0 +1,90 @@
+package eval
+
+import (
+	"path/filepath"
+	"strings"
+
+	"github.com/kunchenguid/no-mistakes/internal/types"
+)
+
+// Score is one candidate's finding-level confusion matrix against gold.
+// Pending is unmatched candidate findings: queued, never punished as FP.
+type Score struct {
+	TruePositive  int
+	FalseNegative int
+	FalsePositive int
+	Pending       int
+}
+
+// ScoreCandidate matches a candidate finding list against recorded gold.
+//
+//   - TP: the candidate raises the same underlying issue as a true-issue gold
+//     (human-accepted Fix, or a human-added miss the candidate also found)
+//   - FN: the candidate misses a true-issue gold
+//   - FP: only an explicit false-positive gold that the candidate still raised
+//   - Pending: unmatched candidate findings, never inferred as invalid
+func ScoreCandidate(labels Labels, findingsJSON string) Score {
+	candidate := parseFindingItems(findingsJSON)
+	used := make([]bool, len(candidate))
+	var score Score
+	for _, gold := range labels.Findings {
+		match := firstUnusedMatch(gold, candidate, used)
+		switch {
+		case isTrueIssueGold(gold.Kind) && match >= 0:
+			score.TruePositive++
+			used[match] = true
+		case isTrueIssueGold(gold.Kind):
+			score.FalseNegative++
+		case gold.Kind == GoldFalsePositive && match >= 0:
+			score.FalsePositive++
+			used[match] = true
+		}
+	}
+	for i := range candidate {
+		if !used[i] {
+			score.Pending++
+		}
+	}
+	return score
+}
+
+func parseFindingItems(raw string) []types.Finding {
+	if strings.TrimSpace(raw) == "" {
+		return nil
+	}
+	findings, err := types.ParseFindingsJSON(raw)
+	if err != nil {
+		return nil
+	}
+	return findings.Items
+}
+
+func firstUnusedMatch(gold FindingGold, candidate []types.Finding, used []bool) int {
+	for i, finding := range candidate {
+		if used[i] {
+			continue
+		}
+		if sameUnderlyingIssue(gold, finding) {
+			return i
+		}
+	}
+	return -1
+}
+
+func sameUnderlyingIssue(gold FindingGold, finding types.Finding) bool {
+	if gold.ID != "" && gold.ID == finding.ID {
+		return true
+	}
+	goldFile, goldDesc := normalizeIssue(gold.File, gold.Description)
+	candFile, candDesc := normalizeIssue(finding.File, finding.Description)
+	if goldFile == "" || candFile == "" || goldDesc == "" || candDesc == "" {
+		return false
+	}
+	return goldFile == candFile && goldDesc == candDesc
+}
+
+func normalizeIssue(file, description string) (string, string) {
+	file = filepath.ToSlash(strings.TrimSpace(file))
+	description = strings.Join(strings.Fields(strings.ToLower(strings.TrimSpace(description))), " ")
+	return file, description
+}

internal/eval/store.go

@@ -51,6 +51,9 @@ func (s *Store) migrate() error {
 	if s == nil || s.db == nil {
 		return fmt.Errorf("eval registry is closed")
 	}
+	if err := s.dropParkPassSchema(); err != nil {
+		return err
+	}
 	_, err := s.db.Exec(`
 CREATE TABLE IF NOT EXISTS cases (
     id TEXT PRIMARY KEY,
@@ -62,8 +65,7 @@ CREATE TABLE IF NOT EXISTS cases (
     language TEXT NOT NULL,
     size_bucket TEXT NOT NULL,
     severity TEXT NOT NULL,
-    verdict_known INTEGER NOT NULL,
-    verdict_should_park INTEGER NOT NULL,
+    gold_count INTEGER NOT NULL,
     path TEXT NOT NULL UNIQUE
 );
 CREATE TABLE IF NOT EXISTS pending_case_deletions (
@@ -86,8 +88,11 @@ CREATE TABLE IF NOT EXISTS evaluations (
     started_at INTEGER NOT NULL,
     completed_at INTEGER NOT NULL,
     status TEXT NOT NULL,
-    expected_park INTEGER,
-    candidate_parked INTEGER NOT NULL,
+    gold_count INTEGER NOT NULL,
+    true_positive INTEGER NOT NULL,
+    false_negative INTEGER NOT NULL,
+    false_positive INTEGER NOT NULL,
+    pending INTEGER NOT NULL,
     tokens_reported INTEGER NOT NULL,
     input_tokens INTEGER NOT NULL,
     output_tokens INTEGER NOT NULL,
@@ -113,6 +118,26 @@ CREATE INDEX IF NOT EXISTS idx_eval_evaluations_case ON evaluations(case_id, com
 	return nil
 }
 
+func (s *Store) dropParkPassSchema() error {
+	var parkColumn int
+	if err := s.db.QueryRow(`SELECT count(*) FROM pragma_table_info('cases') WHERE name = 'verdict_should_park'`).Scan(&parkColumn); err != nil {
+		return fmt.Errorf("inspect eval case schema: %w", err)
+	}
+	if parkColumn == 0 {
+		return nil
+	}
+	_, err := s.db.Exec(`
+DROP TABLE IF EXISTS evaluations;
+DROP TABLE IF EXISTS replay_case_reservations;
+DROP TABLE IF EXISTS pending_case_deletions;
+DROP TABLE IF EXISTS cases;
+`)
+	if err != nil {
+		return fmt.Errorf("replace park/pass eval registry: %w", err)
+	}
+	return nil
+}
+
 func (s *Store) Close() error {
 	if s == nil || s.db == nil {
 		return nil
@@ -129,25 +154,18 @@ func (s *Store) registerCase(c Case) error {
 		return fmt.Errorf("eval registry is closed")
 	}
 	language, size, severity := caseComposition(c)
-	known, shouldPark := 0, 0
-	if c.Labels.Verdict.Known {
-		known = 1
-	}
-	if c.Labels.Verdict.ShouldPark {
-		shouldPark = 1
-	}
 	_, err := s.db.Exec(`INSERT OR IGNORE INTO cases
-(id, source_run_id, source_round_id, captured_at, repo_fingerprint, branch, language, size_bucket, severity, verdict_known, verdict_should_park, path)
-VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
-		c.ID, c.SourceRunID, c.SourceRoundID, c.CapturedAt, c.RepoFingerprint, c.Branch, language, size, severity, known, shouldPark, c.Dir)
+(id, source_run_id, source_round_id, captured_at, repo_fingerprint, branch, language, size_bucket, severity, gold_count, path)
+VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
+		c.ID, c.SourceRunID, c.SourceRoundID, c.CapturedAt, c.RepoFingerprint, c.Branch, language, size, severity, len(c.Labels.Findings), c.Dir)
 	if err != nil {
 		return fmt.Errorf("register eval case: %w", err)
 	}
 	return nil
 }
 
-// ListCases resolves the three MVP logical sets. Diversified is deterministic:
-// it retains the lexicographically first case in each repo/language/size/verdict
+// ListCases resolves the three logical sets. Diversified is deterministic:
+// it retains the lexicographically first case in each repo/language/size/gold
 // bucket, making its composition visible and stable before a user spends tokens.
 func (s *Store) ListCases(set string) ([]Case, error) {
 	if s == nil || s.db == nil {
@@ -187,7 +205,7 @@ func (s *Store) ListCases(set string) ([]Case, error) {
 	case "labeled":
 		out := make([]Case, 0, len(all))
 		for _, c := range all {
-			if c.Labels.Verdict.Known {
+			if c.Labels.HasGold() {
 				out = append(out, c)
 			}
 		}
@@ -340,7 +358,7 @@ func loadCase(dir string) (Case, error) {
 		return Case{}, err
 	}
 	if labels.Version != labelsVersion {
-		return Case{}, fmt.Errorf("unsupported case labels version %d", labels.Version)
+		return Case{}, fmt.Errorf("unsupported case labels version %d (finding-level gold replaced the park/pass verdict; remove the eval directory to start a fresh corpus)", labels.Version)
 	}
 	var decision Decision
 	if err := readJSON(filepath.Join(dir, "original", "decision.json"), &decision); err != nil {
@@ -378,14 +396,29 @@ func writeJSON(path string, value any) error {
 
 func diversifiedKey(c Case) string {
 	language, size, _ := caseComposition(c)
-	verdict := "unlabeled"
-	if c.Labels.Verdict.Known {
-		verdict = "pass"
-		if c.Labels.Verdict.ShouldPark {
-			verdict = "park"
+	return strings.Join([]string{c.RepoFingerprint, language, size, goldBucket(c.Labels)}, "\x00")
+}
+
+func goldBucket(labels Labels) string {
+	hasTP, hasFN := false, false
+	for _, finding := range labels.Findings {
+		switch finding.Kind {
+		case GoldTruePositive:
+			hasTP = true
+		case GoldFalseNegative:
+			hasFN = true
 		}
 	}
-	return strings.Join([]string{c.RepoFingerprint, language, size, verdict}, "\x00")
+	switch {
+	case hasTP && hasFN:
+		return "true-positive+false-negative"
+	case hasTP:
+		return GoldTruePositive
+	case hasFN:
+		return GoldFalseNegative
+	default:
+		return "unlabeled"
+	}
 }
 
 func caseComposition(c Case) (language, size, severity string) {

internal/pipeline/executor.go

@@ -394,13 +394,12 @@ func (e *Executor) Resume(ctx context.Context, run *db.Run, repo *db.Repo, workD
 		if gate.lastRoundID != "" {
 			allSelectedIDs := combineSelectedFindingIDs(response.findingIDs, merged)
 			if idsJSON := marshalFindingIDs(allSelectedIDs); idsJSON != "" {
-				if dbErr := e.db.SetStepRoundSelection(gate.lastRoundID, &idsJSON, db.RoundSelectionSourceUser); dbErr != nil {
-					slog.Warn("failed to record recovered selected finding ids", "step", gate.step.Name(), "round", gate.round, "error", dbErr)
+				var userFindingsJSON *string
+				if merged != "" && merged != selected {
+					userFindingsJSON = &merged
 				}
-			}
-			if merged != "" && merged != selected {
-				if dbErr := e.db.SetStepRoundUserFindings(gate.lastRoundID, &merged); dbErr != nil {
-					slog.Warn("failed to record recovered user findings", "step", gate.step.Name(), "round", gate.round, "error", dbErr)
+				if dbErr := e.db.SetStepRoundUserDecision(gate.lastRoundID, &idsJSON, db.RoundSelectionSourceUser, userFindingsJSON); dbErr != nil {
+					slog.Warn("failed to record recovered user decision", "step", gate.step.Name(), "round", gate.round, "error", dbErr)
 				}
 			}
 		}
@@ -946,14 +945,12 @@ func (e *Executor) executeStep(ctx context.Context, step Step, sr *db.StepResult
 			if currentRoundID != "" {
 				allSelectedIDs := combineSelectedFindingIDs(response.findingIDs, mergedFindings)
 				if idsJSON := marshalFindingIDs(allSelectedIDs); idsJSON != "" {
-					if dbErr := e.db.SetStepRoundSelection(currentRoundID, &idsJSON, db.RoundSelectionSourceUser); dbErr != nil {
-						slog.Warn("failed to record selected finding ids", "step", stepName, "round", roundNum, "error", dbErr)
+					var userFindingsJSON *string
+					if mergedFindings != "" && mergedFindings != selectedFindings {
+						userFindingsJSON = &mergedFindings
 					}
-				}
-				if mergedFindings != "" && mergedFindings != selectedFindings {
-					merged := mergedFindings
-					if dbErr := e.db.SetStepRoundUserFindings(currentRoundID, &merged); dbErr != nil {
-						slog.Warn("failed to record user findings", "step", stepName, "round", roundNum, "error", dbErr)
+					if dbErr := e.db.SetStepRoundUserDecision(currentRoundID, &idsJSON, db.RoundSelectionSourceUser, userFindingsJSON); dbErr != nil {
+						slog.Warn("failed to record user decision", "step", stepName, "round", roundNum, "error", dbErr)
 					}
 				}
 			}