-
[OPIK-7460] [OPIK-7589] [FE] fix: don't label still-evaluating optimization trials "Discarded" + one trial numbering on the progress chart (#7652)
frostbyte_neo 发布于
2026-08-04 08:48:44 +00:00 - [OPIK-7460] [FE] fix: don't label still-evaluating optimization trials "Discarded"
During a live run an experiment's feedback score is a partial average over the
items scored so far, so computeInProgressStatus saw a low provisional score and
immediately returned "pruned" (rendered "Discarded") for a trial that was still
mid-evaluation.- Gate both pruned branches on item completion: a candidate whose
totalDatasetItemCount is below the step-0 baseline's count reads "evaluating". - Derive the denominator from the baseline rather than max() over candidates: a
candidate aggregates all its experiments, so a double-counted one would inflate
the denominator and freeze finished trials on "Evaluating". A 0 denominator
fails open, preserving the previous behaviour.
The API exposes no planned item count (every count comes from rows already in
experiment_items), and Experiment.status is always created "completed" by the
SDK, so the run's own baseline count is the only available in-band signal.Scoped to v2; v1 carries the same logic and stays untouched.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7460] [FE] fix: stop the chart painting a still-evaluating trial as passed
Follow-up to the status gate in the previous commit. On a dataset run (the
ticket's repro) getTrialDotColor collapsed every non-pruned, non-failed status
into the solid "passed" fuchsia, and the dataset legend was hardcoded to
"Passed trial" / "Discarded trial". So the gate made the trials table read
"Evaluating" while the chart painted that same trial in the colour the legend
declares to be a pass — the two surfaces contradicted each other and the chart
asserted a result that did not exist yet.- Keep in-progress statuses (evaluating, running) in their own orange/yellow on
dataset runs; only real outcomes collapse to passed vs discarded. - Add the matching legend entries, but only when such a point is on the chart,
so a dataset run with no in-progress trial still shows just the two outcomes. - Update the dataset-run dot-colour test, which asserted the old collapse.
This also covers "running" trials, which had the same mislabelling before this
ticket: an unscored in-flight trial already rendered as a solid pass.Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7460] [FE] fix: drop the baseline metric delta while a trial is evaluating
Found by rendering the repro against real data: with the status gate in place
the trials table read "Evaluating" on a trial with 5 of 30 items scored, but the
score cell still showed "-75%" next to it — a partial 5-item average compared
against the fully evaluated 30-item baseline. The status was honest while the
metric beside it was not, and "Evaluating -75%" reads as a failing trial.This is the ticket's AC 3 (metric diffs must compare equal denominators): a
mid-evaluation trial has no comparable denominator yet, so the comparison is
withheld until the evaluation completes.- Suppress the trend pill on the score, cost and latency cells while a trial is
in progress; the provisional value itself still renders, and the existing
"Trial items" column already shows the 5-vs-30 progress. - Thread statusMap into those three columns' meta, reusing the status the chart
and status pill already derive, so all three surfaces agree.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7460] [FE] fix: a partial average must not win the run either
Code review of the previous commits found the fix was one-directional. The gate
stopped an unfinished trial being pruned, but bestScore/bestCandidate were still
reduced over every scored candidate, so a partial average could WIN — and become
the threshold everything else is pruned against.Reproduced on dev with baseline 30/30 @ 0.50, candA 30/30 @ 0.90 and candC
2/30 @ 0.99. Before this commit candC took "Best trial", drove the header KPI to
99% and put its prompt in the "Best trial prompt" panel, while candA — the real
winner, fully evaluated — was labelled "Discarded" against candC's 0.99. That is
the reported symptom again, sourced from a different row.- Exclude still-evaluating candidates from best-selection and from bestScore,
falling back to the unfiltered set when nothing has completed so a run never
loses its best marker (with a scored baseline the filtered set is non-empty by
construction — the baseline defines the denominator). - Derive the page-level bestCandidate from the same shared selectBestCandidate.
It was a second, independent reduce; filtering only one of them would have put
the header and the trials table in disagreement.
Also from the review: "failed" had no dataset-run legend entry even though
getTrialDotColor keeps it red, so the chart carried a colour the legend did not
explain — the exact gap the previous commit fixed for the in-progress statuses.
Legend building moves into a pure buildTrialLegendItems so the invariant "every
colour the chart can paint is in the legend" is unit-testable without recharts.Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7589] [FE] fix: label the progress chart by trial numbers, not steps
The single-run progress chart labelled its x axis "Step N" (zero-based)
while the card over the same dot said "Trial #N+1" — two correct but
different numbering systems that read as an off-by-one bug ("Step 3"
under a "Trial #4" card).Trial numbers are the identity everything else in the run view uses
(trials table, sidebar title, deep links, prompt-diff tags), so they
become the chart's one user-facing numbering:- Axis ticks stay positioned by step (branching runs stack sibling
trials on one x) but are labelled by trial: "Baseline", "Trial 2",
and "Trials 3–5" for a fan-out step (buildStepTickLabels). The ghost
dot's tick is numbered after every plotted trial. - The trial card's status label drops its step suffix ("Passed step 3"
→ "Passed"), removing the second sequence from the chart entirely and
matching the trials-table status tags. getTrialStatusLabel and the
card's stepIndex prop threading go away with it.
Reported by Rodrigo Miranda. v2 only — v1 is frozen.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- [OPIK-7589] [FE] fix: stop counting the baseline as Trial #1
The baseline is not a trial, but it took number 1: every candidate was
shifted by one (the dot on step 3 carried a "Trial #4" card) and a run
configured with max_trials=N ended on Trial #N+1 — a second off-by-one,
now between the user's config and the result.v2 now leaves the baseline unnumbered (trialNumber: null) and counts
candidates 1..N, so the last trial number matches max_trials. Every
surface that says "Trial #N" says "Baseline" for the baseline instead:
the chart card, the trials table, the sidebar header, the axis ticks
(which already did), and the prompt-diff parent caption. v1 is frozen
on the old numbering — the new behaviour is opt-in via
aggregateCandidates options (precedent: 4c5ec9a4ca).Nullable trialNumber over a boolean flag or a falsy 0: the type forces
every call site to decide what an unnumbered candidate renders as, and
0 would silently fall through falsy checks like the sidebar title's.Known trade-off: trialNumber is a URL param, so pre-existing v2 deep
links open the neighbouring trial (their numbers included the
baseline). Accepted — Studio links are young and short-lived; a compat
shim costs more than it saves.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
- [OPIK-7589] [FE] fix: don't count the evaluating trial twice on its axis tick
buildStepTickLabels numbered the ghost dot
maxTrialNumber + 1
unconditionally, on the assumption that the evaluating candidate is not
among the plotted points. It is: buildCandidateChartData maps every
candidate, scored or not, and inProgressInfo is derived from that same
candidate list — so the ghost always arrives already plotted and already
numbered.The extra number therefore widened its step to "Trials N-N+1" for a step
holding one trial. Every in-progress linear run hit it: with candidate 3
evaluating the tick read "Trials 3-4" against a "Trial #3" card — the same
class of off-by-one this ticket set out to remove, one surface over.Number the ghost only when its step carries no numbered trial at all.
Found in review by baz-reviewer. Two tests encoded the old assumption
(building chartData without the ghost) and are replaced by ones matching
what the caller actually passes; the synthesised-number path keeps its own
test.Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7589] [FE] fix: resolve trial deep links by experiment id, not trial number
Unnumbering the baseline shifted what
?trialNumber=means, and
activeTrialCandidate trusted that param ahead of thetrialsexperiment
ids. So a link minted before the change opened the neighbouring trial —
and the baseline's own link,trials=[baselineId]&trialNumber=1, opened
the first candidate while the URL named the baseline.The previous commit accepted that as unavoidable breakage. It is not:
trialsis what opens the sidebar at all, so the ids are always present,
and experiment ids never drifted. Preferring them resolves the old links
correctly rather than off-by-one, and changes nothing for current links,
where the two agree. trialNumber stays as the fallback for a candidate
whose experiments are outside the loaded page.Extracted to findActiveTrialCandidate so the resolution order is testable
rather than inline in a page useMemo.Found in review by baz-reviewer.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
- [OPIK-7460] [FE] chore: single-source the in-progress trend rule + aggregation edge cases
Two review follow-ups, no behaviour change.
- Score, cost and latency each repeated `getIsRowInProgress(...) ? undefined
- getBaselinePercentage(...)`, so a future change to how provisional rows
present their comparison could land on one metric and miss the other two.
Extracted as getTrendPercentage.
aggregateCandidates' unnumberedBaseline path only had populated-run
coverage; added the empty run and the no-step-0 page (nothing to exempt,
so candidates still count from 1).Found in review by baz-reviewer.
Co-Authored-By: Claude Opus 5 (1M context) noreply@anthropic.com
Co-authored-by: Claude Opus 5 (1M context) noreply@anthropic.com
下载附件