-
[OPIK-5734] [BE][FE] feat: allow adding assertions when adding traces to evaluation suite (#6150)
frostbyte_neo 发布于
2026-04-12 06:38:21 +00:00 - [OPIK-5734] [BE][FE] feat: allow adding assertions when adding traces to evaluation suite
Extend from-traces and from-spans endpoints to accept optional evaluators
and execution_policy fields. When provided, these are set on the individual
dataset items created, allowing users to define assertions inline from the
traces view without navigating to the evaluation suite items page.BE: Extended CreateDatasetItemsFromTracesRequest/FromSpansRequest with
optional evaluators and executionPolicy. DatasetItemService sets these on
each created item. Added parameterized integration tests.FE: Added collapsible "Evaluation criteria" section to AddToDatasetDialog
with AssertionsField, runs per item, and pass threshold inputs. Syncs
defaults from the selected suite's latest version. Execution policy is
only sent when the user changes it from the suite default.Implements OPIK-5734: Unable to add assertion when adding a trace to an
evaluation suite.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix(review): address PR review comments
- Guard useEffect sync to only run on dataset selection change, not refetch
- Pass suite evaluator to packAssertions to inherit llm_judge config
- Skip reset when re-clicking already-selected dataset
- Simplify test assertions with isEqualTo(executionPolicy)
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- feat(dialog): show config by dataset type and auto-scroll on selection
- Metadata config (enrichment checkboxes) shown only for legacy DATASET type
- Evaluation criteria (assertions + policy) shown only for EVALUATION_SUITE type
- Both sections expanded by default
- Auto-scroll to config section when selecting a dataset
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix(dialog): sync execution policy when version data loads
The useEffect now also depends on suiteExecutionPolicy so it re-runs
when version data arrives after dataset selection, correctly syncing
the suite's actual runs_per_item and pass_threshold defaults.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- test(dialog): fix tests for conditional config sections by dataset type
Tests now select a legacy dataset (type: "dataset") before checking
metadata config, matching the new behavior where config sections are
shown conditionally based on dataset type. Added mock for
useDatasetVersionsList.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix(dialog): reduce dataset list height and hide explainer on selection
Shrink dataset list max-height to 225px (was 300/400px) to prevent
double scrollbars. Hide explainer description once a dataset is
selected to free vertical space for the config section below.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix(tests): reorder import statements in DatasetsResourceCreateFromSpansTest
Co-authored-by: Claude Opus 4.6 (1M context) noreply@anthropic.com
下载附件