github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.55-5533 cdc62de420
    比较

    [OPIK-5269] [BE] feat: expose trace metadata on experiment item compare (#6940)

    frostbyte_neo 发布于 2026-06-02 09:33:59 +00:00

    • [OPIK-5269] [BE] feat: expose trace metadata on experiment item compare

    Surfaces the trace's metadata JSON on the experiment-item join
    returned by GET /v1/private/datasets/{id}/items/experiments/items.
    The ClickHouse query already pulled tfs.metadata into the per-item
    tuple (position 16) for both the live and the version-pinned paths;
    the mapper just dropped it on the floor.

    This unblocks opik.evaluate_resume (OPIK-5269) — the SDK reads
    trace metadata from the item-compare response to distinguish trials
    that completed the full happy path from those whose task succeeded
    but whose scoring crashed mid-way (the marker is set by the SDK and
    round-trips through this field). Without this projection, resume
    would have to fetch each trace individually, one round-trip per row.

    Adds ExperimentItem.traceMetadata: JsonNode (Compare view) and
    the matching mapper line in DatasetItemResultMapper. No schema
    migration, no new join, no query change.

    Includes the regenerated openapi.yaml for both the docs spec and
    Fern's input. The Python and TypeScript rest_api regens land in a
    follow-up auto-PR after this merges.

    • chore: drop openapi.yaml regen; rely on main's autogen PR

    openapi.yaml lands via the automated regen PR (see history: 8930b66fcb, 5c618b7f33, dc90a9116b et al). Feature BE PRs shouldn't carry it.

    • test(be): cover trace_metadata projection on the experiment-item compare

    Two focused tests under DatasetsResourceTest.TraceMetadataOnExperimentItemCompareTest:

    • Trace metadata round-trips through the compare endpoint into
      ExperimentItem.traceMetadata verbatim — the SDK consumer uses
      this for evaluate_resume's marker-based completion check
      (OPIK-5269).
    • traceMetadata is null when the trace has no metadata.

    Locks in the new mapper line (tuple position 16) so a future ordering
    change in the ClickHouse query tuple would be caught here, not later
    in SDK integration.

    • test(be): ignore traceMetadata in shape-comparison list

    The 16 failing assertions in FindDatasetItemsWithExperimentItems
    compared a PODAM-built expected ExperimentItem (no traceMetadata
    override → null) against the actual response where the new mapper
    correctly projects the trace's metadata. Same pattern already in place
    for sibling fields the tests don't drive (feedbackScores,
    comments, createdAt…); traceMetadata joins them.

    The dedicated TraceMetadataOnExperimentItemCompareTest class still
    exercises the projection explicitly.

    • test(be): consolidate trace-metadata projection tests via @ParameterizedTest

    Address baz-reviewer feedback on #6940: collapses the two @Test
    methods into a single @ParameterizedTest backed by traceMetadataCases,
    parameterizing over the (trace.metadata, expected) pair. The
    workspace/dataset/experiment scaffolding is now defined once.

    下载附件