发布

RSS 订阅源

1.10.47-4616 a6a99b526c

比较
[OPIK-4518] [BE] Restore stable dataset item IDs in versioned API (#5370)

frostbyte_neo 发布于 2026-03-24 11:56:11 +00:00
- [OPIK-4518] [BE] Restore stable dataset item IDs in versioned API
After dataset versioning was introduced, the old API endpoints started
returning version-specific row IDs (changing per version) as the id
field instead of the stable dataset_item_id. This broke user flows
where items would get new IDs with each new version snapshot.

Fix: expose dataset_item_id AS id in all versioned item queries so
the id field is stable across versions. Update streaming pagination
cursor to use dataset_item_id instead of row id. Remove the
row-ID-to-dataset-item-ID mapping logic in the service since incoming
id values are now already stable dataset_item_ids.
- fix(backend): clean up review issues in stable dataset item IDs
- Remove unused workspaceId param from getDatasetItemWorkspace DAO
  (query is intentionally unscoped for cross-workspace validation)
- Fix allMatch validation gap in ExperimentItemService: verify all
  requested item IDs were found before checking workspace ownership
- Remove identity map indirection from editItemsViaSelectInsert now
  that IDs are stable dataset_item_ids
- fix(backend): renumber migration files to avoid conflicts with main
000062 → 000065, 000063 → 000066 (skip indexes and projection for
dataset_item_id). Updated changeset IDs inside files to match.
- fix(backend): clarify DatasetItemEdit.id is the stable dataset_item_id
- Fix warnings on updated code
- feat(backend): resolve experiment dataset_item_id in trace queries
Resolve old physical row IDs to stable dataset_item_ids in the
experiments_agg CTE using a targeted LEFT JOIN to
dataset_item_versions. No dedup needed since id and dataset_item_id
are immutable columns.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- chore(backend): remove manual data migration for experiment items
The dual-join approach handles both old (physical row ID) and new
(stable dataset_item_id) experiment items at query time, making
the manual migration unnecessary.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- feat(backend): dual-join all experiment-item queries for backward compat
All queries joining experiment_items.dataset_item_id with
dataset_item_versions now match on both physical row id AND stable
dataset_item_id, so old experiment items (storing row IDs) and new
ones (storing stable IDs) both resolve correctly without requiring
a data migration.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): renumber migrations 000065/000066 → 000070/000071
Avoid collision with main's 000065-000069 added since last rebase.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): renumber migrations 000070/000071 → 000073/000074
Main now has 000070-000072, so bump to next available.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): use if() instead of COALESCE for FixedString LEFT JOIN
ClickHouse FixedString(36) columns return null bytes (not SQL NULL)
on LEFT JOIN miss, so COALESCE never falls through. Use if(div.id != '')
to check whether the join actually matched.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- chore(backend): remove ClickHouse projection migration
Defer the dataset_item_id projection to a follow-up. The skip indexes
(000073) provide sufficient coverage for now. The projection can be
added later if large-version performance requires it.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- perf(backend): use arrayJoin to avoid duplicate CTE evaluation
Consolidate OR-expanded filter conditions that referenced the same CTE
twice into single arrayJoin([col1, col2]) calls. Each CTE is now
evaluated exactly once per usage site.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): rename skip indexes to follow naming convention
Use idx_{table}_{column} pattern: idx_dataset_item_versions_dataset_item_id_bf
and idx_dataset_item_versions_dataset_item_id_minmax.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): restore optimized SELECT_EXPERIMENT_ITEMS_OUTPUT_COLUMNS
Revert to main's simpler query that resolves trace_ids directly from
experiment_items without joining dataset_item_versions. The dual-join
treatment is unnecessary here — output column discovery only needs
trace_ids, not dataset item mapping.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): add two-level dedup to aggregated CTEs, fix naming/javadoc
- Add LIMIT 1 BY dataset_item_id to dataset_items_agg_resolved,
  dataset_items_aggr_resolved, and ExperimentAggregatesDAO's
  dataset_item_versions_resolved CTEs. Without this, the OR-condition
  joins could match one experiment item to multiple dataset item rows
  from different versions, inflating groupArray results.
- Remove orphaned Javadoc from deleted validateMappingsBelongToSameDataset
- Rename deletedRowIds to deletedIds (now holds stable IDs, not row IDs)
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): fix stats query item_agg filter for dual-ID compat
The dataset_item_filters in item_agg selected only physical id,
missing new experiment items that store stable dataset_item_id.
Use arrayJoin([id, dataset_item_id]) and add two-level dedup.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): renumber migration 000073 → 000074
Main added 000073_add_minmax_index_trace_threads_last_updated_at.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- fix(backend): unify CTE column aliases to dataset_item_id AS id, id AS row_id
All aggregated CTEs (dataset_items_agg_resolved, dataset_items_aggr_resolved,
dataset_item_versions_resolved) now use the same convention as
dataset_items_resolved: dataset_item_id AS id (stable), id AS row_id (physical).
Updated all join conditions, arrayJoin filters, and GROUP BY references.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- test(backend): add multi-dataset rejection tests for delete and batch update
Verify that delete and batch update requests with item IDs spanning
multiple datasets (without explicit datasetId) return 400.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

Co-authored-by: Claude Opus 4.6 noreply@anthropic.com
下载附件
- 源代码 (ZIP)
- 源代码 (TAR.GZ)

0 发布 8,340 Git 标签

[OPIK-4518] [BE] Restore stable dataset item IDs in versioned API (#5370)