-
[OPIK-7230] [BE] Optimize experiment-refs ClickHouse lookups + fix missing log_comment (#7337)
frostbyte_neo 发布于
2026-07-06 11:13:45 +00:00 - [OPIK-7230] [BE] Optimize experiment-refs ClickHouse lookups + fix missing log_comment
- Route getExperimentRefsByIds through getSTWithLogComment so the three
GET_EXPERIMENT_REFS_BY_* queries emit a real log_comment breadcrumb
instead of the literal placeholder. - Add a conditional project_id filter to the trace- and span-refs queries;
thread projectId through the aggregation events and split per project in
ExperimentAggregateEventListener so each query targets a single project. - Add minmax skip indexes on experiment_items.trace_id and spans.id
(migration 000095) so these lookups prune granules instead of a
generic-exclusion scan.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Cover per-project split + projectId threading in listener tests
Add ExperimentAggregateEventListenerTest cases asserting the new behavior:
- traces/spans created split trace_ids per project (one aggregation trigger per project)
- traces updated split per project via the traceId->projectId mapping
- traces/spans deleted pass the event projectId through
- span feedback-score passes projectId to getExperimentRefsBySpanIds
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Address review: error isolation, SpansDeleted projectId, @Nullable, migration EOL
- triggerByTraceIdsPerProject: wrap each per-project trigger in onErrorResume so one
project's failure no longer short-circuits the remaining per-project aggregations. - SpanService: pass projectId into the SpansDeleted event (delete-by-traces flow) so
span-delete driven experiment-ref lookups keep project_id pruning. - Mark the optional projectId @Nullable on the trigger helpers and on the projectId
field/constructor param of the six entity/project events + SpansDeleted. - Migration 000095: end file with a trailing blank line per the Liquibase guideline.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Extract EntityProjectEvent base + renumber migration to 000096
- Extract EntityProjectEvent (entityIds, entityType, nullable projectId + the two
delegating constructors); FeedbackScoresCreated/Deleted, CommentsCreated/Updated/Deleted
and AssertionResultsCreated now just extend it, removing the duplicated fields/ctors. - Renumber the skip-index migration 000095 -> 000096 (main added 000095_add_session_id_
to_cipx_trace_identity); changeset ids updated to match.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Resolve project for single-entity score/comment events (review)
Address the high-severity review findings about lost project pruning on
single-entity paths:- FeedbackScoreService.deleteSpanScore/deleteTraceScore now resolve the entity's
project (getProjectIdFrom{Span,Trace}) and pass it into FeedbackScoresDeleted;
the delete still happens even if the project can't be resolved (best-effort). - CommentService.update resolves the comment entity's project (shared
resolveProjectId helper, also reused by create) and passes it into CommentsUpdated.
Batch paths that can span projects (CommentsDeleted, assertion batches) intentionally
keep projectId null and fall back to the trace_id/id skip index.Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Simplify per-project fallback + assert projectId in listener tests
- Collapse the onTracesUpdated fallback (no traceId->projectId mapping) to a single
workspace-scoped trigger relying on the skip index, instead of re-running the trigger
once per project over the full trace-id set; remove the now-unused Set-based overload. - ExperimentAggregateEventListenerTest: assert the exact projectId threaded to the
service (eq(projectId), or isNull() for SpansUpdated which carries no project) instead
of any(), across the traces/spans and feedback/comment handlers.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Renumber skip-index migration 000096 -> 000097 (main took 000096_create_deletion_events_local)
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Best-effort project resolution on score delete + dedup helper (review)
- Extract deleteScoreAndNotify(entityType, id, score): resolves the entity's project
lazily inside the deferred chain via a switch (no eager/Mono-as-param), then deletes
and emits FeedbackScoresDeleted. deleteSpanScore/deleteTraceScore delegate to it. - onErrorResume on the project lookup so a transient lookup failure no longer blocks
the delete; it proceeds with a null projectId (listener falls back to the skip index).
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Route trace ids not covered by the mapping through the fallback (review)
onTracesUpdated: the traceId->projectId mapping (from getProjectIdsByTraceIds) may not
cover every id in event.traceIds(); previously the uncovered ids were silently dropped.
Now the mapped ids are split per project and any uncovered ids are triggered via the
workspace-scoped fallback (null projectId) so their aggregation is not skipped.
Covered by a new listener test.Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Quote entityType placeholder in score-delete warn log (review)
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- Update apps/opik-backend/src/main/resources/liquibase/db-app-analytics/migrations/000097_add_minmax_indexes_experiment_refs_lookup.sql
Co-authored-by: Daniel Dimenshtein danield@comet.com
- [OPIK-7230] [BE] Address review: migration changeset author + redundant terminal error handler
- Align migration 000097 changeset author to the repo convention (thiagoh -> thiagohora).
- triggerByTraceIdsPerProject: drop the redundant terminal error consumer; per-project
onErrorResume already logs and recovers, so no error reaches the subscriber. subscribe().
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- [OPIK-7230] [BE] Review: javadoc on deleteScoreAndNotify + precise status matcher in tests
- Convert the deleteScoreAndNotify explanatory comment to javadoc.
- ExperimentAggregateEventListenerTest: replace the
any()status matcher with
eq(FINISHED_STATUSES) so the assertions/verifications use real arguments.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Co-authored-by: Claude Opus 4.8 (1M context) noreply@anthropic.com
Co-authored-by: Daniel Dimenshtein danield@comet.com下载附件