-
[OPIK-6720] [FE][BE] Annotation queue claim mechanism (#6973)
frostbyte_neo 发布于
2026-06-12 09:43:04 +00:00 - [OPIK-6720] [FE][BE] Annotation queue claim mechanism
Backend:
- ClickHouse migration for lock_timeout_minutes on annotation_queues
- Redis-based item locking with per-queue hash and distributed mutex
- BE-computed scored count via ClickHouse for lock decisions
- PUT/DELETE/GET lock endpoints on annotation queues resource
- tryLockItem in AnnotationQueueService coordinates queue lookup,
scored count, and Redis lock
Frontend:
- Lock polling every 5s, item list polling stays at 30s
- Lock on item view with heartbeat at half TTL
- IN_REVIEW item state with orange dot for locked items
- Sidebar filter bar (To review / Processed)
- Per-user deterministic shuffle for item ordering
- Lock timeout setting in annotation queue form
- Conflict handling: lock denied shows "being reviewed" message
- Active item pinned in To review list while scoring
- Consistent use of getAnnotationQueueItemId for thread/trace IDs
- Revision: Fix empty feedback names ClickHouse error and heartbeat deps
- Handle empty feedbackDefinitionNames in getDistinctAnnotatorCount by
using a comments-only query (ClickHouse rejects empty IN clauses) - Add lock_timeout_minutes to heartbeat useEffect deps so interval
updates when queue TTL setting changes - Fix eslint warnings: memoize locks, eslint-disable for intentional
dep omissions
- Revision 2: Skip lock for non-DEFAULT items, fix stale lock denied state
- Only attempt lock for items in DEFAULT state (to review). Scored,
completed, and in-review items don't need locks. - Reset mutation state when navigating to non-DEFAULT items to clear
stale acquired:false from previous lock attempts. - Reorder CommentAndScoreViewer checks: completedByOthers first,
then in-review/lock-denied. - Rename isLockedForUser to cannotAnnotate for clarity.
- Revision 3: Sidebar UX improvements and per-user sort ordering
- Sort processed items by current user's last annotation timestamp
(not global last_updated_at) so other users' activity doesn't shift
the order - Split sidebar memo: getLastAnnotationByUser runs on 30s item poll
only, not on 5s lock poll - Use segmented-primary Tabs component matching trace detail panel
- Remove opacity for non-DEFAULT items in Processed tab, keep only
when pinned in To review tab - Hide "To review" tag label for DEFAULT items
- Rename isLockedForUser to cannotAnnotate, reorder checks:
completedByOthers first, then in-review/lock-denied - Add getLastAnnotationByUser unit tests (7 cases)
- Revision 4: Filter sort by queue feedback definition names
- getLastAnnotationByUser now takes feedbackDefinitionNames and only
considers matching scores. Unrelated scores (SDK, other queues) no
longer affect processed item ordering. - Empty feedback names = comments-only queue, scores are skipped.
- Added 3 tests for the filtering behavior.
- Revision 5: Address reviewer feedback
- Clarify v1 compat comment: "Opik v1 annotation queues don't have
this setting" - Convert SidebarFilter from string union to enum
- Destructure lockMutation.mutate/reset for stable useEffect deps
- Revision 6: Refactor lock mechanism and address andrescrz review feedback
Lock mechanism refactored to semaphore+MapCache approach:
- RPermitExpirableSemaphoreReactive for atomic capacity-limited locking
- RMapCacheReactive for tracking permit holders with auto-expiry TTL
- addSlotPermits for adjusting capacity on annotatorsPerItem changes
Review feedback from andrescrz:
- Rename lock_timeout_minutes to lock_timeout_seconds (UInt32, default 300)
- Move @NonNull from interfaces to implementations
- Add entity_type filter to feedback_scores queries for index pruning
- Remove FINAL from authored_feedback_scores and comments queries
- Full test assertions using recursive comparison
- Move inner records to top-level classes (LockResponse, LocksResponse, ItemLockInfo)
- Lombok @RequiredArgsConstructor in AnnotationQueueItemLockServiceImpl
- CollectionUtils.isEmpty for null-safe feedbackDefinitionNames in DAO
- Flatten nested flatMap in tryLockItem
- Remove Jakarta validation test, parameterize update tests
- Revision 7: Address remaining andrescrz and baz reviewer feedback
- Remove @NonNull from tryLockItem interface method
- Make update() always validate queue exists (consistent 404)
- Add @Nullable on LockResponse.expiresAt
- Set TTL on semaphore keys and MapCache containers to prevent unbounded growth
- Remove lenient() from behavioral test stubs
- Full object assertions in unit tests
- Revision 8: Switch RMapCache to RMapCacheNative for Valkey compatibility
RMapCache uses Lua struct.pack/struct.unpack scripts that fail on Valkey 8.x
with "bad argument #2 to 'unpack' (data string too short)". RMapCacheNative
uses native Redis 7.4+ HEXPIRE commands for per-entry TTL instead.Also renames test methods from Minutes to Seconds.
- Revision 9: Switch to RMap with manual expiry for Valkey compatibility
RMapCacheNative per-field TTL (HEXPIRE) is a no-op on Redis <7.4 and Valkey <9.0,
causing locks to never expire. RMapCache Lua struct scripts crash on Valkey.Switch to plain RMap with expiry timestamp embedded in value (permitId:expiryMs).
Expired entries filtered on read and lazily cleaned via fastRemove.
Semaphore remains for atomic capacity locking (unchanged).- Revision 10: Fix updateAnnotatorsPerItem test assertion (300 → 5)
下载附件