-
[OPIK-7186] [SDK] feat: surface batch-flush data loss via FlushResult (#7513)
frostbyte_neo 发布于
2026-07-22 10:58:03 +00:00 - [OPIK-7186] [SDK] feat: surface batch-flush data loss via FlushResult
The background sender silently dropped trace/span batches on terminal
failures (e.g. 403), while flush()/end() reported success. Add in-band,
non-blocking, non-throwing visibility:- FlushResult returned by flush()/end() (flushed, drops, pending_replay,
per-drop details); flush() bool now reflects data loss. - Opik.last_flush_result + Opik.get_upload_errors() (sender-wide history).
- DataLossTracker records terminal drops; FlushReporter assembles results.
- Drops recorded across processor (HTTP/serialization/unknown, 401),
queue overflow (generic on_evict hook), and shutdown-replay abandonment.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- fix(sdk): address PR review — idempotent end(), flush() failure result, exact dropped_items
- end(): short-circuit when release() returns None so a repeated end() no
longer overwrites last_flush_result with a spurious not-flushed result - flush(): on failure, record a failed FlushResult instead of leaving a stale
prior success in last_flush_result - manager.release(): return None (not close()'s bool) on the flush=False path,
matching the "no drain outcome" contract - DataLossTracker: keep exact running message/item totals so dropped_items no
longer undercounts once details are evicted from the bounded window - revert recording unparseable-429s as terminal data loss (a 429 is transient,
not lost) — restores prior behavior - docs: qualify end()/flush() for flush=False and attachment-upload scope
- test: build the flush-result client via new to avoid real background
resources; add idempotent-end and stale-result coverage
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- docs(sdk): fix subject-verb agreement in drops_since docstring
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- feat(sdk): add Opik.get_data_loss() sender-wide data-loss history
last_flush_result is scoped to a single flush window, so drops that happened
before or between flushes were recorded but not retrievable in-band. Expose the
DataLossTracker's retained history via Opik.get_data_loss() (tracker ->
FlushReporter -> client). Storage stays a capped deque, so memory is bounded
regardless of run length.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- feat(sdk): return ErrorsReport with timestamps from Opik.get_errors_report()
Replace the bare-list get_data_loss() with get_errors_report() returning an
ErrorsReport instance: exact all-time dropped message/item totals, retained
per-drop details, a generated_at timestamp, and first/last_failure_at helpers
(each FailedMessageInfo already carries its own timestamp). Storage stays a
capped deque, so memory is bounded.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- refactor(sdk): drop ErrorsReport.has_data_loss
Callers can check total_dropped_messages directly; the convenience flag is
redundant.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- docs(sdk): document that ErrorsReport failures are capped
Make the bounded (drop-oldest, default 1000) nature of the report's per-drop
details explicit on ErrorsReport and Opik.get_errors_report(): totals stay
exact, but failures keeps only the most recent entries.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- refactor(sdk): drop the lock in DataLossTracker
Per review: the deque is thread-safe and an exact-to-the-message data-loss
tally isn't worth lock contention on the hot sending path. Counters are plain
ints now (best-effort under concurrent drops); reads snapshot the deque once.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- refactor(sdk): make FlushResult/ErrorsReport failures an immutable tuple
Address PR review: frozen dataclasses now expose failures as
Tuple[FailedMessageInfo, ...] so the value is truly immutable; clarify that
flush()'s bool does reflect upload completion (only the data-loss detail
excludes uploads); strengthen the eviction test to assert retained identities
and order.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Co-authored-by: Claude Opus 4.8 noreply@anthropic.com
下载附件