github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.1.15-5897 fb8d8e87a4
    比较

    [OPIK-6892] [BE] feat: NaN-aware aggregate validation + sentinel translation framework (#7307)

    frostbyte_neo 发布于 2026-07-03 07:59:23 +00:00

    • [OPIK-6892] [BE] feat: NaN-aware aggregate validation + sentinel translation framework

    • fix(traces): guard OptimizationDAO duration read for the non-nullable cutover

    OptimizationDAO reads the materialized traces.duration column unguarded
    (quantiles(0.5)(t.duration) -> sum(duration_p50 * trace_count)); post-cutover an
    experiment whose traces all lack end_time yields quantiles(all-NaN)=NaN, which
    sum propagates (isNotNull(NaN) is true), returning NaN for weighted_duration
    where it was NULL-safe before. Apply the same if(isNaN(duration), NULL, duration)
    guard used on every other traces.duration read. No-op pre-cutover.

    • fix(traces): guard experiment-comparison duration filter, thread V2-client flag, align sentinel precision doc
    • NEW_DURATION_ANALYTICS_DB (ExperimentsComparisonValidKnownField.DURATION) wrapped
      in if(isNaN(duration), NULL, duration): post-cutover a != filter on an unfinished
      trace would otherwise leak (NaN != x is true), mirroring the ttft filter guard.
    • toAnalyticsDbFiltersV2Client gains a traceColumnsNonNullable overload so the
      v2-client filter path can opt into sentinel-aware end_time (was structurally off).
    • SentinelTranslation epoch-sentinel doc precision 6 -> 9 to match the DateTime64(9)
      column and every SQL fragment.

    All no-ops while the columns are Nullable.

    • test(traces): cover flag-on thread end_time sort mapping

    The thread ORDER BY change + flag-gated SORT_FIELD_MAPPING_END_TIME_SENTINEL had no
    sort test (the trace side does). Add one asserting an all-absent thread (end_time =
    epoch → nullIf → NULL) sorts last ascending, not first as 1970 — the discriminator
    for the flag-gated thread sort mapping.

    • fix(traces): guard ttft dedup-merge against the non-nullable join miss

    The INSERT/INSERT_UPDATE dedup merge keeps old_trace.ttft when NOT isNaN(...). Post-
    cutover ttft is non-nullable Float64, so a brand-new-trace LEFT-JOIN miss fills it
    with the type zero 0.0 (not the NaN default) — NOT isNaN(0.0) is true, so the merge
    keeps 0.0 and discards the client's ttft (and later updates too). end_time/start_time
    are safe only because their type zero is the epoch, caught by notEquals(epoch).

    Add an old_trace.id != '' row-presence guard so a join miss falls through to
    new_trace.ttft. No-op pre-cutover (a Nullable miss is NULL; id is already '').
    NaNAwareAggregateIntegrationTest pins the join-miss fill and the guard's effect.

    • refactor(traces): delegate filterNan/nullIfEpoch to SentinelTranslation

    KpiCardDAO.filterNan, WorkspaceMetricsDAO.filterNan and DatasetItemResultMapper
    .nullIfEpoch re-implemented the sentinel->null logic that SentinelTranslation now
    owns (three NaN->null copies, two epoch->null). Make them thin delegators to
    nanToNull/epochToNull so there is a single definition and no drift, without churning
    the 16 call sites. Behavior-preserving (verified byte-identical); covered by
    SentinelTranslationTest and the existing DAO suites.

    • test(traces): assert thread sort with full-object assertThreads

    Replace the endTimes field assertion with the canonical order-sensitive
    TraceAssertions.assertThreads, using each thread retrieved via getTraceThread as
    the expected in sorted order (present, then absent). Threads have no materialized-
    duration artifact (thread duration is the epoch-guarded if(...)->NULL), so a full
    object comparison is valid here (unlike the trace sort test). The run also confirms
    the retrieve and list endpoints return field-consistent thread objects.

    • test(traces): cover experiment-comparison duration filter and v2-client flag threading

    Close the two coverage gaps from the review fixes:

    • FilterQueryBuilderSentinelTest: assert ExperimentsComparisonValidKnownField.DURATION
      (EXPERIMENT_ITEM) renders if(isNaN(duration), NULL, duration) so a != filter excludes
      unfinished traces post-cutover (mirrors the trace ttft/duration filter coverage).
    • FilterQueryBuilderV2ClientTest: assert toAnalyticsDbFiltersV2Client threads
      traceColumnsNonNullable — trace end_time uses the sentinel-aware nullIf under the flag
      and the raw column without it, in v2 {name:Type} placeholder form.
    下载附件