github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.11.6-4931 9739e7f13f
    比较

    [OPIK-5849] [BE] fix: exclude skipped items from pass_rate calculation (#6252)

    frostbyte_neo 发布于 2026-04-14 16:06:02 +00:00

    • [OPIK-5849] [BE] fix: exclude skipped items from pass_rate calculation

    Items without assertion results (skipped) were treated as passed,
    inflating pass_rate. Now they are excluded from both numerator and
    denominator via has_assertions flag + countIf/sumIf filtering.

    Applied to all 3 query locations: ExperimentAggregatesDAO
    (GET_PASS_RATE_AGGREGATION) and ExperimentDAO (2x pass_rate_agg CTEs).

    Updated existing test expectation and added new test for mixed
    scored/unscored items scenario.

    Implements OPIK-5849: Exclude skipped items from pass_rate calculation
    in evaluation suite experiments

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    • fix(test): expect passedCount=0, totalCount=0 for all-skipped items

    The raw query returns 0 (not null) for passedCount and totalCount
    when no items have assertions. Only passRate becomes null via the
    explicit if-check. Updated test expectation accordingly.

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    • fix(sql): use nullIf for division-safe pass_rate calculation

    ClickHouse evaluates both if() branches — when countIf(has_assertions)
    is 0, the division 0/0 produces NaN which toDecimal64 may reject.
    Replace with nullIf(toFloat64(countIf(...)), 0) so division by NULL
    returns NULL directly, avoiding any NaN/toDecimal64 risk.

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    • fix(sql): use CAST Nullable(Decimal64) for pass_rate in ExperimentDAO

    toDecimal64(NULL, 9) throws when all items are skipped because
    toDecimal64 cannot handle NULL input. Use CAST(... AS Nullable(
    Decimal64(9))) instead, which preserves NULL for the API response.

    ExperimentAggregatesDAO is unaffected — it uses ifNull(..., 0)
    before toDecimal64 so NULL never reaches the conversion.

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    • fix(sql): revert ExperimentDAO pass_rate to if() guard

    CAST(... AS Nullable(Decimal64(9))) is not supported in our
    ClickHouse version, causing 500 errors. Revert to the if() guard
    which was proven working in CI. The ExperimentAggregatesDAO keeps
    ifNull+nullIf since NULL never reaches toDecimal64 there.

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    • fix(sql): make pass_rate division safe inside if() false branch

    ClickHouse evaluates both if() branches. When countIf=0, the false
    branch computed 0/0 → NaN → toDecimal64(NaN) which could throw.

    Fix: wrap division with nullIf+ifNull inside the false branch so it
    always produces a valid toDecimal64 input (0 as fallback). The if()
    still returns NULL from the true branch for the all-skipped case.

    Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com

    Co-authored-by: Claude Opus 4.6 (1M context) noreply@anthropic.com

    下载附件