github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.18-5172 cc10aad002
    比较

    [NA] [SDK] test: stabilize e2e verifiers + rate-limit unit test, drop unit-test coverage (#6516)

    frostbyte_neo 发布于 2026-04-28 13:15:53 +00:00

    • [NA] [SDK] test(e2e): poll verify_trace/verify_span until expected fields match

    Existence-only polling (get_*_content(...) is not None) was satisfied
    right after the initial create flush, so a later update (e.g. a message
    replayed from the offline queue, or a follow-up SDK call) could miss the
    ClickHouse ingest window and the verifier would assert on a stale snapshot.
    Under xdist load that lag widens and the race becomes flaky (surfaced by
    test_failed_message_replay__update_span on OPIK-6217 parallel CI).

    Refactor: extract _retry_until_assertions_pass(check), run the full
    assertion body inside it, and rely on synchronization.until to retry
    until either the asserts pass or the timeout fires (then re-run once so
    the real failure reaches pytest with its traceback). No parallel
    "matches expected" predicate — the asserts already encode what counts
    as a match. Same broad-catch policy as the rest of this file.

    Also: convert if not until(...): raise AssertionError(...) patterns to
    assert until(...), "..." for consistency.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • fix(verifiers): retry on pytest_deepassert.equal failures (BaseException)

    Polling was a no-op for assert_equal mismatches because
    pytest_deepassert.equal raises pytest.fail.Exception
    (_pytest.outcomes.Failed) — which inherits from BaseException, not
    Exception. synchronization.until(allow_errors=True) only catches
    Exception, so the Failed escaped the loop on the first iteration and
    the verifier reported the stale snapshot exactly as before the fix.

    Convert that one BaseException-derived class into a falsy return inside
    _attempt so until() can retry. Plain assert (AssertionError, an
    Exception) was already covered by until's broad catch. No widening of
    until itself — keeps SystemExit/KeyboardInterrupt etc. propagating.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • [NA] [SDK] test(rate_limiting): decouple retry_after from test sleep window

    The flake in
    test_dynamic_rate_limiting__rate_limited__check_queue_messages_are_put_back
    came from retry_after and time.sleep(retry_after) using the same value
    (SLEEP_BETWEEN_LOOP_ITERATIONS * 3 = 0.3s). When the test asserted, the
    consumer was waking up from its rate-limit wait at roughly the same
    instant, racing into a second pop-and-rate-limit; if the assert landed
    between the pop and the push-back, queue_size dipped to 4 instead of 5.

    Fix: bump retry_after to 10× the loop interval and shorten the test's
    observation sleep to 3× the loop interval. The consumer is now firmly
    inside its wait branch when the test asserts, with no boundary race.

    Verified with 20 sequential runs locally: all green.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • [NA] [SDK] ci: drop coverage computation from python unit tests

    pytest --cov=src/opik was emitting CoverageWarning: No data was collected. and adding wall-clock without a downstream consumer (the
    result was never published or gated on). Drop it from the unit-test
    job so runs are faster and the noise goes away.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • test(rate_limiting): use synchronization.until for polling

    Replace hand-rolled while time.monotonic() < deadline loops with the
    existing synchronization.until helper. Same polling semantics, less
    local boilerplate, and we lean on the shared utility used elsewhere
    in the test suite. Validated with 20 consecutive runs — no flakes.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • ci(e2e): reduce xdist workers from 4 to 3

    Drop -n from 4 to 3. Past ~3 the per-runner gains flatten because the
    docker-compose backend on the same runner starts to thrash and we
    become tail-bound on the slowest single test file regardless. Updated
    the comment to reflect the new worker count.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    Co-authored-by: Claude Opus 4.7 (1M context) noreply@anthropic.com

    下载附件