github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.53-5514 5783e5ff2c
    比较

    [NA] [SDK] [CI] test: stabilize ADK + GenAI CI and drop flaky ADK test (#6916)

    frostbyte_neo 发布于 2026-05-29 16:13:49 +00:00

    • [NA] [PSDK] test: drop flaky ADK transfer_to_agent test

    test_adk__transfer_to_agent__tracked_and_span_created has failed
    across consecutive CI runs (26582543507, 26632552400, 26637545466) on
    every supported Python version. Two failure modes, same effect — no
    Translator span and root.output=None:

    • Gemini 2.5 Flash decides to translate inline instead of invoking
      transfer_to_agent, despite an instruction that forbids it.
    • Vertex AI returns 429 RESOURCE_EXHAUSTED under CI quota pressure
      (visible in 26637545466's sequential-agents test, same project).

    Prior fixes — hardening the delegation instruction (926c71d1fa),
    relaxing output assertions (8c5f63ce94), and pytest.mark.flaky
    reruns (05318dd8a2) — were all unsuccessful; the rerun attempt
    explicitly confirmed all three attempts failed deterministically and
    was reverted (a21abf5d4e).

    The unique coverage was the execute_tool transfer_to_agent span,
    which can't be exercised without an LLM choosing to delegate.
    Sub-agent span tracing is already covered deterministically by
    test_adk__sequential_agent_with_subagents__every_subagent_has_its_own_span.
    Dropping the test rather than carrying an indefinitely red check.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • [NA] [CI] chore: shrink ADK matrix to first + last Python

    The ADK suite runs ~46 live Gemini calls × 5 Python versions = ~230
    concurrent Vertex AI requests against a single GCP project per CI run.
    us-east1 / gemini-2.5-flash quota tops out well below that, and run
    26637545466 already surfaced a 429 RESOURCE_EXHAUSTED that masqueraded
    as a test bug. Intermediate Python versions virtually never surface
    ADK-specific regressions the bookends miss, so collapse to 3.10 + 3.14
    and reclaim that quota for retries.

    Applies to both the current and legacy ADK 1.3.0 workflows.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • chore(ci): cap ADK matrix concurrency at max-parallel 2

    Today's matrix already collapses to 3.10 + 3.14, so this is a no-op in
    isolation. The value is forward-defensive: if anyone re-expands the
    matrix later (a new Python version, a parameterised ADK pin), the cap
    keeps concurrent Vertex AI Gemini calls from re-multiplying back into
    429 RESOURCE_EXHAUSTED territory without anyone having to re-discover
    the quota math.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • chore(ci): apply same matrix shrink + concurrency cap to GenAI tests

    The Google GenAI library integration tests share the opik-sdk-tests
    GCP project with the ADK suite, so an uncapped 5× matrix here works
    against the same Vertex AI quota pool the ADK changes just relieved.
    Same justification, same defaults: matrix = oldest + newest Python,
    max-parallel = 2.

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    • chore(ci): restore full Python matrix; rely on max-parallel: 2 to throttle

    Earlier in this PR the matrix collapsed to 3.10 + 3.14 to relieve
    Vertex AI quota. With max-parallel: 2 now in place, throttling does
    the same job without losing coverage on intermediate Python versions —
    restore PYTHON_VERSIONS on all three Gemini-touching workflows (ADK,
    ADK legacy 1.3.0, GenAI).

    Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

    Co-authored-by: Claude Opus 4.7 (1M context) noreply@anthropic.com

    下载附件