github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 2.0.13-5098 00a0df8883
    比较

    [OPIK-6099] [SDK] fix: infer provider from LitellmModel prefix in OpikTracingProcessor (#6432)

    frostbyte_neo 发布于 2026-04-22 16:50:30 +00:00

    • [OPIK-6099] [SDK] fix: infer LLM provider from LitellmModel prefix in OpikTracingProcessor

    When the OpenAI Agents SDK routes calls through LitellmModel to a non-OpenAI
    provider, the model string arrives in LiteLLM form (e.g. "gemini/gemini-2.5-flash").
    Previously the provider was hardcoded to OPENAI on every span, so the backend cost
    lookup used the wrong (model, provider) key and returned DEFAULT_COST = 0.

    In openai-agents 0.14+, LitellmModel emits GenerationSpanData (not ResponseSpanData),
    and the existing generation branch was dropping model/provider/usage entirely —
    so non-OpenAI calls through openai-agents were receiving zero cost regardless of
    provider attribution. Both the response and generation paths now extract
    model/provider/usage and infer the provider from the LiteLLM prefix.

    The LiteLLM prefix→LLMProvider mapping is extracted into opik.llm_usage.litellm_provider_mapping
    and reused by the LiteLLM decorator. Unknown prefixes pass through as raw strings so
    the backend can still attempt a lookup and future catalog additions start working
    without another SDK release.

    Implements OPIK-6099.

    • test(openai-agents): use vertex_ai in integration test to avoid Gemini free-tier quota

    Gemini free tier rate-limits gemini-2.5-flash aggressively enough that
    repeated local runs consistently fail with 429/503. The configured VertexAI
    credentials have looser quota and exercise the same GenerationSpanData
    code path that OPIK-6099 fixes — the only observable difference is the
    inferred provider (GOOGLE_VERTEXAI instead of GOOGLE_AI).

    • test(openai-agents): add litellm[google] to openai tests requirements

    Pulls in google-cloud-aiplatform so the new VertexAI integration test can
    authenticate via the configured service-account credentials without a
    manual install step.

    • ci(openai-tests): add GCP credentials for VertexAI integration test

    The new LiteLLM VertexAI test under tests/library_integration/openai/agents_tests
    needs Application Default Credentials to authenticate to google-cloud-aiplatform.
    Mirror the auth setup from lib-adk-tests.yml: GCP_CREDENTIALS_JSON secret,
    GOOGLE_CLOUD_{LOCATION,PROJECT} + GOOGLE_GENAI_USE_VERTEXAI env, and the
    google-github-actions/auth@v2 step that writes ADC for the pytest invocation.

    • refactor(openai-agents): extract helpers, drop verbose comments

    Factor out _infer_provider, _wrap_as_dict, and _build_opik_usage so the
    response/generation parsers read as plain data transformations instead of
    multi-branch bodies with long explanatory comment blocks. Behavior is
    unchanged; all 25 existing tests still pass.

    下载附件