github-featured / topoteretes--cognee

发布

0 发布 152 Git 标签

RSS 订阅源
  • v1.0.7 0dd56ab7d8
    比较

    fix(llm): preflight uses litellm ping instead of structured-output (1.0.7) (#2784)

    frostbyte_neo 发布于 2026-05-05 18:36:36 +00:00

    Summary

    The startup connection check in
    cognee/infrastructure/llm/utils.py::test_llm_connection ran a full
    structured-output call (LLMGateway.acreate_structured_output(..., response_model=str)) for what is supposed to be a lightweight ping. For
    non-OpenAI providers this routes through the provider-specific adapter,
    whose tenacity retry decorator (stop_after_delay(128) with
    wait_exponential_jitter(8, 128)) can mask a fast underlying failure
    behind exponential backoff, tripping the 30 s asyncio.wait_for and
    surfacing a misleading "endpoint unreachable" message even when the
    endpoint is healthy (verified with the same key/model via raw curl in
    <1 s in the original report).

    This PR replaces the preflight with a uniform litellm.acompletion ping
    (max_tokens=1, num_retries=0, no structured-output wrapper).
    Authentication or routing errors now surface immediately instead of
    being eaten by the inner retry stack. Provider routing is made explicit
    via custom_llm_provider so it's correct regardless of whether
    LLM_MODEL is written as claude-haiku-4-5 or
    anthropic/claude-haiku-4-5.

    Stacking

    This PR is stacked on top of #2783 (the max_tokens Anthropic fix,
    base main). The two are related: the original 30 s timeout on
    LLM_PROVIDER=anthropic was triggered by the missing max_tokens
    causing the adapter retry loop to spin. The first PR fixes the symptom;
    this PR fixes the preflight design so future provider-specific bugs
    don't get masked the same way. Merge #2783 first, then this. Otherwise
    expect a pyproject.toml / uv.lock version conflict on 1.0.6.

    Test plan

    • LLM_PROVIDER=anthropic, bad key: now raises
      AuthenticationError in ~0.2 s instead of TimeoutError in 30 s
    • LLM_PROVIDER=openai, valid key: preflight returns successfully
    • ruff check + ruff format --check clean on changed file
    • LLM_MODEL containing duplicated provider prefix (e.g.,
      anthropic/claude-haiku-4-5) routes correctly
    • CI on PR: full unit + integration suite
    • Cross-provider sanity: gemini, ollama, custom (OpenAI-compatible)
      endpoints

    Note on branching

    Repo convention is to branch from dev. Targeting main per the
    request to ship as a hotfix release alongside #2783. A follow-up
    cherry-pick / merge into dev is needed for both PRs.

    🤖 Generated with Claude Code

    Summary by CodeRabbit

    • Bug Fixes

    • More reliable LLM connectivity check: uses a lightweight ping, better
      provider/model routing, treats token/output-limit responses as
      successful health checks, and improves error handling.

    • Chores

      • Package version bumped to 1.0.7.
    下载附件