发布

  • llm: Don't always evict models on CPU-only systems

    frostbyte_neo 发布于 2025-12-02 18:58:08 +00:00

    Model eviction happens when we have at least one other model
    loaded and are unable to load all layers into VRAM. However, on
    CPU-only systems we can never load layers into VRAM, so this
    constantly triggered eviction.

    Fixes #13227

    下载附件