github-featured / kvcache-ai--ktransformers

发布

0 发布 40 Git 标签

RSS 订阅源
  • v0.6.2.post1 d7b5b49a3e
    比较

    [release]: 0.6.2.post1

    frostbyte_neo 发布于 2026-05-03 13:07:23 +00:00 | 73 次提交 在此版本后已推送到 main

    V4-Flash MXFP4 full-GPU prefill fallback now works:

    • Previously crashed all TP schedulers with StopIteration/AttributeError
      whenever --kt-gpu-prefill-token-threshold was low enough to actually
      fire (path was hardcoded for FP8/INT4 layouts).
    • Now detects MXFP4, re-runs the V4 swizzle on the 256-expert gpu_layer,
      caches the load across prefill chunks.
    • Measured on 8x RTX 5090 (threshold=1024, chunked=1024):
      16k input -> 2011 tok/s, 65k -> 2798, 262k -> 2154 prefill TPS.
    下载附件