github-featured / ollama--ollama

发布

0 发布 616 Git 标签

RSS 订阅源
  • v0.23.1 15e6076d79
    比较

    mlx: Gemma4 MTP speculative decoding (#15980)

    frostbyte_neo 发布于 2026-05-05 15:55:04 +00:00

    This change adds support for MTP (multi-token prediction) speculative decoding for the
    gemma4 model family.

    It includes:

    • support for importing safetensors based gemma4 draft models with ollama create
    • a new DRAFT command in the Modelfile for specifying draft models
    • a --quantize-draft flag for the ollama create command to quantize the draft model
    • cache support for speculation
    • changes to the rotating cache to be able to handle MTP correctly
    • sampling support for draft model token prediction

    Co-authored-by: Daniel Hiltgen daniel@ollama.com

    下载附件