diff --git a/README.md b/README.md index 6332548..d1341b6 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ * [Bloome — build & run AI agent teams in the cloud, zero setup](https://bloome.im/app?ref=G6BYnov0&utm_medium=github&utm_source=lyogavin-airllm-ivor-202606) ## Updates -[2026/08] **Qwen3.8-27B** support: Qwen's new dense VL (Gated DeltaNet + Gated Attention, native vision). Same one-liner as every other model. Needs `transformers` 5.8+. +[2026/08] **Qwen3.8-27B** support: Qwen's new dense VL (Gated DeltaNet + Gated Attention, native vision) runs in **3.33GB** of VRAM, measured end to end on one RTX 3090. Needs `transformers` 5.8+. [2026/07] **Kimi K3 (2.8T)** support: the largest open-source model runs on a single card in **3.72GB** of VRAM, measured end to end on one RTX 6000 Ada. Per-expert streaming loads only the experts a token actually routes to. K3 brings three requirements of its own: `pip install compressed-tensors flash-attn` (its model code mandates flash attention regardless of what you request), a CUDA 12 build of torch, since no prebuilt flash-attn wheel exists for CUDA 13 yet, and `transformers` 4.56.x, as its remote code does not load on 5.x. @@ -105,7 +105,7 @@ MAX_LENGTH = 128 model = AutoModel.from_pretrained("Qwen/Qwen3-32B") # go bigger with the exact same one line: -#model = AutoModel.from_pretrained("Qwen/Qwen3.8-27B") # 27B dense VL +#model = AutoModel.from_pretrained("Qwen/Qwen3.8-27B") # 27B dense VL, 3.33GB #model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B") # 235B, runs in ~3GB #model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3") # 671B, runs in ~12GB @@ -285,6 +285,7 @@ The trick: AirLLM only ever keeps **one layer on the GPU at a time**, so the VRA |---|---|---| | Qwen3 / Mistral / Phi (≈8B) | 8B | **~1–2 GB** | | Qwen3-30B / Mixtral (MoE) | 30–47B | **~1–3 GB** | +| Qwen3.8-27B (dense VL) | 27B | **3.33 GB** | | Qwen3-235B (MoE) | 235B | **~3 GB** | | Llama 3.x 70B (full precision) | 70B | **~4 GB** | | Llama 3.1 405B | 405B | **~8 GB** |