chore: refresh star history chart

Support Qwen3.8-27B (3.2.0)

.github/workflows/star-history.yml

@@ -6,12 +6,14 @@ name: Refresh Star History
 # star-history.com SVG API, which is rate-limited and returns an empty chart
 # for large repos. The README embeds the committed PNG, so it always renders.
 #
-# Since 2026-06-30 GitHub restricts GET /repos/{owner}/{repo}/stargazers to
-# admins/collaborators. The Actions GITHUB_TOKEN is an installation token and
-# gets 403, so this workflow needs a collaborator PAT in STAR_HISTORY_TOKEN:
-#   fine-grained: Metadata (read) on this repo, or
-#   classic: public_repo
-#   gh secret set STAR_HISTORY_TOKEN
+# The chart is built from GraphQL, not REST. GitHub restricted REST /stargazers to
+# admins/collaborators in 2026-06, and then refused fine-grained tokens on it outright
+# ("Resource not accessible by personal access token"), which broke this job twice on
+# token permissions alone. GraphQL returns the same public starredAt data without that
+# gate, so the default Actions GITHUB_TOKEN is enough and no PAT is required.
+#
+# The STAR_HISTORY_TOKEN secret this used to need is now unused and can be deleted; do
+# not add a PAT back if this fails, since a token was never the real fix.
 
 on:
   schedule:
@@ -35,15 +37,12 @@ jobs:
 
       - name: Generate star history charts (light + dark)
         env:
-          # Collaborator/admin PAT; Actions GITHUB_TOKEN cannot list stargazers.
-          GITHUB_TOKEN: ${{ secrets.STAR_HISTORY_TOKEN }}
+          # The built-in token is enough for GraphQL, so no secret is involved.
+          GITHUB_TOKEN: ${{ github.token }}
         run: |
-          if [ -z "$GITHUB_TOKEN" ]; then
-            echo "::error::Missing secret STAR_HISTORY_TOKEN. Create a collaborator PAT (fine-grained Metadata:read, or classic public_repo) and run: gh secret set STAR_HISTORY_TOKEN"
-            exit 1
-          fi
-          python scripts/gen_star_history.py "${{ github.repository }}" assets/star-history.png light
-          python scripts/gen_star_history.py "${{ github.repository }}" assets/star-history-dark.png dark
+          python scripts/gen_star_history.py "${{ github.repository }}" \
+            assets/star-history.png light \
+            assets/star-history-dark.png dark
 
       - name: Commit if changed
         uses: stefanzweifel/git-auto-commit-action@v5

README.md

@@ -6,7 +6,7 @@
 [**Example notebooks**](#example-python-notebook) | 
 [**FAQ**](#faq)
 
-**AirLLM** dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run **405B Llama 3.1** on **8GB**, and **DeepSeek-V3 (671B)** on **~12GB**.
+**AirLLM** dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run **405B Llama 3.1** on **8GB**, **DeepSeek-V3 (671B)** on **~12GB**, and **Kimi K3 (2.8T)** — the largest open-source model released to date — on **under 4GB**, because sparse MoE models stream one expert at a time rather than a whole layer.
 
 <a href="https://github.com/lyogavin/airllm/stargazers">![GitHub Repo stars](https://img.shields.io/github/stars/lyogavin/airllm?style=social)</a>
 [![Downloads](https://static.pepy.tech/personalized-badge/airllm?period=total&units=international_system&left_color=grey&right_color=blue&left_text=downloads)](https://pepy.tech/project/airllm)
@@ -31,6 +31,10 @@
 * [Bloome — build & run AI agent teams in the cloud, zero setup](https://bloome.im/app?ref=G6BYnov0&utm_medium=github&utm_source=lyogavin-airllm-ivor-202606)
 
 ## Updates
+[2026/08] **Qwen3.8-27B** support: Qwen's new dense VL (Gated DeltaNet + Gated Attention, native vision) runs in **3.33GB** of VRAM, measured end to end on one RTX 3090. Needs `transformers` 5.8+.
+
+[2026/07] **Kimi K3 (2.8T)** support: the largest open-source model runs on a single card in **3.72GB** of VRAM, measured end to end on one RTX 6000 Ada. Per-expert streaming loads only the experts a token actually routes to. K3 brings three requirements of its own: `pip install compressed-tensors flash-attn` (its model code mandates flash attention regardless of what you request), a CUDA 12 build of torch, since no prebuilt flash-attn wheel exists for CUDA 13 yet, and `transformers` 4.56.x, as its remote code does not load on 5.x.
+
 [2026/06] **v3.0**: FP8 model support + the latest models. Run **DeepSeek-V3 (671B) on ~12GB** and **Qwen3-235B on ~3GB**, plus Qwen3, Llama 3.x/4, DeepSeek V2/V3, Phi-4, Gemma and more — all through a single `AutoModel`.
 
 [2024/08/20] v2.11.0: Support Qwen2.5
@@ -101,6 +105,7 @@ MAX_LENGTH = 128
 model = AutoModel.from_pretrained("Qwen/Qwen3-32B")
 
 # go bigger with the exact same one line:
+#model = AutoModel.from_pretrained("Qwen/Qwen3.8-27B")          # 27B dense VL, 3.33GB
 #model = AutoModel.from_pretrained("Qwen/Qwen3-235B-A22B")     # 235B, runs in ~3GB
 #model = AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3")  # 671B, runs in ~12GB
 
@@ -270,7 +275,7 @@ model.tokenizer.decode(generation_output.sequences[0])
 
 AirLLM works out of the box with **virtually every popular open LLM** — just pass its Hugging Face ID to `AutoModel.from_pretrained(...)`. That covers all the major families:
 
-**Llama** (2 / 3 / 3.1 / 3.3 / 4) · **Qwen** (1 / 2 / 2.5 / 3, including MoE and FP8) · **DeepSeek** (V2 / V3 / R1) · **Mistral & Mixtral** · **Phi** · **Gemma** · **ChatGLM** · **Baichuan** · **InternLM** · **Yi** — and most new models the day they're released.
+**Llama** (2 / 3 / 3.1 / 3.3 / 4) · **Qwen** (1 / 2 / 2.5 / 3 / 3.5 / 3.8, including MoE, FP8, and native VL) · **DeepSeek** (V2 / V3 / R1) · **Mistral & Mixtral** · **Phi** · **Gemma** · **ChatGLM** · **Baichuan** · **InternLM** · **Yi** · **Kimi K3** — and most new models the day they're released.
 
 ### Tiny GPU, huge models
 
@@ -280,6 +285,7 @@ The trick: AirLLM only ever keeps **one layer on the GPU at a time**, so the VRA
 |---|---|---|
 | Qwen3 / Mistral / Phi (≈8B) | 8B | **~1–2 GB** |
 | Qwen3-30B / Mixtral (MoE) | 30–47B | **~1–3 GB** |
+| Qwen3.8-27B (dense VL) | 27B | **3.33 GB** |
 | Qwen3-235B (MoE) | 235B | **~3 GB** |
 | Llama 3.x 70B (full precision) | 70B | **~4 GB** |
 | Llama 3.1 405B | 405B | **~8 GB** |

air_llm/airllm/__init__.py

@@ -31,6 +31,8 @@ else:
         ("AirLLMInternLM", ".airllm_internlm"),
         ("AirLLMMistral", ".airllm_mistral"),
         ("AirLLMMixtral", ".airllm_mixtral"),
+        ("AirLLMKimiK3", ".airllm_kimi_k3"),
+        ("AirLLMQwen3_5", ".airllm_qwen3_5"),
     ):
         try:
             _mod = __import__(__name__ + _module, fromlist=[_name])

air_llm/airllm/airllm_base.py

@@ -6,15 +6,18 @@ import time
 from concurrent.futures import ThreadPoolExecutor
 
 import torch
-from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer, GenerationConfig
+import transformers
+from transformers import AutoConfig, AutoTokenizer, GenerationConfig
+from transformers.generation import GenerationMixin
 from accelerate import init_empty_weights
 from accelerate.utils.modeling import set_module_tensor_to_device
 from transformers.quantizers import AutoHfQuantizer
 
 from .profiler import LayeredProfiler
 
-from .utils import clean_memory, load_layer, \
+from .utils import clean_memory, load_layer, layer_tensor_names, load_layer_subset, \
     find_or_create_local_splitted_path
+from .persist import ModelPersister
 
 try:
     import bitsandbytes as bnb
@@ -25,6 +28,32 @@ except ImportError:
     bitsandbytes_installed = False
 
 
+# Helpers that transformers 5.0 moved out of transformers.utils.generic. Remote model code is
+# routinely written against an older transformers and still imports them from the old location,
+# which makes such models fail to import at all. Re-exporting is enough to load them.
+_RELOCATED_TRANSFORMERS_SYMBOLS = {
+    'OutputRecorder': 'transformers.utils.output_capturing',
+    'check_model_inputs': 'transformers.utils.output_capturing',
+}
+
+
+def restore_relocated_transformers_symbols():
+    """Re-export moved transformers helpers under their old names, where they are missing."""
+    import importlib
+    import transformers.utils.generic as generic
+
+    for name, new_home in _RELOCATED_TRANSFORMERS_SYMBOLS.items():
+        if hasattr(generic, name):
+            continue
+        try:
+            module = importlib.import_module(new_home)
+        except ImportError:
+            continue
+        symbol = getattr(module, name, None)
+        if symbol is not None:
+            setattr(generic, name, symbol)
+
+
 class AirLLMBaseModel:
     """
     Memory-frugal wrapper around a Hugging Face ``*ForCausalLM`` model.
@@ -40,6 +69,10 @@ class AirLLMBaseModel:
     them.
     """
 
+    # Upper bound on how much pinned (page-locked) host memory a single prefetched layer may use.
+    # Layers larger than this are loaded into ordinary pageable memory instead.
+    max_pinned_layer_bytes = 2 * 1024 ** 3
+
     # Subclasses override this to point at non-standard module names.
     def set_layer_names_dict(self):
         self.layer_names_dict = {'embed': 'model.embed_tokens',
@@ -92,6 +125,8 @@ class AirLLMBaseModel:
         self.compression = compression
         self.hf_token = hf_token
 
+        restore_relocated_transformers_symbols()
+
         self.set_layer_names_dict()
 
         self.model_local_path, self.checkpoint_path = find_or_create_local_splitted_path(
@@ -124,6 +159,11 @@ class AirLLMBaseModel:
         # avoids it. Users can still override via dtype=.
         if dtype is None:
             cfg_dtype = getattr(self.config, "torch_dtype", None)
+            if cfg_dtype is None:
+                cfg_dtype = getattr(self.config, "dtype", None)
+            if cfg_dtype is None:
+                text_cfg = getattr(self.config, "text_config", None)
+                cfg_dtype = getattr(text_cfg, "torch_dtype", None) or getattr(text_cfg, "dtype", None)
             if isinstance(cfg_dtype, str):
                 cfg_dtype = getattr(torch, cfg_dtype, None)
             dtype = cfg_dtype if isinstance(cfg_dtype, torch.dtype) else torch.float16
@@ -157,6 +197,7 @@ class AirLLMBaseModel:
         self.max_seq_len = max_seq_len
 
         self.set_layers_from_layer_names()
+        self._load_resident_modules()
         self._install_streaming_hooks()
 
     # ---- customization hooks for subclasses -------------------------------------------------
@@ -175,29 +216,136 @@ class AirLLMBaseModel:
 
     # ---- model construction -----------------------------------------------------------------
 
+    def _propagate_attn_implementation(self, impl):
+        """Push the attention choice down into nested sub-configs.
+
+        Multimodal wrappers keep the real decoder under a sub-config -- Kimi K3 uses ``text_config``
+        -- and transformers records the request only on the config it was handed. The sub-model then
+        reads an unset value and falls through to a flash-attention path, which fails outright on a
+        machine without flash-attn installed.
+        """
+        from transformers import PretrainedConfig
+
+        def walk(cfg, depth=0):
+            if depth > 2:
+                return
+            for sub in vars(cfg).values():
+                if isinstance(sub, PretrainedConfig):
+                    sub._attn_implementation = impl
+                    walk(sub, depth + 1)
+
+        walk(self.config)
+
+    def _model_matches_layer_names(self, model):
+        """True when the instantiated class actually has the modules we plan to stream.
+
+        Auto factories key off ``model_type``, so a VL checkpoint can land on a text-only or
+        backbone class whose tree doesn't match ``layer_names_dict``. Reject those and keep trying.
+        """
+        try:
+            for key in ('embed', 'layer_prefix', 'norm', 'lm_head'):
+                mod = model
+                for attr in self.layer_names_dict[key].split('.'):
+                    mod = getattr(mod, attr)
+            return True
+        except AttributeError:
+            return False
+
+    def _auto_model_classes(self):
+        """Auto* factories to try, in order, when building the empty model.
+
+        ``AutoModelForCausalLM`` maps some VL model_types (notably ``qwen3_5``) onto a text-only
+        ``*ForCausalLM`` class. That class expects a text config and either crashes or builds a
+        model whose module names don't match the checkpoint. Conditional-generation architectures
+        therefore try the image-text factories first.
+        """
+        archs = getattr(self.config, "architectures", None) or []
+        arch = archs[0] if archs else ""
+        names = []
+        if any(tag in arch for tag in ("ConditionalGeneration", "ImageTextToText", "Multimodal")):
+            names.extend(["AutoModelForImageTextToText", "AutoModelForMultimodalLM"])
+        names.extend([
+            "AutoModelForCausalLM",
+            "AutoModelForImageTextToText",
+            "AutoModelForMultimodalLM",
+            "AutoModel",
+        ])
+        seen = set()
+        classes = []
+        for name in names:
+            if name in seen:
+                continue
+            seen.add(name)
+            cls = getattr(transformers, name, None)
+            if cls is not None:
+                classes.append((name, cls))
+        return classes
+
+    def _instantiate_on_meta(self, attn_implementation):
+        """Build the transformers model on the meta device for one attention implementation."""
+        self._propagate_attn_implementation(attn_implementation)
+        kwargs = {
+            "attn_implementation": attn_implementation,
+            "trust_remote_code": self.trust_remote_code,
+        }
+        errors = []
+        for name, cls in self._auto_model_classes():
+            try:
+                with init_empty_weights(include_buffers=False):
+                    model = cls.from_config(self.config, **kwargs)
+            except TypeError:
+                # Older Auto factories don't take attn_implementation.
+                try:
+                    with init_empty_weights(include_buffers=False):
+                        model = cls.from_config(self.config, trust_remote_code=self.trust_remote_code)
+                except Exception as e:  # noqa: BLE001 - try the next factory
+                    errors.append(f"{name}: {type(e).__name__}: {e}")
+                    continue
+            except Exception as e:  # noqa: BLE001 - try the next factory
+                errors.append(f"{name}: {type(e).__name__}: {e}")
+                continue
+            if not self._model_matches_layer_names(model):
+                errors.append(
+                    f"{name}: built {type(model).__name__} but it is missing "
+                    f"{self.layer_names_dict['layer_prefix']}"
+                )
+                continue
+            print(f"built empty {type(model).__name__} via {name} (attn={attn_implementation})")
+            return model
+        raise RuntimeError(
+            "Could not instantiate the model on meta from config. "
+            f"architecture={getattr(self.config, 'architectures', None)} "
+            f"model_type={getattr(self.config, 'model_type', None)}. "
+            f"Tried: {'; '.join(errors)}"
+        )
+
     def init_model(self):
         # Build the real model on meta (no memory). include_buffers=False so non-persistent
         # buffers such as rotary inv_freq are actually computed (they aren't in the checkpoint).
-        self.model = None
         try:
-            with init_empty_weights(include_buffers=False):
-                self.model = AutoModelForCausalLM.from_config(
-                    self.config, attn_implementation="sdpa", trust_remote_code=self.trust_remote_code)
-        except (ValueError, TypeError) as e:
+            self.model = self._instantiate_on_meta("sdpa")
+        except Exception as e:
             print(f"attn_implementation='sdpa' not available ({e}), falling back to eager attention")
-            self.model = None
-        if self.model is None:
             # Some (often remote-code) architectures don't support sdpa and also default to it, so we
             # must request eager explicitly; otherwise transformers re-selects sdpa and errors again.
-            with init_empty_weights(include_buffers=False):
-                self.model = AutoModelForCausalLM.from_config(
-                    self.config, attn_implementation="eager", trust_remote_code=self.trust_remote_code)
+            self.model = self._instantiate_on_meta("eager")
 
         quantization_config = getattr(self.config, "quantization_config", None)
+        if quantization_config is None:
+            # Nested multimodal configs (Kimi K3) keep it under text_config.
+            quantization_config = getattr(getattr(self.config, "text_config", None),
+                                          "quantization_config", None)
         if quantization_config is not None:
             self.hf_quantizer = AutoHfQuantizer.from_config(quantization_config, pre_quantized=True)
             device_map = self.hf_quantizer.update_device_map(None)
             self.hf_quantizer.preprocess_model(model=self.model, device_map=device_map)
+            # compressed-tensors registers a hook that expands every packed module on the first
+            # forward. That undoes per-expert streaming (a K3 layer becomes ~56GB) and is also
+            # unnecessary: we decompress each expert ourselves as it loads. Remove the hook.
+            hook = getattr(self.model, "ct_decompress_hook", None)
+            if hook is not None:
+                hook.remove()
+                delattr(self.model, "ct_decompress_hook")
 
         self.model.eval()
         self.model.tie_weights()
@@ -220,7 +368,16 @@ class AirLLMBaseModel:
         running_dtype = self.running_dtype
         base_cls = type(self.model)
 
-        class _AirLLMRuntimeModel(base_cls):
+        # transformers >= 4.50 removed GenerationMixin from PreTrainedModel, so model classes that
+        # predate that change (or ship as remote code, like Kimi K3's multimodal wrapper) no longer
+        # have .generate(). They still define prepare_inputs_for_generation, so mixing the class
+        # back in restores generation. It must come after the model class, per transformers.
+        extra_bases = () if isinstance(self.model, GenerationMixin) else (GenerationMixin,)
+        if extra_bases:
+            print(f"{base_cls.__name__} does not inherit GenerationMixin; mixing it in so "
+                  f"generate() works.")
+
+        class _AirLLMRuntimeModel(base_cls, *extra_bases):
             @property
             def device(self):
                 return running_device
@@ -270,12 +427,165 @@ class AirLLMBaseModel:
             state_dict = load_layer_output
 
         if self.prefetching and torch.cuda.is_available():
-            for k in state_dict.keys():
-                state_dict[k].pin_memory()
+            # pin_memory() returns a pinned copy rather than pinning in place, so the result has to
+            # be kept for the faster host->device copy to actually happen. Pinned memory can't be
+            # paged out, so we only spend it on layers small enough to be safe: a frontier MoE
+            # checkpoint has ~17GB layers, and with prefetching two are in flight at once, which
+            # would lock up ~34GB of RAM for a copy that is dwarfed by the disk read anyway.
+            total_bytes = sum(v.numel() * v.element_size() for v in state_dict.values())
+            if total_bytes <= self.max_pinned_layer_bytes:
+                try:
+                    for k in state_dict.keys():
+                        state_dict[k] = state_dict[k].pin_memory()
+                except RuntimeError:
+                    # Out of pinned memory: fall back to pageable, which is slower but always works.
+                    pass
 
         return state_dict
 
+    def _restore_plain_weight_modules(self, state_dict):
+        """Undo CT's packed-parameter layout when the checkpoint still ships a plain ``weight``.
+
+        Some checkpoints (Kimi K3) list residual/router Linears under the MXFP4 target list but
+        store them as bf16. After ``preprocess_model`` those modules expose ``weight_packed`` and
+        reject the real ``weight`` tensor. Restore a meta ``weight`` Parameter so the shard can load.
+        """
+        plain = {k[:-len('.weight')] for k in state_dict if k.endswith('.weight')}
+        packed = {k[:-len('.weight_packed')] for k in state_dict if k.endswith('.weight_packed')}
+        for prefix in plain - packed:
+            try:
+                module = self.model.get_submodule(prefix)
+            except AttributeError:
+                continue
+            names = list(module._parameters.keys())
+            if 'weight' in names or 'weight_packed' not in names:
+                continue
+            weight = state_dict[f'{prefix}.weight']
+            for name in names:
+                if name == 'weight' or name.startswith('weight_'):
+                    module._parameters.pop(name, None)
+            module.register_parameter(
+                'weight',
+                torch.nn.Parameter(torch.empty(weight.shape, device='meta', dtype=weight.dtype),
+                                   requires_grad=False),
+            )
+            for attr in ('quantization_scheme', 'quantization_status', 'quantization_format'):
+                if hasattr(module, attr):
+                    delattr(module, attr)
+
+    def _decompress_state_dict(self, state_dict):
+        """Expand packed payloads into the plain weights the modules expect.
+
+        compressed-tensors has no quantized compute kernels: it registers a hook that decompresses
+        the whole model before the first forward, after which every quantized module wants a plain
+        ``weight``. Our shards still hold ``weight_packed``/``weight_scale``, so we expand them here.
+
+        The expansion happens on the GPU, after transferring the *packed* bytes. For MXFP4 that
+        moves 4x less data across PCIe than transferring an already-expanded weight would.
+        """
+        if self.hf_quantizer is None:
+            return state_dict
+
+        packed_prefixes = {k[: -len('.weight_packed')]
+                           for k in state_dict if k.endswith('.weight_packed')}
+        if not packed_prefixes:
+            return state_dict
+
+        from compressed_tensors.compressors.base import BaseCompressor
+        try:
+            from compressed_tensors.linear.compressed_linear import CompressedLinear
+        except ImportError:
+            CompressedLinear = None
+
+        out = dict(state_dict)
+        for prefix in packed_prefixes:
+            try:
+                module = self.model.get_submodule(prefix)
+            except AttributeError:
+                continue
+            # CompressedLinear's own forward consumes the packed payload, so leave it packed.
+            if CompressedLinear is not None and isinstance(module, CompressedLinear):
+                continue
+            scheme = getattr(module, 'quantization_scheme', None)
+            if scheme is None or getattr(scheme, 'format', None) is None:
+                continue
+
+            local = {k[len(prefix) + 1:]: v.to(self.running_device)
+                     for k, v in state_dict.items() if k.startswith(prefix + '.')}
+            # scheme.format is an enum on some compressed-tensors versions and a plain str on others.
+            fmt = getattr(scheme.format, 'value', scheme.format)
+            compressor = BaseCompressor.get_value_from_registry(fmt)
+            decompressed = compressor.decompress(local, scheme)
+
+            for k in local:
+                out.pop(f'{prefix}.{k}', None)
+            if 'weight' in decompressed:
+                # Once expanded, the module reads only `weight`; the scales that came back merely
+                # describe how the checkpoint stored it, and keeping them would waste VRAM.
+                out[f'{prefix}.weight'] = decompressed['weight']
+                self._expose_plain_weight(module, decompressed['weight'])
+            else:
+                for k, v in decompressed.items():
+                    out[f'{prefix}.{k}'] = v
+        return out
+
+    def _expose_plain_weight(self, module, weight):
+        """Swap a module's packed parameters for the plain ``weight`` its forward reads.
+
+        compressed-tensors patches a ``quantized_forward`` onto quantized Linears that reads
+        ``self.weight``; the packed parameters only describe how the checkpoint stores the value.
+        Marking the module COMPRESSED tells that forward the weight is already on the quantization
+        grid, so it skips a fake-quantize that would only reproduce what we just decompressed.
+        """
+        existing = module._parameters.get('weight')
+        if existing is None or existing.shape != weight.shape:
+            for name in [n for n in list(module._parameters) if n.startswith('weight')]:
+                module._parameters.pop(name, None)
+            module.register_parameter(
+                'weight',
+                torch.nn.Parameter(torch.empty(weight.shape, device='meta', dtype=weight.dtype),
+                                   requires_grad=False),
+            )
+        try:
+            from compressed_tensors.quantization import QuantizationStatus
+        except ImportError:
+            return
+        module.quantization_status = QuantizationStatus.COMPRESSED
+
+    def _adopt_checkpoint_shape(self, param_name, value):
+        """Resize a meta placeholder whose shape disagrees with the checkpoint.
+
+        A model class builds its parameters from the config, and that construction can disagree
+        with the weights actually shipped. Kimi K3 sizes ``A_log`` from ``num_heads`` while its
+        checkpoint stores one entry per head channel, which would abort the load. For a parameter
+        still on meta -- i.e. one we have never materialised -- the checkpoint is the source of
+        truth, so adopt its shape.
+        """
+        module_path, _, attr = param_name.rpartition('.')
+        try:
+            module = self.model.get_submodule(module_path) if module_path else self.model
+        except AttributeError:
+            return
+        current = module._parameters.get(attr)
+        if current is None or current.device.type != 'meta' or current.shape == value.shape:
+            return
+
+        if not hasattr(self, '_shape_adoption_warned'):
+            self._shape_adoption_warned = set()
+        if attr not in self._shape_adoption_warned:
+            self._shape_adoption_warned.add(attr)
+            print(f"{attr}: checkpoint ships {tuple(value.shape)} but the model class builds "
+                  f"{tuple(current.shape)}; using the checkpoint shape.")
+
+        module.register_parameter(
+            attr,
+            torch.nn.Parameter(torch.empty(value.shape, device='meta', dtype=current.dtype),
+                               requires_grad=False),
+        )
+
     def move_layer_to_device(self, state_dict):
+        self._restore_plain_weight_modules(state_dict)
+        state_dict = self._decompress_state_dict(state_dict)
         moved = []
         for param_name in self._param_names_from_state_dict(state_dict):
             if self.hf_quantizer is not None and self._needs_quantization(param_name):
@@ -284,12 +594,11 @@ class AirLLMBaseModel:
                 self.hf_quantizer.create_quantized_param(self.model, state_dict[param_name], param_name,
                                                          self.running_device, state_dict)
             else:
-                # Normal load. Pre-quantized weights (fp8) and their block scales must be placed
-                # verbatim: casting an fp8 weight to fp16 silently drops the quantization and the
-                # accompanying weight_scale_inv, producing garbage. Only ordinary high-precision
-                # tensors get cast to the runtime dtype.
+                # Normal load. Only ordinary high-precision tensors get cast to the runtime dtype;
+                # pre-quantized payloads must be placed verbatim (see _should_load_verbatim).
                 value = state_dict[param_name]
-                if value.dtype in (torch.float8_e4m3fn, torch.float8_e5m2) or param_name.endswith("_scale_inv"):
+                self._adopt_checkpoint_shape(param_name, value)
+                if self._should_load_verbatim(param_name, value):
                     set_module_tensor_to_device(self.model, param_name, self.running_device, value=value)
                 else:
                     set_module_tensor_to_device(self.model, param_name, self.running_device,
@@ -297,6 +606,27 @@ class AirLLMBaseModel:
             moved.append(param_name)
         return moved
 
+    # Suffixes of the companion tensors that pre-quantized checkpoints ship alongside a weight
+    # (fp8 block scales, compressed-tensors/MXFP4 packed payloads and their scales, GPTQ indices).
+    _QUANT_COMPANION_SUFFIXES = ("_scale", "_scale_inv", "_packed", "_zero_point", "_g_idx", "_shape")
+
+    def _should_load_verbatim(self, param_name, value):
+        """Whether a checkpoint tensor must be placed on the device without a dtype cast.
+
+        Casting a pre-quantized payload to the runtime dtype destroys it: an fp8 weight loses its
+        quantization, and a 4-bit MXFP4 ``weight_packed`` tensor (stored as packed integers) becomes
+        meaningless floats. Equally important for very large models, decompressing on load would
+        multiply a layer's footprint by ~4x, which is what keeps Kimi-K3-class checkpoints from
+        fitting on a single GPU. So we keep anything that isn't a plain high-precision float as-is.
+        """
+        if not value.is_floating_point():
+            # Packed 4-bit payloads, zero points, g_idx, shape metadata.
+            return True
+        if value.element_size() == 1:
+            # Any 8-bit float: fp8 e4m3/e5m2 weights, and the e8m0 scales MXFP4 uses.
+            return True
+        return param_name.endswith(self._QUANT_COMPANION_SUFFIXES)
+
     def _needs_quantization(self, param_name):
         q = self.hf_quantizer
         # transformers renamed check_quantized_param -> param_needs_quantization.
@@ -319,6 +649,22 @@ class AirLLMBaseModel:
                 names.append(param_name)
         return names
 
+    def _load_resident_modules(self):
+        """Load modules that sit outside the streamed embed -> layers -> norm -> lm_head sequence.
+
+        Multimodal checkpoints carry a vision tower and projector, and some architectures add
+        extra top-level norms. They never get a streaming hook, so without this they would stay on
+        the meta device and fail the moment they run. They are small (well under a GB), so we load
+        them once and leave them resident.
+        """
+        for name in self.layer_names_dict.get('resident', []):
+            try:
+                state_dict = self.load_layer_to_cpu(name)
+            except FileNotFoundError:
+                # Not every checkpoint of a given architecture ships every optional module.
+                continue
+            self.move_layer_to_device(state_dict)
+
     def _install_streaming_hooks(self):
         # Modules execute in this order during a forward: embed -> layers -> norm -> lm_head.
         n = len(self.layer_names)
@@ -339,16 +685,117 @@ class AirLLMBaseModel:
 
         self._streamed_set = set(self._streamed_indices)
 
+        self._setup_expert_streaming()
+
         for idx in self._streamed_indices:
             module = self.layers[idx]
             module._airllm_idx = idx
             module.register_forward_pre_hook(self._pre_hook)
             module.register_forward_hook(self._post_hook)
 
+    # ---- per-expert streaming ---------------------------------------------------------------
+
+    def _setup_expert_streaming(self):
+        """Stream individual MoE experts instead of whole decoder layers, where that is possible.
+
+        A sparse MoE layer holds hundreds of experts but routes each token to a handful of them.
+        Materialising the whole layer is therefore enormously wasteful: for Kimi K3 a layer's
+        experts are ~55GB expanded, of which a token touches ~1GB. Because the model calls each
+        selected expert as its own module (and skips unselected ones), a forward hook per expert
+        loads exactly the experts that actually run.
+
+        This needs `expert_prefix` in layer_names_dict and safetensors shards, since it relies on
+        reading individual tensors out of a shard.
+        """
+        self._expert_streaming = False
+        self._expert_keys = {}
+        self._non_expert_keys = {}
+
+        expert_prefix = self.layer_names_dict.get('expert_prefix')
+        if not expert_prefix:
+            return
+        if type(ModelPersister.get_model_persister()).__name__ != 'SafetensorModelPersister':
+            return
+
+        layer_prefix = self.layer_names_dict['layer_prefix']
+        hooked = 0
+
+        for idx in self._streamed_indices:
+            layer_name = self.layer_names[idx]
+            if not layer_name.startswith(layer_prefix + '.'):
+                continue
+            try:
+                names = layer_tensor_names(self.checkpoint_path, layer_name)
+            except Exception:
+                continue
+
+            marker = f'.{expert_prefix}.'
+            per_expert = {}
+            others = []
+            for key in names:
+                pos = key.find(marker)
+                if pos == -1:
+                    others.append(key)
+                    continue
+                rest = key[pos + len(marker):]
+                head = rest.split('.', 1)[0]
+                if not head.isdigit():
+                    others.append(key)
+                    continue
+                per_expert.setdefault(int(head), []).append(key)
+
+            if not per_expert:
+                continue
+
+            layer_module = self.layers[idx]
+            experts_container = layer_module
+            try:
+                for attr in expert_prefix.split('.'):
+                    experts_container = getattr(experts_container, attr)
+            except AttributeError:
+                continue
+
+            self._non_expert_keys[idx] = others
+            self._expert_keys[idx] = per_expert
+
+            for expert_idx, keys in per_expert.items():
+                if expert_idx >= len(experts_container):
+                    continue
+                expert_module = experts_container[expert_idx]
+                expert_module._airllm_expert = (idx, expert_idx)
+                expert_module.register_forward_pre_hook(self._expert_pre_hook)
+                expert_module.register_forward_hook(self._expert_post_hook)
+                hooked += 1
+
+        if hooked:
+            self._expert_streaming = True
+            n_layers = len(self._expert_keys)
+            print(f"per-expert streaming enabled: {hooked} experts across {n_layers} layers "
+                  f"load on demand, so only the experts a token routes to are materialised.")
+
+    def _expert_pre_hook(self, module, args):
+        layer_idx, expert_idx = module._airllm_expert
+        keys = self._expert_keys[layer_idx][expert_idx]
+        state_dict = load_layer_subset(self.checkpoint_path, self.layer_names[layer_idx], keys)
+        module._airllm_moved = self.move_layer_to_device(state_dict)
+
+    def _expert_post_hook(self, module, args, output):
+        for param_name in getattr(module, '_airllm_moved', []):
+            set_module_tensor_to_device(self.model, param_name, 'meta')
+        module._airllm_moved = []
+        return output
+
     def _next_streamed_idx(self, idx):
         nxt = idx + 1
         return nxt if nxt in self._streamed_set else None
 
+    def _load_streamed_layer(self, idx):
+        """Load one streamed module's weights. Experts are excluded when they stream themselves."""
+        keys = self._non_expert_keys.get(idx) if getattr(self, '_expert_streaming', False) else None
+        if keys is None:
+            return self.load_layer_to_cpu(self.layer_names[idx])
+        return load_layer_subset(self.checkpoint_path, self.layer_names[idx], keys)
+
     def _pre_hook(self, module, args):
         idx = module._airllm_idx
 
@@ -356,18 +803,20 @@ class AirLLMBaseModel:
             state_dict = self._prefetch_future.result()
             self._prefetch_future = None
         else:
-            state_dict = self.load_layer_to_cpu(self.layer_names[idx])
+            state_dict = self._load_streamed_layer(idx)
 
         module._airllm_moved = self.move_layer_to_device(state_dict)
 
         if self.prefetching:
             nxt = self._next_streamed_idx(idx)
             if nxt is not None:
-                self._prefetch_future = self._executor.submit(self.load_layer_to_cpu, self.layer_names[nxt])
+                self._prefetch_future = self._executor.submit(self._load_streamed_layer, nxt)
                 self._prefetched_idx = nxt
 
     def _post_hook(self, module, args, output):
-        if self.hf_quantizer is not None:
+        # module.to('meta') would also evict the experts, which manage their own lifetime, so with
+        # expert streaming we only release exactly what this hook placed.
+        if self.hf_quantizer is not None or getattr(self, '_expert_streaming', False):
             for param_name in getattr(module, '_airllm_moved', []):
                 set_module_tensor_to_device(self.model, param_name, 'meta')
         else:

air_llm/airllm/airllm_kimi_k3.py

@@ -0,0 +1,32 @@
+from .airllm_base import AirLLMBaseModel
+
+
+class AirLLMKimiK3(AirLLMBaseModel):
+    """Kimi K3 (``KimiK3ForConditionalGeneration``).
+
+    K3 is a multimodal MoE checkpoint, so the decoder lives one level deeper than usual, under
+    ``language_model``, and the checkpoint carries a vision tower and projector alongside it.
+    It also adds a pair of top-level Attention Residual modules that sit outside the normal
+    embed -> layers -> norm -> lm_head sequence. Everything else (MXFP4 weights, per-layer
+    streaming) is handled by the generic base class.
+
+    Each layer holds 896 experts and routes a token to 16 of them, so the experts are streamed
+    individually rather than by layer: expanded, a layer's experts are ~55GB but a token needs
+    ~1GB of them.
+    """
+
+    def set_layer_names_dict(self):
+        self.layer_names_dict = {
+            'embed': 'language_model.model.embed_tokens',
+            'layer_prefix': 'language_model.model.layers',
+            'norm': 'language_model.model.norm',
+            'lm_head': 'language_model.lm_head',
+            'expert_prefix': 'block_sparse_moe.experts',
+            # Not streamed: loaded once and kept resident. Together these are well under 1GB.
+            'resident': [
+                'language_model.model.output_attn_res_norm',
+                'language_model.model.output_attn_res_proj',
+                'mm_projector',
+                'vision_tower',
+            ],
+        }

air_llm/airllm/airllm_qwen3_5.py

@@ -0,0 +1,30 @@
+from .airllm_base import AirLLMBaseModel
+
+
+class AirLLMQwen3_5(AirLLMBaseModel):
+    """Qwen3.5 / Qwen3.8 dense VL (``Qwen3_5ForConditionalGeneration``).
+
+    Qwen3.8-27B is this architecture: a native vision-language wrapper whose decoder lives under
+    ``model.language_model``, with a SigLIP-style tower at ``model.visual``. The language stack is
+    a hybrid of Gated DeltaNet (linear attention) and Gated Attention, 64 layers, bf16, ~27B.
+
+    Transformers ignores the checkpoint's ``mtp.*`` Multi-Token Prediction head
+    (``_keys_to_ignore_on_load_unexpected``), so we do not stream or load it. The vision tower is
+    kept resident: text-only ``generate()`` never runs it, but leaving it on meta would crash the
+    moment a caller passes ``pixel_values``.
+
+    Needs transformers 5.8+ (the class is in-tree; this repo ships no remote modeling code).
+    Optional CUDA kernels ``fla`` / ``causal-conv1d`` speed up DeltaNet; transformers falls back
+    to a PyTorch implementation when they are missing.
+    """
+
+    def set_layer_names_dict(self):
+        self.layer_names_dict = {
+            'embed': 'model.language_model.embed_tokens',
+            'layer_prefix': 'model.language_model.layers',
+            'norm': 'model.language_model.norm',
+            'lm_head': 'lm_head',
+            'resident': [
+                'model.visual',
+            ],
+        }

air_llm/airllm/auto_model.py

@@ -21,6 +21,8 @@ ARCH_OVERRIDES = {
     "BaichuanForCausalLM": "AirLLMBaichuan",
     "BaiChuanForCausalLM": "AirLLMBaichuan",
     "InternLMForCausalLM": "AirLLMInternLM",
+    "KimiK3ForConditionalGeneration": "AirLLMKimiK3",
+    "Qwen3_5ForConditionalGeneration": "AirLLMQwen3_5",
 }
 
 

air_llm/airllm/utils.py

@@ -20,6 +20,7 @@ if platform == "darwin":
 
 import torch
 import torch.nn as nn
+from safetensors import safe_open
 from safetensors.torch import load_file, save_file
 
 from .persist import ModelPersister
@@ -112,6 +113,25 @@ def uncompress_layer_state_dict(layer_state_dict):
 
     return layer_state_dict if uncompressed_layer_state_dict is None else uncompressed_layer_state_dict
 
+def layer_tensor_names(local_path, layer_name):
+    """List the tensors in a layer shard without reading any tensor data."""
+    with safe_open(str(Path(local_path) / (layer_name + ".safetensors")), framework="pt") as f:
+        return list(f.keys())
+
+
+def load_layer_subset(local_path, layer_name, keys):
+    """Read only `keys` from a layer shard.
+
+    safetensors can seek to individual tensors, so a single MoE expert costs its own few MB rather
+    than the whole ~16GB layer file. That is what makes per-expert streaming worthwhile.
+    """
+    out = {}
+    with safe_open(str(Path(local_path) / (layer_name + ".safetensors")), framework="pt") as f:
+        for k in keys:
+            out[k] = f.get_tensor(k)
+    return out
+
+
 def load_layer(local_path, layer_name, profiling=False):
     #layer_state_dict = load_file(Path(local_path) / (layer_name + ".safetensors"), device="cpu")
     layer_state_dict = ModelPersister.get_model_persister().load_model(layer_name, local_path)
@@ -185,6 +205,33 @@ def remove_real_and_linked_file(to_delete):
 
 
 
+def link_or_copy_file(src, dst):
+    """Point dst at src's data without duplicating it, falling back to a real copy.
+
+    A hard link is preferred over a symlink because it keeps the data alive even if the original
+    checkpoint file is later deleted (``delete_original``), and because it costs no extra disk.
+    Hard links need both paths on one filesystem, so we degrade to a symlink and finally to a copy.
+    Hugging Face caches store files as symlinks into a blob dir, so we always link the real file.
+    """
+    src = Path(os.path.realpath(str(src)))
+    dst = Path(dst)
+    if dst.exists() or dst.is_symlink():
+        dst.unlink()
+
+    try:
+        os.link(src, dst)
+        return 'hardlink'
+    except OSError:
+        pass
+    try:
+        os.symlink(src, dst)
+        return 'symlink'
+    except OSError:
+        pass
+    shutil.copyfile(src, dst)
+    return 'copy'
+
+
 def split_and_save_layers(checkpoint_path, layer_shards_saving_path=None, splitted_model_dir_name='splitted_model',
                           compression=None, layer_names=None, delete_original=False, repo_id=None, hf_token=None):
     """
@@ -243,6 +290,10 @@ def split_and_save_layers(checkpoint_path, layer_shards_saving_path=None, splitt
 
         if 'rotary_pos_emb' in layer_names:
             layers = [layer_names['rotary_pos_emb']] + layers
+        # Modules that are not part of the streamed sequence but still need their weights on disk,
+        # e.g. a multimodal model's vision tower / projector, or extra top-level norms. They get
+        # their own shard and are loaded once and kept resident.
+        layers = layers + list(layer_names.get('resident', []))
         layers = [l + "." for l in layers]
 
     # Drop layers that have no weights in the checkpoint. This happens for tied embeddings,
@@ -250,6 +301,18 @@ def split_and_save_layers(checkpoint_path, layer_shards_saving_path=None, splitt
     # would try to save an empty shard (which fails) and never detect the split as complete.
     layers = [l for l in layers if any(k.startswith(l) for k in index.keys())]
 
+    # Split in ascending shard order. The loop below only ever walks the shard counter forward, so
+    # a module whose weights sit in an earlier shard than its predecessor's would silently be saved
+    # incomplete. That ordering isn't guaranteed once non-sequential modules (a vision tower, extra
+    # norms) are in the list, so sort by the last shard each module touches. This is a stable sort,
+    # so plain embed -> layers -> norm -> lm_head checkpoints keep their existing order.
+    def _last_shard_of(layer):
+        nums = [int(v.split('-')[1]) for k, v in index.items()
+                if k.startswith(layer) and '-' in v and len(v.split('-')) > 1]
+        return max(nums) if nums else -1
+
+    layers.sort(key=_last_shard_of)
+
 
     # check if splitting exists and all files are there
     found_layers = None
@@ -269,7 +332,36 @@ def split_and_save_layers(checkpoint_path, layer_shards_saving_path=None, splitt
         else:
             print(f"some layer splits found, some are not, re-save all layers in case there's some corruptions.")
 
-    if not delete_original:
+    # Some checkpoints are already sharded exactly one module per file (Kimi K3, for instance, ships
+    # one ~17GB shard per decoder layer). Re-writing those into per-layer files would duplicate the
+    # entire checkpoint on disk -- 1.5TB+ for a 2.8T-parameter model -- and take hours, to produce
+    # byte-identical content. When a shard holds nothing but one module's tensors we link to it
+    # instead of copying.
+    passthrough = {}
+    # Linking only produces a file the loader can read when shards are stored in the same format
+    # the persister writes; the MLX persister, for instance, writes .mlx.npz.
+    persister_is_safetensors = type(ModelPersister.get_model_persister()).__name__ == 'SafetensorModelPersister'
+    if compression is None and safetensors_format and persister_is_safetensors:
+        shard_contents = defaultdict(list)
+        for k, v in index.items():
+            shard_contents[v].append(k)
+        for layer in layers:
+            files = {v for k, v in index.items() if k.startswith(layer)}
+            if len(files) != 1:
+                continue
+            only_file = next(iter(files))
+            if all(k.startswith(layer) for k in shard_contents[only_file]):
+                passthrough[layer] = only_file
+
+    if passthrough:
+        print(f"{len(passthrough)}/{len(layers)} modules are already one-per-shard; "
+              f"linking to the original files instead of copying them.")
+
+    # Must exist before check_space, which stats the filesystem it lives on.
+    saving_path.mkdir(parents=True, exist_ok=True)
+
+    # A copy is only made for the layers we cannot link, so only those need free space.
+    if not delete_original and len(passthrough) < len(layers):
         check_space(checkpoint_path, layer_shards_saving_path, compression, splitted_model_dir_name=splitted_model_dir_name)
 
 
@@ -289,14 +381,29 @@ def split_and_save_layers(checkpoint_path, layer_shards_saving_path=None, splitt
             except ValueError:
                 pass
 
-    if not os.path.exists(saving_path):
-        #os.makedirs(saving_path)
-        saving_path.mkdir(parents=True, exist_ok=True)
-
     single_modelfile = None
 
     for layer in tqdm(layers):
 
+        if layer in passthrough:
+            src = checkpoint_path / passthrough[layer]
+            if not os.path.exists(src):
+                assert repo_id is not None
+                huggingface_hub.snapshot_download(repo_id, allow_patterns=os.path.basename(src),
+                                                  token=hf_token)
+            if not ModelPersister.get_model_persister().model_persist_exist(layer, saving_path):
+                link_or_copy_file(src, saving_path / (layer + 'safetensors'))
+                (saving_path / (layer + 'safetensors.done')).touch()
+            # Keep the shard cursor in step with what we skipped, so a later layer that does need
+            # loading doesn't walk back through (and read) every shard we just linked past.
+            src_parts = passthrough[layer].split('-')
+            if len(src_parts) > 1:
+                try:
+                    shard = max(shard, int(src_parts[1]))
+                except ValueError:
+                    pass
+            continue
+
         # Optionnally load next shard
         # checking whether after spliting from '-', if second element exists. otherwise it throws errors for single 'model.safetensor' files
         shards = [int(v.split('-')[1]) for k, v in index.items() if k.startswith(layer) and '-' in v and len(v.split('-')) > 1]
@@ -395,9 +502,13 @@ def find_or_create_local_splitted_path(model_local_path_or_repo_id, layer_shards
 
     # try local model path, if the model exist split and save there
     if os.path.exists(model_local_path_or_repo_id):
-        if os.path.exists(Path(model_local_path_or_repo_id) / 'pytorch_model.bin.index.json') or \
-           os.path.exists(Path(model_local_path_or_repo_id) / 'model.safetensors.index.json'):
-            print(f"found index file...")
+        # Accept single-file checkpoints too, not just sharded ones with an index: the splitter
+        # handles both, so requiring an index needlessly sent local single-file models down the
+        # "treat it as a repo id" path, where they fail as an invalid repo name.
+        local_weight_files = ('pytorch_model.bin.index.json', 'model.safetensors.index.json',
+                              'model.safetensors', 'pytorch_model.bin')
+        if any(os.path.exists(Path(model_local_path_or_repo_id) / f) for f in local_weight_files):
+            print(f"found local checkpoint...")
             return Path(model_local_path_or_repo_id), split_and_save_layers(model_local_path_or_repo_id, layer_shards_saving_path,
                                                                             compression=compression, layer_names=layer_names, delete_original=delete_original)
         else:

air_llm/setup.py

@@ -15,11 +15,12 @@ for _readme in (os.path.join(here, "README.md"), os.path.join(here, os.pardir, "
 
 setuptools.setup(
     name="airllm",
-    version="3.0.1",
+    version="3.2.0",
     author="Gavin Li",
     author_email="gavinli@animaai.cloud",
     description="AirLLM runs 70B large language models on a single 4GB GPU without quantization, "
-                "distillation or pruning. 405B Llama 3.1 on 8GB, DeepSeek-V3 671B on ~12GB.",
+                "distillation or pruning. 405B Llama 3.1 on 8GB, DeepSeek-V3 671B on ~12GB, "
+                "Kimi K3 2.8T on under 4GB, Qwen3.8-27B on 3.3GB.",
     long_description=long_description,
     long_description_content_type="text/markdown",
     url="https://github.com/lyogavin/airllm",
@@ -36,6 +37,8 @@ setuptools.setup(
         'scipy',
         'sentencepiece',
         # 'bitsandbytes' is optional (used only for --compression); we fall back gracefully when absent.
+        # 'compressed-tensors' is optional too: only checkpoints stored in that format (Kimi K3's
+        # MXFP4 weights) need it, and transformers raises a clear error naming it when it is missing.
     ],
     classifiers=[
         "Programming Language :: Python :: 3",

air_llm/tests/test_kimi_k3_split.py

@@ -0,0 +1,224 @@
+"""Structural tests for Kimi-K3-style checkpoints.
+
+K3 is multimodal (decoder nested under ``language_model``), ships one shard per decoder layer, and
+carries modules outside the streamed sequence (vision tower, projector, extra residual norms).
+These tests build a miniature checkpoint with the same shape and check that the splitter links
+rather than copies the one-module-per-shard files, still materialises the shared shard correctly,
+and never loses a module because of shard ordering.
+"""
+
+import json
+import os
+import sys
+import tempfile
+import types
+import unittest
+from pathlib import Path
+
+import torch
+from safetensors.torch import save_file, load_file
+
+_AIRLLM_DIR = Path(__file__).resolve().parents[1] / 'airllm'
+
+# Bind the package without running airllm/__init__.py: that module pulls in the MLX backend on
+# macOS and the full transformers stack elsewhere, none of which these splitter tests need.
+if 'airllm' not in sys.modules:
+    _pkg = types.ModuleType('airllm')
+    _pkg.__path__ = [str(_AIRLLM_DIR)]
+    sys.modules['airllm'] = _pkg
+
+from airllm.persist import model_persister as _persister_mod
+from airllm.persist.safetensor_model_persister import SafetensorModelPersister
+
+# The persister is otherwise chosen by platform, so pin the safetensors one to exercise the
+# Linux/CUDA path regardless of where the test runs.
+_persister_mod.model_persister = SafetensorModelPersister()
+
+from airllm.utils import split_and_save_layers
+
+N_LAYERS = 3
+LAYER_PREFIX = "language_model.model.layers"
+
+KIMI_LAYER_NAMES = {
+    'embed': 'language_model.model.embed_tokens',
+    'layer_prefix': LAYER_PREFIX,
+    'norm': 'language_model.model.norm',
+    'lm_head': 'language_model.lm_head',
+    'resident': [
+        'language_model.model.output_attn_res_norm',
+        'language_model.model.output_attn_res_proj',
+        'mm_projector',
+        'vision_tower',
+    ],
+}
+
+
+def build_fake_checkpoint(root):
+    """One shard per layer, then a shared shard, then projector and vision tower."""
+    n_shards = N_LAYERS + 3
+    shards = {}
+
+    for i in range(N_LAYERS):
+        name = f"model-{i + 1:05d}-of-{n_shards:06d}.safetensors"
+        shards[name] = {
+            # A packed 4-bit payload plus its scale, like MXFP4 stores.
+            f"{LAYER_PREFIX}.{i}.block_sparse_moe.experts.0.w1.weight_packed":
+                torch.randint(0, 255, (8, 4), dtype=torch.uint8),
+            f"{LAYER_PREFIX}.{i}.block_sparse_moe.experts.0.w1.weight_scale":
+                torch.randn(8, 1, dtype=torch.float32),
+            f"{LAYER_PREFIX}.{i}.input_layernorm.weight": torch.randn(8),
+        }
+
+    shared = f"model-{N_LAYERS + 1:05d}-of-{n_shards:06d}.safetensors"
+    shards[shared] = {
+        "language_model.model.embed_tokens.weight": torch.randn(16, 8),
+        "language_model.model.norm.weight": torch.randn(8),
+        "language_model.lm_head.weight": torch.randn(16, 8),
+        "language_model.model.output_attn_res_norm.weight": torch.randn(8),
+        "language_model.model.output_attn_res_proj.weight": torch.randn(8, 8),
+    }
+    # Deliberately out of order relative to the resident list: the projector lives in an *earlier*
+    # shard than the vision tower, which the splitter must handle without losing tensors.
+    proj = f"model-{N_LAYERS + 2:05d}-of-{n_shards:06d}.safetensors"
+    shards[proj] = {"mm_projector.proj.0.weight": torch.randn(8, 8)}
+    vis = f"model-{N_LAYERS + 3:05d}-of-{n_shards:06d}.safetensors"
+    shards[vis] = {"vision_tower.encoder.blocks.0.mlp.fc0.weight": torch.randn(8, 8)}
+
+    weight_map = {}
+    for fname, sd in shards.items():
+        save_file(sd, str(Path(root) / fname))
+        for k in sd:
+            weight_map[k] = fname
+    with open(Path(root) / "model.safetensors.index.json", "w") as f:
+        json.dump({"metadata": {"total_size": 1}, "weight_map": weight_map}, f)
+    return shards, weight_map
+
+
+class TestKimiK3Split(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.root = Path(self.tmp.name)
+        self.shards, self.weight_map = build_fake_checkpoint(self.root)
+        self.out = Path(split_and_save_layers(self.root, layer_names=KIMI_LAYER_NAMES))
+
+    def tearDown(self):
+        self.tmp.cleanup()
+
+    def _split_file(self, module):
+        return self.out / f"{module}.safetensors"
+
+    def test_every_module_is_split_out(self):
+        expected = ([KIMI_LAYER_NAMES['embed']]
+                    + [f"{LAYER_PREFIX}.{i}" for i in range(N_LAYERS)]
+                    + [KIMI_LAYER_NAMES['norm'], KIMI_LAYER_NAMES['lm_head']]
+                    + KIMI_LAYER_NAMES['resident'])
+        for module in expected:
+            self.assertTrue(self._split_file(module).exists(), f"missing split for {module}")
+
+    def test_one_shard_per_layer_is_linked_not_copied(self):
+        """The whole point: a 1.5TB checkpoint must not be duplicated."""
+        for i in range(N_LAYERS):
+            src = self.root / f"model-{i + 1:05d}-of-{N_LAYERS + 3:06d}.safetensors"
+            dst = self._split_file(f"{LAYER_PREFIX}.{i}")
+            self.assertEqual(os.stat(src).st_ino, os.stat(dst).st_ino,
+                             f"layer {i} was copied instead of linked")
+
+    def test_projector_and_vision_tower_are_linked(self):
+        for module, shard_no in (('mm_projector', N_LAYERS + 2), ('vision_tower', N_LAYERS + 3)):
+            src = self.root / f"model-{shard_no:05d}-of-{N_LAYERS + 3:06d}.safetensors"
+            self.assertEqual(os.stat(src).st_ino, os.stat(self._split_file(module)).st_ino,
+                             f"{module} was copied instead of linked")
+
+    def test_shared_shard_modules_are_materialised_separately(self):
+        """embed / norm / lm_head / residual modules share one shard, so they must be real files."""
+        shared_src = self.root / f"model-{N_LAYERS + 1:05d}-of-{N_LAYERS + 3:06d}.safetensors"
+        for module in ('language_model.model.embed_tokens', 'language_model.model.norm',
+                       'language_model.lm_head', 'language_model.model.output_attn_res_norm',
+                       'language_model.model.output_attn_res_proj'):
+            dst = self._split_file(module)
+            self.assertNotEqual(os.stat(shared_src).st_ino, os.stat(dst).st_ino)
+            keys = set(load_file(str(dst)).keys())
+            self.assertTrue(keys, f"{module} split is empty")
+            self.assertTrue(all(k.startswith(module + '.') for k in keys),
+                            f"{module} split leaked other tensors: {keys}")
+
+    def test_contents_match_the_original_checkpoint(self):
+        """Linked or copied, every tensor must survive the split bit-for-bit."""
+        original = {}
+        for fname in self.shards:
+            original.update(load_file(str(self.root / fname)))
+
+        recovered = {}
+        for f in self.out.glob('*.safetensors'):
+            recovered.update(load_file(str(f)))
+
+        self.assertEqual(set(original.keys()), set(recovered.keys()))
+        for k, v in original.items():
+            self.assertEqual(v.dtype, recovered[k].dtype, f"{k} changed dtype")
+            self.assertTrue(torch.equal(v, recovered[k]), f"{k} changed value")
+
+    def test_packed_4bit_payload_keeps_its_integer_dtype(self):
+        sd = load_file(str(self._split_file(f"{LAYER_PREFIX}.0")))
+        packed = [v for k, v in sd.items() if k.endswith('weight_packed')][0]
+        self.assertEqual(packed.dtype, torch.uint8)
+
+
+class TestStandardCheckpointStillSplits(unittest.TestCase):
+    """Regression guard: the ordinary layout (several layers per shard) must be unaffected.
+
+    Passthrough must not kick in here, because no shard holds exactly one module.
+    """
+
+    N = 4
+
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.root = Path(self.tmp.name)
+
+        shard_a = {"model.embed_tokens.weight": torch.randn(16, 8)}
+        for i in range(2):
+            shard_a[f"model.layers.{i}.self_attn.q_proj.weight"] = torch.randn(8, 8)
+        shard_b = {}
+        for i in range(2, self.N):
+            shard_b[f"model.layers.{i}.self_attn.q_proj.weight"] = torch.randn(8, 8)
+        shard_b["model.norm.weight"] = torch.randn(8)
+        shard_b["lm_head.weight"] = torch.randn(16, 8)
+
+        weight_map = {}
+        for fname, sd in (("model-00001-of-00002.safetensors", shard_a),
+                          ("model-00002-of-00002.safetensors", shard_b)):
+            save_file(sd, str(self.root / fname))
+            for k in sd:
+                weight_map[k] = fname
+        with open(self.root / "model.safetensors.index.json", "w") as f:
+            json.dump({"metadata": {"total_size": 1}, "weight_map": weight_map}, f)
+
+        self.original = dict(shard_a)
+        self.original.update(shard_b)
+        self.out = Path(split_and_save_layers(self.root))
+
+    def tearDown(self):
+        self.tmp.cleanup()
+
+    def test_all_modules_written_as_real_files(self):
+        expected = (['model.embed_tokens'] + [f'model.layers.{i}' for i in range(self.N)]
+                    + ['model.norm', 'lm_head'])
+        src_inodes = {os.stat(self.root / f).st_ino
+                      for f in os.listdir(self.root) if f.endswith('.safetensors')}
+        for module in expected:
+            dst = self.out / f"{module}.safetensors"
+            self.assertTrue(dst.exists(), f"missing split for {module}")
+            self.assertNotIn(os.stat(dst).st_ino, src_inodes,
+                             f"{module} was linked, but this layout should be copied")
+
+    def test_contents_round_trip(self):
+        recovered = {}
+        for f in self.out.glob('*.safetensors'):
+            recovered.update(load_file(str(f)))
+        self.assertEqual(set(self.original.keys()), set(recovered.keys()))
+        for k, v in self.original.items():
+            self.assertTrue(torch.equal(v, recovered[k]), f"{k} changed value")
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

air_llm/tests/test_qwen3_8_split.py

@@ -0,0 +1,166 @@
+"""Structural tests for Qwen3.5 / Qwen3.8 VL checkpoints.
+
+Qwen3.8-27B is ``Qwen3_5ForConditionalGeneration``: the decoder is nested under
+``model.language_model``, the vision tower is ``model.visual``, and the checkpoint also carries an
+``mtp.*`` Multi-Token Prediction head that transformers ignores. These tests build a miniature
+checkpoint with that shape (several layers per shard, like the real 18-file dump) and check that
+the splitter writes every streamed module, keeps the vision tower, and drops MTP.
+"""
+
+import json
+import os
+import sys
+import tempfile
+import types
+import unittest
+from pathlib import Path
+
+import torch
+from safetensors.torch import save_file, load_file
+
+_AIRLLM_DIR = Path(__file__).resolve().parents[1] / 'airllm'
+
+if 'airllm' not in sys.modules:
+    _pkg = types.ModuleType('airllm')
+    _pkg.__path__ = [str(_AIRLLM_DIR)]
+    sys.modules['airllm'] = _pkg
+
+from airllm.persist import model_persister as _persister_mod
+from airllm.persist.safetensor_model_persister import SafetensorModelPersister
+
+_persister_mod.model_persister = SafetensorModelPersister()
+
+from airllm.utils import split_and_save_layers
+
+N_LAYERS = 4
+LAYER_PREFIX = "model.language_model.layers"
+
+QWEN38_LAYER_NAMES = {
+    'embed': 'model.language_model.embed_tokens',
+    'layer_prefix': LAYER_PREFIX,
+    'norm': 'model.language_model.norm',
+    'lm_head': 'lm_head',
+    'resident': [
+        'model.visual',
+    ],
+}
+
+
+def build_fake_checkpoint(root):
+    """Several decoder layers per shard, plus a vision tower mixed into shard 0 and MTP in the last."""
+    shards = {}
+
+    shard0 = {
+        "model.visual.patch_embed.proj.weight": torch.randn(8, 3, 2, 2),
+        "model.visual.blocks.0.attn.qkv.weight": torch.randn(8, 8),
+        "model.visual.merger.linear_fc1.weight": torch.randn(8, 8),
+        f"{LAYER_PREFIX}.0.input_layernorm.weight": torch.randn(8),
+        f"{LAYER_PREFIX}.0.linear_attn.A_log": torch.randn(4),
+        f"{LAYER_PREFIX}.0.linear_attn.in_proj_a.weight": torch.randn(4, 8),
+        f"{LAYER_PREFIX}.0.mlp.gate_proj.weight": torch.randn(8, 8),
+        f"{LAYER_PREFIX}.1.input_layernorm.weight": torch.randn(8),
+        f"{LAYER_PREFIX}.1.self_attn.q_proj.weight": torch.randn(8, 8),
+        f"{LAYER_PREFIX}.1.mlp.down_proj.weight": torch.randn(8, 8),
+    }
+    shards["model-00001-of-00003.safetensors"] = shard0
+
+    shard1 = {
+        "model.language_model.embed_tokens.weight": torch.randn(16, 8),
+        f"{LAYER_PREFIX}.2.input_layernorm.weight": torch.randn(8),
+        f"{LAYER_PREFIX}.2.linear_attn.out_proj.weight": torch.randn(8, 8),
+        f"{LAYER_PREFIX}.3.input_layernorm.weight": torch.randn(8),
+        f"{LAYER_PREFIX}.3.self_attn.o_proj.weight": torch.randn(8, 8),
+        "model.language_model.norm.weight": torch.randn(8),
+    }
+    shards["model-00002-of-00003.safetensors"] = shard1
+
+    shard2 = {
+        "lm_head.weight": torch.randn(16, 8),
+        "mtp.fc.weight": torch.randn(8, 8),
+        "mtp.layers.0.mlp.gate_proj.weight": torch.randn(8, 8),
+        "mtp.norm.weight": torch.randn(8),
+    }
+    shards["model-00003-of-00003.safetensors"] = shard2
+
+    weight_map = {}
+    for fname, sd in shards.items():
+        save_file(sd, str(Path(root) / fname))
+        for k in sd:
+            weight_map[k] = fname
+    with open(Path(root) / "model.safetensors.index.json", "w") as f:
+        json.dump({"metadata": {"total_size": 1}, "weight_map": weight_map}, f)
+    return shards, weight_map
+
+
+class TestQwen38Split(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.root = Path(self.tmp.name)
+        self.shards, self.weight_map = build_fake_checkpoint(self.root)
+        self.out = Path(split_and_save_layers(self.root, layer_names=QWEN38_LAYER_NAMES))
+
+    def tearDown(self):
+        self.tmp.cleanup()
+
+    def _split_file(self, module):
+        return self.out / f"{module}.safetensors"
+
+    def test_architecture_maps_to_the_qwen3_5_subclass(self):
+        src = (_AIRLLM_DIR / 'auto_model.py').read_text()
+        self.assertIn('"Qwen3_5ForConditionalGeneration": "AirLLMQwen3_5"', src)
+        init_src = (_AIRLLM_DIR / '__init__.py').read_text()
+        self.assertIn('AirLLMQwen3_5', init_src)
+
+    def test_every_streamed_and_resident_module_is_split_out(self):
+        expected = ([QWEN38_LAYER_NAMES['embed']]
+                    + [f"{LAYER_PREFIX}.{i}" for i in range(N_LAYERS)]
+                    + [QWEN38_LAYER_NAMES['norm'], QWEN38_LAYER_NAMES['lm_head']]
+                    + QWEN38_LAYER_NAMES['resident'])
+        for module in expected:
+            self.assertTrue(self._split_file(module).exists(), f"missing split for {module}")
+
+    def test_vision_tower_is_one_shard_and_excludes_decoder_weights(self):
+        sd = load_file(str(self._split_file("model.visual")))
+        self.assertTrue(sd, "vision split is empty")
+        self.assertTrue(all(k.startswith("model.visual.") for k in sd),
+                        f"vision split leaked decoder tensors: {set(sd)}")
+        self.assertIn("model.visual.patch_embed.proj.weight", sd)
+        self.assertIn("model.visual.blocks.0.attn.qkv.weight", sd)
+
+    def test_mtp_is_dropped(self):
+        """transformers ignores mtp.*; keeping it would waste a resident slot for a dead module."""
+        recovered = {}
+        for f in self.out.glob('*.safetensors'):
+            recovered.update(load_file(str(f)))
+        self.assertFalse(any(k.startswith("mtp.") for k in recovered),
+                         f"MTP leaked into the split: {[k for k in recovered if k.startswith('mtp.')]}")
+        self.assertFalse(any(p.name.startswith("mtp") for p in self.out.glob('*.safetensors')))
+
+    def test_decoder_and_head_round_trip(self):
+        original = {}
+        for fname in self.shards:
+            original.update(load_file(str(self.root / fname)))
+        keep = {k: v for k, v in original.items() if not k.startswith("mtp.")}
+
+        recovered = {}
+        for f in self.out.glob('*.safetensors'):
+            recovered.update(load_file(str(f)))
+
+        self.assertEqual(set(keep.keys()), set(recovered.keys()))
+        for k, v in keep.items():
+            self.assertEqual(v.dtype, recovered[k].dtype, f"{k} changed dtype")
+            self.assertTrue(torch.equal(v, recovered[k]), f"{k} changed value")
+
+    def test_shared_shards_are_copied_not_linked(self):
+        """The real 27B dump packs several layers per file, so passthrough must not fire."""
+        src_inodes = {os.stat(self.root / f).st_ino
+                      for f in os.listdir(self.root) if f.endswith('.safetensors')}
+        for module in ([f"{LAYER_PREFIX}.{i}" for i in range(N_LAYERS)]
+                       + ['model.language_model.embed_tokens', 'model.visual', 'lm_head']):
+            dst = self._split_file(module)
+            self.assertNotIn(os.stat(dst).st_ino, src_inodes,
+                             f"{module} was linked, but this layout should be copied")
+
+
+if __name__ == '__main__':
+    unittest.main(verbosity=2)

scripts/gen_star_history.py

@@ -7,21 +7,32 @@ so it works reliably for repos with tens of thousands of stars. Output is a PNG,
 always renders in GitHub markdown.
 
 Usage:
-    GITHUB_TOKEN=... python scripts/gen_star_history.py [owner/repo] [output.png] [theme]
+    GITHUB_TOKEN=... python scripts/gen_star_history.py [owner/repo] [output.png theme]...
 
-theme is "light" (default) or "dark".
+theme is "light" (default) or "dark". Pass several output/theme pairs to render them all
+from a single pass over the API.
 """
 import datetime
 import json
-import math
 import os
 import sys
 import urllib.error
 import urllib.request
 
+GRAPHQL_URL = "https://api.github.com/graphql"
+
 REPO = sys.argv[1] if len(sys.argv) > 1 else os.environ.get("REPO", "lyogavin/airllm")
-OUT = sys.argv[2] if len(sys.argv) > 2 else "assets/star-history.png"
-THEME = (sys.argv[3] if len(sys.argv) > 3 else os.environ.get("THEME", "light")).lower()
+
+# Remaining args are output/theme pairs. Walking the stargazer connection is by far the
+# slowest part of this script, so rendering every theme from one walk beats invoking the
+# script once per theme.
+_rest = sys.argv[2:]
+if _rest:
+    TARGETS = [(_rest[i], _rest[i + 1].lower() if i + 1 < len(_rest) else "light")
+               for i in range(0, len(_rest), 2)]
+else:
+    TARGETS = [("assets/star-history.png", os.environ.get("THEME", "light").lower())]
+
 TOKEN = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
 PER_PAGE = 100
 MAX_SAMPLES = 30
@@ -47,48 +58,115 @@ def gh(url, accept="application/vnd.github+json"):
         except json.JSONDecodeError:
             msg = body
         hint = ""
-        if e.code == 403 and "stargazers" in url:
+        if e.code == 401:
             hint = (
-                "\nGitHub restricts /stargazers to admins/collaborators; the "
-                "Actions GITHUB_TOKEN cannot access it. Use a collaborator PAT "
-                "via the STAR_HISTORY_TOKEN secret."
+                "\nThe token was rejected outright, which means it is expired or malformed "
+                "rather than under-permissioned. Issue a new one and re-run: "
+                "gh secret set STAR_HISTORY_TOKEN"
+            )
+        elif e.code == 403 and "personal access token" in msg.lower():
+            # A fine-grained PAT reaching an endpoint its permissions do not cover. This is
+            # distinct from a missing token, and reads identically in logs unless called out:
+            # fine-grained grants have never been enough for /stargazers.
+            hint = (
+                "\nA fine-grained PAT cannot read /stargazers. Use a *classic* token with the "
+                "public_repo scope (https://github.com/settings/tokens) and re-run: "
+                "gh secret set STAR_HISTORY_TOKEN"
+            )
+        elif e.code == 403 and "stargazers" in url:
+            hint = (
+                "\nGitHub restricts /stargazers to admins/collaborators. The token in "
+                "STAR_HISTORY_TOKEN must belong to one, and must be a classic token with "
+                "the public_repo scope."
             )
         raise SystemExit(f"GitHub API {e.code} for {url}: {msg}{hint}") from e
 
 
+def gql(query, variables):
+    payload = json.dumps({"query": query, "variables": variables}).encode()
+    headers = {
+        "Accept": "application/vnd.github+json",
+        "Content-Type": "application/json",
+        "User-Agent": "airllm-star-history",
+    }
+    if TOKEN:
+        headers["Authorization"] = f"Bearer {TOKEN}"
+    req = urllib.request.Request(GRAPHQL_URL, data=payload, headers=headers)
+    try:
+        with urllib.request.urlopen(req, timeout=30) as r:
+            body = json.load(r)
+    except urllib.error.HTTPError as e:
+        detail = e.read().decode("utf-8", errors="replace")
+        extra = "" if TOKEN else "\nNo token found in GITHUB_TOKEN or GH_TOKEN."
+        raise SystemExit(f"GitHub GraphQL {e.code}: {detail}{extra}") from e
+    if body.get("errors"):
+        raise SystemExit(f"GitHub GraphQL error: {json.dumps(body['errors'])}")
+    return body["data"]
+
+
+STARGAZER_QUERY = """
+query($owner:String!, $name:String!, $after:String) {
+  repository(owner:$owner, name:$name) {
+    stargazerCount
+    stargazers(first:%d, after:$after, orderBy:{field:STARRED_AT, direction:ASC}) {
+      pageInfo { hasNextPage endCursor }
+      edges { starredAt }
+    }
+  }
+}
+""" % PER_PAGE
+
+
+def fetch_starred_at():
+    """Every stargazer timestamp, oldest first, plus the repo's current star count.
+
+    Uses GraphQL rather than REST. GitHub restricted REST /stargazers to admins and
+    collaborators in 2026, and then tightened it again so that fine-grained tokens are
+    refused outright -- the same chart broke twice on token permissions. GraphQL serves
+    the identical public data without that gate.
+
+    Its cursors encode a timestamp and user id rather than an offset, so unlike REST we
+    cannot jump to sampled pages and have to walk the whole connection. That is a few
+    hundred requests for a repo this size, which is fine for a once-a-day job.
+    """
+    owner, _, name = REPO.partition("/")
+    stamps, cursor, total = [], None, 0
+    while True:
+        data = gql(STARGAZER_QUERY, {"owner": owner, "name": name, "after": cursor})
+        repo = data.get("repository")
+        if repo is None:
+            raise SystemExit(f"repo {REPO} not found, or the token cannot see it")
+        total = int(repo["stargazerCount"])
+        conn = repo["stargazers"]
+        stamps.extend(e["starredAt"] for e in conn["edges"])
+        if not conn["pageInfo"]["hasNextPage"]:
+            return stamps, total
+        cursor = conn["pageInfo"]["endCursor"]
+
+
 def main():
-    info = gh(f"https://api.github.com/repos/{REPO}")
-    total = int(info["stargazers_count"])
-    if total <= 0:
+    stamps, total = fetch_starred_at()
+    if total <= 0 or not stamps:
         raise SystemExit("repo has no stars")
 
-    max_page = max(1, math.ceil(total / PER_PAGE))
-    if max_page == 1:
-        pages = [1]
-    else:
-        n = min(MAX_SAMPLES, max_page)
-        pages = sorted({1, max_page} | {
-            round(1 + i * (max_page - 1) / (n - 1)) for i in range(n)
-        })
-
+    # Thin the full history down to a readable number of vertices. Walking every
+    # stargazer would plot tens of thousands of points on top of each other.
+    step = max(1, len(stamps) // MAX_SAMPLES)
     points = []
-    for p in pages:
-        data = gh(
-            f"https://api.github.com/repos/{REPO}/stargazers?per_page={PER_PAGE}&page={p}",
-            accept="application/vnd.github.star+json",
-        )
-        if not data:
-            continue
-        starred_at = data[0]["starred_at"]
-        cumulative = (p - 1) * PER_PAGE + 1
-        dt = datetime.datetime.fromisoformat(starred_at.replace("Z", "+00:00"))
-        points.append((dt, cumulative))
+    for i in range(0, len(stamps), step):
+        dt = datetime.datetime.fromisoformat(stamps[i].replace("Z", "+00:00"))
+        points.append((dt, i + 1))
 
     points.append((datetime.datetime.now(datetime.timezone.utc), total))
     points = sorted(set(points))
     if len(points) < 2:
         raise SystemExit("not enough data points to plot")
 
+    for out, theme in TARGETS:
+        render(points, total, out, theme)
+
+
+def render(points, total, out, theme):
     import matplotlib
     matplotlib.use("Agg")
     import matplotlib.dates as mdates
@@ -97,7 +175,7 @@ def main():
     xs = [d for d, _ in points]
     ys = [c for _, c in points]
 
-    c = THEMES.get(THEME, THEMES["light"])
+    c = THEMES.get(theme, THEMES["light"])
     fig, ax = plt.subplots(figsize=(10, 6))
     fig.patch.set_facecolor(c["bg"])
     ax.set_facecolor(c["bg"])
@@ -115,9 +193,10 @@ def main():
     fig.autofmt_xdate()
     fig.tight_layout()
 
-    os.makedirs(os.path.dirname(OUT) or ".", exist_ok=True)
-    fig.savefig(OUT, dpi=130, facecolor=c["bg"])
-    print(f"wrote {OUT} ({THEME}): {len(points)} points, {total} stars")
+    os.makedirs(os.path.dirname(out) or ".", exist_ok=True)
+    fig.savefig(out, dpi=130, facecolor=c["bg"])
+    plt.close(fig)
+    print(f"wrote {out} ({theme}): {len(points)} points, {total} stars")
 
 
 if __name__ == "__main__":