数据集 / 0xSero/GLM-4.7-185B-W4A16

0xSero/GLM-4.7-185B-W4A16 已完整同步

Tip

Support this work → · X · GitHub · REAP paper · Cerebras REAP

GLM-4.7-185B-W4A16

W4A16 quantization of 0xSero/GLM-4.7-185B.

At a glance
Base model 0xSero/GLM-4.7-185B
Format W4A16
Total params 185B
Active / token —
Experts / layer 80
Layers 92
Hidden size 5120
Context 202,752
On-disk size 99 GB

Which variant should I pick?

Variant Format Link
GLM-4.7-185B BF16 link
GLM-4.7-185B-W4A16 (this) W4A16 link
GLM-4.7-202B BF16 link
GLM-4.7-218B-W4A16 W4A16 link
GLM-4.7-REAP-40-W4A16 W4A16 link

𓌳 REAP𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression
📄 Paper • 💻 Code • 📝 Blog

GLM-4.7-REAP-50-W4A16

✨ Highlights

50% Expert-Pruned + INT4 Quantized — Double compression for efficient deployment.

  • ~6.5x Total Compression: 700GB → ~92GB
  • REAP + AutoRound: Expert pruning + weight quantization
  • Optimized for Code & Tools: Calibrated on code generation and function calling
  • Lower VRAM: Fits on 2-4x fewer GPUs than BF16

📋 Model Specifications

Property Value
Base Model GLM-4.7-REAP-50
Original (GLM-4.7) 358B params, ~700GB
After REAP 50% 179B params
After W4A16 Quant ~92GB on disk
Quantization INT4 weights, FP16 activations
Group Size 128
Format GPTQ (AutoRound)
Experts per Layer 80 (was 160)
VRAM Required ~100GB

Compression Pipeline

GLM-4.7 (358B, 700GB)
        │
        ▼  REAP 50% expert pruning
        │
GLM-4.7-REAP-50 (179B)
        │
        ▼  AutoRound W4A16 quantization
        │
GLM-4.7-REAP-50-W4A16 (~92GB)  ◀── This model

Total: ~6.5x compression

🔬 Calibration Dataset: Deep Dive

REAP's effectiveness depends critically on calibration data that represents the target use case. We specifically optimized for code generation, function/tool calling, and agentic workflows.

Why These 3 Datasets?

Dataset Samples Purpose Why It Matters
evol-codealpaca-v1 700 Code generation 51% of mix — Code tasks activate specific expert pathways; pruning without code calibration destroys coding ability
xlam-function-calling-60k 330 Function/tool calling 24% of mix — Tool use requires structured JSON output; experts handling schema generation must be preserved
SWE-smith-trajectories 330 Agentic multi-turn 24% of mix — Real SWE-bench trajectories with tool calls, file edits, and multi-step reasoning

32 个文件

浏览文件

数据集版权信息

本数据集的许可证为 Apache License 2.0。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证