数据集 / cerebras/DeepSeek-V3.2-REAP-345B-A37B

cerebras/DeepSeek-V3.2-REAP-345B-A37B 已完整同步

𓌳 REAP𓌳 the Experts: Why Pruning Prevails for One-Shot MoE Compression

DeepSeek-V3.2-REAP-345B-A37B

✨ Highlights

Introducing DeepSeek-V3.2-REAP-345B-A37B, a memory-efficient compressed variant of DeepSeek-V3.2 that maintains near-identical performance while being 50% lighter.

This model was created using REAP (Router-weighted Expert Activation Pruning), a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include:

  • Near-Lossless Performance: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 671B model
  • 50% Memory Reduction: Compressed from 671B to 345B parameters, significantly lowering deployment costs and memory requirements
  • Preserved Capabilities: Retains all core functionalities including code generation, math & reasoning and tool calling.
  • Drop-in Compatibility: Works with vanilla vLLM - no source modifications or custom patches required

- Optimized for Real-World Use: Particularly effective for resource-constrained environments, local deployments, and academic research

📋 Model Overview

DeepSeek-V3.2-REAP-345B-A37B has the following specifications:

  • Base Model: DeepSeek-V3.2
  • Compression Method: REAP (Router-weighted Expert Activation Pruning)
  • Compression Ratio: 50% expert pruning
  • Type: Sparse Mixture-of-Experts (SMoE) Causal Language Model
  • Number of Parameters: 345B total, 37B activated per token
  • Number of Layers: 61
  • Number of Attention Heads: 128
  • Number of Experts: 128 (uniformly pruned from 256)
  • Number of Activated Experts: 8 per token
  • Context Length: 163,840 tokens
  • License: MIT

📊 Evaluations

Benchmark Score
LiveCodeBench (25.01 - 25.05) 64.64
HumanEval 93.29
HumanEval+ 89.02
MBPP 91.01
MBPP+ 75.13

Evaluations above were run with temperature=0.0; max_tokens=60000 for LiveCodeBench and max_tokens=16384 for the other benchmarks. For more details on the evaluation setup, refer to the REAP arXiv preprint.

🚀 Deployment

You can deploy the model directly using the latest vLLM (that supports DeepSeek-V3.2), no source modifications or custom patches required.

VLLM_USE_DEEP_GEMM=0 vllm serve cerebras/DeepSeek-V3.2-REAP-345B-A37B \
    --tensor-parallel-size 8 \
    --tokenizer-mode deepseek_v32 \
    --tool-call-parser deepseek_v32 \
    --reasoning-parser deepseek_v3 \
    --trust-remote-code \
    --enable-auto-tool-choice

If you encounter insufficient memory when running this model, you might need to set a lower value for --max-num-seqs flag (e.g. set to 8). For more information, refer to the official deployment guide.

🧩 Model Creation

This checkpoint was created by applying the REAP (Router-weighted Expert Activation Pruning) method uniformly across all Mixture-of-Experts (MoE) blocks of DeepSeek-V3.2, with a 50% pruning rate.

How REAP Works

REAP selects experts to prune based on a novel saliency criterion that considers both:

  • Router gate values: How frequently and strongly the router activates each expert
  • **Expert activation n

87 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证