数据集 / XiaomiMiMo/MiMo-V2-Flash

XiaomiMiMo/MiMo-V2-Flash 已完整同步





MiMo-V2-Flash

MiMo-V2-Flash is a Mixture-of-Experts (MoE) language model with 309B total parameters and 15B active parameters. Designed for high-speed reasoning and agentic workflows, it utilizes a novel hybrid attention architecture and Multi-Token Prediction (MTP) to achieve state-of-the-art performance while significantly reducing inference costs.

1. Introduction

MiMo-V2-Flash creates a new balance between long-context modeling capability and inference efficiency. Key features include:

  • Hybrid Attention Architecture: Interleaves Sliding Window Attention (SWA) and Global Attention (GA) with a 5:1 ratio and an aggressive 128-token window. This reduces KV-cache storage by nearly 6x while maintaining long-context performance via learnable attention sink bias.
  • Multi-Token Prediction (MTP): Equipped with a lightweight MTP module (0.33B params/block) using dense FFNs. This triples output speed during inference and will be good to accelerates rollout in RL training.
  • Efficient Pre-Training: Trained on 27T tokens using FP8 mixed precision and native 32k seq length. The context window supports up to 256k length.
  • Agentic Capabilities: Post-training utilizes Multi-Teacher On-Policy Distillation (MOPD) and large-scale agentic RL, achieving superior performance on SWE-Bench and complex reasoning tasks.

2. Model Downloads

Model Total Params Active Params Context Length Download
MiMo-V2-Flash-Base 309B 15B 256k 🤗 HuggingFace
MiMo-V2-Flash 309B 15B 256k 🤗 HuggingFace

Important

We also open-source the 3-layer MTP weights to foster community research.

3. Evaluation Results

Base Model Evaluation

MiMo-V2-Flash-Base demonstrates strong performance across standard benchmarks, surpassing models with significantly larger parameter counts.

Category Benchmark Setting/Length MiMo-V2-Flash Base Kimi-K2 Base DeepSeek-V3.1 Base DeepSeek-V3.2 Exp Base
Params #Activated / #Total - 15B / 309B 32B / 1043B 37B / 671B 37B / 671B
General BBH 3-shot 88.5 88.7 88.2

数据概览

该资源属于模型发布包而非结构化数据集，当前镜像以 safetensors、配置和代码文件为主；使用前应确认显存、运行时和模型许可证要求。

157 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证