数据集 / inclusionAI/Ling-flash-2.0

inclusionAI/Ling-flash-2.0 已完整同步

🤗 Hugging Face   |   🤖 ModelScope

Introduction

Today, Ling-flash-2.0 is officially open-sourced! 🚀 Following the release of the language model Ling-mini-2.0 and the thinking model Ring-mini-2.0, we are now open-sourcing the third MoE LLM under the Ling 2.0 architecture: Ling-flash-2.0, a language model with 100B total parameters and 6.1B activated parameters (4.8B non-embedding). Trained on 20T+ tokens of high-quality data, together with supervised fine-tuning and multi-stage reinforcement learning, Ling-flash-2.0 achieves SOTA performance among dense models under 40B parameters, despite activating only ~6B parameters. Compared to MoE models with larger activation/total parameters, it also demonstrates strong competitiveness. Notably, it delivers outstanding performance in complex reasoning, code generation, and frontend development.

Powerful Complex Reasoning Abilities

We conducted a comprehensive evaluation of Ling-flash-2.0’s reasoning capabilities, reporting strong results on representative benchmarks:

  • Multi-disciplinary knowledge reasoning: GPQA-Diamond, MMLU-Pro
  • Advanced mathematical reasoning: AIME 2025, Omni-MATH, OptMATH (advanced mathematical optimization tasks)
  • Challenging code generation: LiveCodeBench v6, CodeForces-Elo
  • Logical reasoning: KOR-Bench, ARC-Prize
  • Key regulated industries (Finance, Healthcare): FinanceReasoning, HealthBench

Compared with dense models under 40B (e.g., Qwen3-32B-Non-Thinking, Seed-OSS-36B-Instruct (think budget=0)) and larger-activation/total-parameter MoE models (e.g., Hunyuan-A13B-Instruct, GPT-OSS-120B/low), Ling-flash-2.0 demonstrates stronger complex reasoning power. Moreover, it shows high competitiveness on creative tasks (Creative Writing v3).

Efficient Architecture, High-Speed Inference

Guided by Ling Scaling Laws, Ling 2.0 adopts a 1/32 activation-ratio MoE architecture, optimized across multiple design choices: expert granularity, shared-expert ratio, attention balance, aux-loss-free + sigmoid routing strategy, MTP layers, QK-Norm, Partial-RoPE, and more. These refinements enable small-activation MoE models to achieve 7× efficiency gains over equivalent dense architectures. In other words, with just 6.1B activated parameters (4.8B non-embedding), Ling-flash-2.0 can match the performance of ~40B dense models. Thanks to its small activation size, it also delivers major inference speed advantages:

  • On H20 hardware, Ling-flash-2.0 achieves 200+ tokens/s, offering 3× speedups compared to 36B dense models in everyday use.
  • With YaRN extrapolation, it supports 128K context length, and as output length grows, its relative speedup can reach 7× or more.

Model Downloads

You can download

32 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证