数据集 / Skywork/Matrix-Game-2.0

Skywork/Matrix-Game-2.0 已完整同步

Matrix-Game 2.0: An Open-Source, Real-Time, and Streaming Interactive World Model

📝 Overview

Matrix-Game-2.0（1.8B） is an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion

✨ Key Features

  • 🚀 Feature 1: Real-Time Distillation Efficient ​​few-step diffusion​​ for streaming video synthesis at ​​25 FPS​​, producing ​​minute-level, high-fidelity videos​​ across complex environments with ultra-fast speed.
  • 🖱️ Feature 2: Precise Action Injection A ​​mouse/keyboard-to-frame​​ module that embeds user inputs as direct interactions, enabling frame-level control and dynamic response in generated videos.
  • 🎬 Feature 3: Massive Interactive Data Pipeline A scalable production system for ​​Unreal Engine & GTA5​​ that generates ​​~1200 hours​​ of high-quality interactive video data, covering diverse scenes with frame-level realism.

🔥 Latest Updates

  • [2025-08] 🎉 Initial release of Matrix-Game-2.0 Model

Model Overview

Matrix-Game-2.0（1.8B） is derived from the Wan. By removing the text branch and adding action modules, the model predicts next frames only from visual contents and corresponding actions.

Model Overview

📈 Performance Comparison

GameWorld Score Benchmark Comparison

Model Image Quality ↑ Aesthetic Quality ↑ Temporal Cons. ↑ Motion Smooth. ↑ Keyboard Acc. ↑ Mouse Acc. ↑ Object Cons. Scenario Cons.
Oasis 0.27 0.27 0.82 0.99 0.73 0.56 0.18 0.84
Ours 0.61 0.50 0.94 0.98 0.91 0.95 0.64 0.80

Metric Descriptions:

  • Image Quality / Aesthetic: Visual fidelity and perceptual appeal of generated frames

  • Temporal Consistency / Motion Smoothness: Temporal coherence and smoothness between frames

  • Keyboard Accuracy / Mouse Accuracy: Accuracy in following user control signals

  • Object Consistency: Geometric stability and consistency of objects over time

  • Scenario Consistency: Scenario consistency over time

    Please check our GameWorld benchmark for detailed implementation.

🚀 Quick Start

# clone the repository:
git clone https://github.com/SkyworkAI/Matrix-Game.git
cd Matrix-Game/Matrix-Game-2

# install apex and FlashAttention
# Our project also depends on [FlashAttention](https://github.com/Dao-AILab/flash-attention)
# install dependencies:
pip install -r requirements.txt
python setup.py develop

# inference
python inference.py \
    --config_path configs/inference_yaml/{your-config}.yaml \
    --checkpoint_path {path-to-the-checkpoint} \
    --img_path {path-to-the-input-image} \
    --output_folder outputs \
    --num_output_frames 150 \
    --seed 42 \
    --pretrained_model_path {path-to-the-vae-folder}
# inference streaming
python inference_streaming.py \
    --config_path configs/inference_yaml/{you

18 个文件

浏览文件

数据集版权信息

本数据集的许可证为 MIT License。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证