数据集 / Wan-AI/Wan2.2-S2V-14B

Wan-AI/Wan2.2-S2V-14B 已完整同步

Wan2.2-S2V-14B: Audio-Driven Cinematic Video Generation

This repository features the Wan2.2-S2V-14B model, designed for audio-driven cinematic video generation. It was introduced in the paper: Wan-S2V: Audio-Driven Cinematic Video Generation

💜 Wan Homepage    ｜    🖥️ GitHub    |   🤗 Hugging Face Organization   |   🤖 ModelScope Organization   |    📑 Wan-S2V Paper    |    📑 Wan2.2 Base Paper    | 🌐 Project Page    |    📑 Blog    |    💬 Discord  
📕 使用指南(中文)   |    📘 User Guide(English)   |   💬 WeChat(微信)  

Abstract (Wan-S2V Paper)

Current state-of-the-art (SOTA) methods for audio-driven character animation demonstrate promising performance for scenarios primarily involving speech and singing. However, they often fall short in more complex film and television productions, which demand sophisticated elements such as nuanced character interactions, realistic body movements, and dynamic camera work. To address this long-standing challenge of achieving film-level character animation, we propose an audio-driven model, which we refere to as Wan-S2V, built upon Wan. Our model achieves significantly enhanced expressiveness and fidelity in cinematic contexts compared to existing approaches. We conducted extensive experiments, benchmarking our method against cutting-edge models such as Hunyuan-Avatar and Omnihuman. The experimental results consistently demonstrate that our approach significantly outperforms these existing solutions. Additionally, we explore the versatility of our method through its applications in long-form video generation and precise video lip-sync editing.

Wan: Open and Advanced Large-Scale Video Generative Models

We are excited to introduce Wan2.2, a major upgrade to our foundational video models. With Wan2.2, we have focused on incorporating the following innovations:

  • 👍 Effective MoE Architecture: Wan2.2 introduces a Mixture-of-Experts (MoE) architecture into video diffusion models. By separating the denoising process cross timesteps with specialized powerful expert models, this enlarges the overall model capacity while maintaining the same computational cost.

  • 👍 Cinematic-level Aesthetics: Wan2.2 incorporates meticulously curated aesthetic data, complete with detailed labels for lighting, composition, contrast, color tone, and more. This allows for more precise and controllable cinematic style generation, facilitating the creation of videos with customizable aesthetic preferences.

  • 👍 Complex Motion Generation: Compared to Wan2.1, Wan2.2 is trained on a significantly larger data, with +65.6% more images and +83.2% more videos. This expansion notably enhances the model's generalization across multiple dimensions such as motions, semantics, and aesthetics, achieving TOP performance among all open-sourc

49 个文件

浏览文件

数据集版权信息

本数据集的许可证为 Apache License 2.0。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证