Dataset Introduction

OmniThoughtV is a large-scale multimodal long-chain-of-thought dataset distilled from the FineVision dataset using Alibaba Cloud's AI platform (PAI) distillation toolkit, EasyDistill. This dataset establishes a transparent and reproducible data distillation pipeline, enabling efficient construction of multimodal reasoning chains of thought. Fine-tuning smaller models with this dataset effectively endows them with stronger reasoning capabilities and enhances their performance on downstream tasks.

Dataset Usage

from datasets import load_dataset 

data_path='...YOUR_PATH_To.../OmmiThoughtV_Raw_1.8M/'

OmmiThoughtV=load_dataset(data_path, streaming=True)

Dataset Examples

Dataset examples are shown below. In these examples, images are stored in base64-encoded string format.

[
  {
    "role": "system",
    "content": "You are a helpful assistant to think step by step. Provide your reasoning steps within <thinking></thinking> tags and give your final answer within <answer></answer> tags."
  },
  {
    "role": "user",
    "content": 
“
### Question

What is the currency used in the invoice?

### Output Format (Strictly Enforced)

<thinking>
Clearly explain your reasoning step by step. Describe how you arrived at the conclusion.
The reasoning process MUST BE enclosed within <thinking> </thinking> tags.
</thinking>
<answer>
Your final answer to the user's question.
</answer>
<image>
”
  },
  {
    "role": "assistant",
    "content": 
"<thinking>
To determine the currency used in the invoice, I will analyze the document for any indications of currency symbols or references.

1. The invoice is from the CRC Contract Research Center, and it includes prices listed next to each item.
2. Looking at the prices, such as \"5.574,-\", \"10.580,-\", \"6.968,-\", \"689,-\", and \"3.242,-\", these numbers are followed by a comma and a dash, which is typical formatting in some European countries for decimal points and currency notation.
3. The total amount at the bottom is \"24.153,-\". This format is consistent with Belgian francs (BEF), which was the currency used in Belgium before the euro.
4. The document is dated November 24, 1993, and Belgium used the Belgian franc until 2002 when it adopted the euro.
5. Additionally, the company's address is in Belgium (B-1932 Zaventem), and the phone number format is also consistent with Belgium.
6. The use of \"BP\" in the price column likely stands for \"Belgian Francs\" (Belgische Frank), which was the currency code for the Belgian franc.

Given all these observations, the currency used in the invoice is the Belgian franc (BEF).
</thinking>
<answer>
Belgian franc (BEF)
</answer>
“
  }
]

Evaluation

We fine-tuned Qwen3-VL 2B, 4B, and 8B models on a filtered version of the OmniThoughtV dataset containing 500k samples and evaluated them using the LMMs-Eval framework. Since different prompt templates can affect evaluation results, we consistently used a fixed inference prompt throughout our evaluations (see Evaluation Details for specifics). The experimental results are summarized in the table below.

Our experiments successfully validated the effectiveness of the dataset. Results show that all fine-tuned models achieved performance improvements across various benchmarks—both general visual understanding benchmarks (e.g., AI2D, MMStar) and benchmarks more sensitive to reasoning capabilities (e.g., MMMU_Pro_standard, MMMU_Pro_vision, MathVerse, MathVision). Notably, the gains were especially pronounced on reasoning-intensive benchmarks. This clearly demonstrates that distilling high-quality data, carefully filtering it, and scaling it up effectively enhances model reasoning capabilities.

Reference

Wenrui Cai, Chengyu Wang, Junbing Yan, Jun H

alibaba-pai/OmniThoughtV_Filter_0.5M 已完整同步

Dataset Introduction

Dataset Usage

Dataset Examples

Evaluation

Reference

32 个文件

数据集版权信息