数据集 / thu-pacman/PCMind-2.1-Kaiyuan-2B

thu-pacman/PCMind-2.1-Kaiyuan-2B 已完整同步

License arXiv-2512.07612

This repository contains the complete pretraining dataset for PCMind-v2.1-Kaiyuan-2B, a leading fully open-source language model.

Overview

The dataset is organized into 5 training phases, with all phase datasets open-sourced in this repository. Our training methodology employs domain-specific mixing strategies across five primary domains:

  • English: General English text
  • Chinese: General Chinese text
  • Code: Programming and code-related content
  • Math: Mathematical reasoning and problems
  • SFT: Supervised fine-tuning data

The phase-wise mixing ratios are as follows, where we primarily classify datasets into 5 domains: English, Chinese, Code, Math, and SFT data.

Phase Structure

The training process uses two distinct sampling strategies:

Phase Sampling Strategy Data Format
Phase 1-2 Uniform sampling Single column: text
Phase 3-5 Curriculum learning Two columns: text (content), rank (sample order)

Key distinctions:

  • Phases 1-2: Uniform data distribution with random sampling
  • Phases 3-5: Curriculum-based learning with ordered sample progression using the rank field

Each phase employs strategically designed mixing ratios across the five domains. The specific composition and ratios are detailed in our technical report.


Reproducing the Dataset

To construct these phase datasets from scratch, refer to the Kaiyuan-Spark repository, which provides comprehensive documentation on the preprocessing pipeline.

Citation

If you use this dataset, please cite our technical report:

@misc{luo2025pcmind21kaiyuan2btechnicalreport,
  title={PCMind-2.1-Kaiyuan-2B Technical Report}, 
  author={Kairong Luo and Zhenbo Sun and Xinyu Shi and Shengqi Chen and Bowen Yu and Yunyi Chen and Chenyi Dang and Hengtao Tao and Hui Wang and Fangming Liu and Kaifeng Lyu and Wenguang Chen},
  year={2025},
  eprint={2512.07612},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2512.07612},
}

Resources

License

All artifacts (including code, model weights

660 个文件

浏览文件

数据集版权信息

本数据集的许可证为 Apache License 2.0。如有违反相关条款，请联系 WEHUB，我们将及时处理。 查看许可证