Files
amstrongzyf b16c3a77d9 feat: data implement for pre-proposal and proposal and add datasets (#1303)
* feat:(1) support for multi layer dataset extraction (2) add category.json for dataset in datasets/

* fix: fix bug for generate category.json

* feat: add get_dataset_folder_desc

* init data proposal and merge qzli/ft

* update data proposal prompts and add max_position_embeddings and resolve confilcts

* remove sample counts in data proposal

* turn data and train to unified hypo_gen

* refine prompts

* remove category.json and add it to dataset_info

* fix jinja problem and proposal done

* lint

* add ai-generated description and raw readme into dataset_info.json

* update prompt for description

* add datasets

* initial fix for proposal of data

* final version for data proposal

* lint
2025-11-28 20:50:11 +08:00
..