4e7aed45b8
feat: implement finetune data coding, evaluation, and config improvements
2025-11-28 15:52:47 +00:00
b16c3a77d9
feat: data implement for pre-proposal and proposal and add datasets ( #1303 )
...
* feat:(1) support for multi layer dataset extraction (2) add category.json for dataset in datasets/
* fix: fix bug for generate category.json
* feat: add get_dataset_folder_desc
* init data proposal and merge qzli/ft
* update data proposal prompts and add max_position_embeddings and resolve confilcts
* remove sample counts in data proposal
* turn data and train to unified hypo_gen
* refine prompts
* remove category.json and add it to dataset_info
* fix jinja problem and proposal done
* lint
* add ai-generated description and raw readme into dataset_info.json
* update prompt for description
* add datasets
* initial fix for proposal of data
* final version for data proposal
* lint
2025-11-28 20:50:11 +08:00
1f2ca736d9
feat: support creating dataset & multi-eval frame ( #1302 )
...
* feat: add iterative evolve and evaluation support with partial chain stop
* feat: add FTDataEvaluator and support multiple implement functions in finetune
2025-11-28 12:49:40 +08:00
2056c0b30c
remove debug config after execution
2025-11-27 07:07:42 +00:00
3cf9ac9f58
several major update
2025-11-25 08:37:20 +00:00
0c92d348f8
prompts and dockerfile refine for using deepspeed and fa2
2025-11-24 16:02:43 +00:00
a8372826e5
modify the prompt of coder
2025-11-24 14:26:11 +00:00
0e7dc31d1f
user benchmark as input
2025-11-24 06:25:34 +00:00
0b94fbd719
update code
2025-11-18 09:40:31 +00:00
25c295bca5
small update
2025-11-18 09:21:28 +00:00
5049c4f845
simple update on prompts
2025-11-18 07:57:20 +00:00
86d0240ba9
some update to code
2025-11-14 10:14:22 +00:00
190fe2e2cf
update exp_gen
2025-11-12 07:17:07 +00:00
433b1c1ece
refactor: dataset info, and fix some bugs
2025-11-12 07:05:22 +00:00
c686e1f57f
update all code
2025-11-11 08:50:35 +00:00
33d62fe483
expgen gets all params and decides more
2025-11-10 14:08:59 +00:00
a2c3ccd1e5
fix: coder and benchmark bugs
2025-11-08 10:23:01 +00:00
4d3da5bac2
feat(coder): add type and default value in prompt
2025-11-06 09:31:57 +00:00
6d8ea6d30d
some update on the code
2025-11-06 09:28:46 +00:00
7f82cf2e08
Merge branch 'main' into qzli/ft
2025-11-06 07:34:19 +00:00
61052f3010
enable costeer
2025-11-06 07:32:26 +00:00
63ecb3bf26
fix: handle mixed str and dict types in code_list ( #1279 )
...
* fix: handle mixed str and dict types in code_list
* fix: handle missing token_costs entry for loop 0 in summarize_win
2025-11-03 21:52:41 +08:00
ca68896770
refactor: rm model_dump setting
2025-10-27 05:09:37 +00:00
14b259d8d7
refactor: refine exp_feedback
2025-10-24 07:31:59 +00:00
623a656447
fix: enforce queried_knowledge requirement and update type hints for RAGStrategy
2025-10-24 07:03:40 +00:00
5a3d02dc50
Merge remote-tracking branch 'upstream/main' into qzli/ft
2025-10-23 09:14:15 +00:00
417766ee36
fix: refine task scheduling logic in MultiProcessEvolvingStrategy for… ( #1275 )
...
* fix: refine task scheduling logic in MultiProcessEvolvingStrategy for improved handling of feedback in improve mode
* fix: add empty implementation for skipped tasks in MultiProcessEvolvingStrategy
2025-10-23 17:09:18 +08:00
961e56102c
fix: enhance feedback handling in MultiProcessEvolvingStrategy for improved task evolution ( #1274 )
2025-10-23 16:00:23 +08:00
bd024fc8e5
Merge remote-tracking branch 'upstream/xuyang1/fix_improve_mode_bug' into qzli/ft
2025-10-23 07:55:36 +00:00
b845cd6566
fix: enhance feedback handling in MultiProcessEvolvingStrategy for improved task evolution
2025-10-23 07:51:43 +00:00
13d8c2ba4e
refactor: rename evaluator
2025-10-23 05:25:08 +00:00
c5bdf04d15
fix: exclude template field(auto detect)
2025-10-22 11:48:54 +00:00
2afaed88be
Merge branch 'main' into qzli/ft
2025-10-22 09:51:31 +00:00
11c91a9fa3
refactor(coder): remove debug_mode, which is outdated
2025-10-22 09:49:12 +00:00
9344635695
feat: add improve_mode to MultiProcessEvolvingStrategy for selective task implementation ( #1273 )
2025-10-22 17:35:18 +08:00
3366e900e9
fix bugs
2025-10-22 03:24:47 +00:00
61df6d4a4f
fix: filter params, support tensorboard
2025-10-22 03:24:47 +00:00
49d9d2fade
chore: update TODO
2025-10-22 03:17:03 +00:00
71f808ca13
chore: rename, reset prompt
2025-10-22 03:17:03 +00:00
58326beab8
feat: add sample , sample count&data size
2025-10-22 03:17:03 +00:00
5106556773
feat: create_ws_ckp for FT, and clean init file
2025-10-22 03:17:03 +00:00
10cc554291
feat: coder docker cache
2025-10-22 03:17:03 +00:00
a2b7a7001b
refactor: simplify coder evaluator
2025-10-22 03:16:56 +00:00
3b6e559274
chore: rename some parameters
2025-10-22 03:16:28 +00:00
1b31fbf6fe
refactor: simplify llama factory manager
2025-10-22 03:15:01 +00:00
d78154c981
fix: add required fields in yaml&template
2025-10-22 03:15:01 +00:00
1883520776
feat: check commit hash and refresh llamafactory info
2025-10-22 03:15:01 +00:00
fd45c42fe1
feat: extract info from llama-factory in docker, refine model¶meter select(need debug)
2025-10-22 03:15:01 +00:00
7a7ebb86fe
fix: use cache in llamafactory info
2025-10-22 03:15:01 +00:00
17e27c7985
feat: extract llama factory info and save it as file(need debug and develop)
2025-10-22 03:15:01 +00:00
Young