Commit Graph

85 Commits

Author SHA1 Message Date
yqzhishen 28563a1279 Support using e as the base number of the log mel-spectrogram (#175)
* Add `mel_base` and set `e` as default value

* Export dynamic `mel_base`

* Restore old configs

* Refactor mel extraction

* add mel-base check for ddsp vocoder

---------

Co-authored-by: yxlllc <llc1995@sina.com>
2024-03-04 22:39:36 +08:00
yqzhishen c16095bc55 Drop support for some old features and behaviors (#172)
* Drop support for discrete F0 embedding (reserved in ONNX exporter)

* Drop support for `interp_uv` configuration key

* Drop support for `train_set_name` and `valid_set_name` configuration keys

* Drop support for linear domain of random time stretching augmentation

* Drop support for `num_pad_tokens` configuration key

* Drop support for code backup before training

* Drop support for `ffn_padding` configuration key

* Drop support for random seeding

* Add placeholder to load old checkpoint

* Remove duplicate txt_embed layer (resuming may raise errors)

* Remove migration script and error message for transcriptions.txt

* Remove seed from batch shuffling

* Use direct access on some hparam keys

* Fix duplicate keys in YAML

* Rename `pndm_speedup` to `diff_speedup`
2024-02-25 20:57:02 +08:00
yqzhishen e5c79c0c20 New variance parameter: voicing (#170)
* Support voicing in acoustic and variance models (experimental)

* Fix KeyError
2024-02-24 01:17:42 +08:00
yqzhishen b8ba29975a New variance parameter: tension (#169)
* Add `DeconstructedWaveform` class and base harmonic extraction algorithm (by yxlllc)

* Support tension in acoustic and variance models (experimental)

* Support Kth harmonic extraction

* Use square difference instead

* Support weighted multi-harmonic definition

* Unify all tension domains in one function

* Remove `hparams` from pitch extraction APIs

* Fix KeyError
2024-02-19 20:53:52 +08:00
Dachun Sun 36ac5f583b Multi-node batched validation and improvement on strategy selection (#148)
* More metadata when binarize

* Picklable AttrDict

* Support other notion of 'sizes'

* Better strategy getter, Ds TB logger, and eval batch sampler.

* Add title to plots and specify figsize

* Unify batch sampler and bug fixes

* Batch and multi-device validation

* Fix imports

* Fix deadlock under multigpu and resume from ckpt

* Remove unnecessary if in base_dataset

* Move build loss to finish init and ft to build model

* Prevent repeated valid item

* Remove optimizer_idx to support lightning 2.1

* Warning message fix

* Fix val error when aux is off

* Rename fields in metadata and add doc

* Adjust respective duration logging for each speaker

* val persisent worker, module list ordering, update doc

---------

Co-authored-by: yqzhishen <yangqian_1015@icloud.com>
2023-10-29 00:37:14 +08:00
yqzhishen 3ae0e7db53 Implement melody encoder and support glide input (#143)
* Add melody encoder and support glide notes

* Support melody encoder inference

* Add note visualization on TensorBoard

* Remove base pitch embedding when using melody encoder

* Fix KeyError `use_glide_embed`

* Fix invalid access to NoneType

* Support melody encoder ONNX export

* Fix KeyError `use_melody_encoder`

* Support note glide ONNX export

* Support custom glide types

* Support glide embedding scale

* Adjust glide type format

* Remove unnecessary interpolation on frame-level MIDI pitch

* Add glide type coverage checks
2023-10-08 23:24:17 +08:00
yqzhishen 054567a8d8 Fix wrong spk_id assignment in fixed pitch shifting augmentation 2023-08-20 14:21:57 +08:00
yqzhishen 78ae36d9d3 Support training variance models from DS files (#132)
* Support training variance model with DS files

* Prefer full name matching

* Fix attribute error

* Fix NoneType error

* Add error message if attribute is missing

* Trim ds segment index suffix

* Skip parameter smoothing if loading from DS file
2023-08-11 11:50:22 +08:00
yqzhishen f3dd1ecccb Add support for objective evaluation metrics (#123)
* Support objective evaluation metrics

* Add raw pitch accuracy (RPA) logging

* Exclude unvoiced and padding frames
2023-07-30 14:40:12 +08:00
yqzhishen b1304f8164 Fix OOM when turning on augmentation 2023-07-27 00:39:15 +08:00
yqzhishen 7b3c2adf83 Invert wrong logical expr 2023-07-22 14:27:50 +08:00
yqzhishen 75abeae6b5 Fix augmentation error when multiprocessing 2023-07-21 18:02:55 +08:00
yqzhishen f74046db31 Support custom pitch extractor (#118)
* Support custom pitch extractor

* Add doc for `pe` and `pe_ckpt`
2023-07-19 23:57:38 +08:00
yqzhishen cd26ecb143 Add MIDI distribution summary to variance binarizer 2023-07-14 23:29:18 +08:00
yqzhishen 9fcfd2efb1 Add phoneme coverage checks to variance binarizer 2023-07-14 22:49:26 +08:00
yqzhishen 801e05eddf Adjust comments 2023-07-05 23:58:25 +08:00
yqzhishen 21de5a111b Fix missing mel2ph when not training duration predictor 2023-07-03 22:37:37 +08:00
yqzhishen c2e0027b2c Fix conflict of item name when merging speakers 2023-06-28 23:51:55 +08:00
yqzhishen 65b4be379b Modify param names and docs for ds_id and spk_id 2023-06-24 02:36:49 +08:00
yqzhishen 224fd33f39 Support custom spk_id arrangements 2023-06-24 02:20:51 +08:00
yqzhishen 830ee8459e Modification of energy and breathiness definitions (#91)
* Change energy and breathiness definition to dB

* Smoothen energy and breathiness

* Smoothen energy and breathiness (acoustic model)

* Update comments

* Support different values of `dilation_cycle_length`

* Add missing `torch.no_grad()`
2023-05-29 22:58:45 +08:00
yqzhishen 109f56558c Fix base_pitch is all zero when using multiprocessing binarizer on Windows 2023-05-26 16:05:09 +08:00
yqzhishen c8ac07d750 Support variance embeds in acoustic exporter 2023-05-26 00:55:55 +08:00
yqzhishen eb161aa905 Make some attributes optional in binarizer according to requirements 2023-05-25 19:32:55 +08:00
yqzhishen b4dcafa554 Merge branch 'refactor-v2' into variance 2023-05-18 20:09:23 +08:00
yqzhishen c29e073d02 Fix label format to CSV and add migrating scripts 2023-05-18 18:28:22 +08:00
yqzhishen 8c2b7350de Merge branch 'refactor-v2' into variance 2023-05-17 20:38:33 +08:00
yqzhishen 4b0d95ad65 Use direct indexing on augmentation args 2023-05-13 21:30:00 +08:00
yqzhishen 34fe5399ff Augmentations can now be disabled with enabled config keys 2023-05-13 21:17:36 +08:00
ms903x1 cec7760005 Freeze transcriptions generation format to CSV and deprecate meta.json (#85)
* Update acoustic_binarizer.py

add "os.path.exists(path)" to determine whether to read "transcriptions.csv" or "transcriptions.txt".

* Update acoustic_binarizer.py

Delete Meta.json related code.
Use pathlib API replacement of os.path.

* 更新notebook

删除生成txt版transcriptions文件相关代码,删除meta.json相关代码。

* fix

多删了个换行,修复一下
2023-05-09 23:36:17 +08:00
yqzhishen 02b235a3d3 Support variance adaptor in DiffSingerAcoustic 2023-05-02 02:17:32 +08:00
yqzhishen 8c18b66145 Fix breathiness dtype mismatch 2023-05-01 01:29:59 +08:00
yqzhishen 10d0013571 Fix wrong uv mask 2023-04-30 18:59:39 +08:00
yqzhishen 6721eca22d Support breathiness extraction, prediction, embedding and training 2023-04-30 14:53:51 +08:00
yqzhishen 3486795c54 Support energy embed in acoustic model 2023-04-29 13:48:37 +08:00
yqzhishen 88cb146a48 Add energy prediction, re-organize losses 2023-04-28 23:30:17 +08:00
yqzhishen 3ec315931f Round ph_midi 2023-04-19 00:22:16 +08:00
yqzhishen caf3a5c2d7 Deprecate rest notes 2023-04-18 18:10:01 +08:00
yqzhishen 76fb972f70 Introduce ph2word, add modules, implement task 2023-04-15 03:13:58 +08:00
yqzhishen 57ab5131c7 Swap attribute order 2023-04-14 00:48:53 +08:00
yqzhishen d230227868 Finish VarianceBinarizer for pitch predictor 2023-04-13 23:28:53 +08:00
yqzhishen cf93d62624 Merge branch 'refactor-v2' into variance 2023-04-13 20:28:20 +08:00
yqzhishen be6311b6b6 Drop 'f0_coarse' from get_pitch_parselmouth 2023-04-13 20:27:57 +08:00
yqzhishen cea94d4084 Adapt VarianceBinarizer to reuse more code 2023-04-13 20:17:55 +08:00
yqzhishen 5db92dda70 Adjust comments 2023-04-13 20:13:42 +08:00
yqzhishen 45d5dd55c7 Refactor DurationPredictor, add DurationLoss 2023-04-13 20:13:42 +08:00
yqzhishen f547283608 Finish VarianceBinarizer, create module and task for dur predictor training 2023-04-13 20:13:42 +08:00
yqzhishen 0d0b254b26 Reuse more code in binarizer 2023-04-13 20:13:14 +08:00
yqzhishen 0dd4d527e5 Use self.binary_data_dir 2023-04-12 22:49:15 +08:00
yqzhishen a21d1ff2bb Add device to binarizer 2023-04-12 21:08:50 +08:00