This website requires JavaScript.
fc462f25a8
Add definition of mQuaf, close https://github.com/hankcs/HanLP/issues/1965
2025-11-15 15:02:29 -08:00
942ce9306c
TF is deprecated in Transformers and no longer maintained: https://github.com/hankcs/HanLP/issues/1959
2025-10-18 19:28:49 -07:00
e72db55436
Test on ubuntu-latest, macos-latest, windows-latest
2025-09-10 18:28:55 -07:00
5f46aecbff
Fix stdio redirection, fix: https://github.com/hankcs/HanLP/issues/1958
2025-09-07 13:41:23 -07:00
ddb1299bdd
Translate documents to Chinese
2023-11-27 20:15:55 -08:00
199f3f3280
Support reusing existing model and config for finetuning, fix: https://github.com/hankcs/HanLP/issues/1942
2025-01-14 22:08:41 -08:00
81983df45f
Test on ubuntu-20.04, macos-latest, windows-latest
2025-01-14 00:12:12 -08:00
80506cf824
Keep the new hyperparameters when fine-tuning a model, fix: https://github.com/hankcs/HanLP/issues/1942
2025-01-14 00:10:07 -08:00
a639dbcfa1
Revise document
2025-01-13 00:42:06 -08:00
2eb94f2169
Release Ancient Chinese models for tokenization, lemmatization, part-of-speech tagging and dependency parsing
2025-01-12 16:54:03 -08:00
0636dc612d
Revise documents for Ancient Chinese models
2025-01-12 16:28:42 -08:00
f717be146a
Release Ancient Chinese MTL model KYOTO_EVAHAN_TOK_LEM_POS_UDEP_LZH
2025-01-12 16:15:40 -08:00
c1aa002106
Display the main pos by default
2025-01-12 14:47:41 -08:00
7aeb0fc5e7
Support XPOS for CoNLL-U
2025-01-12 14:24:26 -08:00
9988be6d17
Implement TransformerTaggingLemmatizer
2025-01-12 14:19:37 -08:00
e8bad3f574
Release Ancient Chinese tokenization model KYOTO_EVAHAN_TOK_LZH
2025-01-11 17:47:23 -08:00
c0d5d92737
Skip hidden states averaging for tokenization only models
2025-01-10 23:40:11 -08:00
952958700d
Mirror bert-ancient-chinese
2025-01-10 18:48:09 -08:00
5c6d17ba6c
Automatically detect file type by .conllu extension
2025-01-10 17:44:01 -08:00
3185188aa6
Release EN_TOK_LEM_POS_NER_SRL_UDEP_SDP_CON_MODERNBERT_LARGE
2025-01-06 22:01:02 -08:00
7a4b73575f
Replace FloatTensor with sparse_coo_tensor
2025-01-06 21:59:37 -08:00
749ecf6326
Release 2.1.0
2024-12-28 16:11:08 -08:00
d4660c3084
Improve logging
2024-12-28 16:07:22 -08:00
320262345f
TransformList supports config
2024-12-21 23:09:52 -08:00
3aba64935c
Release an English MTL model with ModernBERT encoder
2024-12-28 15:33:12 -08:00
4f7949ac98
Portable同步升级到v1.8.6
2024-12-28 15:16:19 -08:00
e3cfaa0080
Merge branch '1.x' into portable
2024-12-28 15:15:57 -08:00
dadd5c76a5
:checkered_flag:常规维护；小版本+1，发布v1.8.6
2024-12-28 15:14:30 -08:00
03d3e63acb
清理 Predefine
2024-12-28 13:47:45 -08:00
5698f32077
Revise documentation
2024-12-24 16:11:00 -08:00
75edc60c6f
Update eos model config to support new lines
2024-12-24 00:52:45 -08:00
007f29f951
Support coarse tokenization of Japanese
2024-12-23 00:38:16 -08:00
c583768d68
Implement PrependSpace
2024-12-21 14:47:13 -08:00
77481bd1a4
Replace FloatTensor with sparse_coo_tensor
2024-12-20 23:32:58 -08:00
36c9e6fd32
Heuristic to fix lemma for digits
2024-12-20 20:30:47 -08:00
336fd932ab
Support ModernBERT as a tokenizer
2024-12-20 20:25:57 -08:00
d8e6f2a506
Upgrade GitHub Actions to the latest versions
2024-12-17 19:37:18 -08:00
ae3a910b06
Revert binding to the immutable full sha1 as it may prevent us from getting fixes: https://github.com/actions/toolkit/blob/main/docs/action-versioning.md#compatibility
2024-12-17 19:23:14 -08:00
d42012dcc8
fix the risk of permissions and unpinned dependencies in the workflow
2024-12-10 20:00:18 +08:00
c01f4f6482
Fix dep key of to_pretty
2024-12-06 21:03:25 -08:00
1197741509
Fix loading issue of fine-tuned NER models
2024-12-01 19:36:41 -08:00
b8a165cf15
Fix typo
2024-11-29 18:30:41 -08:00
09574c04c3
Convert fea to feats in hanlp_common.document.Document.to_conll
2024-11-29 18:29:56 -08:00
fe47f5a0a4
Move from pkg_resources to packaging, fix: https://github.com/hankcs/HanLP/issues/1925
2024-11-17 01:04:01 -08:00
e68be80af3
Merge remote-tracking branch 'origin/1.x' into 1.x
2024-11-15 23:00:58 -08:00
0df6a5d48b
Portable同步升级到v1.8.5
2024-11-15 22:02:17 -08:00
926e126631
Merge branch '1.x' into portable
2024-11-15 22:01:18 -08:00
a089963f8c
:checkered_flag:常规维护；小版本+1，发布v1.8.5
2024-11-15 22:00:45 -08:00
5008d7b272
Improve the safety of torch.load with weights_only=True
2024-10-08 01:37:15 -07:00
32428a2510
Avoid redundant downloading and decompressing across processes
2024-10-08 01:34:11 -07:00
036f593e85
Please cite our EMNLP paper: https://aclanthology.org/2021.emnlp-main.451/
2024-10-04 18:11:15 -07:00
4280529482
Fix extrapolation in relative transformer, fix: https://github.com/hankcs/HanLP/issues/1906
2024-09-07 17:15:15 -07:00
e64efb850e
Support new versions of tensorflow and numpy
2024-08-21 18:09:40 -07:00
32ed29b4fe
Specify supported tensorflow versions, fix: https://github.com/hankcs/HanLP/issues/1904
2024-08-21 18:08:07 -07:00
022d0fb60c
Parse raw sentences for custom tags, fix: https://github.com/hankcs/HanLP/issues/1903
2024-08-18 16:08:36 -07:00
b32d4b96bd
Update the link to unit tests
2024-07-10 23:57:57 -07:00
77fe054d04
Support Python 3.6 on Windows
2024-07-10 23:27:09 -07:00
b28d38b20a
Test on ubuntu-latest, macos-latest, windows-latest
2024-07-10 23:20:28 -07:00
eab05cab8e
fix:Correcting Documentation Errors
2024-07-07 17:53:59 +08:00
b8742df7a5
Test on ubuntu-latest, macos-latest, windows-latest
2024-07-10 22:48:12 -07:00
dfd8d5eb74
Fix phrasetree fix: https://github.com/hankcs/HanLP/issues/1886
2024-03-22 20:02:47 -07:00
785bed5f27
Fix tokenization of Korean chars fix: https://github.com/hankcs/HanLP/issues/1877
2024-02-23 20:46:42 -08:00
beab770c8f
Mirror tokenizers from our file servers fix: https://github.com/hankcs/HanLP/issues/1865
2023-12-21 18:19:23 -08:00
fa9f6edd48
add test case
2023-12-04 15:40:00 +00:00
08f91c8ba9
let pipline support copy()
2023-12-04 15:01:14 +00:00
0450298721
Pin the version of tokenizers on macOS Python3.6 to compile it
2023-11-27 19:46:15 -08:00
8cbe01e2b8
fix: docs of MSR Tokenization Guidelines of Chinese Text (V5.0)
2023-11-26 21:31:44 +08:00
6bbf1ff107
Update resource & mirror sites
2023-11-27 18:22:18 -08:00
4ac13f1a89
Merge pull request #1853 from webSue/fix/cws_evaluate
2023-10-19 23:09:48 -07:00
69e69b5099
fix:修复中文分词评测工具比较时的计算错误
2023-10-19 21:56:25 +08:00
19eb67aee0
Resize vocab and classifier automatically for taggers
2023-10-18 19:58:47 -07:00
9579160e08
Suppress HF "AdamW is deprecated" warning
2023-10-18 19:06:47 -07:00
4b2686c7a6
修复mini二元文法在JRE初始化后第一次分词可能出现的不一致 fix: https://github.com/hankcs/HanLP/issues/1851#issuecomment-1767808746
2023-10-18 18:35:32 -07:00
4c1f4579bb
Unpin the version of tokenizers fix: https://github.com/hankcs/HanLP/issues/1849
2023-10-13 20:57:41 -07:00
698c7e8a9a
Revise documentations
2023-10-13 20:55:10 -07:00
a27c5644b4
Add explaintion of IC in CTB guideline: https://bbs.hankcs.com/t/ctb-ic/6223
2023-09-18 19:21:23 -07:00
ca70784a1e
Update badges of downloads
2023-08-20 11:21:36 -07:00
9e2c58c805
Merge remote-tracking branch 'origin/1.x' into 1.x
2023-08-12 20:12:16 -07:00
2d3b1bfc1b
Merge pull request #1835 from wxy929629/1.x
2023-08-12 20:11:38 -07:00
9b2ff93794
修复ViterbiSegment分词器中加载自定义词典时未替换DoubleArrayTrie导致分词不符合预期的问题
2023-08-11 14:45:32 +08:00
81795fc6aa
Support TensorFlow on Python 3.10 fix: https://github.com/hankcs/HanLP/issues/1821
2023-05-23 18:03:24 -04:00
47b68812c4
Fix JSON float vs double error: https://bbs.hanlp.com/t/java/5043
2023-04-15 13:39:59 -04:00
bccf0fea7c
Revise documentations
2023-04-14 19:27:45 -04:00
5997c590ec
Support overriding batch_size for a constituency parser fix: https://github.com/hankcs/HanLP/issues/1815
2023-04-14 01:20:48 -04:00
135a399726
Use the original tokens to build constituency tree fix: https://github.com/hankcs/HanLP/issues/1811
2023-04-08 15:03:12 -04:00
5f787ef590
Revise the annotations of 863 pos
2023-04-05 12:19:24 -04:00
e19bc7a5ae
演示如何调整二元文法: https://bbs.hankcs.com/t/topic/5326
2023-04-04 14:02:19 -04:00
ea9565e80a
Lazily load TensorFlow fix: https://github.com/hankcs/HanLP/issues/1810
2023-03-25 13:03:53 -04:00
af70075882
Add protobuf<3.19 to tf dependencies fix: https://github.com/hankcs/HanLP/issues/1809
2023-03-09 21:35:14 -05:00
752b480748
Improve error logging
2023-03-09 21:30:15 -05:00
d57aab29d5
欢迎引用我们的论文：https://aclanthology.org/2021.emnlp-main.451/
2023-02-26 20:08:24 -05:00
6316759f92
Portable同步升级到v1.8.4
2023-02-24 19:46:09 -05:00
08d091e27b
Merge branch '1.x' into portable
2023-02-24 19:45:21 -05:00
9f2646079d
:checkered_flag:常规维护与准确率提升；小版本+1，发布v1.8.4
2023-02-24 19:44:22 -05:00
e1020b0564
修复word2vec文件流关闭问题 fix: https://github.com/hankcs/HanLP/issues/1806
2023-02-24 12:51:59 -05:00
94b41c51bf
调整单测
2023-02-23 11:47:23 +08:00
41b2f3a42e
🙅 补单测
2023-02-23 11:35:30 +08:00
6b4c681754
调整注释
2023-02-23 11:07:40 +08:00
ce07395a5d
增加enableFastBuild
2023-02-23 10:45:13 +08:00
bf77f509a4
Release sentiment_analysis APIs
2023-02-22 15:13:18 -05:00
hankcs