Compare commits

...

33 Commits

Author SHA1 Message Date
Matthew Honnibal 9725c1ada3 Require srsly >=2.5.3 for Python 3.14 cloudpickle fix
tests / Validate (push) Has been cancelled
tests / Test (windows-latest, 3.14) (push) Has been cancelled
tests / Test (macos-latest, 3.10) (push) Has been cancelled
tests / Test (macos-latest, 3.11) (push) Has been cancelled
tests / Test (macos-latest, 3.12) (push) Has been cancelled
tests / Test (macos-latest, 3.13) (push) Has been cancelled
tests / Test (macos-latest, 3.14) (push) Has been cancelled
tests / Test (ubuntu-latest, 3.10) (push) Has been cancelled
tests / Test (ubuntu-latest, 3.11) (push) Has been cancelled
tests / Test (ubuntu-latest, 3.12) (push) Has been cancelled
tests / Test (ubuntu-latest, 3.13) (push) Has been cancelled
tests / Test (ubuntu-latest, 3.14) (push) Has been cancelled
tests / Test (windows-latest, 3.10) (push) Has been cancelled
tests / Test (windows-latest, 3.11) (push) Has been cancelled
tests / Test (windows-latest, 3.12) (push) Has been cancelled
tests / Test (windows-latest, 3.13) (push) Has been cancelled
2026-03-23 13:01:47 +01:00
Matthew Honnibal 7b9fe93d13 Harden GitHub Actions: pin actions to SHA, add least-privilege permissions, remove debug logging 2026-03-23 11:45:46 +01:00
Matthew Honnibal 9abb59cf8c Fix secret exposure in explosionbot workflow pip install command
Use git config insteadOf to authenticate private repo access via env var
rather than interpolating the token directly into the pip install URL,
which would expose it in process listings and potentially in logs.
2026-03-23 11:38:28 +01:00
Matthew Honnibal 34f43bde78 Revert srsly pin to >=2.4.3,<3.0.0 (srsly v3 has msgpack strict_map_key breakage) 2026-03-23 11:27:21 +01:00
Matthew Honnibal ac0c3fe175 Bump ml_datasets pin to >=0.2.1 for srsly v3 compatibility 2026-03-23 11:15:58 +01:00
Matthew Honnibal 1c40afd376 Require srsly v3 (>=3.0.0) for Python 3.14 cloudpickle fix 2026-03-23 10:51:28 +01:00
Matthew Honnibal b05ff52005 Require pydantic v2 (>=2.0.0) now that confection 1.1 has dropped pydantic 2026-03-23 10:28:20 +01:00
Matthew Honnibal 953ee38454 Set fail-fast: false to get full 3.14 CI results across all platforms 2026-03-23 10:25:32 +01:00
Matthew Honnibal 27199bc23f Enable Python 3.14 in CI test matrix 2026-03-23 08:33:03 +01:00
Matthew Honnibal eab3119380 Fix zh tokenizer test: expect ConfigValidationError for invalid segmenter enum 2026-03-22 20:32:46 +01:00
Matthew Honnibal 45db7d772e Skip Python 3.14 in CI until dependencies have wheels 2026-03-21 18:22:48 +01:00
Matthew Honnibal 3ed48a2abe Filter pydantic v1 incompatibility warning for Python 3.14 support 2026-03-21 18:14:38 +01:00
Matthew Honnibal 990577a426 Fix mangled YAML in tests.yml and remove remaining pydantic warning filters 2026-03-21 17:34:37 +01:00
Matthew Honnibal ff1bced640 Set python_requires upper bound to <3.15 2026-03-21 15:37:45 +01:00
Matthew Honnibal 00d26d0795 Add Python 3.14 to CI matrix and classifiers 2026-03-21 15:36:56 +01:00
Matthew Honnibal 4f59963dd8 Fix import sorting in test_textcat.py 2026-03-21 15:33:20 +01:00
Matthew Honnibal 326df3b709 Remove pydantic deprecation warning filters (fixed in confection) 2026-03-21 15:16:50 +01:00
Matthew Honnibal 8b42d87830 Monkeypatch _get_pip_install_cmd in download test for envs without pip on PATH 2026-03-21 14:45:56 +01:00
Matthew Honnibal d434682a46 Fix tests for confection 1.1+: warning filters, zh segmenter test, lazy tok2vec import, str->Path coercion 2026-03-21 14:43:34 +01:00
Matthew Honnibal b11a6e3f42 Add ruff to test ignore list and filter pydantic deprecation warnings from confection 2026-03-21 13:42:13 +01:00
Matthew Honnibal abb0414288 Replace deprecated pydantic .dict() with .model_dump() 2026-03-21 13:16:47 +01:00
Matthew Honnibal dfba77b9b9 Fix import sorting in factories.py 2026-03-21 13:07:05 +01:00
Matthew Honnibal a0f5d7b9eb Fix EntityLinker_v1 import to use legacy module 2026-03-21 13:01:46 +01:00
Matthew Honnibal e17f2e6286 Replace deprecated pydantic Field(help=...) with Field(title=...) 2026-03-21 12:49:59 +01:00
Matthew Honnibal e684b0c1d1 Apply ruff formatting to find_threshold.py and language.py 2026-03-21 09:01:46 +01:00
Matthew Honnibal 4cf4e9e350 Ignore mypy errors (for now) 2026-03-21 09:00:41 +01:00
Matthew Honnibal 30faae59ee Merge branch 'master' into confection1.1 2026-03-21 08:47:21 +01:00
Matthew Honnibal 041f47b3b2 Format 2026-03-21 08:37:39 +01:00
Matthew Honnibal 200a3646c7 Fix ruff isort config: replace unsupported profile with equivalent settings 2026-03-21 08:37:22 +01:00
Matthew Honnibal e9d7df6c3a Remove W503 from ruff ignore list (not a valid ruff rule) 2026-03-21 08:36:53 +01:00
Matthew Honnibal a84a2e6444 Replace black, isort, and flake8 with ruff for linting and formatting
- requirements.txt: remove black, isort, flake8; add ruff
- pyproject.toml: replace [tool.isort] with [tool.ruff] config
- setup.cfg: remove [flake8] section (rules moved to pyproject.toml)
- .pre-commit-config.yaml: replace black/flake8 hooks with ruff/ruff-format
2026-03-21 08:36:11 +01:00
Matthew Honnibal e5fc4d6706 Format 2026-03-21 08:31:48 +01:00
Matthew Honnibal 932727a9cf Update dependency pins: thinc 8.3.12, confection 1.1, weasel 1.0 2026-03-21 08:08:02 +01:00
33 changed files with 93 additions and 64 deletions
+4 -1
View File
@@ -7,9 +7,12 @@ on:
# ** matches 'zero or more of any character'
- 'release-v[0-9]+.[0-9]+.[0-9]+**'
- 'prerelease-v[0-9]+.[0-9]+.[0-9]+**'
permissions: {}
jobs:
build_wheels:
uses: explosion/gha-cibuildwheel/.github/workflows/cibuildwheel.yml@main
uses: explosion/gha-cibuildwheel/.github/workflows/cibuildwheel.yml@2c98f757f13d112cf73fcf4b627249f1fffb5aae # main
permissions:
contents: write
actions: read
+7 -3
View File
@@ -6,6 +6,8 @@ on:
- created
- edited
permissions: {}
jobs:
explosion-bot:
if: github.repository_owner == 'explosion'
@@ -15,13 +17,15 @@ jobs:
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
run: echo "$GITHUB_CONTEXT"
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
- name: Install and run explosion-bot
run: |
pip install git+https://${{ secrets.EXPLOSIONBOT_TOKEN }}@github.com/explosion/explosion-bot
git config --global url."https://x-access-token:${EXPLOSIONBOT_TOKEN}@github.com/".insteadOf "https://github.com/"
pip install git+https://github.com/explosion/explosion-bot
python -m explosionbot
env:
EXPLOSIONBOT_TOKEN: ${{ secrets.EXPLOSIONBOT_TOKEN }}
INPUT_TOKEN: ${{ secrets.EXPLOSIONBOT_TOKEN }}
INPUT_BK_TOKEN: ${{ secrets.BUILDKITE_SECRET }}
ENABLED_COMMANDS: "test_gpu,test_slow,test_slow_gpu"
+5 -1
View File
@@ -11,12 +11,16 @@ on:
types:
- labeled
permissions: {}
jobs:
issue-manager:
permissions:
issues: write
if: github.repository_owner == 'explosion'
runs-on: ubuntu-latest
steps:
- uses: tiangolo/issue-manager@0.4.0
- uses: tiangolo/issue-manager@4d1b7e05935a404dc8337d30bd23be46be8bb8e5 # 0.4.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
config: >
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
if: github.repository_owner == 'explosion'
runs-on: ubuntu-latest
steps:
- uses: dessant/lock-threads@v5
- uses: dessant/lock-threads@1bf7ec25051fe7c00bdd17e6a7cf3d7bfb7dc771 # v5
with:
process-only: 'issues'
issue-inactive-days: '30'
+3 -1
View File
@@ -8,6 +8,8 @@ on:
types:
- published
permissions: {}
jobs:
upload_pypi:
runs-on: ubuntu-latest
@@ -21,7 +23,7 @@ jobs:
# or, alternatively, upload to PyPI on every tag starting with 'v' (remove on: release above to use this)
# if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
steps:
- uses: robinraju/release-downloader@v1
- uses: robinraju/release-downloader@daf26c55d821e836577a15f77d86ddc078948b05 # v1
with:
tag: ${{ github.event.release.tag_name }}
fileName: '*'
+4 -9
View File
@@ -5,21 +5,16 @@ on:
paths:
- "website/meta/universe.json"
permissions: {}
jobs:
build:
if: github.repository_owner == 'explosion'
runs-on: ubuntu-latest
steps:
- name: Dump GitHub context
env:
GITHUB_CONTEXT: ${{ toJson(github) }}
PR_NUMBER: ${{github.event.number}}
run: |
echo "$GITHUB_CONTEXT"
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: '3.10'
- name: Install Bernadette app dependency and send an alert
+10 -8
View File
@@ -19,6 +19,8 @@ on:
- "*.mdx"
- "website/**"
permissions: {}
jobs:
validate:
name: Validate
@@ -26,10 +28,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Configure Python version
uses: actions/setup-python@v4
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.10"
@@ -45,19 +47,19 @@ jobs:
name: Test
needs: Validate
strategy:
fail-fast: true
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python_version: ["3.10", "3.11", "3.12", "3.13"]
python_version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ${{ matrix.os }}
steps:
- name: Check out repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Configure Python version
uses: actions/setup-python@v4
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: ${{ matrix.python_version }}
@@ -93,7 +95,7 @@ jobs:
shell: bash
- name: Test import
run: python -W error -c "import spacy"
run: python -W error -W 'ignore:Core Pydantic V1:UserWarning:pydantic' -c "import spacy"
- name: "Test download CLI"
run: |
@@ -154,7 +156,7 @@ jobs:
- name: "Run CPU tests"
run: |
python -m pytest --pyargs spacy -W error
python -m pytest --pyargs spacy -W error -W 'ignore:Core Pydantic V1:UserWarning:pydantic'
if: "!(startsWith(matrix.os, 'macos') && matrix.python_version == '3.11')"
- name: "Run CPU tests with thinc-apple-ops"
+4 -2
View File
@@ -13,6 +13,8 @@ on:
paths:
- "website/meta/universe.json"
permissions: {}
jobs:
validate:
name: Validate
@@ -20,10 +22,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out repo
uses: actions/checkout@v4
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
- name: Configure Python version
uses: actions/setup-python@v4
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6
with:
python-version: "3.7"
+1 -1
View File
@@ -5,7 +5,7 @@ requires = [
"cymem>=2.0.2,<2.1.0",
"preshed>=3.0.2,<3.1.0",
"murmurhash>=0.28.0,<1.1.0",
"thinc>=8.3.4,<8.4.0",
"thinc>=8.3.12,<8.4.0",
"numpy>=2.0.0,<3.0.0"
]
build-backend = "setuptools.build_meta"
+6 -6
View File
@@ -3,19 +3,19 @@ spacy-legacy>=3.0.11,<3.1.0
spacy-loggers>=1.0.0,<2.0.0
cymem>=2.0.2,<2.1.0
preshed>=3.0.2,<3.1.0
thinc>=8.3.4,<8.4.0
ml_datasets>=0.2.0,<0.3.0
thinc>=8.3.12,<8.4.0
ml_datasets>=0.2.1,<0.3.0
murmurhash>=0.28.0,<1.1.0
wasabi>=0.9.1,<1.2.0
srsly>=2.4.3,<3.0.0
srsly>=2.5.3,<3.0.0
catalogue>=2.0.6,<2.1.0
typer>=0.3.0,<1.0.0
weasel>=0.4.2,<0.5.0
weasel>=1.0.0,<2.0.0
# Third party dependencies
numpy>=2.0.0,<3.0.0
requests>=2.13.0,<3.0.0
tqdm>=4.38.0,<5.0.0
pydantic>=1.7.4,!=1.8,!=1.8.1,<3.0.0
pydantic>=2.0.0,<3.0.0
jinja2
# Official Python utilities
setuptools
@@ -34,4 +34,4 @@ types-requests
types-setuptools>=57.0.0
ruff>=0.9.0
cython-lint>=0.15.0
confection>=0.0.4,<1.0.0
confection>=1.1.0,<2.0.0
+10 -6
View File
@@ -22,6 +22,7 @@ classifiers =
Programming Language :: Python :: 3.11
Programming Language :: Python :: 3.12
Programming Language :: Python :: 3.13
Programming Language :: Python :: 3.14
Topic :: Scientific/Engineering
project_urls =
Release notes = https://github.com/explosion/spaCy/releases
@@ -41,7 +42,7 @@ setup_requires =
cymem>=2.0.2,<2.1.0
preshed>=3.0.2,<3.1.0
murmurhash>=0.28.0,<1.1.0
thinc>=8.3.4,<8.4.0
thinc>=8.3.12,<8.4.0
install_requires =
# Our libraries
spacy-legacy>=3.0.11,<3.1.0
@@ -49,19 +50,19 @@ install_requires =
murmurhash>=0.28.0,<1.1.0
cymem>=2.0.2,<2.1.0
preshed>=3.0.2,<3.1.0
thinc>=8.3.4,<8.4.0
thinc>=8.3.12,<8.4.0
wasabi>=0.9.1,<1.2.0
srsly>=2.4.3,<3.0.0
srsly>=2.5.3,<3.0.0
catalogue>=2.0.6,<2.1.0
weasel>=0.4.2,<0.5.0
confection>=0.0.4,<1.0.0
weasel>=1.0.0,<2.0.0
confection>=1.1.0,<2.0.0
# Third-party dependencies
typer>=0.3.0,<1.0.0
tqdm>=4.38.0,<5.0.0
numpy>=1.15.0; python_version < "3.9"
numpy>=1.19.0; python_version >= "3.9"
requests>=2.13.0,<3.0.0
pydantic>=1.7.4,!=1.8,!=1.8.1,<3.0.0
pydantic>=2.0.0,<3.0.0
jinja2
# Official Python utilities
setuptools
@@ -135,6 +136,9 @@ formats = gztar
markers =
slow: mark a test as slow
issue: reference specific issue
filterwarnings =
error
ignore:Core Pydantic V1:UserWarning:pydantic
[mypy]
ignore_missing_imports = True
+1 -1
View File
@@ -41,7 +41,7 @@ for _schema in (
ConfigSchemaInit,
ConfigSchema,
):
_schema.model_rebuild(_types_namespace=_rebuild_ns)
_schema.model_rebuild(_types_namespace=_rebuild_ns) # type: ignore[attr-defined]
if sys.maxunicode == 65535:
raise SystemError(Errors.E130)
+2 -2
View File
@@ -82,10 +82,10 @@ def debug_config(
config = nlp.config.interpolate()
msg.divider("Config validation for [initialize]")
with show_validation_error(config_path):
T = registry.resolve(config["initialize"], schema=ConfigSchemaInit)
T = registry.resolve(config["initialize"], schema=ConfigSchemaInit) # type: ignore[arg-type]
msg.divider("Config validation for [training]")
with show_validation_error(config_path):
T = registry.resolve(config["training"], schema=ConfigSchemaTraining)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining) # type: ignore[arg-type]
dot_names = [T["train_corpus"], T["dev_corpus"]]
util.resolve_dot_names(config, dot_names)
msg.good("Config is valid")
+1 -1
View File
@@ -137,7 +137,7 @@ def debug_data(
cfg = util.load_config(config_path, overrides=config_overrides)
nlp = util.load_model_from_config(cfg)
config = nlp.config.interpolate()
T = registry.resolve(config["training"], schema=ConfigSchemaTraining)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining) # type: ignore[arg-type]
# Use original config here, not resolved version
sourced_components = get_sourced_components(cfg)
frozen_components = T["frozen_components"]
+1 -1
View File
@@ -89,7 +89,7 @@ def debug_model_cli(
with show_validation_error(config_path):
nlp = util.load_model_from_config(raw_config)
config = nlp.config.interpolate()
T = registry.resolve(config["training"], schema=ConfigSchemaTraining)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining) # type: ignore[arg-type]
seed = T["seed"]
if seed is not None:
msg.info(f"Fixing random seed: {seed}")
+4 -2
View File
@@ -207,8 +207,10 @@ def find_threshold(
),
)
if hasattr(pipe, "cfg"):
nlp.get_pipe(pipe_name).cfg = set_nested_item(
pipe.cfg, config_keys, threshold
nlp.get_pipe(pipe_name).cfg = set_nested_item( # type: ignore[attr-defined]
pipe.cfg,
config_keys,
threshold, # type: ignore[attr-defined]
)
eval_scores = nlp.evaluate(dev_dataset)
+1 -1
View File
@@ -216,7 +216,7 @@ def init_config(
# Filter out duplicates since tok2vec and transformer are added by template
pipeline = [pipe for pipe in pipeline if pipe not in ("tok2vec", "transformer")]
defaults = RECOMMENDATIONS["__default__"]
reco = RecommendationSchema(**RECOMMENDATIONS.get(lang, defaults)).dict()
reco = RecommendationSchema(**RECOMMENDATIONS.get(lang, defaults)).model_dump()
variables = {
"lang": lang,
"components": pipeline,
+2 -2
View File
@@ -35,8 +35,8 @@ else:
try: # Python 3.8+
import importlib.metadata as importlib_metadata
except ImportError:
from catalogue import (
_importlib_metadata as importlib_metadata, # type: ignore[no-redef] # noqa: F401
from catalogue import ( # type: ignore[no-redef]
_importlib_metadata as importlib_metadata, # noqa: F401
)
from thinc.api import Optimizer # noqa: F401
+1 -1
View File
@@ -416,7 +416,7 @@ class SpanishLemmatizer(Lemmatizer):
rule = self.select_rule("verb", features)
verb_lemma = self.lemmatize_verb(
verb,
features - {"PronType=Prs"},
features - {"PronType=Prs"}, # type: ignore[operator]
rule,
index, # type: ignore[operator]
)[0]
+6 -4
View File
@@ -1323,7 +1323,7 @@ class Language:
# Make sure the config is interpolated so we can resolve subsections
config = self.config.interpolate()
# These are the settings provided in the [initialize] block in the config
I = registry.resolve(config["initialize"], schema=ConfigSchemaInit)
I = registry.resolve(config["initialize"], schema=ConfigSchemaInit) # type: ignore[arg-type]
before_init = I["before_init"]
if before_init is not None:
before_init(self)
@@ -1353,7 +1353,7 @@ class Language:
proc.initialize(get_examples, nlp=self, **p_settings)
pretrain_cfg = config.get("pretraining")
if pretrain_cfg:
P = registry.resolve(pretrain_cfg, schema=ConfigSchemaPretrain)
P = registry.resolve(pretrain_cfg, schema=ConfigSchemaPretrain) # type: ignore[arg-type]
init_tok2vec(self, P, I)
self._link_components()
self._optimizer = sgd
@@ -1823,7 +1823,7 @@ class Language:
orig_pretraining = config.pop("pretraining", None)
config["components"] = {}
if auto_fill:
filled = registry.fill(config, validate=validate, schema=ConfigSchema)
filled = registry.fill(config, validate=validate, schema=ConfigSchema) # type: ignore[arg-type]
else:
filled = config
filled["components"] = orig_pipeline
@@ -1832,7 +1832,9 @@ class Language:
filled["pretraining"] = orig_pretraining
config["pretraining"] = orig_pretraining
resolved_nlp = registry.resolve(
filled["nlp"], validate=validate, schema=ConfigSchemaNlp
filled["nlp"],
validate=validate,
schema=ConfigSchemaNlp, # type: ignore[arg-type]
)
create_tokenizer = resolved_nlp["tokenizer"]
create_vectors = resolved_nlp["vectors"]
+1 -1
View File
@@ -57,7 +57,7 @@ cdef class PhraseMatcher:
attr = "ORTH"
if attr == "IS_SENT_START":
attr = "SENT_START"
if attr.lower() not in TokenPattern().dict():
if attr.lower() not in TokenPattern().model_dump():
raise ValueError(Errors.E152.format(attr=attr))
self.attr = IDS.get(attr)
@@ -6,6 +6,7 @@ from pydantic import (
ConfigDict,
Field,
RootModel,
StrictBool,
StrictInt,
StrictStr,
ValidationError,
+2 -1
View File
@@ -14,9 +14,10 @@ from ..pipeline.edit_tree_lemmatizer import (
)
# Import factory default configurations
from ..pipeline.entity_linker import DEFAULT_NEL_MODEL, EntityLinker, EntityLinker_v1
from ..pipeline.entity_linker import DEFAULT_NEL_MODEL, EntityLinker
from ..pipeline.entityruler import DEFAULT_ENT_ID_SEP, EntityRuler
from ..pipeline.functions import DocCleaner, TokenSplitter
from ..pipeline.legacy import EntityLinker_v1
from ..pipeline.lemmatizer import Lemmatizer
from ..pipeline.morphologizer import DEFAULT_MORPH_MODEL, Morphologizer
from ..pipeline.multitask import DEFAULT_MT_MODEL, MultitaskObjective
+3 -3
View File
@@ -251,7 +251,7 @@ class TokenPatternOperatorSimple(str, Enum):
TokenPatternOperatorMinMax = constr(pattern=r"^(\{\d+\}|\{\d+,\d*\}|\{\d*,\d+\})$")
TokenPatternOperator = Union[TokenPatternOperatorSimple, TokenPatternOperatorMinMax]
TokenPatternOperator = Union[TokenPatternOperatorSimple, TokenPatternOperatorMinMax] # type: ignore[valid-type]
StringValue = Union[TokenPatternString, StrictStr]
NumberValue = Union[TokenPatternNumber, StrictInt, StrictFloat]
UnderscoreValue = Union[
@@ -420,8 +420,8 @@ class ConfigSchemaInit(BaseModel):
lookups: Optional[Lookups] = Field(..., title="Vocabulary lookups, e.g. lexeme normalization")
vectors: Optional[StrictStr] = Field(..., title="Path to vectors")
init_tok2vec: Optional[StrictStr] = Field(..., title="Path to pretrained tok2vec weights")
tokenizer: Dict[StrictStr, Any] = Field(..., help="Arguments to be passed into Tokenizer.initialize")
components: Dict[StrictStr, Dict[StrictStr, Any]] = Field(..., help="Arguments for TrainablePipe.initialize methods of pipeline components, keyed by component")
tokenizer: Dict[StrictStr, Any] = Field(..., title="Arguments to be passed into Tokenizer.initialize")
components: Dict[StrictStr, Dict[StrictStr, Any]] = Field(..., title="Arguments for TrainablePipe.initialize methods of pipeline components, keyed by component")
before_init: Optional[Callable[["Language"], "Language"]] = Field(..., title="Optional callback to modify nlp object before initialization")
after_init: Optional[Callable[["Language"], "Language"]] = Field(..., title="Optional callback to modify nlp object after initialization")
# fmt: on
+1 -1
View File
@@ -1,5 +1,5 @@
import pytest
from thinc.api import ConfigValidationError
from confection import ConfigValidationError
from spacy.lang.zh import Chinese, _get_pkuseg_trie_data
+1
View File
@@ -15,6 +15,7 @@ def test_build_dependencies():
"cython-lint",
"black",
"isort",
"ruff",
"mypy",
"types-dataclasses",
"types-mock",
+1
View File
@@ -24,6 +24,7 @@ from spacy.pipeline.textcat_multilabel import (
)
from spacy.pipeline.tok2vec import DEFAULT_TOK2VEC_MODEL
from spacy.scorer import Scorer
from spacy.tests.tok2vec import build_lazy_init_tok2vec as _ # noqa: F401
from spacy.tokens import Doc, DocBin
from spacy.training import Example
from spacy.training.initialize import init_nlp
+3
View File
@@ -1229,6 +1229,9 @@ def test_download_rejects_relative_urls(monkeypatch):
relative path in the filename"""
monkeypatch.setattr(download_module, "run_command", lambda cmd: None)
monkeypatch.setattr(
download_module, "_get_pip_install_cmd", lambda: ["pip", "install"]
)
# Check that normal download works
download_module.download("en_core_web_sm-3.7.1", direct=True)
+1
View File
@@ -4,6 +4,7 @@ from typing import (
Any,
Callable,
Iterable,
Iterator,
List,
Optional,
Sequence,
+2 -1
View File
@@ -50,9 +50,10 @@ def create_jsonl_reader(
@util.registry.readers("spacy.read_labels.v1")
def read_labels(path: Path, *, require: bool = False):
def read_labels(path: Union[str, Path], *, require: bool = False):
# I decided not to give this a generic name, because I don't want people to
# use it for arbitrary stuff, as I want this require arg with default False.
path = Path(path)
if not require and not path.exists():
return None
return srsly.read_json(path)
+1 -1
View File
@@ -50,7 +50,7 @@ def init_nlp(config: Config, *, use_gpu: int = -1) -> "Language":
logger.info("Set up nlp object from config")
config = nlp.config.interpolate()
# Resolve all training-relevant sections using the filled nlp config
T = registry.resolve(config["training"], schema=ConfigSchemaTraining)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining) # type: ignore[arg-type]
dot_names = [T["train_corpus"], T["dev_corpus"]]
if not isinstance(T["train_corpus"], str):
raise ConfigValidationError(
+1 -1
View File
@@ -62,7 +62,7 @@ def train(
allocator = config["training"]["gpu_allocator"]
if use_gpu >= 0 and allocator:
set_gpu_allocator(allocator)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining)
T = registry.resolve(config["training"], schema=ConfigSchemaTraining) # type: ignore[arg-type]
dot_names = [T["train_corpus"], T["dev_corpus"]]
train_corpus, dev_corpus = resolve_dot_names(config, dot_names)
optimizer = T["optimizer"]
+1 -1
View File
@@ -42,7 +42,7 @@ def pretrain(
config["initialize"]["init_tok2vec"] = None
nlp = load_model_from_config(config)
_config = nlp.config.interpolate()
P = registry.resolve(_config["pretraining"], schema=ConfigSchemaPretrain)
P = registry.resolve(_config["pretraining"], schema=ConfigSchemaPretrain) # type: ignore[arg-type]
corpus = dot_to_object(_config, P["corpus"])
corpus = registry.resolve({"corpus": corpus})["corpus"]
batcher = P["batcher"]