* refactor: make Reranker stateless with std::variant value semantics (#461)
Replace class hierarchy (Reranker/ScoreBasedReranker/RrfReranker/
WeightedReranker/CallbackReranker) with std::variant<RrfParams,
WeightedParams, CallbackParams> value type and a stateless free function
reranker::rerank().
Key changes:
- reranker.h: define RerankParams variant + reranker::rerank() API
- query.h: MultiQuery::reranker (shared_ptr) -> MultiQuery::rerank (value)
- schema.h: add CollectionSchema::get_field_ptr() returning FieldSchema::Ptr
- collection.cc: push field lookup to caller, pass vector<FieldSchema::Ptr>
- c_api: remove opaque zvec_reranker_t, add zvec_multi_query_set_rerank_*
- python binding: expose _RrfParams/_WeightedParams/_CallbackParams + setters
- python layer: WeightedReRanker(list[float]), remove Python rerank logic
- all tests updated to new interface
Benefits:
- Thread-safe by design: no mutable state, safe to share across threads
- Collection-decoupled: no bind_schema(), field info passed as parameter
- Simpler lifecycle: value semantics, no shared_ptr management
Closes#461
* chore: remove nightly_build.yml unrelated to reranker refactor
* chore: remove uv.lock unrelated to reranker refactor
* fix: raise ValueError when multi-query has no reranker
After the reranker stateless refactor the C++ MultiQuery rerank
strategy uses a std::variant with a default value, so the implicit
'reranker required' validation no longer triggered. Restore the
check in QueryExecutor._execute_multi_query so that a hybrid
(multi-query) request without a reranker raises ValueError.
* fix(reranker): use index_type FTS check for non-vector normalization
Replace dynamic_cast nullptr check with explicit IndexType::FTS check
and map FTS/BM25 positive scores to (0.0, 1.0) via 2*atan(score)/pi.
* refactor(reranker): move Params types into reranker namespace and qualify usages
Move RrfParams, WeightedParams, CallbackParams and RerankParams into the
zvec::reranker namespace, and add explicit reranker:: qualification at all
usage sites outside the reranker module (query.h, python/c bindings, tests).
* refactor(query): drop unused PendingQuery wrapper, use std::vector<SearchQuery> directly
* refactor(reranker): make _to_cpp_params non-abstract with default NotImplementedError
Remove @abstractmethod from RerankFunction._to_cpp_params and provide a
default implementation raising NotImplementedError. Drop the redundant
_to_cpp_params overrides from Qwen and Sentence rerankers since they use
the Python rerank path and don't need the C++ conversion.
* refactor: change rerank interface from map-based to vector-based (#452)
- Define QueryResult = list[Doc] type alias in doc.py
- Change C++ Reranker::rerank() signature from map<string, DocPtrList> to vector<DocPtrList>
- Extend bind_schema() to accept field_names for index-based field lookup
- Update ScoreBasedReranker/WeightedReranker/CallbackReranker implementations
- Adapt collection.cc MultiQuery path to use vector<DocPtrList>
- Update Python binding to expose rerank() and use vector<double> weights
- Refactor Python RerankFunction interface to list[QueryResult] -> QueryResult
- Remove Python-layer rerank logic from RrfReRanker/WeightedReRanker (delegate to C++)
- Update query_executor to return list[list[Doc]] instead of dict
- Update all related unit tests (C++ and Python)
* refactor: replace list[Doc] with QueryResult type alias in executor and rerank functions
* refactor: replace list[list[Doc]] with list[QueryResult] in query_executor
* fix: remove unused Doc import in rerank_function.py (ruff F401)
* refactor(query_executor): merge duplicate rerank return paths
* refactor: RrfReRanker/WeightedReRanker.rerank() directly call C++ reranker
* refactor: simplify QueryExecutor into unified class, remove Factory/subclasses/validation/concurrency
* refactor: rename _VectorQuery to _SearchQuery, from_vector_query to from_search_query
* refactor(query_executor): split execute into single/multi paths, rename core_vector to search_query, drop unused core_vectors
* style: apply ruff formatter to test_reranker.py and query_executor.py
* refactor: make rescore() private in ScoreBasedReranker hierarchy
* style: apply clang-format to reranker.h
* style: apply clang-format to all modified C++ files
* refactor: rename private methods in QueryExecutor for clearer semantics
* refactor: rename mvq to multi_query for clarity
* fix: make BasicRRF test order-independent for equal scores
* fix: update collection_test to use vector-based reranker interface
* fix: update reranker tests to expect TypeError instead of NotImplementedError
* refactor: remove PendingQuery wrapper, use SearchQuery directly in MultiQuery path
* refactor: simplify MultiQuery path - remove seen_fields, merge field_names into main loop
* fix: address review comments - defensive checks and remove fields param from C API
- ScoreBasedReranker::rerank(): early return empty list when topn <= 0
- WeightedReranker::rescore(): null-check schema_ before use
- CallbackReranker::rerank(): check callback_ is not empty before invoke
- C API zvec_reranker_create_weighted(): remove unused fields parameter
* fix: remove duplicate field name test (check was intentionally removed)
* fix: address egolearner review comments
- Rename QueryResult to DocList for clarity (见名知义)
- Change docstring to #: comment for type alias
- Fix output_fields check: use 'is not None' instead of truthy check
(None means unset, [] means explicit empty list - different semantics)
- Raise ValueError when search-by-id finds no document
* refactor: remove redundant output_fields assignment in _build_search_query
* refactor: address egolearner review comments (C++ refactoring)
- c_api.cc: simplify weighted reranker creation with inline vector ctor
- python_reranker.cc: refactor unwrap_rerank_result - take by value,
early error return, move semantics
- Rename C API functions for consistent naming:
zvec_reranker_create_rrf -> zvec_create_rrf_reranker
zvec_reranker_create_weighted -> zvec_create_weighted_reranker
zvec_reranker_destroy -> zvec_destroy_reranker
zvec_reranker_get_rank_constant -> zvec_get_reranker_rank_constant
- reranker.h/cc: bind_schema returns Result<void>, caches
vector<const FieldSchema*> to avoid repeated schema lookups in rescore
- python_param.cc: rename py::arg vector_query to search_query
* revert: rollback bind_schema refactoring due to thread-safety concern
The field_schemas_ caching approach introduces a data race when the same
WeightedReranker instance is shared across concurrent queries: bind_schema()
writes field_schemas_ while rerank() reads it concurrently.
Revert to storing schema_ + field_names_ and looking up fields in rescore().
Add @note thread-safety warning to WeightedReranker class documentation.
* fix: unify error message format in collection.cc
Change 'Vector field not found: X' to 'Invalid query: field X not found'
for consistent error formatting as suggested by zhourrr.
* fix: sort __all__ and remove duplicates in __init__.pyi
Fix RUF022 lint error: sort __all__ alphabetically and remove duplicate
entries (DenseEmbeddingFunction, ReRanker).
* style: format query_executor.py with ruff formatter
* fix: resolve Python test failures after FTS rebase integration
- test_query_executor.py: update method names to match refactored API
(_do_build -> _build_queries, _do_merge_rerank_results -> _merge_and_rerank)
- test_reranker.py: fix expected exception type (TypeError from pybind11)
- test_collection_fts.py: update error message match patterns
- test_collection_fts_vector_hybrid.py: remove obsolete 'metrics' param,
update weights from dict to positional list, adapt validation tests
for multi-vector queries (now supported with reranker)
- test_collection_dql.py: remove 'metrics' param, update weights format
- collection.cc: distinguish FTS vs vector fields in MultiQuery path
using get_fts_clause() to route field lookup correctly
- reranker.cc: use get_field() instead of get_vector_field() in rescore
to support FTS+vector hybrid weighted reranking
* refactor: pass topn as rerank() parameter, move rerank_field to model rerankers
* fix: address review comments - rename test functions and restore duplicate field check
* refactor: simplify MultiQuery field lookup, let validate_and_sanitize handle type check
* feat: migrate multi-vector query and reranker logic to C++
- Add Reranker base class with RrfReRanker and WeightedReRanker implementations
- Add Collection::MultiQuery interface for multi-vector queries with reranking
- Add MultiVectorQuery struct in doc.h with forward declaration for Reranker
- Add C API bindings for reranker and MultiQuery (zvec_reranker_*, zvec_multi_vector_query_*, zvec_collection_multi_query)
- Add Python binding for reranker classes with py::function bridge for callback
- Validate duplicate field names in multi-vector queries (C++ and Python consistent)
- Remove TODO comment about concurrent execution (SQLEngine is not thread-safe)
- Update collection.h MultiQuery doc comment from concurrently to sequentially
- Add C++ collection tests (6 MultiQuery test cases)
- Add C API tests (reranker functions + multi_vector_query end-to-end)
- Implement Python test cases (11 previously skipped tests now active)
- Simplify Python query_executor validation for unified duplicate field check
* style: format Python files with ruff
* fix: adapt to main branch API changes (VectorQuery->Query rename, validate_and_sanitize)
* fix: multi_vector tests now use multiple same-type vector fields (dense2, sparse2)
* fix: suppress RET501 for intentional default return None in RerankFunction._get_object
* style: ruff format test_collection.py
* refact multi vector query
* format code
* fix(multi-vector): expose SubVectorQuery in Python binding, fix tests
- Register _SubVectorQuery in pybind11 with from_vector_query() factory
- Convert _VectorQuery to _SubVectorQuery in MultiVectorQueryExecutor
- Relax RRF/Weighted score assertion tolerance from 1e-10 to 1e-6
- Fix WeightedReRanker test metric to IP (matching HnswIndexParam default)
* style: ruff format query_executor.py
* fix: define _USE_MATH_DEFINES for M_PI on Windows (MSVC)
* refactor: include reranker.h directly in query.h instead of forward declaration
* refactor(reranker): move topn from member variable to rerank() parameter
* refact code
* style(python): fix ruff UP035/UP037 in multi_vector_reranker
- import Callable from collections.abc instead of typing (UP035)
- remove redundant quotes around MetricType annotations (UP037)
* chore: trigger PR sync
* refact code
* fix(examples): restore CMakeLists.txt formatting broken by clang-format
* refactor(reranker): remove redundant metrics_ map by querying schema directly, and use insert return value to avoid duplicate set lookup
* refactor(reranker): defer schema binding to query time and remove C API callback reranker
lichen2015