Add streaming full-collection traversal across C++/C/Python (relates to #380).
## API
- C++: `Collection::create_iterator(IteratorOptions)` → `DocIterator` (`next()` returns `Result<Doc::Ptr>`: error / `nullptr` EOF / doc; `close()`, idempotent, also run by the destructor).
- C API: opaque `zvec_doc_iterator_t` + `zvec_iterator_options_t` handles; `zvec_collection_create_iterator` / `zvec_doc_iterator_next` / `zvec_doc_iterator_close`; errors mapped precisely to `zvec_error_code_t`.
- Python: `collection.iter_docs()` returns a `DocIterator` (iterator protocol + context manager; prefer `with collection.iter_docs() as docs:`). The snapshot is taken at call time; the iterator closes itself when exhausted, on `with` exit, or via `close()`, releasing the native slot on every path (including early break and exceptions).
## Snapshot semantics
- `create_iterator()` seals the current writing segment on writable collections (read-only collections scan directly, including their writing segment — no flush, no data loss); the snapshot captures the segment set, a deep copy of the delete bitmap, and the schema. Writes after creation are invisible to the iterator; deletions after creation do not affect it.
- `IteratorOptions.output_fields_` selects forward fields (unknown/duplicate names rejected with an error); `include_vector_` controls vector materialization.
## Concurrency (admission control + active-iterator count)
- Iterators and maintenance operations are mutually exclusive via `maintenance_mtx_`: `create_iterator()` fails fast with `FailedPrecondition` while a maintenance operation (optimize, schema DDL, close, destroy) is running.
- While any iterator is open (active-iterator count under the schema lock): schema DDL (create/drop index, add/alter/drop column), destroy and close return `FailedPrecondition`; the destructor path instead logs and waits for open iterators (it cannot report errors); optimize fails at its start. flush, writes and queries are not affected; Stats/Schema/Options (shared lock) are not affected.
- The collection must outlive its iterators: close every iterator before closing/releasing the collection (documented in the C++/C API headers).
## Implementation notes
- Per-segment readers opened lazily (at most one open at a time); deleted rows filtered by `FilteringReader`, wrapped only when the snapshot's delete bitmap is non-empty (`src/db/index/segment/filtering_reader.*`).
- Each batch is materialized column by column in bounded windows of at most 4096 rows (`kMaxRecordBatchNumRows`): a Parquet scan returns a whole row group per ReadNext (up to ~1M rows), so windows cap doc materialization and keep memory constant. Column indices are resolved and validated once per segment reader; scalars/arrays go through the shared column-level converter; vectors are fetched per field using segment-local row ids (`_zvec_row_id_`, correct for compacted segments). Materialization and reader failures are sticky (the error keeps being returned; no partial docs are ever handed out).
- Shared converters in `src/db/index/common/doc_field_converter.*` also serve the SQL engine (`ConvertVectorDataBufferToDocField` / `ConvertArrowColumnToDocFields`; `SegmentImpl::Fetch` keeps its pre-existing implementation).
- Known limitation: docs without a value for a vector field are accepted by insert (the write path only warns and skips) and are indistinguishable from fetch errors today, so iteration with `include_vector` fails on such docs — documented in code with a TODO; tracked in a separate issue.
## Tests
- C++ iterator_test: 22 tests — basic/empty/deleted/close-then-next, include/exclude vector, output_fields selection + rejection, scalar type mapping, 1000-doc integration, read-only collection, performance (100k docs, constant memory), Parquet large-row-group windowed materialization, and concurrency: snapshot isolation under writes, optimize/DDL/close/destroy rejected while open (recovered after the iterator closes), create-iterator rejected while optimize runs, the destructor waits for open iterators, create-iterator-on-closed-collection rejected, slot released by destructor, multiple iterators.
- C c_api_test: 6 iterator tests (basic/concurrent-semantics incl. destroy rejection + FAILED_PRECONDITION mapping, exclude-vector, output-fields, empty, null-args) inside the 75-test C API suite.
- Python test_iter_docs.py: 11 tests (basic fields/vectors, deletion filtering, output fields, isolation, snapshot-at-call-time, iterator protocol, context manager, early close releases the slot).
- Add FtsIndexParam/FtsQueryParam class stubs to model/param/__init__.pyi
- Add Fts/FtsIndexParam/FtsQueryParam to zvec/__init__.pyi exports
- Extend Query.param type to include FtsQueryParam
- Update Query._validate to allow fts + FtsQueryParam combination
- Extend FieldSchema.index_param type to support FtsIndexParam
- Extend Collection.create_index type to support FtsIndexParam
* refactor: change rerank interface from map-based to vector-based (#452)
- Define QueryResult = list[Doc] type alias in doc.py
- Change C++ Reranker::rerank() signature from map<string, DocPtrList> to vector<DocPtrList>
- Extend bind_schema() to accept field_names for index-based field lookup
- Update ScoreBasedReranker/WeightedReranker/CallbackReranker implementations
- Adapt collection.cc MultiQuery path to use vector<DocPtrList>
- Update Python binding to expose rerank() and use vector<double> weights
- Refactor Python RerankFunction interface to list[QueryResult] -> QueryResult
- Remove Python-layer rerank logic from RrfReRanker/WeightedReRanker (delegate to C++)
- Update query_executor to return list[list[Doc]] instead of dict
- Update all related unit tests (C++ and Python)
* refactor: replace list[Doc] with QueryResult type alias in executor and rerank functions
* refactor: replace list[list[Doc]] with list[QueryResult] in query_executor
* fix: remove unused Doc import in rerank_function.py (ruff F401)
* refactor(query_executor): merge duplicate rerank return paths
* refactor: RrfReRanker/WeightedReRanker.rerank() directly call C++ reranker
* refactor: simplify QueryExecutor into unified class, remove Factory/subclasses/validation/concurrency
* refactor: rename _VectorQuery to _SearchQuery, from_vector_query to from_search_query
* refactor(query_executor): split execute into single/multi paths, rename core_vector to search_query, drop unused core_vectors
* style: apply ruff formatter to test_reranker.py and query_executor.py
* refactor: make rescore() private in ScoreBasedReranker hierarchy
* style: apply clang-format to reranker.h
* style: apply clang-format to all modified C++ files
* refactor: rename private methods in QueryExecutor for clearer semantics
* refactor: rename mvq to multi_query for clarity
* fix: make BasicRRF test order-independent for equal scores
* fix: update collection_test to use vector-based reranker interface
* fix: update reranker tests to expect TypeError instead of NotImplementedError
* refactor: remove PendingQuery wrapper, use SearchQuery directly in MultiQuery path
* refactor: simplify MultiQuery path - remove seen_fields, merge field_names into main loop
* fix: address review comments - defensive checks and remove fields param from C API
- ScoreBasedReranker::rerank(): early return empty list when topn <= 0
- WeightedReranker::rescore(): null-check schema_ before use
- CallbackReranker::rerank(): check callback_ is not empty before invoke
- C API zvec_reranker_create_weighted(): remove unused fields parameter
* fix: remove duplicate field name test (check was intentionally removed)
* fix: address egolearner review comments
- Rename QueryResult to DocList for clarity (见名知义)
- Change docstring to #: comment for type alias
- Fix output_fields check: use 'is not None' instead of truthy check
(None means unset, [] means explicit empty list - different semantics)
- Raise ValueError when search-by-id finds no document
* refactor: remove redundant output_fields assignment in _build_search_query
* refactor: address egolearner review comments (C++ refactoring)
- c_api.cc: simplify weighted reranker creation with inline vector ctor
- python_reranker.cc: refactor unwrap_rerank_result - take by value,
early error return, move semantics
- Rename C API functions for consistent naming:
zvec_reranker_create_rrf -> zvec_create_rrf_reranker
zvec_reranker_create_weighted -> zvec_create_weighted_reranker
zvec_reranker_destroy -> zvec_destroy_reranker
zvec_reranker_get_rank_constant -> zvec_get_reranker_rank_constant
- reranker.h/cc: bind_schema returns Result<void>, caches
vector<const FieldSchema*> to avoid repeated schema lookups in rescore
- python_param.cc: rename py::arg vector_query to search_query
* revert: rollback bind_schema refactoring due to thread-safety concern
The field_schemas_ caching approach introduces a data race when the same
WeightedReranker instance is shared across concurrent queries: bind_schema()
writes field_schemas_ while rerank() reads it concurrently.
Revert to storing schema_ + field_names_ and looking up fields in rescore().
Add @note thread-safety warning to WeightedReranker class documentation.
* fix: unify error message format in collection.cc
Change 'Vector field not found: X' to 'Invalid query: field X not found'
for consistent error formatting as suggested by zhourrr.
* fix: sort __all__ and remove duplicates in __init__.pyi
Fix RUF022 lint error: sort __all__ alphabetically and remove duplicate
entries (DenseEmbeddingFunction, ReRanker).
* style: format query_executor.py with ruff formatter
* fix: resolve Python test failures after FTS rebase integration
- test_query_executor.py: update method names to match refactored API
(_do_build -> _build_queries, _do_merge_rerank_results -> _merge_and_rerank)
- test_reranker.py: fix expected exception type (TypeError from pybind11)
- test_collection_fts.py: update error message match patterns
- test_collection_fts_vector_hybrid.py: remove obsolete 'metrics' param,
update weights from dict to positional list, adapt validation tests
for multi-vector queries (now supported with reranker)
- test_collection_dql.py: remove 'metrics' param, update weights format
- collection.cc: distinguish FTS vs vector fields in MultiQuery path
using get_fts_clause() to route field lookup correctly
- reranker.cc: use get_field() instead of get_vector_field() in rescore
to support FTS+vector hybrid weighted reranking
* refactor: pass topn as rerank() parameter, move rerank_field to model rerankers
* fix: address review comments - rename test functions and restore duplicate field check
* refactor: simplify MultiQuery field lookup, let validate_and_sanitize handle type check
Yin Yongqi