Commit Graph

62 Commits

Author SHA1 Message Date
David S. Batista 36af5e38bf test: adding extra Mixin docstore tests (#12355) 2026-08-17 09:28:29 +02:00
David S. Batista a42f62a820 fix: InMemoryDocumentStore adding optional parameter filters to get_metadata_field_unique_values() (#12258) 2026-08-06 14:06:04 +00:00
David S. Batista 7d980a4bd0 fix: in memory doc store get_unique_metadata_values() pagination and case-insensitive + Mixin tests updated (#12168)
Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2026-08-04 09:28:02 +02:00
iridescentWen 6acc8805d4 docs: fix docstring parameter names in three components (#12204) 2026-07-31 13:28:30 +02:00
Sebastian Husch Lee 580905cd05 test: Strengthen some of the doc store to mixin tests (#12047)
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2026-07-17 09:47:37 +00:00
Sebastian Husch Lee 07b8d9337e fix: fix doc store mixin to use correct doc store method (#12045) 2026-07-17 09:24:34 +00:00
bogdankostic 3464f29ec9 feat!: Gate pipeline deserialization through a module allowlist (#11432)
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2026-06-12 10:25:23 +02:00
Stefano Fiorucci 0faec86833 test: fix wrong placement for some async Document Store tests (#11315) 2026-05-15 08:36:30 +02:00
David S. Batista 5333208b10 test: making all write_documents be write_documents_async (#11174) 2026-04-23 14:21:18 +02:00
David S. Batista 828594ec31 tests: moving async Mixin tests from document_store to document_store_async (#11020)
* moving async Mixin tests from document_store to document_store_async

* fixing InMemoryDocumentStore tests
2026-04-01 17:16:29 +02:00
Stefano Fiorucci e1281af827 chore: fix types for mypy 1.20 (#11003) 2026-04-01 13:20:46 +02:00
David S. Batista b02b7ce2f1 test: adding mixing filter async tests + implementing them in InMemoryDocumentStore tests (#10975) 2026-03-30 11:47:02 +02:00
David S. Batista 23c25d4f53 feat: add get_meta_data async mixin tests to haystack.testing + InMemoryDocumentStore async operations and tests (#10963)
* adding get_metadata async related Mixin tests

* adding get_metadata async methods to the InMemoryDocumentStore

* using Mixin async metadata tests to InMemoryDocumentstore tests

* adding release notes

* double ticks in release notes

* Update haystack/testing/document_store_async.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2026-03-27 16:47:48 +00:00
Miguel Miranda Dias d047e93863 fix: UpdateByFilterAsyncTest, CountDocumentsByFilterAsyncTest, CountUniqueMetadataByFilterAsyncTest (#10953)
* fix: address #10920

* formatting

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2026-03-27 16:55:04 +01:00
Miguel Miranda Dias d7c51f6d72 fix: DeleteAllAsyncTest, DeleteByFilterAsyncTest, (#10952)
* fix: address #10919

* adding delete_all_documents_async missing in InMemoryDocumentStore

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2026-03-27 16:22:41 +01:00
Miguel Miranda Dias 3f01f22cee fix: CountDocumentsAsyncTest, WriteDocumentsAsyncTest, WriteDocumentsAsyncTest (#10948)
* fix: address #10917

* removing lazyimport + solving MRO conflict

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2026-03-27 15:24:09 +01:00
David S. Batista 9a5024b316 test: extend document stores Mixin tests (#10806)
* adding new Mixin tests

* adding more operations + using tests from Mixin

* reverting changes that were wrongly introduced from another branch

* adding release notes

* removing release notes

* renaming reserved field name in test

* renaming reserved field name in test + removing test

* cleanin up some tests

* moving one more test into Mixin

* adding two more tests

* reverting
2026-03-18 16:04:56 +01:00
Julian Risch 1794405984 docs: Add docstrings and type annotations (#10824)
* add type annotations to run,warm_up, __str__

* none and any type annotations

* type annotations

* __init__ return type annotation

* list[Document] instead of Any return type annotations

* fmt

* typing

* add run_async return type annotation for LinkContentFetcher

* run_async return type annotations

* fixing docstrings

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2026-03-16 18:16:05 +01:00
Stefano Fiorucci 819ed7244b chore: drop pylint (#10640)
* draft

* progress

* more fixes

* improvs

* strict

* rm pylint

* rm pylint comments

* fix and simplify

* standardize

* more

* alpha order

* final touch

* remove template

* rm ellipsis comments
2026-02-24 07:46:20 +00:00
Sebastian Husch Lee 7d4ea2a326 feat: Support using from __future__ import annotations in files that define components (#10547)
* Support usge of from __future__ import annotations by resolving type hints in our calc of __haystack_inputs__

* Add reno

* Prefer inspect signature over type hints. Only use type hints if the annoation isinstance string

* Fix failing tests

* Add unit test

* add license header

* fix
2026-02-11 10:24:20 +01:00
David S. Batista 1257102535 test: add delete_all_documents, delete_by_filter and update_by_filter generic tests to haystack.testing.document_store (#10477)
* feat: add delete_all_documents tests to DeleteDocumentsTest mixin

* adding delete_all_documents to DocumentStoreProtocol

* type ignore since delete_all_documents no part of Protocol

* adding tests for delete_by_filter and update_by_filter

* converting methods to static

* updating tests

* adding test_delete_by_filter_advanced_filters() test

* test_update_by_filter_advanced_filters

* extending tests to support refresh option if present - for ElasticSearch and OpenSearch

* adding release notes

* adding safeguard for DocumentStores that don't implement delete_all_documents

* moving all tests into a new class DocumentStoreBaseExtendedTests

* further drilling down new tests into separate classes

* Update haystack/testing/document_store.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* type ignore

* changing pattern in sig.inspect

* adding skip to delete_all_docs tests based on recreate_index/collection param

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2026-02-05 16:35:20 +01:00
Stefano Fiorucci 7c6323b960 refactor: support PEP604 typing syntax (X|Y and X|None) (#10255)
* adopt X|Y syntax: draft

* cast Union

* fix pylint + state testing

* use X|Y

* rm unused imports

* trigger e2e tests

* fix + simplification

* add compatibility tests

* rm e2e tests trigger

* fix

* add relnote

* simplify/fix pep604 union parsing

* fix comments

* test _is_optional_type

* introduce _build_pep604_union_type; make _is_union_type private

* try removing problematic test
2025-12-19 15:47:51 +01:00
Stefano Fiorucci c05d3f0051 chore: remove unused type:ignore and cast (#9690)
* chore: remove unused type:ignore and casts

* rm unused imports
2025-08-07 15:41:00 +02:00
Abdelrahman Kaseb 5f3c37d287 chore: adopt PEP 585 type hints (#9678)
* chore(lint): enforce and apply PEP 585 type hinting

* Run fmt fixes

* Fix all typing imports using some regex

* Fix all typing written in string in tests

* undo changes in the e2e tests

* make e2e test use list instead of List

* type fixes

* remove type:ignore

* pylint

* Remove typing from Usage example comments

* Remove typing from most of comments

* try to fix e2e tests on comm PRs

* fix

* Add tests typing.List in to adjust test compatiplity
- test/components/agents/test_state_class.py
- test/components/converters/test_output_adapter.py
- test/components/joiners/test_list_joiner.py

* simplify pyproject

* improve relnote

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-08-07 10:23:14 +02:00
Stefano Fiorucci c5cde40d3a unpin ruff and update code (#9040) 2025-03-14 14:53:25 +00:00
Stefano Fiorucci f3c44be904 refactor!: remove dataframe field from Document and ExtractedTableAnswer; make pandas optional (#8906)
* remove dataframe

* release note

* small fix

* group imports

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* address feedback

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-03-04 11:06:07 +00:00
mathislucka 1a91365cc8 fix: callables can be deserialized from fully qualified import path (#8788)
* fix: callables can be deserialized from fully qualified import path

* fix: license header

* fix: format

* fix: types

* fix? types

* test: extend test case

* format

* add release notes
2025-02-03 12:35:37 +01:00
Stefano Fiorucci bc30105fbc test: reorganize docstore test suite to isolate dataframe tests (#8684)
* reorganize docstore test suite to isolate dataframe tests

* improve docstring

* include FilterDocumentsTestWithDataframe in InMemoryDocumentStore tests
2025-01-08 14:58:52 +00:00
David S. Batista 248dccbdd3 chore: fixing pylint issues (#8610)
* initial import

* fixing internal methods

* fixing some internal methods

* modify _preprocess

* fixed internal methods

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-12-09 16:53:37 +00:00
Amna Mubashar 4e6c7967d9 fix: Remove deprecated Legacy Filter Tests (#8567) 2024-11-22 14:39:22 +01:00
Silvano Cerza 8205724395 feat: Rework Pipeline.run() to better handle cycles (#8431)
* draft

* Enhance

* Almost works

* Simplify some parts and handle intermediate outputs

* Handle connections with default

* Handle cycles with multiple connections from two components

* Update distributed outputs at the correct time

* Remove Component inputs after it runs

* Add agent pipeline test case

* Fix infite loop test

* Handle some corner cases with loops checking and inputs deletion

* Fix tests

* Add new behavioral test

* Remove unused code in behavioural test

* Fix behavioural test

* Fix max run check

* Simplify outputs distribution

* Simplify subgraph run check

* Remove unused _init_run_queue function

* Remove commented code

* Add some missing type hints

* Simplify cycles breaking

* Fix _distribute_output test

* Fix _find_components_that_will_receive_no_input test

* Fix validation test

* Fix tracer losing Component inputs

* Fix some linting issues

* Remove ignore pylint rule

* Rename method that break cycles and make it raise

* Add docstring to _run_subgraph

* Update Pipeline.run() docstring

* Update comment to clarify cycles execution

* Remove SelfLoop sample Component

* Add behavioural test for unsupported cycles

* Rename behavioural test to be more specific

* Add new behavioural test

* Add release notes

* Remove commented out code and random pass

* Use more efficient function to find cycles

* Simplify _break_supported_cycles_in_graph by using defaultdict

* Stop breaking edges as soon as we make the graph acyclic

* Fix docstring and add some more comments

* Fix _distribute_output docstring

* Fix _find_receivers_from docstring

* More detailed release notes

* Minimize calls to networkx.is_directed_acyclic_graph

* Add some more info on edges keys

* Adjust components_in_cycles comment

* Add new Pipeline behavioural test

* Enhance _find_components_that_will_receive_no_input to cover more cases

* Explain why run_queue is reset after running a subgraph cycle

* Rename _init_inputs_state to _normalize_input_data

* Better explain the subgraph output distribution

* Remove for else

* Fix some comments and docstrings

* Fix linting

* Add missing return type

* Fix typo

* Rename _normalize_input_data to _normalize_varidiac_input_data and add more documentation

* Remove unused import

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2024-10-29 15:43:16 +01:00
Ajit Singh 6cf13e8b98 enhancement: reduced usage of numpy and substituted built-in libraries (#8418)
* reduced usage of numpy and substituted built-in libraries

* added release note

* edited expit function to support both float as well as list (this case was giving error CI)

* revert code , numpy can't be removed here

* more cleaning

* fix relnote

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-10-18 15:42:19 +02:00
Massimiliano Pippi 3a03fce71c ci: Add code formatting checks (#7882)
* ruff settings

enable ruff format and re-format outdated files

feat: `EvaluationRunResult` add parameter to specify columns to keep in the comparative `Dataframe`  (#7879)

* adding param to explictily state which cols to keep

* adding param to explictily state which cols to keep

* adding param to explictily state which cols to keep

* updating tests

* adding release notes

* Update haystack/evaluation/eval_run_result.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update releasenotes/notes/add-keep-columns-to-EvalRunResult-comparative-be3e15ce45de3e0b.yaml

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* updating docstring

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

add format-check

fail on format and linting failures

fix string formatting

reformat long lines

fix tests

fix typing

linter

pull from main

* reformat

* lint -> check

* lint -> check
2024-06-18 15:52:46 +00:00
Massimiliano Pippi 10c675d534 chore: add license header to all modules (#7675)
* add license header to modules
* check license header at linting time
2024-05-09 13:40:36 +00:00
David S. Batista d9c861e223 Fixing docstrings (#7513)
* wip

* some more docstring fixes

* reverting to make type checker pass
2024-04-09 16:06:56 +02:00
Massimiliano Pippi 174ac79b35 chore: add missing docstrings in document store base tests (#7509)
* add missing docstrings in document store base tests

* also tell pylint to ignore empty docstrings for legacy
2024-04-09 09:25:17 +02:00
Massimiliano Pippi caa67a5fb8 fix docstring for the sample_components package (#7499) 2024-04-09 08:51:22 +02:00
Massimiliano Pippi 47f38c340e Ensure test_comparison_in checks an actual subset of documents (#7427)
* fix test_comparison_in

* relnotes
2024-03-27 17:31:05 +01:00
Tobias Wochinger 655d4a1a8d test: test for missing dependencies (#7278)
* tests: import test for missing libraries

* build: add missing dependencies

* refactor: use glob instead of tree walk

* test: extract constants + more documentation
2024-03-05 12:14:10 +01:00
Tobias Wochinger fe0ac5c4a2 chore: enforce kwarg logging (#7207)
* chore: add logger which eases logging of extras

* chore: start migrating to key value

* fix: import fixes

* tests: temporarily comment out breaking test

* refactor: move to kwarg based logging

* style: fix import order

* chore: implement self-review comments

* test: drop failing test

* chore: fix more import orders

* docs: add changelog

* tests: fix broken tests

* chore: fix getting the frames

* chore: add comment

* chore: cleanup

* chore: adapt remaining `%s` usages
2024-02-29 14:31:20 +01:00
Stefano Fiorucci 380052a963 replace text with content (#7211) 2024-02-28 09:28:14 +01:00
David S. Batista 55785d553b fixing typo (#6967) 2024-02-09 17:18:22 +01:00
Silvano Cerza 0191b1e6e4 feat: Change Component's I/O dunder type (#6916)
* Add Pipeline.get_component_name() method

* Add utility class to ease discoverability of Component I/O

* Move InputOutput in component package

* Rename InputOutput to _InputOutput

* Raise if inputs or outputs field already exist

* Fix tests

* Add release notes

* Move InputSocket and OutputSocket in types package

* Move _InputOutput in socket package

* Rename _InputOutput class to Sockets

* Simplify Sockets class

* Dictch I/O dunder fields in favour of inputs and outputs fields

* Update Sockets docstrings

* Update release notes

* Fix mypy

* Remove unnecessary assignment

* Remove unused logging

* Change SocketsType to SocketsIOType to avoid confusion

* Change sockets type and name

* Change Sockets.__repr__ to return component instance

* Fix linting

* Fix sockets tests

* Revert to dunder fields for Component IO

* Use singular in IO dunder fields

* Delete release notes

* Update haystack/core/component/types.py

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-05 17:46:45 +01:00
Silvano Cerza d4f6531c52 feat: Refactor Pipeline.run() (#6729)
* First rough implementation of refactored run

* Further improve run logic

* Properly handle variadic input in run

* Further work

* Enhance names and add more documentation

* Fix issue with output distribution

* This works

* Enhance run comments

* Mark Multiplexer as greedy

* Remove MergeLoop in favour of Multiplexer in tests

* Remove FirstIntSelector in favour of Multiplexer

* Handle corner when waiting for input is stuck

* Remove unused import

* Handle mutable input data in run and misbehaving components

* Handle run input validation

* Test validation

* Fix pylint

* Fix mypy

* Call warm_up in run to fix tests
2024-01-18 17:53:47 +01:00
Massimiliano Pippi e1ec4e5e4d refact!: Remove symbols under the haystack.document_stores namespace (#6714)
* remove symbols under the haystack.document_stores namespace

* Update haystack/document_stores/types/protocol.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* fix

* same for retrievers

* leftovers

* more leftovers

* add relnote

* leftovers

* one more

* fix examples

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-01-10 21:20:42 +01:00
Stefano Fiorucci 4912f7cb58 refactor!: improve the deserialization logic for components that use a Document Store (#6466)
* improve deserialization

* rm ds decorator

* improve tests

* fix pylint

* rm decorator from module init

* rm decorator

* rm decorator from factory

* fix tests

* release note

* rm print
2023-12-04 15:17:28 +01:00
Massimiliano Pippi 7c05f37a53 remove unit marker (#6450) 2023-11-29 19:24:25 +01:00
Silvano Cerza 3f7cea19ab Reworked Document Store testing comparison to be more versatile (#6442) 2023-11-29 10:57:28 +01:00
Silvano Cerza 831d0611d9 feat: Change default DuplicatePolicy in DocumentStore.write_documents() (#6438)
* Change default DuplicatePolicy in DocumentStore.write_documents()

* Add release notes
2023-11-28 12:30:17 +01:00
Massimiliano Pippi 00e1dd6eb8 chore: rearrange the core package, move tests and clean up (#6427)
* rearrange code

* fix tests

* relnote

* merge test modules

* remove extra

* rearrange draw tests

* forgot

* remove unused import
2023-11-28 09:58:56 +01:00