Commit Graph

452 Commits

Author SHA1 Message Date
Gaudy Blanco 5bf8ce6a56 fix changes 2025-12-05 19:20:50 -06:00
Gaudy Blanco 4877ba675b change implementation for context api 2025-12-05 19:15:52 -06:00
Gaudy Blanco 1eb8fb0cf1 context api implementation 2025-12-05 14:58:54 -06:00
Nathan Evans 20a96cb35e Init command asks for models (#2137)
* Add init prompting for models

* Remove hard-coded model config validation

* Switch to typer option prompt for full CLI use with models

* Update getting started for init model input

* Bump request timeout and overall smoke test timeout
2025-11-24 10:05:47 -08:00
Nathan Evans 7bf82b78ab Add empty checks for NLP graphs (#2133) 2025-11-17 15:45:07 -08:00
Nathan Evans d6e6191d84 Format 2025-11-17 13:59:58 -08:00
gaudyb a4ffc3d34c Remove embeddings optional new (#2128)
* remove optional embeddings

* fix test

* fix tests

* fix pipeline

* fix test

* fix test

* fix test

* fix tests

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-11-17 13:10:54 -06:00
Nathan Evans 4512ce027f Empty graph guards (#2126)
* Remove networkx from graph_extractor and clean out redundancy

* Bubble pipeline error to console
2025-11-11 10:11:08 -08:00
Derek Worthen e0cce31f54 Graphrag config (#2119)
* Add load_config to graphrag-common package.
2025-11-10 07:57:03 -08:00
Nathan Evans ae1f5e1811 Nov 2025 housekeeping (#2120)
* Remove gensim sideload

* Split CI build/type checks from unit tests

* Thorough review of docs to align with v3

* Format

* Fix version

* Fix type
2025-11-06 10:03:22 -08:00
Nathan Evans 6033e4ffa2 Storage fixes and cleanup (#2118)
* Fix pipeline recursion

* Remove base_dir from storage.find

* Remove max_count from storage.find

* Remove prefix on storage integ test

* Add base_dir in creation_date test

* Wrap base_dir in Path

* Use constants for input/update directories
2025-11-05 13:06:09 -08:00
Nathan Evans 6b03af6277 Fix formatting 2025-11-04 10:48:09 -08:00
Derek Worthen 619269243d Restructure project as monorepo. (#2111)
* Restructure project as monorepo.
2025-11-04 09:51:56 -08:00
Gaudy Blanco c43a58c353 fix for container name 2025-10-23 10:40:21 -06:00
gaudyb 542d3db2a9 Prefix vector store (#2106)
* add prefix to vector store configuration and removal of container name

* docs updated

* change prefix property name

* change prefix property name

* feedback implemented

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-10-21 15:30:50 -06:00
Nathan Evans 1bb9fa8e13 Unified factory (#2105)
* Simplify Factory interface

* Migrate CacheFactory to standard base class

* Migrate LoggerFactory to standard base class

* Migrate StorageFactory to standard base class

* Migrate VectorStoreFactory to standard base class

* Update vector store example notebook

* Delete notebook outputs

* Move default providers into factories

* Move retry/limit tests into integ

* Split language model factories

* Set smoke test tpm/rpm

* Fix factory integ tests

* Add method to smoke test, switch text to 'fast'

* Fix text smoke config for fast workflow

* Add new workflows to text smoke test

* Convert input readers to a proper factory

* Remove covariates from fast smoke test

* Update docs for input factory

* Bump smoke runtime

* Even longer runtime

* min-csv timeout

* Remove unnecessary lambdas
2025-10-20 12:05:27 -07:00
gaudyb 0436405962 Remove document overwrite (#2101)
* remove document overwrite from vector store configuration

* remove document overwrite and refactor load documents method

* fix test

* fix test

* fix test

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-10-16 07:56:54 -06:00
Nathan Evans 5ec49fd39c V3 docs and cleanup (#2100)
* Remove community contrib notebooks

* Add migration notebook and breaking changes page edits

* Update/polish docs

* Make model instance name configurable

* Add vector schema updates to v3 migration notebook

* Spellcheck

* Bump smoke test runtimes
2025-10-15 13:47:19 -07:00
Nathan Evans b732445535 Remove multi search (#2093)
* Remove multi-search from CLI

* Remove multi-search from API

* Flatten vector_store config

* Push hydrated vector store down to embed_text

* Remove outputs from config

* Remove multi-search notebook/docs

* Add missing response_type in basic search API

* Fix basic search context and id mapping

* Fix v1 migration notebook

* Fix query entity search tests
2025-10-10 17:20:53 -07:00
Nathan Evans 715be61404 Sort deps alpha 2025-10-10 17:10:42 -07:00
Nathan Evans f7a8a0888d Merge branch 'main' into v3/main 2025-10-10 17:03:52 -07:00
Nathan Evans 6284cdd110 Remove fnllm (#2095) 2025-10-10 16:59:25 -07:00
Nathan Evans eb0dfe376b Remove strategy dicts (#2090)
* Remove "strategy" from community reports config/workflow

* Remove extraction strategy from extract_graph

* Remove summarization strategy from extract_graph

* Remove strategy from claim extraction

* Strongly type prompt templates

* Remove strategy from embed_text

* Push hydrated params into community report workflows

* Push hyrdated params into extract covariates

* Push hydrated params into extract graph NLP

* Push hydrated params into extract graph

* Push hydrated params into text embeddings

* Remove a few more low-level defaults

* Semver

* Remove configurable prompt delimiters

* Update smoke tests
2025-10-10 12:15:23 -07:00
gaudyb 79ad9b96f3 reduce schema fields (#2089)
* reduce schema fields

* fix launch.json

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-10-09 13:41:31 -06:00
Alonso Guevara fdb7e3835b Release v2.7.0 (#2087) v2.7.0 2025-10-08 21:33:34 -07:00
Nathan Evans 2b5284ca1b Merge branch 'main' into v3/main 2025-10-07 16:24:15 -07:00
Nathan Evans ac8a7f5eef Housekeeping (#2086)
* Add deprecation warnings for fnllm and multi-search

* Fix dangling token_encoder refs

* Fix local_search notebook

* Fix global search dynamic notebook

* Fix global search notebook

* Fix drift notebook

* Switch example notebooks to use LiteLLM config

* Properly annotate dev deps as a group

* Semver

* Remove --extra dev

* Remove llm_model variable

* Ignore ruff ASYNC240

* Add note about expected broken notebook in docs

* Fix custom vector store notebook

* Push tokenizer throughout
2025-10-07 16:21:24 -07:00
Nathan Evans 4364d678dd Merge branch 'main' into v3/main 2025-10-06 12:41:12 -07:00
Nathan Evans 6c86b0a7bb Init config cleanup (#2084)
* Spruce up init_config output, including LiteLLM default

* Remove deployment_name requirement for Azure

* Semver

* Add model_provider

* Add default model_provider

* Remove OBE test

* Update minimal config for tests

* Add model_provider to verb tests
2025-10-06 12:06:41 -07:00
Nathan Evans 2bd3922d8d Litellm auth fix (#2083)
* Fix scope for Azure auth with LiteLLM

* Change internal language on max_attempts to max_retries

* Rework model config connectivity validation

* Semver

* Swtich smoke tests to LiteLLM

* Take out temporary retry_strategy = none since it is not fnllm compatible

* Bump smoke test timeout

* Bump smoke timeout further

* Tune smoke params

* Update smoke test bounds

* Remove covariates from min-csv smoke

* Smoke: adjust communities, remove drift

* Remove secrets where they aren't necessary

* Clean out old env var references
2025-10-06 10:54:21 -07:00
Andres Morales ebe959a9db Update v3/main missing config + functions (#2082) 2025-09-30 15:48:40 -06:00
gaudyb d7773bd15c Clean vector store (#2077)
* clean vector store code

* fix

* fix launch.json

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-09-25 21:17:10 -06:00
Nathan Evans d7516825e2 Update comment 2025-09-24 14:51:07 -07:00
Nathan Evans de767cc881 Use 4.1 and text-embedding-3-large as defaults 2025-09-24 14:49:55 -07:00
Nathan Evans 7f996cf584 Docs/2.6.0 (#2070)
* Add basic search to overview

* Add info on input documents DataFrame

* Add info on factories to docs

* Add consumption warning and switch to "christmas" for folder name

* Add logger to factories list

* Add litellm docs. (#2058)

* Fix version for input docs

* Spelling

---------

Co-authored-by: Derek Worthen <worthend.derek@gmail.com>
2025-09-23 14:48:28 -07:00
Nathan Evans b73053010e Merge branch 'main' into v3/main 2025-09-23 11:07:42 -07:00
Alonso Guevara 9bc899fe95 Release v2.6.0 (#2068) v2.6.0 2025-09-22 16:16:54 -06:00
Derek Worthen 2b70e4a4f3 Tokenizer (#2051)
* Add LiteLLM chat and embedding model providers.

* Fix code review findings.

* Add litellm.

* Fix formatting.

* Update dictionary.

* Update litellm.

* Fix embedding.

* Remove manual use of tiktoken and replace with
Tokenizer interface. Adds support for encoding
and decoding the models supported by litellm.

* Update litellm.

* Configure litellm to drop unsupported params.

* Cleanup semversioner release notes.

* Add num_tokens util to Tokenizer interface.

* Update litellm service factories.

* Cleanup litellm chat/embedding model argument assignment.

* Update chat and embedding type field for litellm use and future migration away from fnllm.

* Flatten litellm service organization.

* Update litellm.

* Update litellm factory validation.

* Flatten litellm rate limit service organization.

* Update rate limiter - disable with None/null instead of 0.

* Fix usage of get_tokenizer.

* Update litellm service registrations.

* Add jitter to exponential retry.

* Update validation.

* Update validation.

* Add litellm request logging layer.

* Update cache key.

* Update defaults.

---------

Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
2025-09-22 13:55:14 -06:00
gaudyb 82cd3b7df2 Custom vector store schema implementation (#2062)
* progress on vector customization

* fix for lancedb vectors

* cosmosdb implementation

* uv run poe format

* clean test for vector store

* semversioner update

* test_factory.py integration test fixes

* fixes for cosmosdb test

* integration test fix for lancedb

* uv fix for format

* test fixes

* fixes for tests

* fix cosmosdb bug

* print statement

* test

* test

* fix cosmosdb bug

* test validation

* validation cosmosdb

* validate cosmosdb

* fix cosmosdb

* fix small feedback from PR

---------

Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
2025-09-19 10:11:34 -07:00
Nathan Evans 075cadd59a Clarify managed auth setup in Azure documentation (#2064)
Updated instructions for using managed auth on Azure.
2025-09-18 14:58:09 -07:00
Nathan Evans 6d7a50b7f0 Remove community reports rate limiter (#2056)
* Remove hard-coded community reports rate limiter

* Semver

* Format

* Add memory cache factory
2025-09-18 13:40:24 -07:00
Nathan Evans 2bf7e7c018 Fix multi-index search (#2063) 2025-09-18 12:49:56 -07:00
Nathan Evans 84e600871e Merge branch 'main' into v3/main 2025-09-17 16:05:28 -07:00
Nathan Evans 6c66b7c30f Configure async for NLP extraction (#2059)
* Make async mode configurable for NLP extraction

* Semver
2025-09-16 11:52:18 -07:00
Nathan Evans 8720accf0b Merge branch 'main' into v3/main 2025-09-15 12:09:08 -07:00
Nathan Evans 04d9f585c8 Re-implement hierarchical Leiden (#2049)
* Use graspologic-native hierarchical leiden

* Re-implement largest_connected_component

* Copy in modularity

* Use graspologic-native directly in pyproject

* Remove directed graph tests (we don't use this)

* Semver

* Remove graspologic dep
2025-09-09 16:26:07 -07:00
Nathan Evans 97704ab933 Remove text unit grouping (#2052)
* Remove text unit group_by_columns

* Semver

* Fix default token split test

* Fix models in config test samples

* Fix token length in context sort test

* Fix document sort
2025-09-09 16:04:04 -07:00
Chenghua Duan a398cc38bb Update command to use no-discover-entity-types (#2038)
"no-entity-types" is an incorrect configuration parameter.

Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
2025-09-09 16:46:06 -06:00
Nathan Evans 978e79875e Remove file filtering (#2050)
* Remove document filtering

* Semver

* Fix integ tests

* Fix file find tuple

* Fix another dangling find tuple
2025-09-09 15:36:25 -07:00
Nathan Evans 429e1b1f9e Remove graph embedding and UMAP (#2048)
* Remove umap/layout operation

* Remove graph embedding

* Bump unified-search to GR 2.5.0

* Remove graph vis from unified-search
2025-09-09 15:35:43 -07:00