* Sync docs for schema and metadata fields
* Clarify prepend_metadata raw_data note
* Align docs notebooks with v3 model APIs
* Clean notebook formatting in v3 model cells
* Use clean global_search params without pop workaround
* Fix dynamic global search notebook JSON mode param conflict
* Remove unused imports in docs drift notebook
* Restore drift notebook data setup variables
* Apply ruff format to drift notebook
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* remove outputs
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Move document ID, human_readable_id, and raw_data setup from create_final_documents into load workflows
Consolidates core document field initialization (id string cast, human_readable_id index, raw_data default) into load_input_documents and load_update_documents so that create_final_documents only handles the text unit join. Also applies the same setup in the run_pipeline input_documents bypass paths.
* Remove overzealous input document assignment
* Semver
* Format
* add notebook example support for each package
* add notebook example support for each package
* semversioner change
* feedback implemented for notebooks
* feedback implemented for notebooks
* feedback implemented for notebooks
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Remove graph embedding and UMAP (#2048)
* Remove umap/layout operation
* Remove graph embedding
* Bump unified-search to GR 2.5.0
* Remove graph vis from unified-search
* Remove file filtering (#2050)
* Remove document filtering
* Semver
* Fix integ tests
* Fix file find tuple
* Fix another dangling find tuple
* Remove text unit grouping (#2052)
* Remove text unit group_by_columns
* Semver
* Fix default token split test
* Fix models in config test samples
* Fix token length in context sort test
* Fix document sort
* Re-implement hierarchical Leiden (#2049)
* Use graspologic-native hierarchical leiden
* Re-implement largest_connected_component
* Copy in modularity
* Use graspologic-native directly in pyproject
* Remove directed graph tests (we don't use this)
* Semver
* Remove graspologic dep
* Use 4.1 and text-embedding-3-large as defaults
* Update comment
* Clean vector store (#2077)
* clean vector store code
* fix
* fix launch.json
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Update v3/main missing config + functions (#2082)
* reduce schema fields (#2089)
* reduce schema fields
* fix launch.json
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Remove strategy dicts (#2090)
* Remove "strategy" from community reports config/workflow
* Remove extraction strategy from extract_graph
* Remove summarization strategy from extract_graph
* Remove strategy from claim extraction
* Strongly type prompt templates
* Remove strategy from embed_text
* Push hydrated params into community report workflows
* Push hyrdated params into extract covariates
* Push hydrated params into extract graph NLP
* Push hydrated params into extract graph
* Push hydrated params into text embeddings
* Remove a few more low-level defaults
* Semver
* Remove configurable prompt delimiters
* Update smoke tests
* Remove fnllm (#2095)
* Sort deps alpha
* Remove multi search (#2093)
* Remove multi-search from CLI
* Remove multi-search from API
* Flatten vector_store config
* Push hydrated vector store down to embed_text
* Remove outputs from config
* Remove multi-search notebook/docs
* Add missing response_type in basic search API
* Fix basic search context and id mapping
* Fix v1 migration notebook
* Fix query entity search tests
* V3 docs and cleanup (#2100)
* Remove community contrib notebooks
* Add migration notebook and breaking changes page edits
* Update/polish docs
* Make model instance name configurable
* Add vector schema updates to v3 migration notebook
* Spellcheck
* Bump smoke test runtimes
* Remove document overwrite (#2101)
* remove document overwrite from vector store configuration
* remove document overwrite and refactor load documents method
* fix test
* fix test
* fix test
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Unified factory (#2105)
* Simplify Factory interface
* Migrate CacheFactory to standard base class
* Migrate LoggerFactory to standard base class
* Migrate StorageFactory to standard base class
* Migrate VectorStoreFactory to standard base class
* Update vector store example notebook
* Delete notebook outputs
* Move default providers into factories
* Move retry/limit tests into integ
* Split language model factories
* Set smoke test tpm/rpm
* Fix factory integ tests
* Add method to smoke test, switch text to 'fast'
* Fix text smoke config for fast workflow
* Add new workflows to text smoke test
* Convert input readers to a proper factory
* Remove covariates from fast smoke test
* Update docs for input factory
* Bump smoke runtime
* Even longer runtime
* min-csv timeout
* Remove unnecessary lambdas
* Prefix vector store (#2106)
* add prefix to vector store configuration and removal of container name
* docs updated
* change prefix property name
* change prefix property name
* feedback implemented
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* fix for container name
* Restructure project as monorepo. (#2111)
* Restructure project as monorepo.
* Fix formatting
* Storage fixes and cleanup (#2118)
* Fix pipeline recursion
* Remove base_dir from storage.find
* Remove max_count from storage.find
* Remove prefix on storage integ test
* Add base_dir in creation_date test
* Wrap base_dir in Path
* Use constants for input/update directories
* Nov 2025 housekeeping (#2120)
* Remove gensim sideload
* Split CI build/type checks from unit tests
* Thorough review of docs to align with v3
* Format
* Fix version
* Fix type
* Graphrag config (#2119)
* Add load_config to graphrag-common package.
* Empty graph guards (#2126)
* Remove networkx from graph_extractor and clean out redundancy
* Bubble pipeline error to console
* Remove embeddings optional new (#2128)
* remove optional embeddings
* fix test
* fix tests
* fix pipeline
* fix test
* fix test
* fix test
* fix tests
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Format
* Add empty checks for NLP graphs (#2133)
* Init command asks for models (#2137)
* Add init prompting for models
* Remove hard-coded model config validation
* Switch to typer option prompt for full CLI use with models
* Update getting started for init model input
* Bump request timeout and overall smoke test timeout
* Add graphrag-storage. (#2127)
* Add graphrag-storage.
* Python update (3.13) (#2149)
* Update to python 3.14 as default, with range down to 3.10
* Fix enum value in query cli
* Update pyarrow
* Update py version for storage package
* Remove 3.10
* add fastuuid
* Update Python support to 3.11-3.14 with stricter dependency constraints
- Set minimum Python version to 3.11 (removed 3.10 support)
- Added support for Python 3.14
- Updated CI workflows: single-version jobs use 3.14, matrix jobs use 3.11 and 3.14
- Fixed license format to use SPDX-compatible format for Python 3.14
- Updated pyarrow to >=22.0.0 for Python 3.14 wheel support
- Added explicit fastuuid~=0.14 and blis~=1.3 for Python 3.14 compatibility
- Replaced all loose version constraints (>=) with compatible release (~=) for better lock file control
- Applied stricter versioning to all packages: graphrag, graphrag-common, graphrag-storage, unified-search-app
* update uv lock
* Pin blis to ~=1.3.3 to ensure Python 3.14 wheel availability
* Update uv lock
* Update numpy to >=2.0.0 for Python 3.14 Windows compatibility
Numpy 1.25.x has access violation issues on Python 3.14 Windows.
Numpy 2.x has proper Python 3.14 support including Windows wheels.
* update uv lock
* Update pandas to >=2.3.0 for numpy 2.x compatibility
Pandas 2.2.x was compiled against numpy 1.x and causes ABI
incompatibility errors with numpy 2.x. Pandas 2.3.0+ supports
numpy 2.x properly.
* update uv.lock
* Add scipy>=1.15.0 for numpy 2.x compatibility
Scipy versions < 1.15.0 have C extensions built against numpy 1.x
and are incompatible with numpy 2.x, causing dtype size errors.
* update uv lock
* Update Python support to 3.11-3.13 with compatible dependencies
- Set Python version range to 3.11-3.13 (removed 3.14 support)
- Updated CI workflows: single-version jobs use 3.13, matrix jobs use 3.11 and 3.13
- Dependencies optimized for Python 3.13 compatibility:
- pyarrow~=22.0 (has Python 3.13 wheels)
- numpy~=1.26
- pandas~=2.2
- blis~=1.0
- fastuuid~=0.13
- Applied stricter version constraints using ~= operator throughout
- Updated uv.lock with resolved dependencies
* Update numpy to 2.1+ and pandas to 2.3+ for Python 3.13 Windows compatibility
Numpy 1.26.x causes access violations on Python 3.13 Windows.
Numpy 2.1+ has proper Python 3.13 support with Windows wheels.
Pandas 2.3+ is required for numpy 2.x compatibility.
* update vsts.yml python version
* Add GraphRAG Cache package. (#2153)
* Add GraphRAG Cache package.
* Fix a bunch of module comments and function visibility (#2154)
* Issue #2004 fix (#2159)
* fix issue #2004 using KeenhoChu idea in his PR
* add unit test for dynamic community selection
* add unit test for dynamic community selection implementing #2158 logic
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Mismatch between header in community report generation prompt examples and input data (id vs human_readable_id) (#2161)
* fix issue #860 for mismatch in prompts and input
* fix format
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Chunker factory (#2156)
* Delete NoopTextSplitter
* Delete unused check_token_limit
* Add base chunking factory and migrate workflow to use it
* Split apart chunker module
* Co-locate chunking/splitting
* Collapse token splitting functionality into one class/function
* Restore create_base_text_units parameterization
* Move Tokenizer base class to common package
* Move pre-pending into chunkers
* Streamline config
* Fix defaults construction
* Add prepending tests
* Remove chunk_size_includes_metadata config
* Revert ChunkingDocument interface
* Move metadata prepending to a util
* Move Tokenizer back to GR core
* Fix tokenizer removal from chunker
* Set defaults for chunking config
* Move chunking to monorepo package
* Format
* Typo
* Add ChunkResult model
* Streamline chunking config
* Add missing version updates for graphrag_chunking
* Input factory (#2168)
* Update input factory to match other factories
* Move input config alongside input readers
* Move file pattern logic into InputReader
* Set encoding default
* Clean up optional column configs
* Combine structured data extraction
* Remove pandas from input loading
* Throw if empty documents
* Add json lines (jsonl) input support
* Store raw data
* Fix merge imports
* Move metadata handling entirely to chunking
* Nicer automatic title
* Typo
* Add get_property utility for nested dictionary access with dot notation
* Update structured_file_reader to use get_property utility
* Extract input module into new graphrag-input monorepo package
- Create new graphrag-input package with input loading utilities
- Move InputConfig, InputFileType, InputReader, TextDocument, and file readers (CSV, JSON, JSONL, Text)
- Add get_property utility for nested dictionary access with dot notation
- Include hashing utility for document ID generation
- Update all imports throughout codebase to use graphrag_input
- Add package to workspace configuration and release tasks
- Remove old graphrag.index.input module
* Rename ChunkResult to TextChunk and add transformer support
- Rename chunk_result.py to text_chunk.py with ChunkResult -> TextChunk
- Add 'original' field to TextChunk to track pre-transform text
- Add optional transform callback to chunker.chunk() method
- Add add_metadata transformer for prepending metadata to chunks
- Update create_chunk_results to apply transforms and populate original
- Update sentence_chunker and token_chunker with transform support
- Refactor create_base_text_units to use new transformer pattern
- Rename pluck_metadata to get/collect methods on TextDocument
* Back-compat comment
* Align input config type name with other factory configs
* Add MarkItDown support
* Remove pattern default from MarkItDown reader
* Remove plugins flag (implicit disabled)
* Format
* Update verb tests
* Separate storage from input config
* Add empty objects for NaN raw_data
* Fix smoke tests
* Fix BOM in csv smoke
* Format
* DRIFT fixes (#2171)
* Use stable ids for community reports
* Remove deprecated title from embedding flow
* Remove embedding column from df loaders
* Fix lancedb insertion
* Add drift back to smoke tests
* Fix mock embedder to match default embedding length
* Fix DRIFT notebook
* Push drift_k_followups through to prompt
* Format
* Vector package (#2172)
* Extract graphrag-vectors package
* Simplify vector factory usage and config defaults
* Update factory integ initializers
* Fix mock patch
* Format
* Register vector stores in tests
* Set a default vector store name
* Update vector readme
* Remove impls from init
* Move some validation into impls
* Remove index_prefix
* Move duplicate method to base class
* Fix smoke vector config
* Update index bug (#2173)
* fix update index bug
* blob storage bug fix
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Add GraphRAG LLM package. (#2174)
* Update documentation for v3 release (#2176)
update documentation for v3 release
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Graphrag llm cleanup (#2181)
* Migration update (#2180)
* fix formatting.
---------
Co-authored-by: Nathan Evans <github@talkswithnumbers.com>
Co-authored-by: gaudyb <85708998+gaudyb@users.noreply.github.com>
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
Co-authored-by: Andres Morales <86074752+andresmor-ms@users.noreply.github.com>
* Fix scope for Azure auth with LiteLLM
* Change internal language on max_attempts to max_retries
* Rework model config connectivity validation
* Semver
* Swtich smoke tests to LiteLLM
* Take out temporary retry_strategy = none since it is not fnllm compatible
* Bump smoke test timeout
* Bump smoke timeout further
* Tune smoke params
* Update smoke test bounds
* Remove covariates from min-csv smoke
* Smoke: adjust communities, remove drift
* Remove secrets where they aren't necessary
* Clean out old env var references
* Add basic search to overview
* Add info on input documents DataFrame
* Add info on factories to docs
* Add consumption warning and switch to "christmas" for folder name
* Add logger to factories list
* Add litellm docs. (#2058)
* Fix version for input docs
* Spelling
---------
Co-authored-by: Derek Worthen <worthend.derek@gmail.com>
* Add models page
* Update config docs for new params
* Spelling
* Add comment on CoT with o-series
* Add notes about managed identity
* Update the viz guide
* Spruce up the getting started wording
* Capitalization
* Add BYOG page
* More BYOG edits
* Update dictionary
* Change example model name
* Update API overview
* Fix global search example
* Fix local search example
* Fix global dynamic example
* Fix drift example
* Update multi-index example
* Semver
* Add children to the community tables
* Replace NaN children with empty list
* Replace subcommunity logic with built-in parent/child fields
* Remove restore_community_hierarchy
* Add children and frequency to migration notebook
* Format
* Semver
* Add children to reports
* Update tests
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* consolidate query api functions and remove code duplication
* refactor and remove more code duplication
* Add semversioner file
* fix basic search
* fix drift search and update base class function names
* update example notebooks
Corrected a missing backtick in a note within the `GRAPHRAG_API_KEY` description. This ensures proper code formatting and improves readability in the documentation. No content was altered aside from formatting adjustments.
Co-authored-by: Nathan Evans <github@talkswithnumbers.com>
Updated the auto prompt tuning doc with `--selection-method` instead of only `--method` as per the latest API.
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Refactor config
- Add new ModelConfig to represent LLM settings
- Combines LLMParameters, ParallelizationParameters, encoding_model, and async_mode
- Add top level models config that is a list of available LLM ModelConfigs
- Remove LLMConfig inheritance and delete LLMConfig
- Replace the inheritance with a model_id reference to the ModelConfig listed in the top level models config
- Remove all fallbacks and hydration logic from create_graphrag_config
- This removes the automatic env variable overrides
- Support env variables within config files using Templating
- This requires "$" to be escaped with extra "$" so ".*\\.txt$" becomes ".*\\.txt$$"
- Update init content to initialize new config file with the ModelConfig structure
* Use dict of ModelConfig instead of list
* Add model validations and unit tests
* Fix ruff checks
* Add semversioner change
* Fix unit tests
* validate root_dir in pydantic model
* Rename ModelConfig to LanguageModelConfig
* Rename ModelConfigMissingError to LanguageModelConfigMissingError
* Add validationg for unexpected API keys
* Allow skipping pydantic validation for testing/mocking purposes.
* Add default lm configs to verb tests
* smoke test
* remove config from flows to fix llm arg mapping
* Fix embedding llm arg mapping
* Remove timestamp from smoke test outputs
* Remove unused "subworkflows" smoke test properties
* Add models to smoke test configs
* Update smoke test output path
* Send logs to logs folder
* Fix output path
* Fix csv test file pattern
* Update placeholder
* Format
* Instantiate default model configs
* Fix unit tests for config defaults
* Fix migration notebook
* Remove create_pipeline_config
* Remove several unused config models
* Remove indexing embedding and input configs
* Move embeddings function to config
* Remove skip_workflows
* Remove skip embeddings in favor of explicit naming
* fix unit test spelling mistake
* self.models[model_id] is already a language model. Remove redundant casting.
* update validation errors to instruct users to rerun graphrag init
* instantiate LanguageModelConfigs with validation
* skip validation in unit tests
* update verb tests to use default model settings instead of skipping validation
* test using llm settings
* cleanup verb tests
* remove unsafe default model config
* remove the ability to skip pydantic validation
* remove None union types when default values are set
* move vector_store from embeddings to top level of config and delete resolve_paths
* update vector store settings
* fix vector store and smoke tests
* fix serializing vector_store settings
* fix vector_store usage
* fix vector_store type
* support cli overrides for loading graphrag config
* rename storage to output
* Add --force flag to init
* Remove run_id and resume, fix Drift config assignment
* Ruff
---------
Co-authored-by: Nathan Evans <github@talkswithnumbers.com>
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Add new inputs and missing vector store for retrieving vectors
* Format
* Semver
* Remove .Identifier files
* Fix spellcheck
* Remove unnecessary input file for notebooks
* update index api to accept callbacks
* fix hardcoded folder name that was creating an empty folder
* add API notebook
* add semversioner file
* filename change
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Fix local question gen and example notebook
* Update global search notebook
* Add lazy blog post
* Update breaking changes doc for migration notes
* Simplify Getting Started page
* Semver
* Spellcheck
* Fix types
* Add comments on cache-free migration
* Update wording
* Spelling
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
Derek Worthen