* Fix JSONL loader for blank and invalid rows
* Sync docs for schema and metadata fields
* Revert "Sync docs for schema and metadata fields"
This reverts commit 55f170f7d25f58ba4494f2dee8c5690fd282c778.
* feat(graphrag-vectors): add filtering, timestamps, and CRUD operations
Implement the vector store enhancements from the graphrag-vectors-design spec:
New modules:
- filtering.py: Pydantic-based filter expression system with F builder,
operator overloads, JSON serialization, client-side evaluate(), and
per-backend compilation (SQL for LanceDB/CosmosDB, OData for Azure AI Search)
- timestamp.py: ISO 8601 timestamp explosion into filterable component fields
Enhanced VectorStoreDocument:
- data: dict for user-defined metadata fields
- create_date / update_date: automatic ISO 8601 timestamps
Enhanced VectorStore base class:
- fields config for typed metadata columns
- insert / count / remove / update CRUD methods
- select, filters, include_vectors params on search methods
- Automatic timestamp explosion on insert/update
- User-defined date field explosion
Backend implementations (LanceDB, Azure AI Search, CosmosDB):
- Full filter compilation to native query languages
- Typed schema creation with user-defined fields
- All new CRUD operations
Breaking changes:
- search_by_id raises IndexError when document not found
- Updated indexer_adapters.py caller to handle the new exception
Tests:
- 54 unit tests for filtering and timestamp modules
- 28 LanceDB integration tests covering CRUD, filters, timestamps, select,
include_vectors, and user-defined date field explosion
* fix: resolve CI build failures (formatting, lint, pyright, test mocks)
- Fix ruff formatting and lint errors across all changed files
- Refactor filtering.py: move operator overloads from monkey-patching to
direct class methods for pyright visibility
- Use validation_alias/serialization_alias with populate_by_name for
Pydantic AND/OR/NOT models (pyright + runtime compatible)
- Use Operator enum members instead of string literals in FieldRef
- Add missing abstract methods (insert, count, remove, update) to test
mock VectorStore classes
- Update mock method signatures to match base class (select, filters,
include_vectors params)
- Add docstrings to FieldRef magic methods (ruff D105)
- Fix noqa:S608 placement in cosmosdb.py
* feat: add top-level vector_size to VectorStoreConfig
Add a vector_size field (default 3072) to VectorStoreConfig so users
can set it once instead of on every individual index schema. The value
is propagated to new IndexSchema entries during validation.
* chore: add semversioner patch entry
* chore: add ismatch and ftype to spellcheck dictionary
* Add example notebooks for LanceDB, Azure AI Search, and CosmosDB vector stores
- Three notebooks demonstrating: document loading, similarity search, metadata
filtering with F builder, timestamp filtering, document update/removal
- Sample data files (text_units.parquet, embeddings.text_unit_text.parquet)
- Add CPY001, SLF001, DTZ005 to notebook lint ignores in pyproject.toml
* refactor: extract model/tokenizer creation from generate_text_embeddings into callers
* Replace NX-based compute_degree with DataFrame-only implementation
- Add graphrag.graphs package with compute_degree operating directly on
relationships DataFrames instead of building NetworkX graphs
- Update finalize_entities and finalize_relationships to use the new
utility, eliminating NX graph construction in those paths
- Remove the old compute_degree operation from index/operations
- Add side-by-side tests validating parity with NetworkX degree output
* Add DataFrame-based connected components and LCC utilities
- Add connected_components and largest_connected_component to
graphrag.graphs using union-find on edge list DataFrames
- Fix compute_degree to normalise edge direction so (A,B) and (B,A)
are treated as the same undirected edge
- Replace NX largest_connected_component in prune_graph operation with
the new DataFrame utility via graph_to_dataframes
- Add realistic A Christmas Carol graph fixture (529 nodes, 978 edges)
converted from verb test parquet data
- Add side-by-side tests for connected components and fixture-based
test for compute_degree, all validated against NetworkX
* Add DataFrame-based stable LCC utility with side-by-side tests
* Wire stable_lcc into cluster_graph, replacing NX stable_largest_connected_component
* Remove NetworkX from clustering pipeline
- cluster_graph now accepts a DataFrame instead of nx.Graph
- hierarchical_leiden now accepts list[tuple[str, str, float]] edge list
- create_communities passes relationships DataFrame directly, removing
create_graph dependency
- Edge direction normalization and deduplication (keep='last') replaces
implicit NX dedup behavior
- Modularity helper callers convert to edge list via _nx_to_edge_list
* Remove NetworkX from prune_graph
- prune_graph operation now accepts (entities, relationships) DataFrames
instead of nx.Graph, returns pruned DataFrames directly
- Uses compute_degree for degree calculation, largest_connected_component
for LCC filtering — no NetworkX
- Workflow no longer round-trips through create_graph/graph_to_dataframes
- Reset index on returned DataFrames to avoid downstream alignment errors
* Move old NX utilities out of production code
- Move stable_lcc (NX version) to tests/unit/graphs/nx_stable_lcc.py
for side-by-side comparison tests only
- Delete graph_to_dataframes.py (dead code, zero imports)
- Update test imports to use the new test helper location
* Delete create_graph, inline into snapshot_graphml
- snapshot_graphml now accepts edges DataFrame directly and calls
nx.from_pandas_edgelist internally
- finalize_graph workflow passes relationships DataFrame to snapshot
- Removed create_graph.py (no remaining callers)
* Move graph utilities from index/utils/graphs.py to graphrag/graphs/ modules
- hierarchical_leiden, first/final_level_hierarchical_clustering → graphs/hierarchical_leiden.py
- calculate_pmi/rrf_edge_weights → graphs/edge_weights.py
- calculate_* modularity functions, _df_to_edge_list → graphs/modularity.py
- NX-based modularity/LCC/edge-list helpers removed (replaced by DF-based equivalents)
- Delete index/utils/graphs.py (no remaining callers)
- Update cluster_graph.py and build_noun_graph.py to import from new locations
- Inline NX largest_connected_component into test helper nx_stable_lcc.py
- Add side-by-side modularity tests (9 tests comparing DF vs NX)
* Add semversioner patch for NetworkX removal
* Spelling
* Spelling config
* Fix British English spellings to American English
* Remove graph embedding and UMAP (#2048)
* Remove umap/layout operation
* Remove graph embedding
* Bump unified-search to GR 2.5.0
* Remove graph vis from unified-search
* Remove file filtering (#2050)
* Remove document filtering
* Semver
* Fix integ tests
* Fix file find tuple
* Fix another dangling find tuple
* Remove text unit grouping (#2052)
* Remove text unit group_by_columns
* Semver
* Fix default token split test
* Fix models in config test samples
* Fix token length in context sort test
* Fix document sort
* Re-implement hierarchical Leiden (#2049)
* Use graspologic-native hierarchical leiden
* Re-implement largest_connected_component
* Copy in modularity
* Use graspologic-native directly in pyproject
* Remove directed graph tests (we don't use this)
* Semver
* Remove graspologic dep
* Use 4.1 and text-embedding-3-large as defaults
* Update comment
* Clean vector store (#2077)
* clean vector store code
* fix
* fix launch.json
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Update v3/main missing config + functions (#2082)
* reduce schema fields (#2089)
* reduce schema fields
* fix launch.json
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Remove strategy dicts (#2090)
* Remove "strategy" from community reports config/workflow
* Remove extraction strategy from extract_graph
* Remove summarization strategy from extract_graph
* Remove strategy from claim extraction
* Strongly type prompt templates
* Remove strategy from embed_text
* Push hydrated params into community report workflows
* Push hyrdated params into extract covariates
* Push hydrated params into extract graph NLP
* Push hydrated params into extract graph
* Push hydrated params into text embeddings
* Remove a few more low-level defaults
* Semver
* Remove configurable prompt delimiters
* Update smoke tests
* Remove fnllm (#2095)
* Sort deps alpha
* Remove multi search (#2093)
* Remove multi-search from CLI
* Remove multi-search from API
* Flatten vector_store config
* Push hydrated vector store down to embed_text
* Remove outputs from config
* Remove multi-search notebook/docs
* Add missing response_type in basic search API
* Fix basic search context and id mapping
* Fix v1 migration notebook
* Fix query entity search tests
* V3 docs and cleanup (#2100)
* Remove community contrib notebooks
* Add migration notebook and breaking changes page edits
* Update/polish docs
* Make model instance name configurable
* Add vector schema updates to v3 migration notebook
* Spellcheck
* Bump smoke test runtimes
* Remove document overwrite (#2101)
* remove document overwrite from vector store configuration
* remove document overwrite and refactor load documents method
* fix test
* fix test
* fix test
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Unified factory (#2105)
* Simplify Factory interface
* Migrate CacheFactory to standard base class
* Migrate LoggerFactory to standard base class
* Migrate StorageFactory to standard base class
* Migrate VectorStoreFactory to standard base class
* Update vector store example notebook
* Delete notebook outputs
* Move default providers into factories
* Move retry/limit tests into integ
* Split language model factories
* Set smoke test tpm/rpm
* Fix factory integ tests
* Add method to smoke test, switch text to 'fast'
* Fix text smoke config for fast workflow
* Add new workflows to text smoke test
* Convert input readers to a proper factory
* Remove covariates from fast smoke test
* Update docs for input factory
* Bump smoke runtime
* Even longer runtime
* min-csv timeout
* Remove unnecessary lambdas
* Prefix vector store (#2106)
* add prefix to vector store configuration and removal of container name
* docs updated
* change prefix property name
* change prefix property name
* feedback implemented
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* fix for container name
* Restructure project as monorepo. (#2111)
* Restructure project as monorepo.
* Fix formatting
* Storage fixes and cleanup (#2118)
* Fix pipeline recursion
* Remove base_dir from storage.find
* Remove max_count from storage.find
* Remove prefix on storage integ test
* Add base_dir in creation_date test
* Wrap base_dir in Path
* Use constants for input/update directories
* Nov 2025 housekeeping (#2120)
* Remove gensim sideload
* Split CI build/type checks from unit tests
* Thorough review of docs to align with v3
* Format
* Fix version
* Fix type
* Graphrag config (#2119)
* Add load_config to graphrag-common package.
* Empty graph guards (#2126)
* Remove networkx from graph_extractor and clean out redundancy
* Bubble pipeline error to console
* Remove embeddings optional new (#2128)
* remove optional embeddings
* fix test
* fix tests
* fix pipeline
* fix test
* fix test
* fix test
* fix tests
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Format
* Add empty checks for NLP graphs (#2133)
* Init command asks for models (#2137)
* Add init prompting for models
* Remove hard-coded model config validation
* Switch to typer option prompt for full CLI use with models
* Update getting started for init model input
* Bump request timeout and overall smoke test timeout
* Add graphrag-storage. (#2127)
* Add graphrag-storage.
* Python update (3.13) (#2149)
* Update to python 3.14 as default, with range down to 3.10
* Fix enum value in query cli
* Update pyarrow
* Update py version for storage package
* Remove 3.10
* add fastuuid
* Update Python support to 3.11-3.14 with stricter dependency constraints
- Set minimum Python version to 3.11 (removed 3.10 support)
- Added support for Python 3.14
- Updated CI workflows: single-version jobs use 3.14, matrix jobs use 3.11 and 3.14
- Fixed license format to use SPDX-compatible format for Python 3.14
- Updated pyarrow to >=22.0.0 for Python 3.14 wheel support
- Added explicit fastuuid~=0.14 and blis~=1.3 for Python 3.14 compatibility
- Replaced all loose version constraints (>=) with compatible release (~=) for better lock file control
- Applied stricter versioning to all packages: graphrag, graphrag-common, graphrag-storage, unified-search-app
* update uv lock
* Pin blis to ~=1.3.3 to ensure Python 3.14 wheel availability
* Update uv lock
* Update numpy to >=2.0.0 for Python 3.14 Windows compatibility
Numpy 1.25.x has access violation issues on Python 3.14 Windows.
Numpy 2.x has proper Python 3.14 support including Windows wheels.
* update uv lock
* Update pandas to >=2.3.0 for numpy 2.x compatibility
Pandas 2.2.x was compiled against numpy 1.x and causes ABI
incompatibility errors with numpy 2.x. Pandas 2.3.0+ supports
numpy 2.x properly.
* update uv.lock
* Add scipy>=1.15.0 for numpy 2.x compatibility
Scipy versions < 1.15.0 have C extensions built against numpy 1.x
and are incompatible with numpy 2.x, causing dtype size errors.
* update uv lock
* Update Python support to 3.11-3.13 with compatible dependencies
- Set Python version range to 3.11-3.13 (removed 3.14 support)
- Updated CI workflows: single-version jobs use 3.13, matrix jobs use 3.11 and 3.13
- Dependencies optimized for Python 3.13 compatibility:
- pyarrow~=22.0 (has Python 3.13 wheels)
- numpy~=1.26
- pandas~=2.2
- blis~=1.0
- fastuuid~=0.13
- Applied stricter version constraints using ~= operator throughout
- Updated uv.lock with resolved dependencies
* Update numpy to 2.1+ and pandas to 2.3+ for Python 3.13 Windows compatibility
Numpy 1.26.x causes access violations on Python 3.13 Windows.
Numpy 2.1+ has proper Python 3.13 support with Windows wheels.
Pandas 2.3+ is required for numpy 2.x compatibility.
* update vsts.yml python version
* Add GraphRAG Cache package. (#2153)
* Add GraphRAG Cache package.
* Fix a bunch of module comments and function visibility (#2154)
* Issue #2004 fix (#2159)
* fix issue #2004 using KeenhoChu idea in his PR
* add unit test for dynamic community selection
* add unit test for dynamic community selection implementing #2158 logic
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Mismatch between header in community report generation prompt examples and input data (id vs human_readable_id) (#2161)
* fix issue #860 for mismatch in prompts and input
* fix format
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Chunker factory (#2156)
* Delete NoopTextSplitter
* Delete unused check_token_limit
* Add base chunking factory and migrate workflow to use it
* Split apart chunker module
* Co-locate chunking/splitting
* Collapse token splitting functionality into one class/function
* Restore create_base_text_units parameterization
* Move Tokenizer base class to common package
* Move pre-pending into chunkers
* Streamline config
* Fix defaults construction
* Add prepending tests
* Remove chunk_size_includes_metadata config
* Revert ChunkingDocument interface
* Move metadata prepending to a util
* Move Tokenizer back to GR core
* Fix tokenizer removal from chunker
* Set defaults for chunking config
* Move chunking to monorepo package
* Format
* Typo
* Add ChunkResult model
* Streamline chunking config
* Add missing version updates for graphrag_chunking
* Input factory (#2168)
* Update input factory to match other factories
* Move input config alongside input readers
* Move file pattern logic into InputReader
* Set encoding default
* Clean up optional column configs
* Combine structured data extraction
* Remove pandas from input loading
* Throw if empty documents
* Add json lines (jsonl) input support
* Store raw data
* Fix merge imports
* Move metadata handling entirely to chunking
* Nicer automatic title
* Typo
* Add get_property utility for nested dictionary access with dot notation
* Update structured_file_reader to use get_property utility
* Extract input module into new graphrag-input monorepo package
- Create new graphrag-input package with input loading utilities
- Move InputConfig, InputFileType, InputReader, TextDocument, and file readers (CSV, JSON, JSONL, Text)
- Add get_property utility for nested dictionary access with dot notation
- Include hashing utility for document ID generation
- Update all imports throughout codebase to use graphrag_input
- Add package to workspace configuration and release tasks
- Remove old graphrag.index.input module
* Rename ChunkResult to TextChunk and add transformer support
- Rename chunk_result.py to text_chunk.py with ChunkResult -> TextChunk
- Add 'original' field to TextChunk to track pre-transform text
- Add optional transform callback to chunker.chunk() method
- Add add_metadata transformer for prepending metadata to chunks
- Update create_chunk_results to apply transforms and populate original
- Update sentence_chunker and token_chunker with transform support
- Refactor create_base_text_units to use new transformer pattern
- Rename pluck_metadata to get/collect methods on TextDocument
* Back-compat comment
* Align input config type name with other factory configs
* Add MarkItDown support
* Remove pattern default from MarkItDown reader
* Remove plugins flag (implicit disabled)
* Format
* Update verb tests
* Separate storage from input config
* Add empty objects for NaN raw_data
* Fix smoke tests
* Fix BOM in csv smoke
* Format
* DRIFT fixes (#2171)
* Use stable ids for community reports
* Remove deprecated title from embedding flow
* Remove embedding column from df loaders
* Fix lancedb insertion
* Add drift back to smoke tests
* Fix mock embedder to match default embedding length
* Fix DRIFT notebook
* Push drift_k_followups through to prompt
* Format
* Vector package (#2172)
* Extract graphrag-vectors package
* Simplify vector factory usage and config defaults
* Update factory integ initializers
* Fix mock patch
* Format
* Register vector stores in tests
* Set a default vector store name
* Update vector readme
* Remove impls from init
* Move some validation into impls
* Remove index_prefix
* Move duplicate method to base class
* Fix smoke vector config
* Update index bug (#2173)
* fix update index bug
* blob storage bug fix
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Add GraphRAG LLM package. (#2174)
* Update documentation for v3 release (#2176)
update documentation for v3 release
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Graphrag llm cleanup (#2181)
* Migration update (#2180)
* fix formatting.
---------
Co-authored-by: Nathan Evans <github@talkswithnumbers.com>
Co-authored-by: gaudyb <85708998+gaudyb@users.noreply.github.com>
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
Co-authored-by: Andres Morales <86074752+andresmor-ms@users.noreply.github.com>
* Fix scope for Azure auth with LiteLLM
* Change internal language on max_attempts to max_retries
* Rework model config connectivity validation
* Semver
* Swtich smoke tests to LiteLLM
* Take out temporary retry_strategy = none since it is not fnllm compatible
* Bump smoke test timeout
* Bump smoke timeout further
* Tune smoke params
* Update smoke test bounds
* Remove covariates from min-csv smoke
* Smoke: adjust communities, remove drift
* Remove secrets where they aren't necessary
* Clean out old env var references
* progress on vector customization
* fix for lancedb vectors
* cosmosdb implementation
* uv run poe format
* clean test for vector store
* semversioner update
* test_factory.py integration test fixes
* fixes for cosmosdb test
* integration test fix for lancedb
* uv fix for format
* test fixes
* fixes for tests
* fix cosmosdb bug
* print statement
* test
* test
* fix cosmosdb bug
* test validation
* validation cosmosdb
* validate cosmosdb
* fix cosmosdb
* fix small feedback from PR
---------
Co-authored-by: Gaudy Blanco <gaudy-microsoft@MacBook-Pro-m4-Gaudy-For-Work.local>
* Turn down blob/cosmos exception reporting to match file storage
* Restore indexing-engine.log
* Restore some basic console logging and progress for index CLI
* Semver
* Ignore small ruff complaints
* Fix CLI console printing
* Move covariate run conditional
* All pipeline registration
* Fix method name construction
* Rename context storage -> output_storage
* Rename OutputConfig as generic StorageConfig
* Reuse Storage model under InputConfig
* Move input storage creation out of document loading
* Move document loading into workflows
* Semver
* Fix smoke test config for new workflows
* Fix unit tests
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Update tiktoken
* Add max_completion_tokens to model config
* Update/remove outdated comments
* Remove max_tokens from report generation
* Remove max_tokens from entity summarization
* Remove logit_bias from graph extraction
* Remove logit_bias from claim extraction
* Swap params if reasoning model
* Add reasoning model support to basic search
* Add reasoning model support for local and global search
* Support reasoning models with dynamic community selection
* Support reasoning models in DRIFT search
* Remove unused num_threads entry
* Semver
* Update openai
* Add reasoning_effort param
* Rework update output structure
* Semver
* Fix unit test
* Update frequency in incremental
---------
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Add vector store id reference to embeddings config.
* changed structure of output config section
* added cli integration for multi index global
* added cli integration for multi index local
* added cli integration for multi index drift and basic
* finished local testing of multi-index cli
* ruff fixes
* partially refactored test code to align with new output section
* more test changes for new output structure
* semversioner
* refactored to align with new multi index config proposal
* locally tested new multi-index output proposal
* cleaned up tests to align with new structure
---------
Co-authored-by: Derek Worthen <worthend.derek@gmail.com>
* remove unused columns and change property document_attribute_columns to metadata
* format file
* fix 'metadata' column on output
* run check
* fix test on nltk
* remove docs changes
* Require explicit azure auth settings when using AOI.
- Must set LanguageModel.azure_auth_type to either
"api_key" or "managed_identity" when using AOI.
* Fix smoke tests
* Use general auth_type property instead of azure_auth_type
* Remove unused error type
* Update validation
* Update validation comment
* added multi-global-query function header
* ported over code for merging dataframes
* added connection to global streaming api function
* added function header for update context helper
* implemented and incorperated update_context function
* Updated to make sure 'parent' column in final_communities gets incremented for multi index.
* first cut at multi_local_seach function
* several minor changes and fixes
* Updated multi index local search.
* Cleaned up code.
* fixed lambda function ruff errors
* fixed more ruff errors
* moved query api helpers to util file
* moved index api helpers to util file
* merged in code left out of conflict
* changed GraphRagConfig object to support lists of vector stores
* Updated with fixes for multi_local_search.
* Minor updates.
* Minor updates.
* Updates for ruff check.
* Minor updates.
* removed redundant vector_store_configs arg
* ruff formatting changes
* semversioner
* Minor fix.
* spellcheck fixes
* ruff
* test fix for cicd errors
* another test fix
* added explicit typing for ci tests
* added dict type check for vector_store during indexing
* more ruff fixes
* moved type check
* Removed streaming. Added multi drift and basic searches.
* Formatting changes.
* Updates for pyright.
* Update for ruff.
* Ruff formatted.
* first cut at fixing vector store typing errors
* got multi local search working with new config
* ruff and test fixes
* added fix for embeddings type error
* renamed multi index api functions
* ruff
* convert config model to dict[VectorStoreConfig]
* modified tests to support new vector_store model
* ruff fixes
* changed some test setups to match new model
* changed ci/cd settings files to match new structure
* Fix stderror check
* fixed bug in vector_store_config validation
* ruff
* add database_name field to vectorstoreconfig
* removed print statements
* small refactoring for PR comments
* modified default config in test
* modified vector store config unit test
---------
Co-authored-by: dorbaker <dorbaker@microsoft.com>
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* Refactor config
- Add new ModelConfig to represent LLM settings
- Combines LLMParameters, ParallelizationParameters, encoding_model, and async_mode
- Add top level models config that is a list of available LLM ModelConfigs
- Remove LLMConfig inheritance and delete LLMConfig
- Replace the inheritance with a model_id reference to the ModelConfig listed in the top level models config
- Remove all fallbacks and hydration logic from create_graphrag_config
- This removes the automatic env variable overrides
- Support env variables within config files using Templating
- This requires "$" to be escaped with extra "$" so ".*\\.txt$" becomes ".*\\.txt$$"
- Update init content to initialize new config file with the ModelConfig structure
* Use dict of ModelConfig instead of list
* Add model validations and unit tests
* Fix ruff checks
* Add semversioner change
* Fix unit tests
* validate root_dir in pydantic model
* Rename ModelConfig to LanguageModelConfig
* Rename ModelConfigMissingError to LanguageModelConfigMissingError
* Add validationg for unexpected API keys
* Allow skipping pydantic validation for testing/mocking purposes.
* Add default lm configs to verb tests
* smoke test
* remove config from flows to fix llm arg mapping
* Fix embedding llm arg mapping
* Remove timestamp from smoke test outputs
* Remove unused "subworkflows" smoke test properties
* Add models to smoke test configs
* Update smoke test output path
* Send logs to logs folder
* Fix output path
* Fix csv test file pattern
* Update placeholder
* Format
* Instantiate default model configs
* Fix unit tests for config defaults
* Fix migration notebook
* Remove create_pipeline_config
* Remove several unused config models
* Remove indexing embedding and input configs
* Move embeddings function to config
* Remove skip_workflows
* Remove skip embeddings in favor of explicit naming
* fix unit test spelling mistake
* self.models[model_id] is already a language model. Remove redundant casting.
* update validation errors to instruct users to rerun graphrag init
* instantiate LanguageModelConfigs with validation
* skip validation in unit tests
* update verb tests to use default model settings instead of skipping validation
* test using llm settings
* cleanup verb tests
* remove unsafe default model config
* remove the ability to skip pydantic validation
* remove None union types when default values are set
* move vector_store from embeddings to top level of config and delete resolve_paths
* update vector store settings
* fix vector store and smoke tests
* fix serializing vector_store settings
* fix vector_store usage
* fix vector_store type
* support cli overrides for loading graphrag config
* rename storage to output
* Add --force flag to init
* Remove run_id and resume, fix Drift config assignment
* Ruff
---------
Co-authored-by: Nathan Evans <github@talkswithnumbers.com>
Co-authored-by: Alonso Guevara <alonsog@microsoft.com>
* add text_splitting unit test
* change folder test text splitting
* fix chunk fn
* test new function
* run formatter
* run spell check
* run semver
* remove tiktoken mocked from tests
* change progress ticker
* fix ruff check
Derek Worthen