github-featured/deusdata--codebase-memory-mcp
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

Initial release: codebase-memory-mcp v0.0.1

Browse Source 
MCP server that indexes codebases into a queryable knowledge graph.
Single Go binary, SQLite storage, 12 languages via tree-sitter.

Features: call graph, cross-service HTTP linking, dead code detection,
route nodes, Cypher queries, incremental reindex, JSON config scanning.
This commit is contained in:
Martin Vogel
2026-02-25 00:04:52 +01:00
commit f63c70865e
59 changed files with 11435 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+28
View File
@@ -0,0 +1,28 @@
# Binaries
code-graph-mcp
codebase-memory-mcp
bin/
*.exe
*.dll
*.so
*.dylib
# Test artifacts
*.test
*.out
coverage.txt
# IDE
.idea/
.vscode/
*.swp
*.swo
# OS
.DS_Store
Thumbs.db
# Database files (local cache)
*.db
*.db-wal
*.db-shm
LICENSE
+22
View File
@@ -0,0 +1,22 @@
MIT License
Copyright (c) 2025 DeusData
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Makefile
+19
View File
@@ -0,0 +1,19 @@
.PHONY: build test lint clean install
BINARY=codebase-memory-mcp
MODULE=github.com/DeusData/codebase-memory-mcp
build:
go build -o bin/$(BINARY) ./cmd/codebase-memory-mcp/
test:
go test ./... -v
lint:
golangci-lint run ./...
clean:
rm -rf bin/
install:
go install ./cmd/codebase-memory-mcp/
README.md
+362
View File
@@ -0,0 +1,362 @@
# codebase-memory-mcp
An MCP server that remembers your codebase structure. Indexes source code into a queryable knowledge graph — functions, classes, call chains, cross-service HTTP links — all stored in embedded SQLite. Single Go binary, no Docker, no external databases.
Parses source code with [tree-sitter](https://tree-sitter.github.io/tree-sitter/), extracts functions, classes, modules, call relationships, and cross-service HTTP links. Exposes the graph through 11 MCP tools for use with Claude Code or any MCP-compatible client.
## Features
- **12 languages**: Python, Go, JavaScript, TypeScript, TSX, Rust, Java, C++, C#, PHP, Lua, Scala
- **Call graph**: Resolves function calls across files and packages (import-aware, type-inferred)
- **Cross-service HTTP linking**: Discovers REST routes (FastAPI, Gin, Express) and matches them to HTTP call sites with confidence scoring
- **Incremental reindex**: Content-hash based — only re-parses changed files
- **Cypher-like queries**: `MATCH (f:Function)-[:CALLS]->(g) WHERE f.name = 'main' RETURN g.name`
- **Dead code detection**: Finds functions with zero callers, excluding entry points (route handlers, `main()`, framework-decorated functions)
- **Route nodes**: REST endpoints are first-class graph entities, queryable by path/method
- **JSON config scanning**: Extracts URLs from config/payload JSON files for cross-service linking
- **Single binary, zero infrastructure**: SQLite WAL mode, persists to `~/.cache/codebase-memory-mcp/`
## How It Works
codebase-memory-mcp is a **structural analysis backend** — it builds and queries the knowledge graph. It does **not** include an LLM. Instead, it relies on the MCP client (Claude Code, or any MCP-compatible AI assistant) to be the intelligence layer.
When you ask Claude Code a question like *"what calls ProcessOrder?"*, this is what happens:
1. **Claude Code** understands your natural language question
2. **Claude Code** decides which MCP tool to call — in this case `trace_call_path(function_name="ProcessOrder", direction="inbound")`
3. **codebase-memory-mcp** executes the graph query against SQLite and returns structured results
4. **Claude Code** interprets the results and presents them in plain English
For complex graph patterns, Claude Code writes Cypher queries on the fly:
```
You: "Show me all cross-service HTTP calls with confidence above 0.5"
Claude Code generates and sends:
query_graph(query="MATCH (a)-[r:HTTP_CALLS]->(b) WHERE r.confidence > 0.5
RETURN a.name, b.name, r.url_path, r.confidence
ORDER BY r.confidence DESC LIMIT 20")
codebase-memory-mcp returns the matching edges.
Claude Code formats and explains the results.
```
**Why no built-in LLM?** Other code graph tools embed an LLM to translate natural language into graph queries. This means extra API keys, extra cost per query, and another model to configure. With MCP, the AI assistant you're already talking to *is* the query translator — no duplication needed.
**Token efficiency**: Compared to having an AI agent grep through your codebase file by file, graph queries return precise results in a single tool call. In benchmarks on a multi-service project (2,348 nodes, 3,853 edges), five structural queries consumed ~3,400 tokens via codebase-memory-mcp versus ~412,000 tokens via file-by-file exploration — a **99.2% reduction**.
## Installation
### Quick Install via Claude Code
The fastest way: paste the repo URL directly into Claude Code and ask it to install:
```
You: "Install this MCP server: https://github.com/DeusData/codebase-memory-mcp"
```
Claude Code will clone, build, and configure the MCP server automatically.
### Prerequisites
| Requirement | Version | Check | Install |
|-------------|---------|-------|---------|
| **Go** | 1.23+ | `go version` | [go.dev/dl](https://go.dev/dl/) |
| **C compiler** | gcc or clang | `gcc --version` or `clang --version` | See below |
| **Git** | any | `git --version` | Pre-installed on most systems |
**C compiler** is needed because tree-sitter uses CGO (C bindings for AST parsing):
- **macOS**: Install Xcode command line tools — `xcode-select --install`. This provides `clang` and is likely already installed.
- **Linux (Debian/Ubuntu)**: `sudo apt install build-essential`
- **Linux (Fedora/RHEL)**: `sudo dnf install gcc`
- **Windows**: Not currently supported (CGO cross-compilation is complex). Use WSL2 with the Linux instructions above.
### Build from Source
```bash
# Clone the repository
git clone https://github.com/DeusData/codebase-memory-mcp.git
cd codebase-memory-mcp
# Build the binary (CGO_ENABLED=1 is the default, but be explicit)
CGO_ENABLED=1 go build -o codebase-memory-mcp ./cmd/codebase-memory-mcp/
# Option A: Move to a directory on your PATH
sudo mv codebase-memory-mcp /usr/local/bin/
# Option B: Or keep it in place and use the absolute path in MCP config
```
### Verify
```bash
# Should print nothing and wait for stdio input (Ctrl+C to exit)
codebase-memory-mcp
```
### Configure Claude Code
Add the MCP server to your project's `.mcp.json` (per-project) or `~/.claude/settings.json` (global):
**Per-project** (`.mcp.json` in project root — recommended):
```json
{
"mcpServers": {
"codebase-memory-mcp": {
"type": "stdio",
"command": "/usr/local/bin/codebase-memory-mcp"
}
}
}
```
**Global** (`~/.claude/settings.json`):
```json
{
"mcpServers": {
"codebase-memory-mcp": {
"type": "stdio",
"command": "/usr/local/bin/codebase-memory-mcp"
}
}
}
```
If you kept the binary in the cloned directory, use the full path instead:
```json
{
"command": "/path/to/codebase-memory-mcp/codebase-memory-mcp"
}
```
Restart Claude Code after adding the config. Verify with `/mcp` — you should see `codebase-memory-mcp` listed with 11 tools.
### First Use
```
You: "Index this project"
```
Claude Code will call `index_repository` and build the knowledge graph. After indexing, you can ask structural questions like *"what calls main?"*, *"find dead code"*, or *"show cross-service HTTP calls"*.
## MCP Tools
### Indexing
| Tool | Description |
|------|-------------|
| `index_repository` | Index a repository into the graph. Supports incremental reindex via content hashing. |
| `list_projects` | List all indexed projects with timestamps and node/edge counts. |
| `delete_project` | Remove a project and all its graph data. |
### Querying
| Tool | Description |
|------|-------------|
| `search_graph` | Structured search with filters: label, name pattern (regex), file pattern (glob), relationship type, degree (fan-in/fan-out), entry point exclusion. |
| `trace_call_path` | BFS traversal from/to a function. Returns call chains with signatures, constants, and edge types. |
| `query_graph` | Execute Cypher-like graph queries (read-only). |
| `get_graph_schema` | Node/edge counts, relationship patterns, sample names. |
| `get_code_snippet` | Read source code for a function by qualified name (reads from disk). |
### File Access
| Tool | Description |
|------|-------------|
| `search_code` | Grep-like text search within indexed project files. |
| `read_file` | Read any file from an indexed project (with optional line range). |
| `list_directory` | List files/directories with glob filtering. |
## Usage Examples
### Index a project
```
index_repository(repo_path="/path/to/your/project")
```
### Find all functions matching a pattern
```
search_graph(label="Function", name_pattern=".*Handler")
```
### Trace what a function calls
```
trace_call_path(function_name="ProcessOrder", depth=3, direction="outbound")
```
### Find what calls a function
```
trace_call_path(function_name="ProcessOrder", depth=2, direction="inbound")
```
### Dead code detection
```
search_graph(
label="Function",
relationship="CALLS",
direction="inbound",
max_degree=0,
exclude_entry_points=true
)
```
### Cross-service HTTP calls
```
search_graph(label="Function", relationship="HTTP_CALLS", direction="outbound")
```
### Query all REST routes
```
search_graph(label="Route")
```
### Cypher queries
```
query_graph(query="MATCH (f:Function)-[:CALLS]->(g:Function) WHERE f.name = 'main' RETURN g.name, g.qualified_name LIMIT 20")
```
```
query_graph(query="MATCH (a)-[r:HTTP_CALLS]->(b) RETURN a.name, b.name, r.url_path, r.confidence LIMIT 10")
```
### High fan-out functions (calling 10+ others)
```
search_graph(label="Function", relationship="CALLS", direction="outbound", min_degree=10)
```
## Graph Data Model
### Node Labels
`Project`, `Package`, `Folder`, `File`, `Module`, `Class`, `Function`, `Method`, `Interface`, `Enum`, `Type`, `Route`
### Edge Types
`CONTAINS_PACKAGE`, `CONTAINS_FOLDER`, `CONTAINS_FILE`, `CONTAINS_MODULE`, `DEFINES`, `DEFINES_METHOD`, `IMPORTS`, `CALLS`, `HTTP_CALLS`, `INHERITS`, `IMPLEMENTS`, `DEPENDS_ON_EXTERNAL`, `HANDLES`
### Node Properties
- **Function/Method**: `signature`, `return_type`, `receiver`, `decorators`, `is_exported`, `is_entry_point`
- **Module**: `constants` (list of module-level constants)
- **Route**: `method`, `path`, `handler`
- **All nodes**: `name`, `qualified_name`, `file_path`, `start_line`, `end_line`
## Teaching Claude Code to Use the Graph
Claude Code can use the tools without any configuration — the MCP tool descriptions are self-documenting. However, without a hint, Claude Code will default to its built-in Grep/Glob/Read tools for code questions instead of the faster graph queries.
Add one of the following to tell Claude Code to **prefer graph tools for structural questions**.
### Option A: Global CLAUDE.md (recommended — works across all projects)
Add to `~/.claude/CLAUDE.md`:
```markdown
## Codebase Memory (codebase-memory-mcp)
When this MCP server is available, **prefer graph tools over grep/Explore for structural code questions**.
Graph queries return precise results in a single tool call (~500 tokens) vs file-by-file exploration (~80K tokens).
- **Before exploration/planning**: Run `index_repository` to ensure the graph is current
- **"Who calls X?"**: `trace_call_path(function_name="X", direction="inbound")`
- **"What does X call?"**: `trace_call_path(function_name="X", direction="outbound")`
- **Find functions by pattern**: `search_graph(label="Function", name_pattern=".*Pattern.*")`
- **Dead code**: `search_graph(label="Function", relationship="CALLS", direction="inbound", max_degree=0, exclude_entry_points=true)`
- **Cross-service calls**: `search_graph(relationship="HTTP_CALLS")` or `query_graph` with Cypher
- **REST routes**: `search_graph(label="Route")`
- **Understand structure first**: `get_graph_schema` before writing complex queries
- **Read source**: `get_code_snippet(qualified_name="...")` after finding functions via search
- **Complex patterns**: `query_graph` with Cypher for multi-hop graph traversals
Use grep/Glob for text search (string literals, error messages, config values) — the graph doesn't index text content.
```
### Option B: Per-project CLAUDE.md
Add the same snippet to a specific project's `CLAUDE.md` if you only want it active for that project.
### Option C: Claude Code skill file
Create `~/.claude/skills/codebase-memory.md` for automatic activation when relevant:
```markdown
# codebase-memory-mcp Skill
## When to use
- Structural code questions: "who calls X?", "what does X depend on?", "show me the call chain"
- Dead code analysis: functions with zero callers
- Cross-service tracing: HTTP call paths between microservices
- Architecture overview: understanding module boundaries and dependencies
- Pre-planning: index before designing changes to understand blast radius
## When NOT to use
- Text search (use grep/Glob instead)
- Single file reads (use Read tool instead)
- Syntax/formatting questions (not a graph concern)
## Workflow
1. **Ensure freshness**: `list_projects` to check `indexed_at`. If stale, `index_repository`.
2. **Understand schema**: `get_graph_schema` to see what's indexed (node counts, edge types).
3. **Search**: `search_graph` for filtered queries, `trace_call_path` for call chains.
4. **Deep dive**: `get_code_snippet` to read source of interesting functions.
5. **Complex queries**: `query_graph` with Cypher for multi-hop patterns.
## Tips
- `trace_call_path` with `direction="both"` shows full context (callers + callees)
- `search_graph` with `file_pattern` scopes results to a service/directory
- Route nodes (`label="Route"`) let you query REST endpoints as graph entities
- Edge properties on HTTP_CALLS include `confidence` and `url_path`
- Reindex after significant code changes (new files, moved functions)
```
## Persistence
The SQLite database is stored at `~/.cache/codebase-memory-mcp/codebase-memory.db`. It persists across restarts automatically (WAL mode, ACID-safe).
To reset everything:
```bash
rm -rf ~/.cache/codebase-memory-mcp/
```
## Development
```bash
make build # Build binary to bin/
make test # Run all tests
make lint # Run golangci-lint
make install # go install
```
## Architecture
```
cmd/codebase-memory-mcp/ Entry point (MCP stdio server)
internal/
store/ SQLite graph storage (nodes, edges, traversal, search)
lang/ Language specs (12 languages, tree-sitter node types)
parser/ Tree-sitter grammar loading and AST parsing
pipeline/ 4-pass indexing (structure -> definitions -> calls -> HTTP links)
httplink/ Cross-service HTTP route/call-site matching
cypher/ Cypher query lexer, parser, planner, executor
tools/ MCP tool handlers (11 tools)
discover/ File discovery with .cgrignore support
fqn/ Qualified name computation
```
## License
MIT
cmd/codebase-memory-mcp/main.go
+24
View File
@@ -0,0 +1,24 @@
package main
import (
"context"
"log"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/DeusData/codebase-memory-mcp/internal/tools"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func main() {
s, err := store.Open("codebase-memory")
if err != nil {
log.Fatalf("store open err=%v", err)
}
defer s.Close()
srv := tools.NewServer(s)
if err := srv.MCPServer().Run(context.Background(), &mcp.StdioTransport{}); err != nil {
log.Fatalf("server err=%v", err)
}
}
go.mod
+40
View File
@@ -0,0 +1,40 @@
module github.com/DeusData/codebase-memory-mcp
go 1.23.0
require (
github.com/modelcontextprotocol/go-sdk v1.3.1
github.com/tree-sitter-grammars/tree-sitter-lua v0.4.1
github.com/tree-sitter/go-tree-sitter v0.25.0
github.com/tree-sitter/tree-sitter-cpp v0.23.4
github.com/tree-sitter/tree-sitter-go v0.25.0
github.com/tree-sitter/tree-sitter-java v0.23.5
github.com/tree-sitter/tree-sitter-javascript v0.25.0
github.com/tree-sitter/tree-sitter-php v0.24.2
github.com/tree-sitter/tree-sitter-python v0.25.0
github.com/tree-sitter/tree-sitter-rust v0.24.0
github.com/tree-sitter/tree-sitter-scala v0.24.0
github.com/tree-sitter/tree-sitter-typescript v0.23.2
modernc.org/sqlite v1.37.1
)
require (
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/jsonschema-go v0.4.2 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-pointer v0.0.1 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.3 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.15.0 // indirect
golang.org/x/sys v0.35.0 // indirect
modernc.org/libc v1.65.7 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
go.sum
+105
View File
@@ -0,0 +1,105 @@
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/golang-jwt/jwt/v5 v5.2.2 h1:Rl4B7itRWVtYIHFrSNd7vhTiz9UpLdi6gZhZ3wEeDy8=
github.com/golang-jwt/jwt/v5 v5.2.2/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/jsonschema-go v0.4.2 h1:tmrUohrwoLZZS/P3x7ex0WAVknEkBZM46iALbcqoRA8=
github.com/google/jsonschema-go v0.4.2/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-pointer v0.0.1 h1:n+XhsuGeVO6MEAp7xyEukFINEa+Quek5psIR/ylA6o0=
github.com/mattn/go-pointer v0.0.1/go.mod h1:2zXcozF6qYGgmsG+SeTZz3oAbFLdD3OWqnUbNvJZAlc=
github.com/modelcontextprotocol/go-sdk v1.3.1 h1:TfqtNKOIWN4Z1oqmPAiWDC2Jq7K9OdJaooe0teoXASI=
github.com/modelcontextprotocol/go-sdk v1.3.1/go.mod h1:DgVX498dMD8UJlseK1S5i1T4tFz2fkBk4xogC3D15nw=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc=
github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg=
github.com/segmentio/encoding v0.5.3 h1:OjMgICtcSFuNvQCdwqMCv9Tg7lEOXGwm1J5RPQccx6w=
github.com/segmentio/encoding v0.5.3/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tree-sitter-grammars/tree-sitter-lua v0.4.1 h1:DOxLDVTEmkLuoIHXJ65h5oyHcEK89mn9DOhlnPQRAmQ=
github.com/tree-sitter-grammars/tree-sitter-lua v0.4.1/go.mod h1:hIOfn+lxpU4SRrtejLVrU2+8SAoRwNC01m3XaR/Cw0A=
github.com/tree-sitter/go-tree-sitter v0.25.0 h1:sx6kcg8raRFCvc9BnXglke6axya12krCJF5xJ2sftRU=
github.com/tree-sitter/go-tree-sitter v0.25.0/go.mod h1:r77ig7BikoZhHrrsjAnv8RqGti5rtSyvDHPzgTPsUuU=
github.com/tree-sitter/tree-sitter-c v0.23.4 h1:nBPH3FV07DzAD7p0GfNvXM+Y7pNIoPenQWBpvM++t4c=
github.com/tree-sitter/tree-sitter-c v0.23.4/go.mod h1:MkI5dOiIpeN94LNjeCp8ljXN/953JCwAby4bClMr6bw=
github.com/tree-sitter/tree-sitter-cpp v0.23.4 h1:LaWZsiqQKvR65yHgKmnaqA+uz6tlDJTJFCyFIeZU/8w=
github.com/tree-sitter/tree-sitter-cpp v0.23.4/go.mod h1:doqNW64BriC7WBCQ1klf0KmJpdEvfxyXtoEybnBo6v8=
github.com/tree-sitter/tree-sitter-embedded-template v0.23.2 h1:nFkkH6Sbe56EXLmZBqHHcamTpmz3TId97I16EnGy4rg=
github.com/tree-sitter/tree-sitter-embedded-template v0.23.2/go.mod h1:HNPOhN0qF3hWluYLdxWs5WbzP/iE4aaRVPMsdxuzIaQ=
github.com/tree-sitter/tree-sitter-go v0.25.0 h1:cEB0Q3LHgZtS+ECHx9wcP7AwzoOddJFQCVmytX42cVU=
github.com/tree-sitter/tree-sitter-go v0.25.0/go.mod h1:Jrx8QqYN0v7npv1fJRH1AznddllYiCMUChtVjxPK040=
github.com/tree-sitter/tree-sitter-html v0.23.2 h1:1UYDV+Yd05GGRhVnTcbP58GkKLSHHZwVaN+lBZV11Lc=
github.com/tree-sitter/tree-sitter-html v0.23.2/go.mod h1:gpUv/dG3Xl/eebqgeYeFMt+JLOY9cgFinb/Nw08a9og=
github.com/tree-sitter/tree-sitter-java v0.23.5 h1:J9YeMGMwXYlKSP3K4Us8CitC6hjtMjqpeOf2GGo6tig=
github.com/tree-sitter/tree-sitter-java v0.23.5/go.mod h1:NRKlI8+EznxA7t1Yt3xtraPk1Wzqh3GAIC46wxvc320=
github.com/tree-sitter/tree-sitter-javascript v0.25.0 h1:ZkWETb66/w8cc13yhfnNuHOLDQWl3BnKlH6f9AdR88c=
github.com/tree-sitter/tree-sitter-javascript v0.25.0/go.mod h1:lmGD1EJdCA+v0S1u2fFgepMg/opzSg/4pgFym2FPGAs=
github.com/tree-sitter/tree-sitter-json v0.24.8 h1:tV5rMkihgtiOe14a9LHfDY5kzTl5GNUYe6carZBn0fQ=
github.com/tree-sitter/tree-sitter-json v0.24.8/go.mod h1:F351KK0KGvCaYbZ5zxwx/gWWvZhIDl0eMtn+1r+gQbo=
github.com/tree-sitter/tree-sitter-php v0.24.2 h1:yy+COnaaHUNDTKODfNbHhVRD4mQpFELTnBK9+EhpO+w=
github.com/tree-sitter/tree-sitter-php v0.24.2/go.mod h1:cEzabPRy4doSxaP9CF9u4FXc3L9Q3Mek3XPVCfqJQRw=
github.com/tree-sitter/tree-sitter-python v0.25.0 h1:O6XD9v8U1LOcRc3cNj9nM7XufrtEBezE6VrpRrHZDf0=
github.com/tree-sitter/tree-sitter-python v0.25.0/go.mod h1:cpdthSy/Yoa28aJFBscFHlGiU+cnSiSh1kuDVtI8YeM=
github.com/tree-sitter/tree-sitter-ruby v0.23.1 h1:T/NKHUA+iVbHM440hFx+lzVOzS4dV6z8Qw8ai+72bYo=
github.com/tree-sitter/tree-sitter-ruby v0.23.1/go.mod h1:kUS4kCCQloFcdX6sdpr8p6r2rogbM6ZjTox5ZOQy8cA=
github.com/tree-sitter/tree-sitter-rust v0.24.0 h1:nr3ga5ThXyPR5n/DiMq4Zh3e8pMR+sfzk088QE809+g=
github.com/tree-sitter/tree-sitter-rust v0.24.0/go.mod h1:hfeGWic9BAfgTrc7Xf6FaOAguCFJRo3RBbs7QJ6D7MI=
github.com/tree-sitter/tree-sitter-scala v0.24.0 h1:F8UcZQdNQSkOGtkW8tUsFrqifOVXzmzJ19/JSbB+X3E=
github.com/tree-sitter/tree-sitter-scala v0.24.0/go.mod h1:BmDV0f9rgsnGuG9QtKXQZnqJvECyR9fM8wVg984ulBo=
github.com/tree-sitter/tree-sitter-typescript v0.23.2 h1:/Odvphn18PniVixb9e97X0DbNVsU6Qocv9mfkyzdXwU=
github.com/tree-sitter/tree-sitter-typescript v0.23.2/go.mod h1:zjzMXT/Ulffel2xfOcAkQQkiAkmgnbtPGlFQw/5X4xA=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 h1:R84qjqJb5nVJMxqWYb3np9L5ZsaDtB+a39EqjV0JSUM=
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0/go.mod h1:S9Xr4PYopiDyqSyp5NjCrhFrqg6A5zA2E/iPHPhqnS8=
golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/oauth2 v0.30.0 h1:dnDm7JmhM45NNpd8FDDeLhK6FwqbOf4MLCM9zb1BOHI=
golang.org/x/oauth2 v0.30.0/go.mod h1:B++QgG3ZKulg6sRPGD/mqlHQs5rB3Ml9erfeDY7xKlU=
golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.26.1 h1:+X5NtzVBn0KgsBCBe+xkDC7twLb/jNVj9FPgiwSQO3s=
modernc.org/cc/v4 v4.26.1/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.28.0 h1:rjznn6WWehKq7dG4JtLRKxb52Ecv8OUGah8+Z/SfpNU=
modernc.org/ccgo/v4 v4.28.0/go.mod h1:JygV3+9AV6SmPhDasu4JgquwU81XAKLd3OKTUDNOiKE=
modernc.org/fileutil v1.3.1 h1:8vq5fe7jdtEvoCf3Zf9Nm0Q05sH6kGx0Op2CPx1wTC8=
modernc.org/fileutil v1.3.1/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/libc v1.65.7 h1:Ia9Z4yzZtWNtUIuiPuQ7Qf7kxYrxP1/jeHZzG8bFu00=
modernc.org/libc v1.65.7/go.mod h1:011EQibzzio/VX3ygj1qGFt5kMjP0lHb0qCW5/D/pQU=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.37.1 h1:EgHJK/FPoqC+q2YBXg7fUmES37pCHFc97sI7zSayBEs=
modernc.org/sqlite v1.37.1/go.mod h1:XwdRtsE1MpiBcL54+MbKcaDvcuej+IYSMfLN6gSKV8g=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
internal/cypher/ast.go
+74
View File
@@ -0,0 +1,74 @@
package cypher
// Query represents a parsed Cypher query.
type Query struct {
Match *MatchClause
Where *WhereClause
Return *ReturnClause
}
// MatchClause holds the MATCH pattern.
type MatchClause struct {
Pattern *Pattern
}
// Pattern is a sequence of alternating nodes and relationships.
type Pattern struct {
Elements []PatternElement
}
// PatternElement is either a NodePattern or a RelPattern.
type PatternElement interface {
patternElement()
}
// NodePattern matches a graph node with optional label and inline properties.
type NodePattern struct {
Variable string // e.g. "f"
Label string // e.g. "Function" (optional)
Props map[string]string // inline property filters (optional)
}
func (*NodePattern) patternElement() {}
// RelPattern matches a graph relationship with optional types, direction, and hops.
type RelPattern struct {
Variable string // (optional)
Types []string // relationship types, e.g. ["CALLS", "HTTP_CALLS"]
Direction string // "outbound", "inbound", "any"
MinHops int // for variable-length, default 1
MaxHops int // for variable-length, default 1 (0 means unbounded)
}
func (*RelPattern) patternElement() {}
// WhereClause holds filter conditions joined by AND/OR.
type WhereClause struct {
Conditions []Condition
Operator string // "AND" or "OR"
}
// Condition is a single property comparison.
type Condition struct {
Variable string // "f"
Property string // "name"
Operator string // "=", "=~", "CONTAINS", "STARTS WITH", ">", "<", ">=", "<="
Value string // the comparison value
}
// ReturnClause specifies which data to return from the query.
type ReturnClause struct {
Items []ReturnItem
OrderBy string // "f.name" (optional)
OrderDir string // "ASC" or "DESC"
Limit int // 0 means no limit
Distinct bool
}
// ReturnItem is a single item in the RETURN clause.
type ReturnItem struct {
Variable string // "f"
Property string // "name" (empty = return whole node)
Alias string // "AS call_count" (optional)
Func string // "COUNT" (optional aggregation)
}
internal/cypher/cypher_test.go
+700
View File
@@ -0,0 +1,700 @@
package cypher
import (
"testing"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
// --- Lexer tests ---
func TestLexBasicQuery(t *testing.T) {
tokens, err := Lex(`MATCH (f:Function) WHERE f.name = "Hello" RETURN f.name`)
if err != nil {
t.Fatalf("lex: %v", err)
}
expected := []TokenType{
TokMatch, TokLParen, TokIdent, TokColon, TokIdent, TokRParen,
TokWhere, TokIdent, TokDot, TokIdent, TokEQ, TokString,
TokReturn, TokIdent, TokDot, TokIdent, TokEOF,
}
if len(tokens) != len(expected) {
t.Fatalf("expected %d tokens, got %d", len(expected), len(tokens))
}
for i, tok := range tokens {
if tok.Type != expected[i] {
t.Errorf("token[%d]: expected type %d, got %d (%q)", i, expected[i], tok.Type, tok.Value)
}
}
}
func TestLexRegexOperator(t *testing.T) {
tokens, err := Lex(`f.name =~ ".*Handler"`)
if err != nil {
t.Fatalf("lex: %v", err)
}
// f, ., name, =~, ".*Handler"
if tokens[3].Type != TokRegex {
t.Errorf("expected TokRegex, got type %d (%q)", tokens[3].Type, tokens[3].Value)
}
}
func TestLexVariableLengthPath(t *testing.T) {
tokens, err := Lex(`[:CALLS*1..3]`)
if err != nil {
t.Fatalf("lex: %v", err)
}
expected := []TokenType{
TokLBracket, TokColon, TokIdent, TokStar, TokNumber, TokDotDot, TokNumber, TokRBracket, TokEOF,
}
if len(tokens) != len(expected) {
t.Fatalf("expected %d tokens, got %d", len(expected), len(tokens))
}
for i, tok := range tokens {
if tok.Type != expected[i] {
t.Errorf("token[%d]: expected type %d, got %d (%q)", i, expected[i], tok.Type, tok.Value)
}
}
}
// --- Parser tests ---
func TestParseNodePattern(t *testing.T) {
q, err := Parse(`MATCH (f:Function {name: "Hello"}) RETURN f`)
if err != nil {
t.Fatalf("parse: %v", err)
}
if q.Match == nil || q.Match.Pattern == nil {
t.Fatal("expected match pattern")
}
elems := q.Match.Pattern.Elements
if len(elems) != 1 {
t.Fatalf("expected 1 element, got %d", len(elems))
}
node := elems[0].(*NodePattern)
if node.Variable != "f" {
t.Errorf("expected variable 'f', got %q", node.Variable)
}
if node.Label != "Function" {
t.Errorf("expected label 'Function', got %q", node.Label)
}
if node.Props["name"] != "Hello" {
t.Errorf("expected prop name='Hello', got %q", node.Props["name"])
}
}
func TestParseRelationship(t *testing.T) {
q, err := Parse(`MATCH (f)-[:CALLS]->(g) RETURN f.name, g.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
elems := q.Match.Pattern.Elements
if len(elems) != 3 {
t.Fatalf("expected 3 elements (node-rel-node), got %d", len(elems))
}
rel := elems[1].(*RelPattern)
if len(rel.Types) != 1 || rel.Types[0] != "CALLS" {
t.Errorf("expected CALLS type, got %v", rel.Types)
}
if rel.Direction != "outbound" {
t.Errorf("expected outbound, got %q", rel.Direction)
}
if rel.MinHops != 1 || rel.MaxHops != 1 {
t.Errorf("expected hops 1..1, got %d..%d", rel.MinHops, rel.MaxHops)
}
}
func TestParseVariableLength(t *testing.T) {
q, err := Parse(`MATCH (f)-[:CALLS*1..3]->(g) RETURN g.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
rel := q.Match.Pattern.Elements[1].(*RelPattern)
if rel.MinHops != 1 {
t.Errorf("expected minHops=1, got %d", rel.MinHops)
}
if rel.MaxHops != 3 {
t.Errorf("expected maxHops=3, got %d", rel.MaxHops)
}
}
func TestParseWhereRegex(t *testing.T) {
q, err := Parse(`MATCH (f:Function) WHERE f.name =~ ".*Handler" RETURN f.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
if q.Where == nil {
t.Fatal("expected WHERE clause")
}
if len(q.Where.Conditions) != 1 {
t.Fatalf("expected 1 condition, got %d", len(q.Where.Conditions))
}
c := q.Where.Conditions[0]
if c.Operator != "=~" {
t.Errorf("expected =~, got %q", c.Operator)
}
if c.Value != ".*Handler" {
t.Errorf("expected '.*Handler', got %q", c.Value)
}
}
func TestParseReturnWithCount(t *testing.T) {
q, err := Parse(`MATCH (f)-[:CALLS]->(g) RETURN f.name, COUNT(g) AS cnt ORDER BY cnt DESC LIMIT 10`)
if err != nil {
t.Fatalf("parse: %v", err)
}
if q.Return == nil {
t.Fatal("expected RETURN clause")
}
if len(q.Return.Items) != 2 {
t.Fatalf("expected 2 return items, got %d", len(q.Return.Items))
}
// First item: f.name
if q.Return.Items[0].Variable != "f" || q.Return.Items[0].Property != "name" {
t.Errorf("expected f.name, got %s.%s", q.Return.Items[0].Variable, q.Return.Items[0].Property)
}
// Second item: COUNT(g) AS cnt
if q.Return.Items[1].Func != "COUNT" {
t.Errorf("expected COUNT, got %q", q.Return.Items[1].Func)
}
if q.Return.Items[1].Variable != "g" {
t.Errorf("expected variable 'g', got %q", q.Return.Items[1].Variable)
}
if q.Return.Items[1].Alias != "cnt" {
t.Errorf("expected alias 'cnt', got %q", q.Return.Items[1].Alias)
}
// ORDER BY
if q.Return.OrderBy != "cnt" {
t.Errorf("expected ORDER BY cnt, got %q", q.Return.OrderBy)
}
if q.Return.OrderDir != "DESC" {
t.Errorf("expected DESC, got %q", q.Return.OrderDir)
}
if q.Return.Limit != 10 {
t.Errorf("expected LIMIT 10, got %d", q.Return.Limit)
}
}
func TestParseBidirectional(t *testing.T) {
q, err := Parse(`MATCH (f:Function)-[:CALLS]-(g) RETURN f.name, g.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
rel := q.Match.Pattern.Elements[1].(*RelPattern)
if rel.Direction != "any" {
t.Errorf("expected 'any' direction, got %q", rel.Direction)
}
}
func TestParseInbound(t *testing.T) {
q, err := Parse(`MATCH (f:Function)<-[:CALLS]-(g) RETURN f.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
rel := q.Match.Pattern.Elements[1].(*RelPattern)
if rel.Direction != "inbound" {
t.Errorf("expected inbound, got %q", rel.Direction)
}
}
func TestParseMultipleRelTypes(t *testing.T) {
q, err := Parse(`MATCH (f)-[:CALLS|HTTP_CALLS]->(g) RETURN g.name`)
if err != nil {
t.Fatalf("parse: %v", err)
}
rel := q.Match.Pattern.Elements[1].(*RelPattern)
if len(rel.Types) != 2 {
t.Fatalf("expected 2 types, got %d", len(rel.Types))
}
if rel.Types[0] != "CALLS" || rel.Types[1] != "HTTP_CALLS" {
t.Errorf("expected [CALLS, HTTP_CALLS], got %v", rel.Types)
}
}
func TestParseWhereStartsWith(t *testing.T) {
q, err := Parse(`MATCH (f:Function) WHERE f.name STARTS WITH "Send" RETURN f`)
if err != nil {
t.Fatalf("parse: %v", err)
}
c := q.Where.Conditions[0]
if c.Operator != "STARTS WITH" {
t.Errorf("expected 'STARTS WITH', got %q", c.Operator)
}
if c.Value != "Send" {
t.Errorf("expected 'Send', got %q", c.Value)
}
}
func TestParseWhereContains(t *testing.T) {
q, err := Parse(`MATCH (f:Function) WHERE f.name CONTAINS "Handler" RETURN f`)
if err != nil {
t.Fatalf("parse: %v", err)
}
c := q.Where.Conditions[0]
if c.Operator != "CONTAINS" {
t.Errorf("expected CONTAINS, got %q", c.Operator)
}
}
func TestParseWhereNumericComparison(t *testing.T) {
q, err := Parse(`MATCH (f:Function) WHERE f.start_line > 10 RETURN f`)
if err != nil {
t.Fatalf("parse: %v", err)
}
c := q.Where.Conditions[0]
if c.Operator != ">" {
t.Errorf("expected '>', got %q", c.Operator)
}
if c.Value != "10" {
t.Errorf("expected '10', got %q", c.Value)
}
}
func TestParseWhereAnd(t *testing.T) {
q, err := Parse(`MATCH (f) WHERE f.label = "Function" AND f.name = "Foo" RETURN f`)
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(q.Where.Conditions) != 2 {
t.Fatalf("expected 2 conditions, got %d", len(q.Where.Conditions))
}
if q.Where.Operator != "AND" {
t.Errorf("expected AND, got %q", q.Where.Operator)
}
}
func TestParseDistinct(t *testing.T) {
q, err := Parse(`MATCH (f:Function) RETURN DISTINCT f.label`)
if err != nil {
t.Fatalf("parse: %v", err)
}
if !q.Return.Distinct {
t.Error("expected DISTINCT to be true")
}
}
// --- Integration test ---
func setupTestStore(t *testing.T) *store.Store {
t.Helper()
s, err := store.OpenMemory()
if err != nil {
t.Fatalf("open memory store: %v", err)
}
if err := s.UpsertProject("test", "/tmp/test"); err != nil {
t.Fatalf("upsert project: %v", err)
}
// Create nodes
idA, _ := s.UpsertNode(&store.Node{
Project: "test", Label: "Function", Name: "HandleOrder",
QualifiedName: "test.main.HandleOrder", FilePath: "main.go",
StartLine: 10, EndLine: 30,
Properties: map[string]any{"signature": "func HandleOrder(w, r)"},
})
idB, _ := s.UpsertNode(&store.Node{
Project: "test", Label: "Function", Name: "ValidateOrder",
QualifiedName: "test.service.ValidateOrder", FilePath: "service.go",
StartLine: 5, EndLine: 20,
Properties: map[string]any{"signature": "func ValidateOrder(o Order) error"},
})
idC, _ := s.UpsertNode(&store.Node{
Project: "test", Label: "Function", Name: "SubmitOrder",
QualifiedName: "test.service.SubmitOrder", FilePath: "service.go",
StartLine: 25, EndLine: 50,
Properties: map[string]any{"signature": "func SubmitOrder(o Order) error"},
})
idD, _ := s.UpsertNode(&store.Node{
Project: "test", Label: "Module", Name: "main",
QualifiedName: "test.main", FilePath: "main.go",
})
idE, _ := s.UpsertNode(&store.Node{
Project: "test", Label: "Function", Name: "LogError",
QualifiedName: "test.util.LogError", FilePath: "util.go",
StartLine: 1, EndLine: 5,
})
// Edges: HandleOrder -> ValidateOrder -> SubmitOrder
// HandleOrder -> LogError
s.InsertEdge(&store.Edge{Project: "test", SourceID: idA, TargetID: idB, Type: "CALLS"})
s.InsertEdge(&store.Edge{Project: "test", SourceID: idB, TargetID: idC, Type: "CALLS"})
s.InsertEdge(&store.Edge{Project: "test", SourceID: idA, TargetID: idE, Type: "CALLS"})
s.InsertEdge(&store.Edge{Project: "test", SourceID: idD, TargetID: idA, Type: "DEFINES"})
return s
}
func TestExecuteSimpleMatch(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 4 {
t.Errorf("expected 4 functions, got %d", len(result.Rows))
}
}
func TestExecuteRelationshipQuery(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function)-[:CALLS]->(g:Function) RETURN f.name, g.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
// HandleOrder -> ValidateOrder, HandleOrder -> LogError, ValidateOrder -> SubmitOrder
if len(result.Rows) != 3 {
t.Fatalf("expected 3 rows, got %d", len(result.Rows))
}
// Verify columns
if len(result.Columns) != 2 {
t.Errorf("expected 2 columns, got %d", len(result.Columns))
}
// Check that HandleOrder -> ValidateOrder is in the results
found := false
for _, row := range result.Rows {
if row["f.name"] == "HandleOrder" && row["g.name"] == "ValidateOrder" {
found = true
break
}
}
if !found {
t.Error("expected HandleOrder -> ValidateOrder in results")
}
}
func TestExecuteWhereFilter(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.name = "HandleOrder" RETURN f.name, f.file_path`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
if result.Rows[0]["f.name"] != "HandleOrder" {
t.Errorf("expected HandleOrder, got %v", result.Rows[0]["f.name"])
}
}
func TestExecuteWhereRegex(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.name =~ ".*Order" RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
// HandleOrder, ValidateOrder
if len(result.Rows) != 2 {
t.Errorf("expected 2 rows, got %d", len(result.Rows))
}
}
func TestExecuteWhereStartsWith(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.name STARTS WITH "Send" RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
if result.Rows[0]["f.name"] != "SubmitOrder" {
t.Errorf("expected SubmitOrder, got %v", result.Rows[0]["f.name"])
}
}
func TestExecuteWhereContains(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.name CONTAINS "Order" RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 2 {
t.Errorf("expected 2 rows (HandleOrder, ValidateOrder), got %d", len(result.Rows))
}
}
func TestExecuteWhereNumeric(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.start_line > 10 RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
// SubmitOrder (start_line=25)
if len(result.Rows) != 1 {
t.Errorf("expected 1 row, got %d", len(result.Rows))
}
}
func TestExecuteVariableLength(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
// HandleOrder calls ValidateOrder (hop 1), ValidateOrder calls SubmitOrder (hop 2)
result, err := exec.Execute(`MATCH (f:Function {name: "HandleOrder"})-[:CALLS*1..2]->(g:Function) RETURN g.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
// Should include ValidateOrder (hop 1), LogError (hop 1), SubmitOrder (hop 2)
if len(result.Rows) < 2 {
t.Errorf("expected at least 2 rows for variable-length path, got %d", len(result.Rows))
}
}
func TestExecuteWithLimit(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) RETURN f.name LIMIT 2`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 2 {
t.Errorf("expected 2 rows, got %d", len(result.Rows))
}
}
func TestExecuteWithOrderBy(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) RETURN f.name ORDER BY f.name ASC`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) < 2 {
t.Fatalf("expected at least 2 rows, got %d", len(result.Rows))
}
// First should be HandleOrder (alphabetically first)
firstName := result.Rows[0]["f.name"]
if firstName != "HandleOrder" {
t.Errorf("expected first row 'HandleOrder', got %v", firstName)
}
}
func TestExecuteCountAggregation(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function)-[:CALLS]->(g:Function) RETURN f.name, COUNT(g) AS call_count ORDER BY call_count DESC`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) < 1 {
t.Fatalf("expected at least 1 row, got %d", len(result.Rows))
}
// HandleOrder calls 2 functions (ValidateOrder, LogError)
for _, row := range result.Rows {
if row["f.name"] == "HandleOrder" {
count, ok := row["call_count"].(int)
if !ok {
t.Errorf("expected int count, got %T", row["call_count"])
} else if count != 2 {
t.Errorf("expected call_count=2 for HandleOrder, got %d", count)
}
}
}
}
func TestExecuteInboundRelationship(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
// Who calls ValidateOrder?
result, err := exec.Execute(`MATCH (f:Function)<-[:CALLS]-(g:Function) WHERE f.name = "ValidateOrder" RETURN g.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 caller, got %d", len(result.Rows))
}
if result.Rows[0]["g.name"] != "HandleOrder" {
t.Errorf("expected HandleOrder, got %v", result.Rows[0]["g.name"])
}
}
func TestExecuteDistinct(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) RETURN DISTINCT f.label`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Errorf("expected 1 distinct label, got %d", len(result.Rows))
}
if result.Rows[0]["f.label"] != "Function" {
t.Errorf("expected 'Function', got %v", result.Rows[0]["f.label"])
}
}
func TestExecuteInlinePropertyFilter(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function {name: "SubmitOrder"}) RETURN f.name, f.qualified_name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
if result.Rows[0]["f.name"] != "SubmitOrder" {
t.Errorf("expected SubmitOrder, got %v", result.Rows[0]["f.name"])
}
}
func TestExecuteNoResults(t *testing.T) {
s := setupTestStore(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (f:Function) WHERE f.name = "NonExistent" RETURN f.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 0 {
t.Errorf("expected 0 rows, got %d", len(result.Rows))
}
}
func TestParseError(t *testing.T) {
_, err := Parse(`NOT A VALID QUERY`)
if err == nil {
t.Error("expected parse error for invalid query")
}
}
// --- Edge property tests (Feature 2) ---
func setupTestStoreWithHTTPCalls(t *testing.T) *store.Store {
t.Helper()
s := setupTestStore(t)
// Add HTTP_CALLS edge with confidence
callerNode, _ := s.FindNodeByQN("test", "test.main.HandleOrder")
targetNode, _ := s.FindNodeByQN("test", "test.service.SubmitOrder")
if callerNode == nil || targetNode == nil {
t.Fatal("expected test nodes to exist")
}
s.InsertEdge(&store.Edge{
Project: "test",
SourceID: callerNode.ID,
TargetID: targetNode.ID,
Type: "HTTP_CALLS",
Properties: map[string]any{
"url_path": "/api/orders",
"confidence": 0.85,
"method": "POST",
},
})
return s
}
func TestExecuteEdgePropertyAccess(t *testing.T) {
s := setupTestStoreWithHTTPCalls(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (a:Function)-[r:HTTP_CALLS]->(b:Function) RETURN a.name, b.name, r.url_path, r.confidence`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
row := result.Rows[0]
if row["a.name"] != "HandleOrder" {
t.Errorf("a.name = %v, want HandleOrder", row["a.name"])
}
if row["b.name"] != "SubmitOrder" {
t.Errorf("b.name = %v, want SubmitOrder", row["b.name"])
}
if row["r.url_path"] != "/api/orders" {
t.Errorf("r.url_path = %v, want /api/orders", row["r.url_path"])
}
conf, ok := row["r.confidence"].(float64)
if !ok {
t.Errorf("r.confidence type = %T, want float64", row["r.confidence"])
} else if conf != 0.85 {
t.Errorf("r.confidence = %v, want 0.85", conf)
}
}
func TestExecuteEdgePropertyInWhere(t *testing.T) {
s := setupTestStoreWithHTTPCalls(t)
defer s.Close()
exec := &Executor{Store: s}
// Filter by confidence > 0.8
result, err := exec.Execute(`MATCH (a)-[r:HTTP_CALLS]->(b) WHERE r.confidence > 0.8 RETURN a.name, b.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
// Filter by confidence > 0.9 — should return nothing
result2, err := exec.Execute(`MATCH (a)-[r:HTTP_CALLS]->(b) WHERE r.confidence > 0.9 RETURN a.name`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result2.Rows) != 0 {
t.Errorf("expected 0 rows for confidence > 0.9, got %d", len(result2.Rows))
}
}
func TestExecuteEdgeType(t *testing.T) {
s := setupTestStoreWithHTTPCalls(t)
defer s.Close()
exec := &Executor{Store: s}
result, err := exec.Execute(`MATCH (a)-[r:HTTP_CALLS]->(b) RETURN r.type`)
if err != nil {
t.Fatalf("execute: %v", err)
}
if len(result.Rows) != 1 {
t.Fatalf("expected 1 row, got %d", len(result.Rows))
}
if result.Rows[0]["r.type"] != "HTTP_CALLS" {
t.Errorf("r.type = %v, want HTTP_CALLS", result.Rows[0]["r.type"])
}
}
internal/cypher/executor.go
+846
View File
@@ -0,0 +1,846 @@
package cypher
import (
"fmt"
"regexp"
"sort"
"strconv"
"strings"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
const maxResultRows = 200
// Executor runs Cypher execution plans against a store.
type Executor struct {
Store *store.Store
}
// Result holds the tabular output of a query.
type Result struct {
Columns []string `json:"columns"`
Rows []map[string]any `json:"rows"`
}
// binding maps variable names to matched nodes and edges.
type binding struct {
nodes map[string]*store.Node
edges map[string]*store.Edge
}
func newBinding() binding {
return binding{
nodes: make(map[string]*store.Node),
edges: make(map[string]*store.Edge),
}
}
// adjacentResult pairs a matched node with the edge that reached it.
type adjacentResult struct {
Node *store.Node
Edge *store.Edge
}
// Execute parses, plans, and executes a Cypher query across all projects.
func (e *Executor) Execute(query string) (*Result, error) {
q, err := Parse(query)
if err != nil {
return nil, fmt.Errorf("parse: %w", err)
}
plan, err := BuildPlan(q)
if err != nil {
return nil, fmt.Errorf("plan: %w", err)
}
return e.executePlan(plan)
}
func (e *Executor) executePlan(plan *Plan) (*Result, error) {
projects, err := e.Store.ListProjects()
if err != nil {
return nil, fmt.Errorf("list projects: %w", err)
}
var allBindings []binding
for _, proj := range projects {
bindings, err := e.executeStepsForProject(proj.Name, plan.Steps)
if err != nil {
continue // skip projects that error
}
allBindings = append(allBindings, bindings...)
if len(allBindings) > maxResultRows*2 {
allBindings = allBindings[:maxResultRows*2]
break
}
}
return e.projectResults(allBindings, plan.ReturnSpec)
}
func (e *Executor) executeStepsForProject(project string, steps []PlanStep) ([]binding, error) {
var bindings []binding
for i, step := range steps {
var err error
switch s := step.(type) {
case *ScanNodes:
bindings, err = e.execScan(project, s, bindings)
case *ExpandRelationship:
bindings, err = e.execExpand(s, bindings)
case *FilterWhere:
bindings, err = e.execFilter(s, bindings)
default:
return nil, fmt.Errorf("unknown step type: %T", step)
}
if err != nil {
return nil, err
}
// Only cap after the last step or after expand (which can explode).
// Never cap between scan and filter — the filter needs all candidates.
isLastStep := i == len(steps)-1
_, isExpand := step.(*ExpandRelationship)
if isLastStep || isExpand {
if len(bindings) > maxResultRows*2 {
bindings = bindings[:maxResultRows*2]
}
}
}
return bindings, nil
}
func (e *Executor) execScan(project string, s *ScanNodes, _ []binding) ([]binding, error) {
var nodes []*store.Node
var err error
if s.Label != "" {
nodes, err = e.Store.FindNodesByLabel(project, s.Label)
} else {
nodes, err = e.Store.AllNodes(project)
}
if err != nil {
return nil, fmt.Errorf("scan nodes: %w", err)
}
// Apply inline property filters
if len(s.Props) > 0 {
nodes = filterNodesByProps(nodes, s.Props)
}
var bindings []binding
for _, n := range nodes {
b := newBinding()
if s.Variable != "" {
b.nodes[s.Variable] = n
}
bindings = append(bindings, b)
}
return bindings, nil
}
func (e *Executor) execExpand(s *ExpandRelationship, bindings []binding) ([]binding, error) {
if len(bindings) == 0 {
return nil, nil
}
isVariableLength := s.MinHops != 1 || s.MaxHops != 1
var result []binding
for _, b := range bindings {
fromNode, ok := b.nodes[s.FromVar]
if !ok {
continue
}
if isVariableLength {
expanded, err := e.expandVariableLength(b, fromNode, s)
if err != nil {
return nil, err
}
result = append(result, expanded...)
} else {
expanded, err := e.expandFixedLength(b, fromNode, s)
if err != nil {
return nil, err
}
result = append(result, expanded...)
}
if len(result) > maxResultRows*2 {
result = result[:maxResultRows*2]
break
}
}
return result, nil
}
func (e *Executor) expandFixedLength(b binding, fromNode *store.Node, s *ExpandRelationship) ([]binding, error) {
adjacents, err := e.findAdjacentNodes(fromNode.ID, s.EdgeTypes, s.Direction)
if err != nil {
return nil, err
}
var result []binding
for _, adj := range adjacents {
if s.ToLabel != "" && adj.Node.Label != s.ToLabel {
continue
}
if len(s.ToProps) > 0 && !nodeMatchesProps(adj.Node, s.ToProps) {
continue
}
newB := copyBinding(b)
if s.ToVar != "" {
newB.nodes[s.ToVar] = adj.Node
}
if s.RelVar != "" && adj.Edge != nil {
newB.edges[s.RelVar] = adj.Edge
}
result = append(result, newB)
}
return result, nil
}
func (e *Executor) expandVariableLength(b binding, fromNode *store.Node, s *ExpandRelationship) ([]binding, error) {
maxDepth := s.MaxHops
if maxDepth == 0 {
maxDepth = 10 // cap unbounded at 10
}
direction := s.Direction
if direction == "" {
direction = "outbound"
}
edgeTypes := s.EdgeTypes
if len(edgeTypes) == 0 {
edgeTypes = []string{"CALLS"} // default
}
bfsResult, err := e.Store.BFS(fromNode.ID, direction, edgeTypes, maxDepth, maxResultRows)
if err != nil {
return nil, fmt.Errorf("bfs: %w", err)
}
var result []binding
for _, nh := range bfsResult.Visited {
if nh.Hop < s.MinHops {
continue
}
if s.MaxHops > 0 && nh.Hop > s.MaxHops {
continue
}
if s.ToLabel != "" && nh.Node.Label != s.ToLabel {
continue
}
if len(s.ToProps) > 0 && !nodeMatchesProps(nh.Node, s.ToProps) {
continue
}
newB := copyBinding(b)
if s.ToVar != "" {
newB.nodes[s.ToVar] = nh.Node
}
// Note: variable-length BFS doesn't bind individual edges
result = append(result, newB)
}
return result, nil
}
func (e *Executor) findAdjacentNodes(nodeID int64, edgeTypes []string, direction string) ([]adjacentResult, error) {
var allEdges []*store.Edge
switch direction {
case "outbound":
if len(edgeTypes) > 0 {
for _, et := range edgeTypes {
edges, err := e.Store.FindEdgesBySourceAndType(nodeID, et)
if err != nil {
return nil, err
}
allEdges = append(allEdges, edges...)
}
} else {
edges, err := e.Store.FindEdgesBySource(nodeID)
if err != nil {
return nil, err
}
allEdges = edges
}
case "inbound":
if len(edgeTypes) > 0 {
for _, et := range edgeTypes {
edges, err := e.Store.FindEdgesByTargetAndType(nodeID, et)
if err != nil {
return nil, err
}
allEdges = append(allEdges, edges...)
}
} else {
edges, err := e.Store.FindEdgesByTarget(nodeID)
if err != nil {
return nil, err
}
allEdges = edges
}
case "any":
outEdges, err := e.Store.FindEdgesBySource(nodeID)
if err != nil {
return nil, err
}
inEdges, err := e.Store.FindEdgesByTarget(nodeID)
if err != nil {
return nil, err
}
if len(edgeTypes) > 0 {
typeSet := make(map[string]bool, len(edgeTypes))
for _, et := range edgeTypes {
typeSet[et] = true
}
for _, edge := range outEdges {
if typeSet[edge.Type] {
allEdges = append(allEdges, edge)
}
}
for _, edge := range inEdges {
if typeSet[edge.Type] {
allEdges = append(allEdges, edge)
}
}
} else {
allEdges = append(outEdges, inEdges...)
}
default:
edges, err := e.Store.FindEdgesBySource(nodeID)
if err != nil {
return nil, err
}
allEdges = edges
}
// Resolve edge targets/sources to nodes, preserving the edge
seen := make(map[int64]bool)
var results []adjacentResult
for _, edge := range allEdges {
var targetID int64
switch direction {
case "inbound":
targetID = edge.SourceID
case "any":
if edge.SourceID == nodeID {
targetID = edge.TargetID
} else {
targetID = edge.SourceID
}
default:
targetID = edge.TargetID
}
if seen[targetID] {
continue
}
seen[targetID] = true
node, err := e.Store.FindNodeByID(targetID)
if err != nil || node == nil {
continue
}
results = append(results, adjacentResult{Node: node, Edge: edge})
}
return results, nil
}
func (e *Executor) execFilter(s *FilterWhere, bindings []binding) ([]binding, error) {
var result []binding
for _, b := range bindings {
match, err := evaluateConditions(b, s.Conditions, s.Operator)
if err != nil {
return nil, err
}
if match {
result = append(result, b)
}
}
return result, nil
}
func evaluateConditions(b binding, conditions []Condition, op string) (bool, error) {
if op == "OR" {
for _, c := range conditions {
ok, err := evaluateCondition(b, c)
if err != nil {
return false, err
}
if ok {
return true, nil
}
}
return false, nil
}
// AND (default)
for _, c := range conditions {
ok, err := evaluateCondition(b, c)
if err != nil {
return false, err
}
if !ok {
return false, nil
}
}
return true, nil
}
func evaluateCondition(b binding, c Condition) (bool, error) {
// Try node first, then edge
var actual any
if node, ok := b.nodes[c.Variable]; ok {
actual = getNodeProperty(node, c.Property)
} else if edge, ok := b.edges[c.Variable]; ok {
actual = getEdgeProperty(edge, c.Property)
} else {
return false, nil
}
switch c.Operator {
case "=":
return fmt.Sprintf("%v", actual) == c.Value, nil
case "=~":
s, ok := actual.(string)
if !ok {
return false, nil
}
matched, err := regexp.MatchString(c.Value, s)
if err != nil {
return false, fmt.Errorf("regex %q: %w", c.Value, err)
}
return matched, nil
case "CONTAINS":
s, ok := actual.(string)
if !ok {
return false, nil
}
return strings.Contains(s, c.Value), nil
case "STARTS WITH":
s, ok := actual.(string)
if !ok {
return false, nil
}
return strings.HasPrefix(s, c.Value), nil
case ">", "<", ">=", "<=":
return compareNumeric(actual, c.Value, c.Operator)
default:
return false, fmt.Errorf("unsupported operator: %s", c.Operator)
}
}
func compareNumeric(actual any, expected string, op string) (bool, error) {
expectedNum, err := strconv.ParseFloat(expected, 64)
if err != nil {
return false, nil
}
var actualNum float64
switch v := actual.(type) {
case int:
actualNum = float64(v)
case int64:
actualNum = float64(v)
case float64:
actualNum = v
case string:
n, err := strconv.ParseFloat(v, 64)
if err != nil {
return false, nil
}
actualNum = n
default:
return false, nil
}
switch op {
case ">":
return actualNum > expectedNum, nil
case "<":
return actualNum < expectedNum, nil
case ">=":
return actualNum >= expectedNum, nil
case "<=":
return actualNum <= expectedNum, nil
default:
return false, nil
}
}
func getNodeProperty(n *store.Node, prop string) any {
switch prop {
case "name":
return n.Name
case "qualified_name":
return n.QualifiedName
case "label":
return n.Label
case "file_path":
return n.FilePath
case "start_line":
return n.StartLine
case "end_line":
return n.EndLine
case "id":
return n.ID
case "project":
return n.Project
default:
if n.Properties != nil {
if v, ok := n.Properties[prop]; ok {
return v
}
}
return nil
}
}
// getEdgeProperty returns a property value from an edge.
func getEdgeProperty(edge *store.Edge, prop string) any {
switch prop {
case "type":
return edge.Type
case "id":
return edge.ID
case "source_id":
return edge.SourceID
case "target_id":
return edge.TargetID
default:
if edge.Properties != nil {
if v, ok := edge.Properties[prop]; ok {
return v
}
}
return nil
}
}
func (e *Executor) projectResults(bindings []binding, ret *ReturnClause) (*Result, error) {
if ret == nil {
return e.defaultProjection(bindings)
}
// Check if we have a COUNT aggregation
hasCount := false
for _, item := range ret.Items {
if item.Func == "COUNT" {
hasCount = true
break
}
}
if hasCount {
return e.aggregateResults(bindings, ret)
}
return e.simpleProjection(bindings, ret)
}
func (e *Executor) defaultProjection(bindings []binding) (*Result, error) {
if len(bindings) == 0 {
return &Result{Columns: []string{}, Rows: []map[string]any{}}, nil
}
// Collect all variable names from nodes and edges
varSet := make(map[string]bool)
edgeVarSet := make(map[string]bool)
for _, b := range bindings {
for k := range b.nodes {
varSet[k] = true
}
for k := range b.edges {
edgeVarSet[k] = true
}
}
var cols []string
for k := range varSet {
cols = append(cols, k+".name", k+".qualified_name", k+".label")
}
for k := range edgeVarSet {
cols = append(cols, k+".type")
}
sort.Strings(cols)
var rows []map[string]any
for _, b := range bindings {
row := make(map[string]any)
for varName, node := range b.nodes {
row[varName+".name"] = node.Name
row[varName+".qualified_name"] = node.QualifiedName
row[varName+".label"] = node.Label
}
for varName, edge := range b.edges {
row[varName+".type"] = edge.Type
}
rows = append(rows, row)
}
if len(rows) > maxResultRows {
rows = rows[:maxResultRows]
}
return &Result{Columns: cols, Rows: rows}, nil
}
func (e *Executor) simpleProjection(bindings []binding, ret *ReturnClause) (*Result, error) {
var cols []string
for _, item := range ret.Items {
col := item.Variable
if item.Property != "" {
col = item.Variable + "." + item.Property
}
if item.Alias != "" {
col = item.Alias
}
cols = append(cols, col)
}
seen := make(map[string]bool)
var rows []map[string]any
for _, b := range bindings {
row := make(map[string]any)
for i, item := range ret.Items {
// Try node first, then edge
if node, ok := b.nodes[item.Variable]; ok {
if item.Property == "" {
row[cols[i]] = map[string]any{
"name": node.Name,
"qualified_name": node.QualifiedName,
"label": node.Label,
"file_path": node.FilePath,
"start_line": node.StartLine,
"end_line": node.EndLine,
}
} else {
row[cols[i]] = getNodeProperty(node, item.Property)
}
} else if edge, ok := b.edges[item.Variable]; ok {
if item.Property == "" {
row[cols[i]] = map[string]any{
"type": edge.Type,
"source_id": edge.SourceID,
"target_id": edge.TargetID,
}
} else {
row[cols[i]] = getEdgeProperty(edge, item.Property)
}
} else {
row[cols[i]] = nil
}
}
// DISTINCT check
if ret.Distinct {
key := fmt.Sprintf("%v", row)
if seen[key] {
continue
}
seen[key] = true
}
rows = append(rows, row)
}
// ORDER BY
if ret.OrderBy != "" {
orderCol := ret.OrderBy
// Find the matching column name
for i, item := range ret.Items {
if item.Alias == orderCol {
orderCol = cols[i]
break
}
}
sortRows(rows, orderCol, ret.OrderDir)
}
// LIMIT
limit := ret.Limit
if limit <= 0 || limit > maxResultRows {
limit = maxResultRows
}
if len(rows) > limit {
rows = rows[:limit]
}
return &Result{Columns: cols, Rows: rows}, nil
}
func (e *Executor) aggregateResults(bindings []binding, ret *ReturnClause) (*Result, error) {
// Group by non-COUNT items
var groupItems []ReturnItem
var countItem ReturnItem
for _, item := range ret.Items {
if item.Func == "COUNT" {
countItem = item
} else {
groupItems = append(groupItems, item)
}
}
// Build grouping key -> count
type groupEntry struct {
key string
row map[string]any
count int
}
groups := make(map[string]*groupEntry)
var order []string
for _, b := range bindings {
row := make(map[string]any)
var keyParts []string
for _, item := range groupItems {
col := item.Variable
if item.Property != "" {
col = item.Variable + "." + item.Property
}
if item.Alias != "" {
col = item.Alias
}
var val any
if node, ok := b.nodes[item.Variable]; ok {
val = getNodeProperty(node, item.Property)
} else if edge, ok := b.edges[item.Variable]; ok {
val = getEdgeProperty(edge, item.Property)
}
row[col] = val
keyParts = append(keyParts, fmt.Sprintf("%v", val))
}
key := strings.Join(keyParts, "\x00")
if g, ok := groups[key]; ok {
g.count++
} else {
groups[key] = &groupEntry{key: key, row: row, count: 1}
order = append(order, key)
}
}
// Build columns
var cols []string
for _, item := range ret.Items {
col := item.Variable
if item.Property != "" {
col = item.Variable + "." + item.Property
}
if item.Alias != "" {
col = item.Alias
}
cols = append(cols, col)
}
// Build result rows
countCol := countItem.Alias
if countCol == "" {
countCol = "COUNT(" + countItem.Variable + ")"
}
var rows []map[string]any
for _, key := range order {
g := groups[key]
row := g.row
row[countCol] = g.count
rows = append(rows, row)
}
// ORDER BY
if ret.OrderBy != "" {
sortRows(rows, ret.OrderBy, ret.OrderDir)
}
// LIMIT
limit := ret.Limit
if limit <= 0 || limit > maxResultRows {
limit = maxResultRows
}
if len(rows) > limit {
rows = rows[:limit]
}
return &Result{Columns: cols, Rows: rows}, nil
}
// sortRows sorts rows by the given column.
func sortRows(rows []map[string]any, col string, dir string) {
sort.SliceStable(rows, func(i, j int) bool {
a, b := rows[i][col], rows[j][col]
cmp := compareValues(a, b)
if dir == "DESC" {
return cmp > 0
}
return cmp < 0
})
}
func compareValues(a, b any) int {
// Try numeric
aNum, aOK := toFloat(a)
bNum, bOK := toFloat(b)
if aOK && bOK {
if aNum < bNum {
return -1
}
if aNum > bNum {
return 1
}
return 0
}
// Fall back to string
aStr := fmt.Sprintf("%v", a)
bStr := fmt.Sprintf("%v", b)
if aStr < bStr {
return -1
}
if aStr > bStr {
return 1
}
return 0
}
func toFloat(v any) (float64, bool) {
switch n := v.(type) {
case int:
return float64(n), true
case int64:
return float64(n), true
case float64:
return n, true
default:
return 0, false
}
}
// copyBinding makes a shallow copy of a binding.
func copyBinding(b binding) binding {
c := newBinding()
for k, v := range b.nodes {
c.nodes[k] = v
}
for k, v := range b.edges {
c.edges[k] = v
}
return c
}
// filterNodesByProps filters nodes by inline property key-value pairs.
func filterNodesByProps(nodes []*store.Node, props map[string]string) []*store.Node {
var filtered []*store.Node
for _, n := range nodes {
if nodeMatchesProps(n, props) {
filtered = append(filtered, n)
}
}
return filtered
}
// nodeMatchesProps checks if a node matches all given property filters.
func nodeMatchesProps(n *store.Node, props map[string]string) bool {
for key, val := range props {
actual := getNodeProperty(n, key)
if fmt.Sprintf("%v", actual) != val {
return false
}
}
return true
}
internal/cypher/lexer.go
+281
View File
@@ -0,0 +1,281 @@
package cypher
import (
"fmt"
"strings"
"unicode"
)
// TokenType classifies a lexer token.
type TokenType int
const (
// Keywords
TokMatch TokenType = iota // MATCH
TokWhere // WHERE
TokReturn // RETURN
TokOrder // ORDER
TokBy // BY
TokLimit // LIMIT
TokAnd // AND
TokOr // OR
TokAs // AS
TokDistinct // DISTINCT
TokCount // COUNT
TokContains // CONTAINS
TokStarts // STARTS
TokWith // WITH
TokNot // NOT
TokAsc // ASC
TokDesc // DESC
// Symbols
TokLParen // (
TokRParen // )
TokLBracket // [
TokRBracket // ]
TokDash // -
TokGT // >
TokLT // <
TokColon // :
TokDot // .
TokLBrace // {
TokRBrace // }
TokStar // *
TokComma // ,
TokEQ // =
TokRegex // =~
TokGTE // >=
TokLTE // <=
TokPipe // |
TokDotDot // ..
// Literals
TokIdent // identifier
TokString // "..." or '...'
TokNumber // integer
TokEOF // end of input
)
// Token is a single lexer token.
type Token struct {
Type TokenType
Value string
Pos int // byte offset in the input
}
func (t Token) String() string {
return fmt.Sprintf("Token(%d, %q, pos=%d)", t.Type, t.Value, t.Pos)
}
// keywords maps uppercase keyword strings to their token type.
var keywords = map[string]TokenType{
"MATCH": TokMatch,
"WHERE": TokWhere,
"RETURN": TokReturn,
"ORDER": TokOrder,
"BY": TokBy,
"LIMIT": TokLimit,
"AND": TokAnd,
"OR": TokOr,
"AS": TokAs,
"DISTINCT": TokDistinct,
"COUNT": TokCount,
"CONTAINS": TokContains,
"STARTS": TokStarts,
"WITH": TokWith,
"NOT": TokNot,
"ASC": TokAsc,
"DESC": TokDesc,
}
// Lexer tokenizes a Cypher query string.
type Lexer struct {
input string
pos int
tokens []Token
}
// Lex tokenizes the input string into a slice of tokens.
func Lex(input string) ([]Token, error) {
l := &Lexer{input: input}
if err := l.tokenize(); err != nil {
return nil, err
}
return l.tokens, nil
}
func (l *Lexer) tokenize() error {
for l.pos < len(l.input) {
ch := l.input[l.pos]
// Skip whitespace
if unicode.IsSpace(rune(ch)) {
l.pos++
continue
}
// Skip line comments
if ch == '/' && l.pos+1 < len(l.input) && l.input[l.pos+1] == '/' {
for l.pos < len(l.input) && l.input[l.pos] != '\n' {
l.pos++
}
continue
}
// Skip block comments
if ch == '/' && l.pos+1 < len(l.input) && l.input[l.pos+1] == '*' {
l.pos += 2
for l.pos+1 < len(l.input) {
if l.input[l.pos] == '*' && l.input[l.pos+1] == '/' {
l.pos += 2
break
}
l.pos++
}
continue
}
switch {
case ch == '(':
l.emit(TokLParen, "(")
case ch == ')':
l.emit(TokRParen, ")")
case ch == '[':
l.emit(TokLBracket, "[")
case ch == ']':
l.emit(TokRBracket, "]")
case ch == '{':
l.emit(TokLBrace, "{")
case ch == '}':
l.emit(TokRBrace, "}")
case ch == '*':
l.emit(TokStar, "*")
case ch == ',':
l.emit(TokComma, ",")
case ch == '|':
l.emit(TokPipe, "|")
case ch == ':':
l.emit(TokColon, ":")
case ch == '.':
if l.pos+1 < len(l.input) && l.input[l.pos+1] == '.' {
l.pos++
l.emit(TokDotDot, "..")
} else {
l.emit(TokDot, ".")
}
case ch == '-':
l.emit(TokDash, "-")
case ch == '>':
if l.pos+1 < len(l.input) && l.input[l.pos+1] == '=' {
l.pos++
l.emit(TokGTE, ">=")
} else {
l.emit(TokGT, ">")
}
case ch == '<':
if l.pos+1 < len(l.input) && l.input[l.pos+1] == '=' {
l.pos++
l.emit(TokLTE, "<=")
} else {
l.emit(TokLT, "<")
}
case ch == '=':
if l.pos+1 < len(l.input) && l.input[l.pos+1] == '~' {
l.pos++
l.emit(TokRegex, "=~")
} else {
l.emit(TokEQ, "=")
}
case ch == '"' || ch == '\'':
if err := l.lexString(ch); err != nil {
return err
}
continue // lexString advances pos itself
case isDigit(ch):
l.lexNumber()
continue // lexNumber advances pos itself
case isIdentStart(ch):
l.lexIdent()
continue // lexIdent advances pos itself
default:
return fmt.Errorf("unexpected char %q at pos %d", string(ch), l.pos)
}
l.pos++
}
l.tokens = append(l.tokens, Token{Type: TokEOF, Value: "", Pos: l.pos})
return nil
}
func (l *Lexer) emit(typ TokenType, val string) {
l.tokens = append(l.tokens, Token{Type: typ, Value: val, Pos: l.pos})
}
func (l *Lexer) lexString(quote byte) error {
start := l.pos
l.pos++ // skip opening quote
var sb strings.Builder
for l.pos < len(l.input) {
ch := l.input[l.pos]
if ch == '\\' && l.pos+1 < len(l.input) {
l.pos++
sb.WriteByte(l.input[l.pos])
l.pos++
continue
}
if ch == quote {
l.tokens = append(l.tokens, Token{Type: TokString, Value: sb.String(), Pos: start})
l.pos++ // skip closing quote
return nil
}
sb.WriteByte(ch)
l.pos++
}
return fmt.Errorf("unterminated string at pos %d", start)
}
func (l *Lexer) lexNumber() {
start := l.pos
for l.pos < len(l.input) && isDigit(l.input[l.pos]) {
l.pos++
}
// Handle decimal point (e.g. 0.9, 3.14) — but not ".." (range operator)
if l.pos < len(l.input) && l.input[l.pos] == '.' {
if l.pos+1 < len(l.input) && l.input[l.pos+1] != '.' && isDigit(l.input[l.pos+1]) {
l.pos++ // consume '.'
for l.pos < len(l.input) && isDigit(l.input[l.pos]) {
l.pos++
}
}
}
l.tokens = append(l.tokens, Token{Type: TokNumber, Value: l.input[start:l.pos], Pos: start})
}
func (l *Lexer) lexIdent() {
start := l.pos
for l.pos < len(l.input) && isIdentPart(l.input[l.pos]) {
l.pos++
}
word := l.input[start:l.pos]
upper := strings.ToUpper(word)
if tok, ok := keywords[upper]; ok {
l.tokens = append(l.tokens, Token{Type: tok, Value: upper, Pos: start})
} else {
l.tokens = append(l.tokens, Token{Type: TokIdent, Value: word, Pos: start})
}
}
func isDigit(ch byte) bool {
return ch >= '0' && ch <= '9'
}
func isIdentStart(ch byte) bool {
return (ch >= 'a' && ch <= 'z') || (ch >= 'A' && ch <= 'Z') || ch == '_'
}
func isIdentPart(ch byte) bool {
return isIdentStart(ch) || isDigit(ch)
}
internal/cypher/parser.go
+558
View File
@@ -0,0 +1,558 @@
package cypher
import (
"fmt"
"strconv"
)
// Parser converts a token stream into an AST.
type Parser struct {
tokens []Token
pos int
}
// Parse tokenizes and parses a Cypher query string into an AST.
func Parse(input string) (*Query, error) {
tokens, err := Lex(input)
if err != nil {
return nil, fmt.Errorf("lex: %w", err)
}
p := &Parser{tokens: tokens}
return p.parseQuery()
}
func (p *Parser) peek() Token {
if p.pos >= len(p.tokens) {
return Token{Type: TokEOF}
}
return p.tokens[p.pos]
}
func (p *Parser) advance() Token {
t := p.peek()
p.pos++
return t
}
func (p *Parser) expect(typ TokenType) (Token, error) {
t := p.advance()
if t.Type != typ {
return t, fmt.Errorf("expected token %d, got %d (%q) at pos %d", typ, t.Type, t.Value, t.Pos)
}
return t, nil
}
func (p *Parser) parseQuery() (*Query, error) {
q := &Query{}
// MATCH clause (required)
if p.peek().Type != TokMatch {
return nil, fmt.Errorf("expected MATCH at pos %d, got %q", p.peek().Pos, p.peek().Value)
}
m, err := p.parseMatch()
if err != nil {
return nil, err
}
q.Match = m
// WHERE clause (optional)
if p.peek().Type == TokWhere {
w, err := p.parseWhere()
if err != nil {
return nil, err
}
q.Where = w
}
// RETURN clause (optional but common)
if p.peek().Type == TokReturn {
r, err := p.parseReturn()
if err != nil {
return nil, err
}
q.Return = r
}
return q, nil
}
func (p *Parser) parseMatch() (*MatchClause, error) {
if _, err := p.expect(TokMatch); err != nil {
return nil, err
}
pat, err := p.parsePattern()
if err != nil {
return nil, fmt.Errorf("match pattern: %w", err)
}
return &MatchClause{Pattern: pat}, nil
}
func (p *Parser) parsePattern() (*Pattern, error) {
pat := &Pattern{}
// First element must be a node
node, err := p.parseNodePattern()
if err != nil {
return nil, err
}
pat.Elements = append(pat.Elements, node)
// Parse alternating rel-node pairs
for p.isRelStart() {
rel, nextNode, err := p.parseRelAndNode()
if err != nil {
return nil, err
}
pat.Elements = append(pat.Elements, rel, nextNode)
}
return pat, nil
}
// isRelStart checks whether the next tokens begin a relationship pattern.
// Patterns: -[...]-> or <-[...]- or -[...]-
func (p *Parser) isRelStart() bool {
t := p.peek()
return t.Type == TokDash || t.Type == TokLT
}
func (p *Parser) parseRelAndNode() (*RelPattern, *NodePattern, error) {
rel := &RelPattern{MinHops: 1, MaxHops: 1}
// Determine direction by looking at leading token
// Possibilities:
// -[...]->(node) outbound
// <-[...]-(node) inbound
// -[...]-(node) any
leadingArrow := false
if p.peek().Type == TokLT {
leadingArrow = true
p.advance() // consume <
}
// Expect dash
if _, err := p.expect(TokDash); err != nil {
return nil, nil, fmt.Errorf("expected '-' in relationship: %w", err)
}
// Optional bracket section [...]
if p.peek().Type == TokLBracket {
if err := p.parseRelBracket(rel); err != nil {
return nil, nil, err
}
}
// Expect dash
if _, err := p.expect(TokDash); err != nil {
return nil, nil, fmt.Errorf("expected '-' after relationship: %w", err)
}
// Check trailing arrow
trailingArrow := false
if p.peek().Type == TokGT {
trailingArrow = true
p.advance() // consume >
}
// Determine direction
switch {
case !leadingArrow && trailingArrow:
rel.Direction = "outbound"
case leadingArrow && !trailingArrow:
rel.Direction = "inbound"
default:
rel.Direction = "any"
}
// Parse the next node
node, err := p.parseNodePattern()
if err != nil {
return nil, nil, err
}
return rel, node, nil
}
func (p *Parser) parseRelBracket(rel *RelPattern) error {
p.advance() // consume [
// Optional variable name
if p.peek().Type == TokIdent {
rel.Variable = p.advance().Value
}
// Optional :TYPE or :TYPE1|TYPE2
if p.peek().Type == TokColon {
p.advance() // consume :
types, err := p.parseRelTypes()
if err != nil {
return err
}
rel.Types = types
}
// Optional *min..max for variable-length
if p.peek().Type == TokStar {
p.advance() // consume *
if err := p.parseHopRange(rel); err != nil {
return err
}
}
// Expect ]
if _, err := p.expect(TokRBracket); err != nil {
return fmt.Errorf("expected ']' to close relationship: %w", err)
}
return nil
}
func (p *Parser) parseRelTypes() ([]string, error) {
var types []string
t := p.advance()
if t.Type != TokIdent {
return nil, fmt.Errorf("expected relationship type name, got %q at pos %d", t.Value, t.Pos)
}
types = append(types, t.Value)
// Handle TYPE1|TYPE2
for p.peek().Type == TokPipe {
p.advance() // consume |
t = p.advance()
if t.Type != TokIdent {
return nil, fmt.Errorf("expected relationship type after '|', got %q at pos %d", t.Value, t.Pos)
}
types = append(types, t.Value)
}
return types, nil
}
func (p *Parser) parseHopRange(rel *RelPattern) error {
// Possibilities after *:
// *1..3 min=1, max=3
// *..3 min=1, max=3
// *1.. min=1, max=0 (unbounded)
// *3 min=1, max=3 (shorthand)
// (empty) min=1, max=0 (unbounded)
if p.peek().Type == TokNumber {
n, _ := strconv.Atoi(p.advance().Value)
if p.peek().Type == TokDotDot {
// *N..M or *N..
rel.MinHops = n
p.advance() // consume ..
if p.peek().Type == TokNumber {
m, _ := strconv.Atoi(p.advance().Value)
rel.MaxHops = m
} else {
rel.MaxHops = 0 // unbounded
}
} else {
// *N (shorthand for *1..N)
rel.MinHops = 1
rel.MaxHops = n
}
} else if p.peek().Type == TokDotDot {
// *..M
p.advance() // consume ..
rel.MinHops = 1
if p.peek().Type == TokNumber {
m, _ := strconv.Atoi(p.advance().Value)
rel.MaxHops = m
} else {
rel.MaxHops = 0
}
} else {
// Just * with no range: unbounded
rel.MinHops = 1
rel.MaxHops = 0
}
return nil
}
func (p *Parser) parseNodePattern() (*NodePattern, error) {
if _, err := p.expect(TokLParen); err != nil {
return nil, fmt.Errorf("expected '(' for node pattern: %w", err)
}
node := &NodePattern{}
// Optional variable name
if p.peek().Type == TokIdent {
node.Variable = p.advance().Value
}
// Optional :Label
if p.peek().Type == TokColon {
p.advance() // consume :
t := p.advance()
if t.Type != TokIdent {
return nil, fmt.Errorf("expected label name after ':', got %q at pos %d", t.Value, t.Pos)
}
node.Label = t.Value
}
// Optional {key: "val", ...}
if p.peek().Type == TokLBrace {
props, err := p.parseInlineProps()
if err != nil {
return nil, err
}
node.Props = props
}
if _, err := p.expect(TokRParen); err != nil {
return nil, fmt.Errorf("expected ')' to close node pattern: %w", err)
}
return node, nil
}
func (p *Parser) parseInlineProps() (map[string]string, error) {
p.advance() // consume {
props := make(map[string]string)
for p.peek().Type != TokRBrace {
if len(props) > 0 {
if _, err := p.expect(TokComma); err != nil {
return nil, fmt.Errorf("expected ',' between properties: %w", err)
}
}
// key
keyTok := p.advance()
if keyTok.Type != TokIdent {
return nil, fmt.Errorf("expected property key, got %q at pos %d", keyTok.Value, keyTok.Pos)
}
// :
if _, err := p.expect(TokColon); err != nil {
return nil, fmt.Errorf("expected ':' after property key: %w", err)
}
// value (string)
valTok := p.advance()
if valTok.Type != TokString {
return nil, fmt.Errorf("expected string value for property %q, got %q at pos %d", keyTok.Value, valTok.Value, valTok.Pos)
}
props[keyTok.Value] = valTok.Value
}
p.advance() // consume }
return props, nil
}
func (p *Parser) parseWhere() (*WhereClause, error) {
p.advance() // consume WHERE
w := &WhereClause{Operator: "AND"}
cond, err := p.parseCondition()
if err != nil {
return nil, err
}
w.Conditions = append(w.Conditions, cond)
for p.peek().Type == TokAnd || p.peek().Type == TokOr {
op := p.advance()
if op.Type == TokOr {
w.Operator = "OR"
}
cond, err := p.parseCondition()
if err != nil {
return nil, err
}
w.Conditions = append(w.Conditions, cond)
}
return w, nil
}
func (p *Parser) parseCondition() (Condition, error) {
c := Condition{}
// variable.property
varTok := p.advance()
if varTok.Type != TokIdent {
return c, fmt.Errorf("expected variable name in condition, got %q at pos %d", varTok.Value, varTok.Pos)
}
c.Variable = varTok.Value
if _, err := p.expect(TokDot); err != nil {
return c, fmt.Errorf("expected '.' after variable in condition: %w", err)
}
propTok := p.advance()
if propTok.Type != TokIdent {
return c, fmt.Errorf("expected property name in condition, got %q at pos %d", propTok.Value, propTok.Pos)
}
c.Property = propTok.Value
// Operator
op := p.peek()
switch op.Type {
case TokEQ:
c.Operator = "="
p.advance()
case TokRegex:
c.Operator = "=~"
p.advance()
case TokGT:
c.Operator = ">"
p.advance()
case TokLT:
c.Operator = "<"
p.advance()
case TokGTE:
c.Operator = ">="
p.advance()
case TokLTE:
c.Operator = "<="
p.advance()
case TokContains:
c.Operator = "CONTAINS"
p.advance()
case TokStarts:
// STARTS WITH
p.advance() // consume STARTS
if p.peek().Type != TokWith {
return c, fmt.Errorf("expected WITH after STARTS at pos %d", p.peek().Pos)
}
p.advance() // consume WITH
c.Operator = "STARTS WITH"
default:
return c, fmt.Errorf("expected comparison operator, got %q at pos %d", op.Value, op.Pos)
}
// Value (string or number)
valTok := p.advance()
switch valTok.Type {
case TokString:
c.Value = valTok.Value
case TokNumber:
c.Value = valTok.Value
default:
return c, fmt.Errorf("expected value in condition, got %q at pos %d", valTok.Value, valTok.Pos)
}
return c, nil
}
func (p *Parser) parseReturn() (*ReturnClause, error) {
p.advance() // consume RETURN
r := &ReturnClause{OrderDir: "ASC"}
// Optional DISTINCT
if p.peek().Type == TokDistinct {
r.Distinct = true
p.advance()
}
// Parse return items
item, err := p.parseReturnItem()
if err != nil {
return nil, err
}
r.Items = append(r.Items, item)
for p.peek().Type == TokComma {
p.advance() // consume ,
item, err := p.parseReturnItem()
if err != nil {
return nil, err
}
r.Items = append(r.Items, item)
}
// Optional ORDER BY
if p.peek().Type == TokOrder {
p.advance() // consume ORDER
if _, err := p.expect(TokBy); err != nil {
return nil, fmt.Errorf("expected BY after ORDER: %w", err)
}
orderTok := p.advance()
if orderTok.Type != TokIdent {
return nil, fmt.Errorf("expected field name for ORDER BY, got %q", orderTok.Value)
}
orderField := orderTok.Value
if p.peek().Type == TokDot {
p.advance() // consume .
propTok := p.advance()
orderField = orderField + "." + propTok.Value
}
r.OrderBy = orderField
// Optional ASC/DESC
if p.peek().Type == TokAsc {
r.OrderDir = "ASC"
p.advance()
} else if p.peek().Type == TokDesc {
r.OrderDir = "DESC"
p.advance()
}
}
// Optional LIMIT
if p.peek().Type == TokLimit {
p.advance() // consume LIMIT
numTok := p.advance()
if numTok.Type != TokNumber {
return nil, fmt.Errorf("expected number after LIMIT, got %q", numTok.Value)
}
n, _ := strconv.Atoi(numTok.Value)
r.Limit = n
}
return r, nil
}
func (p *Parser) parseReturnItem() (ReturnItem, error) {
item := ReturnItem{}
// Check for COUNT(variable)
if p.peek().Type == TokCount {
p.advance() // consume COUNT
item.Func = "COUNT"
if _, err := p.expect(TokLParen); err != nil {
return item, fmt.Errorf("expected '(' after COUNT: %w", err)
}
varTok := p.advance()
if varTok.Type != TokIdent {
return item, fmt.Errorf("expected variable in COUNT(), got %q", varTok.Value)
}
item.Variable = varTok.Value
if _, err := p.expect(TokRParen); err != nil {
return item, fmt.Errorf("expected ')' after COUNT variable: %w", err)
}
} else {
// variable or variable.property
varTok := p.advance()
if varTok.Type != TokIdent {
return item, fmt.Errorf("expected variable in RETURN item, got %q at pos %d", varTok.Value, varTok.Pos)
}
item.Variable = varTok.Value
if p.peek().Type == TokDot {
p.advance() // consume .
propTok := p.advance()
if propTok.Type != TokIdent {
return item, fmt.Errorf("expected property after '.', got %q", propTok.Value)
}
item.Property = propTok.Value
}
}
// Optional AS alias
if p.peek().Type == TokAs {
p.advance() // consume AS
aliasTok := p.advance()
if aliasTok.Type != TokIdent {
return item, fmt.Errorf("expected alias after AS, got %q", aliasTok.Value)
}
item.Alias = aliasTok.Value
}
return item, nil
}
internal/cypher/planner.go
+129
View File
@@ -0,0 +1,129 @@
package cypher
// Plan represents an execution plan for a parsed Cypher query.
type Plan struct {
Steps []PlanStep
ReturnSpec *ReturnClause
}
// PlanStep is a single step in the execution plan.
type PlanStep interface {
stepType() string
}
// ScanNodes finds nodes matching label and/or inline property filters.
type ScanNodes struct {
Variable string
Label string
Props map[string]string // inline property filters
}
func (*ScanNodes) stepType() string { return "scan" }
// ExpandRelationship follows edges from bound nodes to match target nodes.
type ExpandRelationship struct {
FromVar string // source variable (already bound)
ToVar string // target variable (to bind)
RelVar string // optional relationship variable (to bind edge)
ToLabel string // optional label filter on target
ToProps map[string]string
EdgeTypes []string // required edge types
Direction string // "outbound", "inbound", "any"
MinHops int
MaxHops int
}
func (*ExpandRelationship) stepType() string { return "expand" }
// FilterWhere applies WHERE conditions to the bindings.
type FilterWhere struct {
Conditions []Condition
Operator string // "AND" or "OR"
}
func (*FilterWhere) stepType() string { return "filter" }
// BuildPlan converts a parsed Query AST into an execution Plan.
func BuildPlan(q *Query) (*Plan, error) {
plan := &Plan{ReturnSpec: q.Return}
elements := q.Match.Pattern.Elements
if len(elements) == 0 {
return plan, nil
}
// First element is always a node pattern
firstNode := elements[0].(*NodePattern)
plan.Steps = append(plan.Steps, &ScanNodes{
Variable: firstNode.Variable,
Label: firstNode.Label,
Props: firstNode.Props,
})
// Optimization: push WHERE conditions that reference only the first
// scan variable BEFORE any expand steps. This dramatically reduces the
// number of bindings that need to be expanded.
var earlyFilters []Condition
var lateFilters []Condition
if q.Where != nil {
scanVar := firstNode.Variable
hasExpand := len(elements) > 1
if hasExpand && q.Where.Operator == "AND" {
// Split conditions: those referencing only scanVar go early
for _, c := range q.Where.Conditions {
if c.Variable == scanVar {
earlyFilters = append(earlyFilters, c)
} else {
lateFilters = append(lateFilters, c)
}
}
} else {
// Can't split OR conditions or when there's no expand
lateFilters = q.Where.Conditions
}
}
// Insert early filter right after scan
if len(earlyFilters) > 0 {
plan.Steps = append(plan.Steps, &FilterWhere{
Conditions: earlyFilters,
Operator: "AND",
})
}
// Process relationship-node pairs
for i := 1; i+1 < len(elements); i += 2 {
rel := elements[i].(*RelPattern)
targetNode := elements[i+1].(*NodePattern)
plan.Steps = append(plan.Steps, &ExpandRelationship{
FromVar: elements[i-1].(*NodePattern).Variable,
ToVar: targetNode.Variable,
RelVar: rel.Variable,
ToLabel: targetNode.Label,
ToProps: targetNode.Props,
EdgeTypes: rel.Types,
Direction: rel.Direction,
MinHops: rel.MinHops,
MaxHops: rel.MaxHops,
})
}
// Late WHERE filter (conditions referencing expand variables)
if len(lateFilters) > 0 {
plan.Steps = append(plan.Steps, &FilterWhere{
Conditions: lateFilters,
Operator: q.Where.Operator,
})
} else if q.Where != nil && len(earlyFilters) == 0 {
// No split happened — add all conditions at the end
plan.Steps = append(plan.Steps, &FilterWhere{
Conditions: q.Where.Conditions,
Operator: q.Where.Operator,
})
}
return plan, nil
}
internal/discover/discover.go
+175
View File
@@ -0,0 +1,175 @@
package discover
import (
"bufio"
"os"
"path/filepath"
"strings"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
)
// IGNORE_PATTERNS are directory names to skip during discovery.
var IGNORE_PATTERNS = map[string]bool{
".cache": true, ".claude": true, ".eclipse": true, ".eggs": true,
".env": true, ".git": true, ".gradle": true, ".hg": true,
".idea": true, ".maven": true, ".mypy_cache": true, ".nox": true,
".npm": true, ".nyc_output": true, ".pnpm-store": true,
".pytest_cache": true, ".qdrant_code_embeddings": true,
".ruff_cache": true, ".svn": true, ".tmp": true, ".tox": true,
".venv": true, ".vs": true, ".vscode": true, ".yarn": true,
"__pycache__": true, "bin": true, "bower_components": true,
"build": true, "coverage": true, "dist": true, "env": true,
"htmlcov": true, "node_modules": true, "obj": true, "out": true,
"Pods": true, "site-packages": true, "target": true, "temp": true,
"tmp": true, "vendor": true, "venv": true,
}
// IGNORE_SUFFIXES are file suffixes to skip.
var IGNORE_SUFFIXES = map[string]bool{
".tmp": true, "~": true, ".pyc": true, ".pyo": true,
".o": true, ".a": true, ".so": true, ".dll": true, ".class": true,
}
// FileInfo represents a discovered source file.
type FileInfo struct {
Path string // absolute path
RelPath string // relative to repo root
Language lang.Language // detected language
}
// Options configures file discovery.
type Options struct {
IgnoreFile string // path to .cgrignore file (optional)
}
// Discover walks a repository and returns all source files.
func Discover(repoPath string, opts *Options) ([]FileInfo, error) {
repoPath, err := filepath.Abs(repoPath)
if err != nil {
return nil, err
}
// Load .cgrignore patterns if present
var extraIgnore []string
if opts != nil && opts.IgnoreFile != "" {
extraIgnore, _ = loadIgnoreFile(opts.IgnoreFile)
} else {
ignPath := filepath.Join(repoPath, ".cgrignore")
extraIgnore, _ = loadIgnoreFile(ignPath)
}
var files []FileInfo
err = filepath.Walk(repoPath, func(path string, info os.FileInfo, walkErr error) error {
if walkErr != nil {
return nil // skip errors
}
rel, _ := filepath.Rel(repoPath, path)
if info.IsDir() {
name := info.Name()
// Check ignore patterns
if IGNORE_PATTERNS[name] {
return filepath.SkipDir
}
// Check .cgrignore patterns
for _, pattern := range extraIgnore {
if matched, _ := filepath.Match(pattern, name); matched {
return filepath.SkipDir
}
if matched, _ := filepath.Match(pattern, rel); matched {
return filepath.SkipDir
}
}
return nil
}
// Skip ignored suffixes
for suffix := range IGNORE_SUFFIXES {
if strings.HasSuffix(path, suffix) {
return nil
}
}
// Check if we support this language
ext := filepath.Ext(path)
l, ok := lang.LanguageForExtension(ext)
if ok {
files = append(files, FileInfo{
Path: path,
RelPath: filepath.ToSlash(rel),
Language: l,
})
return nil
}
// JSON files: pick up selectively (skip tool configs, lock files)
if ext == ".json" && !isIgnoredJSON(info.Name()) {
files = append(files, FileInfo{
Path: path,
RelPath: filepath.ToSlash(rel),
Language: lang.JSON,
})
}
return nil
})
return files, err
}
// ignoredJSONFiles are JSON filenames to skip (tool configs, lock files, specs).
var ignoredJSONFiles = map[string]bool{
"package.json": true,
"package-lock.json": true,
"tsconfig.json": true,
"jsconfig.json": true,
"composer.json": true,
"composer.lock": true,
"yarn.lock": true,
"openapi.json": true,
"swagger.json": true,
"jest.config.json": true,
".eslintrc.json": true,
".prettierrc.json": true,
".babelrc.json": true,
"tslint.json": true,
"angular.json": true,
"firebase.json": true,
"renovate.json": true,
"lerna.json": true,
"turbo.json": true,
".stylelintrc.json": true,
"pnpm-lock.json": true,
"deno.json": true,
"biome.json": true,
"devcontainer.json": true,
".devcontainer.json": true,
"launch.json": true,
"settings.json": true,
"extensions.json": true,
"tasks.json": true,
}
func isIgnoredJSON(name string) bool {
return ignoredJSONFiles[name]
}
func loadIgnoreFile(path string) ([]string, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
var patterns []string
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line != "" && !strings.HasPrefix(line, "#") {
patterns = append(patterns, line)
}
}
return patterns, scanner.Err()
}
internal/fqn/fqn.go
+45
View File
@@ -0,0 +1,45 @@
package fqn
import (
"path/filepath"
"strings"
)
// Compute returns the canonical qualified name for a node.
// Format: <project>.<rel_path_parts_dotted>.<name>
// Examples:
// - myproject.cmd.server.main.HandleRequest
// - myproject.pkg.service.ProcessOrder
func Compute(project, relPath, name string) string {
// Remove file extension
relPath = strings.TrimSuffix(relPath, filepath.Ext(relPath))
// Convert path separators to dots
parts := strings.Split(filepath.ToSlash(relPath), "/")
// For Python __init__.py, drop the __init__ part
if len(parts) > 0 && parts[len(parts)-1] == "__init__" {
parts = parts[:len(parts)-1]
}
// For JS/TS index files
if len(parts) > 0 && parts[len(parts)-1] == "index" {
parts = parts[:len(parts)-1]
}
all := append([]string{project}, parts...)
if name != "" {
all = append(all, name)
}
return strings.Join(all, ".")
}
// ModuleQN returns the qualified name for a module (file without function name).
func ModuleQN(project, relPath string) string {
return Compute(project, relPath, "")
}
// FolderQN returns the qualified name for a folder.
func FolderQN(project, relDir string) string {
parts := strings.Split(filepath.ToSlash(relDir), "/")
all := append([]string{project}, parts...)
return strings.Join(all, ".")
}
internal/httplink/httplink.go
+943
View File
@@ -0,0 +1,943 @@
package httplink
import (
"bufio"
"encoding/json"
"fmt"
"log/slog"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
// RouteHandler represents a discovered HTTP route handler.
type RouteHandler struct {
Path string
Method string
FunctionName string
QualifiedName string
}
// HTTPCallSite represents a discovered HTTP call site.
type HTTPCallSite struct {
Path string
Method string // best-effort: "GET", "POST", etc. or "" if unknown
SourceName string
SourceQualifiedName string
SourceLabel string // "Function", "Method", or "Module"
}
// HTTPLink represents a matched HTTP call from caller to handler.
type HTTPLink struct {
CallerQN string
CallerLabel string
HandlerQN string
URLPath string
}
// Linker discovers cross-service HTTP calls and creates HTTP_CALLS edges.
type Linker struct {
store *store.Store
project string
}
// New creates a new HTTP Linker.
func New(s *store.Store, project string) *Linker {
return &Linker{store: s, project: project}
}
// regex patterns for route and URL discovery
var (
// Python decorators: @app.post("/path"), @router.get("/path")
pyRouteRe = regexp.MustCompile(`@\w+\.(get|post|put|delete|patch)\(\s*["']([^"']+)["']`)
// Go gin routes: .POST("/path", .GET("/path"
goRouteRe = regexp.MustCompile(`\.(GET|POST|PUT|DELETE|PATCH)\(\s*["']([^"']+)["']`)
// Express.js routes: app.get("/path", router.post("/path"
expressRouteRe = regexp.MustCompile(`\w+\.(get|post|put|delete|patch)\(\s*["'` + "`" + `]([^"'` + "`" + `]+)["'` + "`" + `]`)
// Java Spring annotations: @GetMapping("/path"), @PostMapping, @RequestMapping
springMappingRe = regexp.MustCompile(`@(Get|Post|Put|Delete|Patch|Request)Mapping\(\s*(?:value\s*=\s*)?["']([^"']+)["']`)
// Rust Actix annotations: #[get("/path")], #[post("/path")]
actixRouteRe = regexp.MustCompile(`#\[(get|post|put|delete|patch)\(\s*"([^"]+)"`)
// PHP Laravel routes: Route::get("/path", Route::post("/path"
laravelRouteRe = regexp.MustCompile(`Route::(get|post|put|delete|patch)\(\s*["']([^"']+)["']`)
// URL patterns in source: https://host/path or http://host/path — captures domain and path
urlRe = regexp.MustCompile(`https?://([a-zA-Z0-9.\-]+)(/[a-zA-Z0-9_/:.\-]+)`)
// Path-only patterns: "/api/something" (quoted paths starting with /)
pathRe = regexp.MustCompile(`["'](/[a-zA-Z0-9_/:.\-]{2,})["']`)
// Path param normalizers
colonParamRe = regexp.MustCompile(`:[a-zA-Z_]+`)
braceParamRe = regexp.MustCompile(`\{[a-zA-Z_]+\}`)
)
// Run executes the HTTP linking pass.
func (l *Linker) Run() ([]HTTPLink, error) {
proj, err := l.store.GetProject(l.project)
if err != nil {
return nil, fmt.Errorf("get project: %w", err)
}
rootPath := proj.RootPath
routes := l.discoverRoutes(rootPath)
slog.Info("httplink.routes", "count", len(routes))
// Insert Route nodes and HANDLES edges
l.insertRouteNodes(routes)
callSites := l.discoverCallSites(rootPath)
slog.Info("httplink.callsites", "count", len(callSites))
links := l.matchAndLink(routes, callSites)
slog.Info("httplink.links", "count", len(links))
return links, nil
}
// insertRouteNodes creates Route nodes for each discovered route handler and
// HANDLES edges from the handler function to the Route node.
func (l *Linker) insertRouteNodes(routes []RouteHandler) {
for _, rh := range routes {
// Build a stable qualified name for the Route node
normalMethod := rh.Method
if normalMethod == "" {
normalMethod = "ANY"
}
normalPath := strings.ReplaceAll(rh.Path, "/", "_")
normalPath = strings.Trim(normalPath, "_")
routeQN := rh.QualifiedName + ".route." + normalMethod + "." + normalPath
routeName := normalMethod + " " + rh.Path
routeID, err := l.store.UpsertNode(&store.Node{
Project: l.project,
Label: "Route",
Name: routeName,
QualifiedName: routeQN,
Properties: map[string]any{
"method": rh.Method,
"path": rh.Path,
"handler": rh.QualifiedName,
},
})
if err != nil || routeID == 0 {
continue
}
// Create HANDLES edge from handler → Route
handlerNode, _ := l.store.FindNodeByQN(l.project, rh.QualifiedName)
if handlerNode != nil {
l.store.InsertEdge(&store.Edge{
Project: l.project,
SourceID: handlerNode.ID,
TargetID: routeID,
Type: "HANDLES",
})
// Mark handler as entry point (for Feature 4)
if handlerNode.Properties == nil {
handlerNode.Properties = map[string]any{}
}
handlerNode.Properties["is_entry_point"] = true
l.store.UpsertNode(handlerNode)
}
}
slog.Info("httplink.route_nodes", "count", len(routes))
}
// discoverRoutes finds route handler registrations from Function nodes.
func (l *Linker) discoverRoutes(rootPath string) []RouteHandler {
var routes []RouteHandler
funcs, err := l.store.FindNodesByLabel(l.project, "Function")
if err != nil {
slog.Warn("httplink.routes.funcs.err", "err", err)
return routes
}
methods, err := l.store.FindNodesByLabel(l.project, "Method")
if err != nil {
slog.Warn("httplink.routes.methods.err", "err", err)
} else {
funcs = append(funcs, methods...)
}
for _, f := range funcs {
// Python: check decorators property
routes = append(routes, extractPythonRoutes(f)...)
// Java: check annotation-based decorators (Spring)
routes = append(routes, extractJavaRoutes(f)...)
// Rust: check attribute decorators (Actix)
routes = append(routes, extractRustRoutes(f)...)
// Source-based route discovery (Go gin, Express.js, PHP Laravel)
if f.FilePath != "" && f.StartLine > 0 && f.EndLine > 0 {
source := readSourceLines(rootPath, f.FilePath, f.StartLine, f.EndLine)
if source != "" {
routes = append(routes, extractGoRoutes(f, source)...)
routes = append(routes, extractExpressRoutes(f, source)...)
routes = append(routes, extractLaravelRoutes(f, source)...)
}
}
}
return routes
}
// extractPythonRoutes extracts route handlers from Python decorator metadata.
func extractPythonRoutes(f *store.Node) []RouteHandler {
var routes []RouteHandler
decs, ok := f.Properties["decorators"]
if !ok {
return routes
}
// decorators is stored as []any (JSON deserialized)
decList, ok := decs.([]any)
if !ok {
return routes
}
for _, d := range decList {
decStr, ok := d.(string)
if !ok {
continue
}
matches := pyRouteRe.FindAllStringSubmatch(decStr, -1)
for _, m := range matches {
routes = append(routes, RouteHandler{
Path: m[2],
Method: strings.ToUpper(m[1]),
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
}
return routes
}
// extractGoRoutes extracts route registrations from Go source code (gin patterns).
func extractGoRoutes(f *store.Node, source string) []RouteHandler {
var routes []RouteHandler
matches := goRouteRe.FindAllStringSubmatch(source, -1)
for _, m := range matches {
routes = append(routes, RouteHandler{
Path: m[2],
Method: strings.ToUpper(m[1]),
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
return routes
}
// extractExpressRoutes extracts route registrations from JS/TS source (Express/Koa patterns).
func extractExpressRoutes(f *store.Node, source string) []RouteHandler {
var routes []RouteHandler
matches := expressRouteRe.FindAllStringSubmatch(source, -1)
for _, m := range matches {
routes = append(routes, RouteHandler{
Path: m[2],
Method: strings.ToUpper(m[1]),
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
return routes
}
// extractJavaRoutes extracts routes from Java Spring annotations in decorators.
func extractJavaRoutes(f *store.Node) []RouteHandler {
var routes []RouteHandler
decs, ok := f.Properties["decorators"]
if !ok {
return routes
}
decList, ok := decs.([]any)
if !ok {
return routes
}
for _, d := range decList {
decStr, ok := d.(string)
if !ok {
continue
}
matches := springMappingRe.FindAllStringSubmatch(decStr, -1)
for _, m := range matches {
method := strings.ToUpper(m[1])
if method == "REQUEST" {
method = "" // RequestMapping doesn't specify method
}
routes = append(routes, RouteHandler{
Path: m[2],
Method: method,
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
}
return routes
}
// extractRustRoutes extracts routes from Rust Actix attribute decorators.
func extractRustRoutes(f *store.Node) []RouteHandler {
var routes []RouteHandler
decs, ok := f.Properties["decorators"]
if !ok {
return routes
}
decList, ok := decs.([]any)
if !ok {
return routes
}
for _, d := range decList {
decStr, ok := d.(string)
if !ok {
continue
}
matches := actixRouteRe.FindAllStringSubmatch(decStr, -1)
for _, m := range matches {
routes = append(routes, RouteHandler{
Path: m[2],
Method: strings.ToUpper(m[1]),
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
}
return routes
}
// extractLaravelRoutes extracts route registrations from PHP Laravel source.
func extractLaravelRoutes(f *store.Node, source string) []RouteHandler {
var routes []RouteHandler
matches := laravelRouteRe.FindAllStringSubmatch(source, -1)
for _, m := range matches {
routes = append(routes, RouteHandler{
Path: m[2],
Method: strings.ToUpper(m[1]),
FunctionName: f.Name,
QualifiedName: f.QualifiedName,
})
}
return routes
}
// discoverCallSites finds HTTP URL references in Module constants and Function source.
func (l *Linker) discoverCallSites(rootPath string) []HTTPCallSite {
var sites []HTTPCallSite
// Module constants
modules, err := l.store.FindNodesByLabel(l.project, "Module")
if err != nil {
slog.Warn("httplink.callsites.modules.err", "err", err)
} else {
for _, m := range modules {
sites = append(sites, extractModuleCallSites(m)...)
}
}
// Function/Method source
funcs, err := l.store.FindNodesByLabel(l.project, "Function")
if err != nil {
slog.Warn("httplink.callsites.funcs.err", "err", err)
} else {
for _, f := range funcs {
sites = append(sites, extractFunctionCallSites(f, rootPath)...)
}
}
methods, err := l.store.FindNodesByLabel(l.project, "Method")
if err != nil {
slog.Warn("httplink.callsites.methods.err", "err", err)
} else {
for _, f := range methods {
sites = append(sites, extractFunctionCallSites(f, rootPath)...)
}
}
return sites
}
// extractModuleCallSites extracts HTTP paths from module constants.
func extractModuleCallSites(m *store.Node) []HTTPCallSite {
var sites []HTTPCallSite
constants, ok := m.Properties["constants"]
if !ok {
return sites
}
constList, ok := constants.([]any)
if !ok {
return sites
}
for _, c := range constList {
cStr, ok := c.(string)
if !ok {
continue
}
paths := extractURLPaths(cStr)
for _, p := range paths {
sites = append(sites, HTTPCallSite{
Path: p,
SourceName: m.Name,
SourceQualifiedName: m.QualifiedName,
SourceLabel: "Module",
})
}
}
return sites
}
// detectHTTPMethod tries to find the HTTP method used near a URL path in source code.
func detectHTTPMethod(source string) string {
upper := strings.ToUpper(source)
for _, verb := range []string{"POST", "PUT", "DELETE", "PATCH", "GET"} {
// Python: requests.post(, httpx.post(
if strings.Contains(upper, "REQUESTS."+verb+"(") || strings.Contains(upper, "HTTPX."+verb+"(") {
return verb
}
// Go: "POST" near http.NewRequest
if strings.Contains(upper, `"`+verb+`"`) && strings.Contains(upper, "HTTP.") {
return verb
}
// JS: method: "POST", method: 'POST'
if strings.Contains(upper, "METHOD") && strings.Contains(upper, verb) {
return verb
}
// Java: HttpMethod.POST, .method(POST
if strings.Contains(upper, "HTTPMETHOD."+verb) {
return verb
}
// Rust: reqwest::Client::new().post(, .get(
if strings.Contains(source, "."+strings.ToLower(verb)+"(") {
return verb
}
// PHP: curl CURLOPT_CUSTOMREQUEST
if strings.Contains(upper, "CURLOPT") && strings.Contains(upper, verb) {
return verb
}
}
return ""
}
// httpClientKeywords are patterns indicating actual HTTP client usage.
// A function must contain at least one of these to be considered an HTTP call site.
var httpClientKeywords = []string{
// Python
"requests.get", "requests.post", "requests.put", "requests.delete", "requests.patch",
"httpx.", "aiohttp.", "urllib.request",
// Go
"http.Get", "http.Post", "http.NewRequest", "client.Do(",
// JavaScript/TypeScript
"fetch(", "axios.", ".ajax(",
// Java
"HttpClient", "RestTemplate", "WebClient", "OkHttpClient",
"HttpURLConnection", "openConnection(",
// Rust
"reqwest::", "hyper::", "surf::", "ureq::",
// PHP
"curl_exec", "curl_init", "Guzzle", "Http::get", "Http::post",
// Scala
"sttp.", "http4s", "HttpClient", "wsClient",
// C++
"curl_easy", "cpr::Get", "cpr::Post", "httplib::",
// Lua
"socket.http", "http.request", "curl.",
// Generic
"send_request", "http_client",
}
// extractFunctionCallSites extracts HTTP paths from function source code.
func extractFunctionCallSites(f *store.Node, rootPath string) []HTTPCallSite {
var sites []HTTPCallSite
if f.FilePath == "" || f.StartLine <= 0 || f.EndLine <= 0 {
return sites
}
// Skip Python dunder methods — they configure, not call
if strings.HasPrefix(f.Name, "__") && strings.HasSuffix(f.Name, "__") {
return sites
}
source := readSourceLines(rootPath, f.FilePath, f.StartLine, f.EndLine)
if source == "" {
return sites
}
// Require at least one HTTP client keyword to avoid false positives
// from functions that merely store URL strings in variables
hasHTTPClient := false
for _, kw := range httpClientKeywords {
if strings.Contains(source, kw) {
hasHTTPClient = true
break
}
}
if !hasHTTPClient {
return sites
}
method := detectHTTPMethod(source)
paths := extractURLPaths(source)
for _, p := range paths {
sites = append(sites, HTTPCallSite{
Path: p,
Method: method,
SourceName: f.Name,
SourceQualifiedName: f.QualifiedName,
SourceLabel: f.Label,
})
}
return sites
}
// externalDomains are well-known external API domains whose paths
// should not be matched against internal route handlers.
var externalDomains = []string{
"googleapis.com",
"google.com",
"github.com",
"gitlab.com",
"docker.com",
"docker.io",
"npmjs.org",
"pypi.org",
"cloudflare.com",
"sentry.io",
"aws.amazon.com",
}
// isExternalDomain checks if a domain is a well-known external API.
func isExternalDomain(domain string) bool {
domain = strings.ToLower(domain)
for _, ext := range externalDomains {
if domain == ext || strings.HasSuffix(domain, "."+ext) {
return true
}
}
return false
}
// extractURLPaths finds URL path segments from text.
func extractURLPaths(text string) []string {
seen := map[string]bool{}
var paths []string
// Full URLs: extract domain and path, skip external domains
for _, m := range urlRe.FindAllStringSubmatch(text, -1) {
domain := m[1]
p := m[2]
if isExternalDomain(domain) {
continue
}
if !seen[p] {
seen[p] = true
paths = append(paths, p)
}
}
// Quoted path literals
for _, m := range pathRe.FindAllStringSubmatch(text, -1) {
p := m[1]
if !seen[p] {
seen[p] = true
paths = append(paths, p)
}
}
// Try to extract URLs from embedded JSON strings (e.g., Cloud Tasks payloads)
for _, p := range extractJSONStringPaths(text) {
if !seen[p] {
seen[p] = true
paths = append(paths, p)
}
}
return paths
}
// extractJSONStringPaths tries to JSON-parse the text (or substrings that look
// like JSON) and extract URL paths from string values within.
func extractJSONStringPaths(text string) []string {
seen := make(map[string]bool)
var paths []string
// Find JSON-like substrings: {...} or [...]
for _, bounds := range findJSONBounds(text) {
var parsed any
if err := json.Unmarshal([]byte(bounds), &parsed); err != nil {
continue
}
var raw []string
walkJSONForURLs(parsed, &raw)
for _, p := range raw {
if !seen[p] {
seen[p] = true
paths = append(paths, p)
}
}
}
return paths
}
// findJSONBounds extracts substrings that look like JSON objects or arrays.
func findJSONBounds(text string) []string {
var results []string
for _, opener := range []byte{'{', '['} {
closer := byte('}')
if opener == '[' {
closer = ']'
}
start := strings.IndexByte(text, opener)
for start >= 0 && start < len(text) {
depth := 0
inStr := false
for i := start; i < len(text); i++ {
ch := text[i]
if inStr {
if ch == '\\' {
i++ // skip escaped char
continue
}
if ch == '"' {
inStr = false
}
continue
}
if ch == '"' {
inStr = true
} else if ch == opener {
depth++
} else if ch == closer {
depth--
if depth == 0 {
candidate := text[start : i+1]
if len(candidate) > 5 { // skip trivially small
results = append(results, candidate)
}
start = i + 1
break
}
}
}
if depth != 0 {
break
}
next := strings.IndexByte(text[start:], opener)
if next < 0 {
break
}
start += next
}
}
return results
}
// walkJSONForURLs recursively walks parsed JSON and extracts URL paths.
func walkJSONForURLs(v any, out *[]string) {
switch val := v.(type) {
case map[string]any:
for _, child := range val {
walkJSONForURLs(child, out)
}
case []any:
for _, child := range val {
walkJSONForURLs(child, out)
}
case string:
// Check if value is a URL or path
for _, m := range urlRe.FindAllStringSubmatch(val, -1) {
if !isExternalDomain(m[1]) {
*out = append(*out, m[2])
}
}
for _, m := range pathRe.FindAllStringSubmatch(`"`+val+`"`, -1) {
*out = append(*out, m[1])
}
}
}
// matchAndLink matches call site paths to route handler paths and creates edges.
// Uses multi-signal probabilistic scoring (path Jaccard, depth, method, source type).
// Only creates edges above the confidence threshold.
func (l *Linker) matchAndLink(routes []RouteHandler, callSites []HTTPCallSite) []HTTPLink {
var links []HTTPLink
for _, cs := range callSites {
for _, rh := range routes {
if sameService(cs.SourceQualifiedName, rh.QualifiedName) {
continue
}
// Multi-signal confidence scoring
pathScore := pathMatchScore(cs.Path, rh.Path)
if pathScore == 0 {
continue
}
score := pathScore*sourceWeight(cs.SourceLabel) + methodBonus(cs.Method, rh.Method)
if score < matchConfidenceThreshold {
continue
}
if score > 1.0 {
score = 1.0
}
// Create HTTP_CALLS edge with confidence score
callerNode, _ := l.store.FindNodeByQN(l.project, cs.SourceQualifiedName)
handlerNode, _ := l.store.FindNodeByQN(l.project, rh.QualifiedName)
if callerNode != nil && handlerNode != nil {
props := map[string]any{
"url_path": cs.Path,
"confidence": score,
}
if rh.Method != "" {
props["method"] = rh.Method
}
_, _ = l.store.InsertEdge(&store.Edge{
Project: l.project,
SourceID: callerNode.ID,
TargetID: handlerNode.ID,
Type: "HTTP_CALLS",
Properties: props,
})
}
links = append(links, HTTPLink{
CallerQN: cs.SourceQualifiedName,
CallerLabel: cs.SourceLabel,
HandlerQN: rh.QualifiedName,
URLPath: cs.Path,
})
}
}
return links
}
// normalizePath normalizes a URL path for comparison.
func normalizePath(path string) string {
path = strings.TrimRight(path, "/")
path = colonParamRe.ReplaceAllString(path, "*")
path = braceParamRe.ReplaceAllString(path, "*")
return strings.ToLower(path)
}
// matchConfidenceThreshold is the minimum score for an HTTP_CALLS edge.
const matchConfidenceThreshold = 0.3
// pathMatchScore returns a confidence score (0.0–1.0) for how well callPath
// matches routePath. Returns 0 if no match.
//
// Multi-signal scoring (inspired by RAD/Code2DFD research):
// confidence = matchBase × (0.5 × jaccard + 0.5 × depthFactor)
//
// Where:
// matchBase: exact=0.95, suffix=0.75, wildcard=0.55
// jaccard: segment Jaccard similarity (non-wildcard segments)
// depthFactor: min(matched_segments / 3.0, 1.0) — longer paths = more specific
func pathMatchScore(callPath, routePath string) float64 {
normCall := normalizePath(callPath)
normRoute := normalizePath(routePath)
if normCall == "" || normRoute == "" {
return 0
}
// Determine structural match type
var matchBase float64
var matchedCallSegs, matchedRouteSegs []string
if normCall == normRoute {
matchBase = 0.95
matchedCallSegs = splitSegments(normCall)
matchedRouteSegs = splitSegments(normRoute)
} else if strings.HasSuffix(normCall, normRoute) {
matchBase = 0.75
matchedCallSegs = splitSegments(normRoute) // use the route portion that matched
matchedRouteSegs = splitSegments(normRoute)
} else {
// Segment-by-segment wildcard matching
callParts := strings.Split(normCall, "/")
routeParts := strings.Split(normRoute, "/")
if len(callParts) != len(routeParts) {
return 0
}
for i := range callParts {
if callParts[i] != routeParts[i] && callParts[i] != "*" && routeParts[i] != "*" {
return 0
}
}
matchBase = 0.55
matchedCallSegs = splitSegments(normCall)
matchedRouteSegs = splitSegments(normRoute)
}
// Jaccard similarity on non-empty, non-wildcard segments
jaccard := segmentJaccard(matchedCallSegs, matchedRouteSegs)
// Depth factor: more segments = more specific match
totalSegs := len(matchedRouteSegs)
depthFactor := float64(totalSegs) / 3.0
if depthFactor > 1.0 {
depthFactor = 1.0
}
score := matchBase * (0.5*jaccard + 0.5*depthFactor)
if score > 1.0 {
score = 1.0
}
return score
}
// splitSegments splits a normalized path into non-empty segments.
func splitSegments(path string) []string {
var segs []string
for _, s := range strings.Split(path, "/") {
if s != "" {
segs = append(segs, s)
}
}
return segs
}
// segmentJaccard computes Jaccard similarity on non-wildcard path segments.
// Wildcards (*) are excluded from both sets since they match anything.
func segmentJaccard(segsA, segsB []string) float64 {
setA := make(map[string]bool)
setB := make(map[string]bool)
for _, s := range segsA {
if s != "*" {
setA[s] = true
}
}
for _, s := range segsB {
if s != "*" {
setB[s] = true
}
}
if len(setA) == 0 && len(setB) == 0 {
return 0
}
intersection := 0
for k := range setA {
if setB[k] {
intersection++
}
}
union := len(setA)
for k := range setB {
if !setA[k] {
union++
}
}
if union == 0 {
return 0
}
return float64(intersection) / float64(union)
}
// methodBonus returns a confidence adjustment based on HTTP method matching.
//
// +0.10 if both methods are known and match
// 0.00 if one or both methods are unknown
// -0.15 if both methods are known and mismatch
func methodBonus(callMethod, routeMethod string) float64 {
if callMethod == "" || routeMethod == "" {
return 0
}
if strings.EqualFold(callMethod, routeMethod) {
return 0.10
}
return -0.15
}
// sourceWeight returns a confidence multiplier based on call site type.
// Function/Method sources are higher confidence (HTTP client in source code)
// than Module sources (URL in constants — may be config, not a call).
func sourceWeight(label string) float64 {
switch label {
case "Function", "Method":
return 1.0
default:
return 0.85
}
}
// pathsMatch is a convenience wrapper for tests — returns true if score >= threshold.
func pathsMatch(callPath, routePath string) bool {
return pathMatchScore(callPath, routePath) >= matchConfidenceThreshold
}
// sameService checks if two qualified names share the same directory path.
// It strips the last 2 segments (module file + function/method name) from each
// QN and compares the remaining directory prefix. If the prefixes are identical,
// the nodes are in the same deployable unit.
//
// Example: "myapp.docker-images.cloud-runs.svcA.module.func" → dir prefix "myapp.docker-images.cloud-runs.svcA"
// "myapp.docker-images.cloud-runs.svcB.routes.handler" → dir prefix "myapp.docker-images.cloud-runs.svcB"
// → different prefix → different service → returns false
func sameService(qn1, qn2 string) bool {
parts1 := strings.Split(qn1, ".")
parts2 := strings.Split(qn2, ".")
// Strip last 2 segments (module + name) to get directory path
const strip = 2
if len(parts1) <= strip || len(parts2) <= strip {
return false
}
dir1 := strings.Join(parts1[:len(parts1)-strip], ".")
dir2 := strings.Join(parts2[:len(parts2)-strip], ".")
return dir1 == dir2
}
// readSourceLines reads specific lines from a file on disk.
func readSourceLines(rootPath, relPath string, startLine, endLine int) string {
absPath := filepath.Join(rootPath, relPath)
f, err := os.Open(absPath)
if err != nil {
return ""
}
defer f.Close()
var lines []string
scanner := bufio.NewScanner(f)
lineNum := 0
for scanner.Scan() {
lineNum++
if lineNum >= startLine && lineNum <= endLine {
lines = append(lines, scanner.Text())
}
if lineNum > endLine {
break
}
}
return strings.Join(lines, "\n")
}
internal/httplink/httplink_test.go
+525
View File
@@ -0,0 +1,525 @@
package httplink
import (
"os"
"path/filepath"
"testing"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
func TestNormalizePath(t *testing.T) {
tests := []struct {
input string
want string
}{
{"/api/orders/", "/api/orders"},
{"/api/orders", "/api/orders"},
{"/api/orders/:id", "/api/orders/*"},
{"/api/orders/{order_id}", "/api/orders/*"},
{"/API/Orders", "/api/orders"},
{"/api/:version/items/:id", "/api/*/items/*"},
{"/api/{version}/items/{id}", "/api/*/items/*"},
{"/", ""},
{"", ""},
}
for _, tt := range tests {
got := normalizePath(tt.input)
if got != tt.want {
t.Errorf("normalizePath(%q) = %q, want %q", tt.input, got, tt.want)
}
}
}
func TestPathsMatch(t *testing.T) {
tests := []struct {
callPath string
routePath string
want bool
}{
// Exact match
{"/api/orders", "/api/orders", true},
{"/api/orders/", "/api/orders", true},
// Case insensitive
{"/API/Orders", "/api/orders", true},
// Suffix match (call has host prefix, route is just path)
{"https://example.com/api/orders", "/api/orders", true},
{"/api/orders", "/api/orders", true},
// Wildcard params
{"/api/orders/:id", "/api/orders/{order_id}", true},
{"/api/orders/123", "/api/orders/:id", true}, // 123 matches * (normalized :id)
// Segment wildcard: :version normalizes to *, matches any segment
{"/api/:version/items", "/api/v1/items", true},
// Different lengths
{"/api/orders", "/api/orders/detail", false},
{"/api", "/api/orders", false},
// Both have wildcards
{"/api/*/items", "/api/*/items", true},
// No match
{"/api/users", "/api/orders", false},
}
for _, tt := range tests {
got := pathsMatch(tt.callPath, tt.routePath)
if got != tt.want {
t.Errorf("pathsMatch(%q, %q) = %v, want %v", tt.callPath, tt.routePath, got, tt.want)
}
}
}
func TestPathsMatchSuffix(t *testing.T) {
// Suffix match: normalized call path ends with normalized route path
got := pathsMatch("/host/prefix/api/orders", "/api/orders")
if !got {
t.Error("expected suffix match for /host/prefix/api/orders -> /api/orders")
}
}
func TestPathMatchScore(t *testing.T) {
tests := []struct {
call string
route string
min float64
max float64
}{
// Exact matches: matchBase=0.95, confidence = 0.95 × (0.5×jaccard + 0.5×depthFactor)
{"/api/orders", "/api/orders", 0.78, 0.82}, // jaccard=1.0, depth=2/3=0.667 → 0.95×0.833 ≈ 0.79
{"/integrate", "/integrate", 0.60, 0.67}, // jaccard=1.0, depth=1/3=0.333 → 0.95×0.667 ≈ 0.63
{"/api/v1/orders/items", "/api/v1/orders/items", 0.93, 0.96}, // jaccard=1.0, depth=4/3→1.0 → 0.95×1.0 = 0.95
// Suffix matches: matchBase=0.75
{"https://host/api/orders", "/api/orders", 0.60, 0.66}, // jaccard=1.0, depth=0.667 → 0.75×0.833 ≈ 0.625
// Wildcard matches: matchBase=0.55
{"/api/orders/123", "/api/orders/:id", 0.43, 0.48}, // jaccard({api,orders,123}∩{api,orders})=2/3=0.667, depth=1.0 → 0.55×0.833 ≈ 0.458
// No match
{"/api/users", "/api/orders", 0.0, 0.0},
{"/", "/api/orders", 0.0, 0.0}, // empty normalized
{"", "/api/orders", 0.0, 0.0},
}
for _, tt := range tests {
got := pathMatchScore(tt.call, tt.route)
if got < tt.min || got > tt.max {
t.Errorf("pathMatchScore(%q, %q) = %.2f, want [%.2f, %.2f]", tt.call, tt.route, got, tt.min, tt.max)
}
}
}
func TestSameService(t *testing.T) {
tests := []struct {
qn1 string
qn2 string
want bool
}{
// Full directory comparison: strip last 2 segments (module+name), compare rest
// "a.b.c.mod.func" → dir="a.b.c", so same dir = same service
{"a.b.c.mod.Func1", "a.b.c.mod.Func2", true}, // same dir (a.b.c)
{"a.b.c.mod.Func1", "a.b.x.mod.Func2", false}, // different dir (a.b.c vs a.b.x)
{"a.b.c.d.mod.Func", "a.b.c.d.mod.Other", true}, // same deep dir (a.b.c.d)
{"a.b.c.d.mod.Func", "a.b.c.e.mod.Other", false}, // different deep dir
{"short.x", "short.y", false}, // only 2 segments → strip leaves empty → false
{"a.b", "a.b", false}, // 2 segments → not enough to determine
{"a.b.c", "a.b.c", true}, // 3 segments: dir="a", same
{"a.b.c", "x.b.c", false}, // 3 segments: dir="a" vs "x"
// Realistic multi-service QN patterns
{"myapp.docker-images.cloud-runs.order-service.main.Func", "myapp.docker-images.cloud-runs.order-service.handlers.Other", true},
{"myapp.docker-images.cloud-runs.order-service.main.Func", "myapp.docker-images.cloud-runs.notification-service.main.health_check", false},
{"myapp.docker-images.cloud-runs.svcA.sub.mod.Func", "myapp.docker-images.cloud-runs.svcA.sub.mod.Other", true},
{"myapp.docker-images.cloud-runs.svcA.sub.mod.Func", "myapp.docker-images.cloud-runs.svcB.sub.mod.Other", false},
}
for _, tt := range tests {
got := sameService(tt.qn1, tt.qn2)
if got != tt.want {
t.Errorf("sameService(%q, %q) = %v, want %v", tt.qn1, tt.qn2, got, tt.want)
}
}
}
func TestExtractURLPaths(t *testing.T) {
tests := []struct {
text string
want int // expected number of paths
}{
{`URL = "https://example.com/api/orders"`, 1},
{`fetch("http://host/api/v1/items")`, 1},
{`path = "/api/orders"`, 1},
{`no urls here`, 0},
{`both = "https://a.com/api/x" and "/api/y"`, 2},
}
for _, tt := range tests {
got := extractURLPaths(tt.text)
if len(got) != tt.want {
t.Errorf("extractURLPaths(%q) returned %d paths, want %d: %v", tt.text, len(got), tt.want, got)
}
}
}
func TestExtractPythonRoutes(t *testing.T) {
node := &store.Node{
Name: "create_order",
QualifiedName: "proj.api.routes.create_order",
Properties: map[string]any{
"decorators": []any{
`@app.post("/api/orders")`,
},
},
}
routes := extractPythonRoutes(node)
if len(routes) != 1 {
t.Fatalf("expected 1 route, got %d", len(routes))
}
if routes[0].Path != "/api/orders" {
t.Errorf("path = %q, want /api/orders", routes[0].Path)
}
if routes[0].Method != "POST" {
t.Errorf("method = %q, want POST", routes[0].Method)
}
if routes[0].QualifiedName != "proj.api.routes.create_order" {
t.Errorf("qn = %q, want proj.api.routes.create_order", routes[0].QualifiedName)
}
}
func TestExtractPythonRoutesMultiple(t *testing.T) {
node := &store.Node{
Name: "handler",
QualifiedName: "proj.api.handler",
Properties: map[string]any{
"decorators": []any{
`@router.get("/api/items/{item_id}")`,
`@router.post("/api/items")`,
},
},
}
routes := extractPythonRoutes(node)
if len(routes) != 2 {
t.Fatalf("expected 2 routes, got %d", len(routes))
}
}
func TestExtractPythonRoutesNoDecorators(t *testing.T) {
node := &store.Node{
Name: "helper",
QualifiedName: "proj.utils.helper",
Properties: map[string]any{},
}
routes := extractPythonRoutes(node)
if len(routes) != 0 {
t.Errorf("expected 0 routes, got %d", len(routes))
}
}
func TestExtractGoRoutes(t *testing.T) {
source := `
r.POST("/api/orders", h.CreateOrder)
r.GET("/api/orders/:id", h.GetOrder)
`
node := &store.Node{
Name: "RegisterRoutes",
QualifiedName: "proj.api.RegisterRoutes",
}
routes := extractGoRoutes(node, source)
if len(routes) != 2 {
t.Fatalf("expected 2 routes, got %d", len(routes))
}
if routes[0].Path != "/api/orders" {
t.Errorf("route[0].Path = %q, want /api/orders", routes[0].Path)
}
if routes[0].Method != "POST" {
t.Errorf("route[0].Method = %q, want POST", routes[0].Method)
}
if routes[1].Path != "/api/orders/:id" {
t.Errorf("route[1].Path = %q, want /api/orders/:id", routes[1].Path)
}
}
func TestReadSourceLines(t *testing.T) {
dir, err := os.MkdirTemp("", "httplink-test-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
content := "line1\nline2\nline3\nline4\nline5\n"
path := filepath.Join(dir, "test.go")
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
got := readSourceLines(dir, "test.go", 2, 4)
want := "line2\nline3\nline4"
if got != want {
t.Errorf("readSourceLines = %q, want %q", got, want)
}
}
func TestReadSourceLinesMissingFile(t *testing.T) {
got := readSourceLines("/nonexistent", "missing.go", 1, 10)
if got != "" {
t.Errorf("expected empty string for missing file, got %q", got)
}
}
func TestLinkerRun(t *testing.T) {
// Set up a temp directory with a Python route handler and a Go caller
dir, err := os.MkdirTemp("", "httplink-e2e-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
// Write a Go file that contains a URL constant
goDir := filepath.Join(dir, "caller")
if err := os.MkdirAll(goDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(goDir, "client.go"), []byte(`package caller
const OrderURL = "https://api.example.com/api/orders"
`), 0o644); err != nil {
t.Fatal(err)
}
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
project := "testproj"
if err := s.UpsertProject(project, dir); err != nil {
t.Fatal(err)
}
// Create a Module node with constants containing a URL
callerID, _ := s.UpsertNode(&store.Node{
Project: project,
Label: "Module",
Name: "client.go",
QualifiedName: "testproj.caller.client",
FilePath: "caller/client.go",
Properties: map[string]any{
"constants": []any{`OrderURL = "https://api.example.com/api/orders"`},
},
})
// Create a Function node with a Python route decorator
handlerID, _ := s.UpsertNode(&store.Node{
Project: project,
Label: "Function",
Name: "create_order",
QualifiedName: "testproj.handler.routes.create_order",
FilePath: "handler/routes.py",
Properties: map[string]any{
"decorators": []any{`@app.post("/api/orders")`},
},
})
if callerID == 0 || handlerID == 0 {
t.Fatal("failed to create test nodes")
}
linker := New(s, project)
links, err := linker.Run()
if err != nil {
t.Fatalf("Run: %v", err)
}
if len(links) == 0 {
t.Fatal("expected at least 1 HTTP link, got 0")
}
// Verify the link
found := false
for _, link := range links {
if link.CallerQN == "testproj.caller.client" && link.HandlerQN == "testproj.handler.routes.create_order" {
found = true
t.Logf("link: %s -> %s (path=%s)", link.CallerQN, link.HandlerQN, link.URLPath)
}
}
if !found {
t.Error("expected link from testproj.caller.client to testproj.handler.routes.create_order")
for _, link := range links {
t.Logf(" got: %s -> %s", link.CallerQN, link.HandlerQN)
}
}
// Verify edge was created in store
callerNode, _ := s.FindNodeByQN(project, "testproj.caller.client")
if callerNode == nil {
t.Fatal("caller node not found")
}
edges, _ := s.FindEdgesBySourceAndType(callerNode.ID, "HTTP_CALLS")
if len(edges) == 0 {
t.Error("expected HTTP_CALLS edge in store, got 0")
}
}
func TestExtractJSONStringPaths(t *testing.T) {
tests := []struct {
name string
text string
want int
}{
{
name: "JSON object with URL",
text: `BODY = '{"target": "https://api.internal.com/api/orders", "method": "POST"}'`,
want: 1, // /api/orders
},
{
name: "JSON object with path",
text: `CONFIG = {"endpoint": "/api/v1/process", "timeout": 30}`,
want: 1, // /api/v1/process
},
{
name: "no JSON",
text: `plain string without json`,
want: 0,
},
{
name: "nested JSON with URL",
text: `{"services": [{"url": "https://svc.example.com/api/health"}]}`,
want: 1,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := extractJSONStringPaths(tt.text)
if len(got) != tt.want {
t.Errorf("extractJSONStringPaths(%q) returned %d paths, want %d: %v", tt.text, len(got), tt.want, got)
}
})
}
}
func TestRouteNodesCreated(t *testing.T) {
dir, err := os.MkdirTemp("", "httplink-route-nodes-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
project := "testproj"
if err := s.UpsertProject(project, dir); err != nil {
t.Fatal(err)
}
// Create a Function node with a Python route decorator
_, _ = s.UpsertNode(&store.Node{
Project: project,
Label: "Function",
Name: "create_order",
QualifiedName: "testproj.handler.routes.create_order",
FilePath: "handler/routes.py",
Properties: map[string]any{
"decorators": []any{`@app.post("/api/orders")`},
},
})
linker := New(s, project)
_, err = linker.Run()
if err != nil {
t.Fatalf("Run: %v", err)
}
// Verify Route node was created
routeNodes, _ := s.FindNodesByLabel(project, "Route")
if len(routeNodes) != 1 {
t.Fatalf("expected 1 Route node, got %d", len(routeNodes))
}
rn := routeNodes[0]
if rn.Name != "POST /api/orders" {
t.Errorf("Route name = %q, want 'POST /api/orders'", rn.Name)
}
if rn.Properties["method"] != "POST" {
t.Errorf("Route method = %v, want POST", rn.Properties["method"])
}
if rn.Properties["path"] != "/api/orders" {
t.Errorf("Route path = %v, want /api/orders", rn.Properties["path"])
}
// Verify HANDLES edge from handler → Route
handlerNode, _ := s.FindNodeByQN(project, "testproj.handler.routes.create_order")
if handlerNode == nil {
t.Fatal("handler node not found")
}
edges, _ := s.FindEdgesBySourceAndType(handlerNode.ID, "HANDLES")
if len(edges) != 1 {
t.Errorf("expected 1 HANDLES edge, got %d", len(edges))
}
// Verify handler marked as entry point
if handlerNode.Properties["is_entry_point"] != true {
t.Error("expected handler to be marked as is_entry_point")
}
}
func TestLinkerSkipsSameService(t *testing.T) {
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
dir, err := os.MkdirTemp("", "httplink-same-svc-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
project := "testproj"
if err := s.UpsertProject(project, dir); err != nil {
t.Fatal(err)
}
// Both in the same service (same first 4 QN segments: testproj.cat.sub.svcA)
_, _ = s.UpsertNode(&store.Node{
Project: project,
Label: "Module",
Name: "client.py",
QualifiedName: "testproj.cat.sub.svcA.internal.client",
FilePath: "cat/sub/svcA/internal/client.py",
Properties: map[string]any{
"constants": []any{`URL = "https://localhost/api/orders"`},
},
})
_, _ = s.UpsertNode(&store.Node{
Project: project,
Label: "Function",
Name: "handle_orders",
QualifiedName: "testproj.cat.sub.svcA.internal.handle_orders",
FilePath: "cat/sub/svcA/internal/routes.py",
Properties: map[string]any{
"decorators": []any{`@app.get("/api/orders")`},
},
})
linker := New(s, project)
links, err := linker.Run()
if err != nil {
t.Fatalf("Run: %v", err)
}
if len(links) != 0 {
t.Errorf("expected 0 links (same service), got %d", len(links))
for _, l := range links {
t.Logf(" %s -> %s", l.CallerQN, l.HandlerQN)
}
}
}
internal/lang/cpp.go
+40
View File
@@ -0,0 +1,40 @@
package lang
func init() {
Register(&LanguageSpec{
Language: CPP,
FileExtensions: []string{".cpp", ".h", ".hpp", ".cc", ".cxx", ".hxx", ".hh", ".ixx", ".cppm", ".ccm"},
FunctionNodeTypes: []string{
"function_definition",
"declaration",
"field_declaration",
"template_declaration",
"lambda_expression",
},
ClassNodeTypes: []string{
"class_specifier",
"struct_specifier",
"union_specifier",
"enum_specifier",
},
ModuleNodeTypes: []string{
"translation_unit",
"namespace_definition",
"linkage_specification",
"declaration",
},
CallNodeTypes: []string{
"call_expression",
"field_expression",
"subscript_expression",
"new_expression",
"delete_expression",
"binary_expression",
"unary_expression",
"update_expression",
},
ImportNodeTypes: []string{"preproc_include", "template_function", "declaration"},
ImportFromTypes: []string{"preproc_include", "template_function", "declaration"},
PackageIndicators: []string{"CMakeLists.txt", "Makefile", "*.vcxproj", "conanfile.txt"},
})
}
internal/lang/csharp.go
+27
View File
@@ -0,0 +1,27 @@
package lang
func init() {
Register(&LanguageSpec{
Language: CSharp,
FileExtensions: []string{".cs"},
FunctionNodeTypes: []string{
"destructor_declaration",
"local_function_statement",
"function_pointer_type",
"constructor_declaration",
"anonymous_method_expression",
"lambda_expression",
"method_declaration",
},
ClassNodeTypes: []string{
"class_declaration",
"struct_declaration",
"enum_declaration",
"interface_declaration",
},
ModuleNodeTypes: []string{"compilation_unit"},
CallNodeTypes: []string{"invocation_expression"},
ImportNodeTypes: []string{"using_directive"},
ImportFromTypes: []string{"using_directive"},
})
}
internal/lang/go_lang.go
+14
View File
@@ -0,0 +1,14 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Go,
FileExtensions: []string{".go"},
FunctionNodeTypes: []string{"function_declaration", "method_declaration"},
ClassNodeTypes: []string{"type_declaration"},
ModuleNodeTypes: []string{"source_file"},
CallNodeTypes: []string{"call_expression"},
ImportNodeTypes: []string{"import_declaration"},
ImportFromTypes: []string{"import_declaration"},
})
}
internal/lang/java.go
+20
View File
@@ -0,0 +1,20 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Java,
FileExtensions: []string{".java"},
FunctionNodeTypes: []string{"method_declaration", "constructor_declaration"},
ClassNodeTypes: []string{
"class_declaration",
"interface_declaration",
"enum_declaration",
"annotation_type_declaration",
"record_declaration",
},
ModuleNodeTypes: []string{"program"},
CallNodeTypes: []string{"method_invocation"},
ImportNodeTypes: []string{"import_declaration"},
ImportFromTypes: []string{"import_declaration"},
})
}
internal/lang/javascript.go
+20
View File
@@ -0,0 +1,20 @@
package lang
func init() {
Register(&LanguageSpec{
Language: JavaScript,
FileExtensions: []string{".js", ".jsx"},
FunctionNodeTypes: []string{
"function_declaration",
"generator_function_declaration",
"function_expression",
"arrow_function",
"method_definition",
},
ClassNodeTypes: []string{"class_declaration", "class"},
ModuleNodeTypes: []string{"program"},
CallNodeTypes: []string{"call_expression"},
ImportNodeTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
ImportFromTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
})
}
internal/lang/lang.go
+72
View File
@@ -0,0 +1,72 @@
package lang
// Language represents a supported programming language.
type Language string
const (
Python Language = "python"
JavaScript Language = "javascript"
TypeScript Language = "typescript"
Go Language = "go"
Rust Language = "rust"
Java Language = "java"
CPP Language = "cpp"
TSX Language = "tsx"
CSharp Language = "c-sharp"
PHP Language = "php"
Lua Language = "lua"
Scala Language = "scala"
JSON Language = "json" // Not in AllLanguages(); no LanguageSpec or tree-sitter grammar
)
// AllLanguages returns all supported languages.
func AllLanguages() []Language {
return []Language{Python, JavaScript, TypeScript, TSX, Go, Rust, Java, CPP, CSharp, PHP, Lua, Scala}
}
// LanguageSpec defines the tree-sitter node types for a language.
type LanguageSpec struct {
Language Language
FileExtensions []string
FunctionNodeTypes []string
ClassNodeTypes []string
ModuleNodeTypes []string
CallNodeTypes []string
ImportNodeTypes []string
ImportFromTypes []string
PackageIndicators []string
}
// registry maps file extensions to language specs.
var registry = map[string]*LanguageSpec{}
// Register adds a LanguageSpec to the global registry.
func Register(spec *LanguageSpec) {
for _, ext := range spec.FileExtensions {
registry[ext] = spec
}
}
// ForExtension returns the LanguageSpec for a file extension (e.g. ".go").
func ForExtension(ext string) *LanguageSpec {
return registry[ext]
}
// ForLanguage returns the LanguageSpec for a language.
func ForLanguage(lang Language) *LanguageSpec {
for _, spec := range registry {
if spec.Language == lang {
return spec
}
}
return nil
}
// LanguageForExtension returns the Language for a file extension.
func LanguageForExtension(ext string) (Language, bool) {
spec := registry[ext]
if spec == nil {
return "", false
}
return spec.Language, true
}
internal/lang/lang_test.go
+77
View File
@@ -0,0 +1,77 @@
package lang
import "testing"
func TestForExtension(t *testing.T) {
tests := []struct {
ext string
lang Language
}{
{".py", Python},
{".go", Go},
{".js", JavaScript},
{".ts", TypeScript},
{".tsx", TSX},
{".rs", Rust},
{".java", Java},
{".cpp", CPP},
{".h", CPP},
{".cs", CSharp},
{".php", PHP},
{".lua", Lua},
{".scala", Scala},
}
for _, tt := range tests {
spec := ForExtension(tt.ext)
if spec == nil {
t.Errorf("ForExtension(%q) = nil, want %s", tt.ext, tt.lang)
continue
}
if spec.Language != tt.lang {
t.Errorf("ForExtension(%q).Language = %s, want %s", tt.ext, spec.Language, tt.lang)
}
}
}
func TestForLanguage(t *testing.T) {
for _, lang := range AllLanguages() {
spec := ForLanguage(lang)
if spec == nil {
t.Errorf("ForLanguage(%s) = nil", lang)
}
}
}
func TestUnknownExtension(t *testing.T) {
if spec := ForExtension(".xyz"); spec != nil {
t.Errorf("ForExtension(.xyz) should be nil, got %v", spec)
}
}
func TestGoSpec(t *testing.T) {
spec := ForLanguage(Go)
if spec == nil {
t.Fatal("Go spec not registered")
}
if len(spec.FunctionNodeTypes) != 2 {
t.Errorf("Go FunctionNodeTypes: got %d, want 2", len(spec.FunctionNodeTypes))
}
// Should contain function_declaration and method_declaration
found := map[string]bool{}
for _, nt := range spec.FunctionNodeTypes {
found[nt] = true
}
if !found["function_declaration"] || !found["method_declaration"] {
t.Errorf("Go FunctionNodeTypes missing expected types: %v", spec.FunctionNodeTypes)
}
}
func TestPythonSpec(t *testing.T) {
spec := ForLanguage(Python)
if spec == nil {
t.Fatal("Python spec not registered")
}
if spec.PackageIndicators[0] != "__init__.py" {
t.Errorf("Python PackageIndicators: got %v, want [__init__.py]", spec.PackageIndicators)
}
}
internal/lang/lua.go
+13
View File
@@ -0,0 +1,13 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Lua,
FileExtensions: []string{".lua"},
FunctionNodeTypes: []string{"function_declaration", "function_definition"},
ClassNodeTypes: []string{},
ModuleNodeTypes: []string{"chunk"},
CallNodeTypes: []string{"function_call"},
ImportNodeTypes: []string{"function_call"},
})
}
internal/lang/php.go
+27
View File
@@ -0,0 +1,27 @@
package lang
func init() {
Register(&LanguageSpec{
Language: PHP,
FileExtensions: []string{".php"},
FunctionNodeTypes: []string{
"function_static_declaration",
"anonymous_function",
"function_definition",
"arrow_function",
},
ClassNodeTypes: []string{
"trait_declaration",
"enum_declaration",
"interface_declaration",
"class_declaration",
},
ModuleNodeTypes: []string{"program"},
CallNodeTypes: []string{
"member_call_expression",
"scoped_call_expression",
"function_call_expression",
"nullsafe_member_call_expression",
},
})
}
internal/lang/python.go
+15
View File
@@ -0,0 +1,15 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Python,
FileExtensions: []string{".py"},
FunctionNodeTypes: []string{"function_definition"},
ClassNodeTypes: []string{"class_definition"},
ModuleNodeTypes: []string{"module"},
CallNodeTypes: []string{"call", "with_statement"},
ImportNodeTypes: []string{"import_statement"},
ImportFromTypes: []string{"import_from_statement"},
PackageIndicators: []string{"__init__.py"},
})
}
internal/lang/rust.go
+26
View File
@@ -0,0 +1,26 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Rust,
FileExtensions: []string{".rs"},
FunctionNodeTypes: []string{
"function_item",
"function_signature_item",
"closure_expression",
},
ClassNodeTypes: []string{
"struct_item",
"enum_item",
"union_item",
"trait_item",
"impl_item",
"type_item",
},
ModuleNodeTypes: []string{"source_file", "mod_item"},
CallNodeTypes: []string{"call_expression", "macro_invocation"},
ImportNodeTypes: []string{"use_declaration", "extern_crate_declaration"},
ImportFromTypes: []string{"use_declaration"},
PackageIndicators: []string{"Cargo.toml"},
})
}
internal/lang/scala.go
+23
View File
@@ -0,0 +1,23 @@
package lang
func init() {
Register(&LanguageSpec{
Language: Scala,
FileExtensions: []string{".scala", ".sc"},
FunctionNodeTypes: []string{"function_definition", "function_declaration"},
ClassNodeTypes: []string{
"class_definition",
"object_definition",
"trait_definition",
},
ModuleNodeTypes: []string{"compilation_unit"},
CallNodeTypes: []string{
"call_expression",
"generic_function",
"field_expression",
"infix_expression",
},
ImportNodeTypes: []string{"import_declaration"},
ImportFromTypes: []string{"import_declaration"},
})
}
internal/lang/tsx.go
+29
View File
@@ -0,0 +1,29 @@
package lang
func init() {
Register(&LanguageSpec{
Language: TSX,
FileExtensions: []string{".tsx"},
FunctionNodeTypes: []string{
"function_declaration",
"generator_function_declaration",
"function_expression",
"arrow_function",
"method_definition",
"function_signature",
},
ClassNodeTypes: []string{
"class_declaration",
"class",
"abstract_class_declaration",
"enum_declaration",
"interface_declaration",
"type_alias_declaration",
"internal_module",
},
ModuleNodeTypes: []string{"program"},
CallNodeTypes: []string{"call_expression"},
ImportNodeTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
ImportFromTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
})
}
internal/lang/typescript.go
+29
View File
@@ -0,0 +1,29 @@
package lang
func init() {
Register(&LanguageSpec{
Language: TypeScript,
FileExtensions: []string{".ts"},
FunctionNodeTypes: []string{
"function_declaration",
"generator_function_declaration",
"function_expression",
"arrow_function",
"method_definition",
"function_signature",
},
ClassNodeTypes: []string{
"class_declaration",
"class",
"abstract_class_declaration",
"enum_declaration",
"interface_declaration",
"type_alias_declaration",
"internal_module",
},
ModuleNodeTypes: []string{"program"},
CallNodeTypes: []string{"call_expression"},
ImportNodeTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
ImportFromTypes: []string{"import_statement", "lexical_declaration", "export_statement"},
})
}
internal/parser/parser.go
+104
View File
@@ -0,0 +1,104 @@
package parser
import (
"fmt"
"sync"
"unsafe"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
tree_sitter_cpp "github.com/tree-sitter/tree-sitter-cpp/bindings/go"
tree_sitter_go "github.com/tree-sitter/tree-sitter-go/bindings/go"
tree_sitter_java "github.com/tree-sitter/tree-sitter-java/bindings/go"
tree_sitter_javascript "github.com/tree-sitter/tree-sitter-javascript/bindings/go"
tree_sitter_lua "github.com/tree-sitter-grammars/tree-sitter-lua/bindings/go"
tree_sitter_php "github.com/tree-sitter/tree-sitter-php/bindings/go"
tree_sitter_python "github.com/tree-sitter/tree-sitter-python/bindings/go"
tree_sitter_rust "github.com/tree-sitter/tree-sitter-rust/bindings/go"
tree_sitter_scala "github.com/tree-sitter/tree-sitter-scala/bindings/go"
tree_sitter_typescript "github.com/tree-sitter/tree-sitter-typescript/bindings/go"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
)
var (
languagesOnce sync.Once
languages map[lang.Language]*tree_sitter.Language
)
func initLanguages() {
languagesOnce.Do(func() {
languages = map[lang.Language]*tree_sitter.Language{
lang.Python: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_python.Language())),
lang.JavaScript: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_javascript.Language())),
lang.TypeScript: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_typescript.LanguageTypescript())),
lang.TSX: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_typescript.LanguageTSX())),
lang.Go: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_go.Language())),
lang.Rust: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_rust.Language())),
lang.Java: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_java.Language())),
lang.CPP: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_cpp.Language())),
// C# skipped: upstream module path mismatch (tree-sitter-c-sharp vs tree-sitter-c_sharp)
lang.PHP: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_php.LanguagePHPOnly())),
lang.Lua: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_lua.Language())),
lang.Scala: tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_scala.Language())),
}
})
}
// GetLanguage returns the tree-sitter Language for a lang.Language.
func GetLanguage(l lang.Language) (*tree_sitter.Language, error) {
initLanguages()
tsLang, ok := languages[l]
if !ok {
return nil, fmt.Errorf("unsupported language: %s", l)
}
return tsLang, nil
}
// Parse parses source code into a tree-sitter AST Tree.
// The caller must call tree.Close() when done.
func Parse(l lang.Language, source []byte) (*tree_sitter.Tree, error) {
tsLang, err := GetLanguage(l)
if err != nil {
return nil, err
}
p := tree_sitter.NewParser()
defer p.Close()
if err := p.SetLanguage(tsLang); err != nil {
return nil, fmt.Errorf("set language %s: %w", l, err)
}
tree := p.Parse(source, nil)
if tree == nil {
return nil, fmt.Errorf("parse failed for language %s", l)
}
return tree, nil
}
// WalkFunc is called for each node during AST traversal.
// Return false to skip children.
type WalkFunc func(node *tree_sitter.Node) bool
// Walk traverses the AST in depth-first order.
func Walk(node *tree_sitter.Node, fn WalkFunc) {
if node == nil {
return
}
if !fn(node) {
return
}
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child != nil {
Walk(child, fn)
}
}
}
// NodeText returns the text content of a node.
func NodeText(node *tree_sitter.Node, source []byte) string {
return string(source[node.StartByte():node.EndByte()])
}
internal/parser/parser_test.go
+119
View File
@@ -0,0 +1,119 @@
package parser
import (
"testing"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
)
func TestParseGo(t *testing.T) {
source := []byte(`package main
func Hello() string {
return "hello"
}
func Add(a, b int) int {
return a + b
}
`)
tree, err := Parse(lang.Go, source)
if err != nil {
t.Fatalf("Parse Go: %v", err)
}
defer tree.Close()
root := tree.RootNode()
if root == nil {
t.Fatal("root node is nil")
}
var funcCount int
Walk(root, func(n *tree_sitter.Node) bool {
if n.Kind() == "function_declaration" {
funcCount++
}
return true
})
if funcCount != 2 {
t.Errorf("expected 2 function_declarations, got %d", funcCount)
}
}
func TestParsePython(t *testing.T) {
source := []byte(`def greet(name):
return f"Hello, {name}"
class MyClass:
def method(self):
pass
`)
tree, err := Parse(lang.Python, source)
if err != nil {
t.Fatalf("Parse Python: %v", err)
}
defer tree.Close()
root := tree.RootNode()
var funcCount, classCount int
Walk(root, func(n *tree_sitter.Node) bool {
switch n.Kind() {
case "function_definition":
funcCount++
case "class_definition":
classCount++
}
return true
})
if funcCount != 2 {
t.Errorf("expected 2 function_definitions, got %d", funcCount)
}
if classCount != 1 {
t.Errorf("expected 1 class_definition, got %d", classCount)
}
}
func TestAllLanguagesLoad(t *testing.T) {
for _, l := range lang.AllLanguages() {
if l == lang.CSharp {
continue // C# grammar has broken Go module path upstream
}
_, err := GetLanguage(l)
if err != nil {
t.Errorf("GetLanguage(%s): %v", l, err)
}
}
}
func TestNodeText(t *testing.T) {
source := []byte(`package main
func Hello() string {
return "hello"
}
`)
tree, err := Parse(lang.Go, source)
if err != nil {
t.Fatalf("Parse: %v", err)
}
defer tree.Close()
root := tree.RootNode()
Walk(root, func(n *tree_sitter.Node) bool {
if n.Kind() == "function_declaration" {
nameNode := n.ChildByFieldName("name")
if nameNode == nil {
t.Error("function has no name node")
return false
}
name := NodeText(nameNode, source)
if name != "Hello" {
t.Errorf("expected Hello, got %s", name)
}
return false
}
return true
})
}
internal/pipeline/implements.go
+167
View File
@@ -0,0 +1,167 @@
package pipeline
import (
"log/slog"
"strings"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
// passImplements detects Go interface satisfaction and creates IMPLEMENTS edges.
// A struct implements an interface if it has methods matching all interface methods.
func (p *Pipeline) passImplements() error {
slog.Info("pass5.implements")
interfaces, err := p.Store.FindNodesByLabel(p.ProjectName, "Interface")
if err != nil || len(interfaces) == 0 {
return nil
}
// Build interface -> method names map
type ifaceInfo struct {
node *store.Node
methods []string
}
var ifaces []ifaceInfo
for _, iface := range interfaces {
// Only process Go interfaces (check file extension)
if !strings.HasSuffix(iface.FilePath, ".go") {
continue
}
// Find methods defined by this interface via DEFINES_METHOD edges
edges, err := p.Store.FindEdgesBySourceAndType(iface.ID, "DEFINES_METHOD")
if err != nil || len(edges) == 0 {
continue
}
var methodNames []string
for _, e := range edges {
methodNode, _ := p.Store.FindNodeByID(e.TargetID)
if methodNode != nil {
methodNames = append(methodNames, methodNode.Name)
}
}
if len(methodNames) > 0 {
ifaces = append(ifaces, ifaceInfo{node: iface, methods: methodNames})
}
}
if len(ifaces) == 0 {
return nil
}
// Build struct -> method names map from Go receiver methods.
// Go methods are stored as "Method" nodes with a "receiver" property
// containing the receiver type like "(h *Handlers)".
methods, err := p.Store.FindNodesByLabel(p.ProjectName, "Method")
if err != nil {
return nil
}
// receiverType -> set of method names
structMethods := make(map[string]map[string]bool)
// receiverType -> one sample method's QN prefix (to find the struct node)
structQNPrefix := make(map[string]string)
for _, m := range methods {
if !strings.HasSuffix(m.FilePath, ".go") {
continue
}
recv, ok := m.Properties["receiver"]
if !ok {
continue
}
recvStr, ok := recv.(string)
if !ok || recvStr == "" {
continue
}
typeName := extractReceiverType(recvStr)
if typeName == "" {
continue
}
if structMethods[typeName] == nil {
structMethods[typeName] = make(map[string]bool)
}
structMethods[typeName][m.Name] = true
// Store QN prefix for finding the struct. The method QN is like
// "project.path.module.MethodName" — we want the module QN prefix.
if _, exists := structQNPrefix[typeName]; !exists {
// Take everything before the last dot (the method name)
if idx := strings.LastIndex(m.QualifiedName, "."); idx > 0 {
structQNPrefix[typeName] = m.QualifiedName[:idx]
}
}
}
// Check each struct against each interface
linkCount := 0
for _, iface := range ifaces {
for typeName, methodSet := range structMethods {
if satisfies(iface.methods, methodSet) {
// Find or create the struct node. In Go, structs are stored
// as "Class" nodes (since tree-sitter classifies type_declaration
// as a class-like node).
structQN := structQNPrefix[typeName] + "." + typeName
structNode, _ := p.Store.FindNodeByQN(p.ProjectName, structQN)
// If we can't find a class node by that QN, try to find it
// by searching Class nodes with the matching name
if structNode == nil {
classes, _ := p.Store.FindNodesByLabel(p.ProjectName, "Class")
for _, c := range classes {
if c.Name == typeName && strings.HasSuffix(c.FilePath, ".go") {
structNode = c
break
}
}
}
if structNode == nil {
continue
}
_, _ = p.Store.InsertEdge(&store.Edge{
Project: p.ProjectName,
SourceID: structNode.ID,
TargetID: iface.node.ID,
Type: "IMPLEMENTS",
})
linkCount++
}
}
}
slog.Info("pass5.implements.done", "links", linkCount)
return nil
}
// extractReceiverType extracts the type name from a Go receiver string.
// "(h *Handlers)" -> "Handlers", "(s Store)" -> "Store"
func extractReceiverType(recv string) string {
recv = strings.TrimSpace(recv)
recv = strings.Trim(recv, "()")
parts := strings.Fields(recv)
if len(parts) == 0 {
return ""
}
// Last field is the type, possibly with * prefix
typeName := parts[len(parts)-1]
typeName = strings.TrimPrefix(typeName, "*")
return typeName
}
// satisfies checks if a set of method names includes all interface methods.
func satisfies(ifaceMethods []string, structMethodSet map[string]bool) bool {
for _, m := range ifaceMethods {
if !structMethodSet[m] {
return false
}
}
return true
}
internal/pipeline/imports.go
+319
View File
@@ -0,0 +1,319 @@
package pipeline
import (
"path/filepath"
"strings"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
"github.com/DeusData/codebase-memory-mcp/internal/fqn"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
"github.com/DeusData/codebase-memory-mcp/internal/parser"
)
// parseImports extracts the import map for a source file.
// Returns localName -> resolvedQN mapping.
func parseImports(
root *tree_sitter.Node,
source []byte,
language lang.Language,
projectName, relPath string,
) map[string]string {
switch language {
case lang.Go:
return parseGoImports(root, source, projectName, relPath)
case lang.Python:
return parsePythonImports(root, source, projectName, relPath)
default:
return nil
}
}
// parseGoImports extracts Go import declarations.
// For each import spec: localName -> module QN (project-relative) or raw path.
//
// Go import AST structure:
//
// import_declaration
// import_spec_list
// import_spec
// name: package_identifier (optional alias)
// path: interpreted_string_literal
func parseGoImports(
root *tree_sitter.Node,
source []byte,
projectName, relPath string,
) map[string]string {
imports := make(map[string]string)
parser.Walk(root, func(node *tree_sitter.Node) bool {
if node.Kind() != "import_declaration" {
return true
}
// Process each import_spec inside this declaration
processGoImportDecl(node, source, projectName, imports)
return false // don't recurse further
})
return imports
}
func processGoImportDecl(node *tree_sitter.Node, source []byte, projectName string, imports map[string]string) {
parser.Walk(node, func(child *tree_sitter.Node) bool {
if child.Kind() != "import_spec" {
return true
}
pathNode := child.ChildByFieldName("path")
if pathNode == nil {
return false
}
importPath := stripQuotes(parser.NodeText(pathNode, source))
if importPath == "" {
return false
}
// Determine the local name: alias if present, else last segment
localName := lastPathSegment(importPath)
nameNode := child.ChildByFieldName("name")
if nameNode != nil {
alias := parser.NodeText(nameNode, source)
if alias != "" && alias != "." && alias != "_" {
localName = alias
}
}
// Resolve the import path to a project-internal QN if possible.
// We check if any part of the import path matches the project name,
// which indicates an internal package.
resolvedQN := resolveGoImportPath(importPath, projectName)
imports[localName] = resolvedQN
return false
})
}
// resolveGoImportPath converts a Go import path to a project-internal QN.
// For internal packages: "github.com/org/project/pkg/foo" -> "project.pkg.foo"
// For external packages: "fmt" -> "fmt", "net/http" -> "http"
func resolveGoImportPath(importPath, projectName string) string {
parts := strings.Split(importPath, "/")
// Check if this is a project-internal import by looking for the project
// name in the path segments (common pattern: github.com/org/project/...)
for i, part := range parts {
if part == projectName {
// Everything after the project name becomes the QN
remaining := parts[i:]
return strings.Join(remaining, ".")
}
}
// External package: use the full path with dots
return strings.Join(parts, ".")
}
// parsePythonImports extracts Python import statements.
//
// Python import AST structures:
//
// import_statement:
// dotted_name children (e.g., "import foo.bar")
// aliased_import with alias (e.g., "import foo as f")
//
// import_from_statement:
// module_name: dotted_name or relative_import
// name: dotted_name (what's being imported)
// Multiple names possible (e.g., "from foo import bar, baz")
func parsePythonImports(
root *tree_sitter.Node,
source []byte,
projectName, relPath string,
) map[string]string {
imports := make(map[string]string)
parser.Walk(root, func(node *tree_sitter.Node) bool {
switch node.Kind() {
case "import_statement":
processPythonImport(node, source, projectName, imports)
return false
case "import_from_statement":
processPythonFromImport(node, source, projectName, relPath, imports)
return false
}
return true
})
return imports
}
// processPythonImport handles "import X" and "import X as Y" statements.
func processPythonImport(node *tree_sitter.Node, source []byte, projectName string, imports map[string]string) {
for i := uint(0); i < node.NamedChildCount(); i++ {
child := node.NamedChild(i)
if child == nil {
continue
}
switch child.Kind() {
case "dotted_name":
name := parser.NodeText(child, source)
localName := lastDotSegment(name)
imports[localName] = resolvePythonModule(name, projectName)
case "aliased_import":
nameNode := child.ChildByFieldName("name")
aliasNode := child.ChildByFieldName("alias")
if nameNode == nil {
continue
}
name := parser.NodeText(nameNode, source)
localName := lastDotSegment(name)
if aliasNode != nil {
localName = parser.NodeText(aliasNode, source)
}
imports[localName] = resolvePythonModule(name, projectName)
}
}
}
// processPythonFromImport handles "from X import Y" statements.
func processPythonFromImport(
node *tree_sitter.Node,
source []byte,
projectName, relPath string,
imports map[string]string,
) {
// Get the module being imported from
moduleNode := node.ChildByFieldName("module_name")
var modulePath string
isRelative := false
if moduleNode != nil {
modulePath = parser.NodeText(moduleNode, source)
isRelative = strings.HasPrefix(modulePath, ".")
} else {
// Check for bare relative import: "from . import X"
text := parser.NodeText(node, source)
if strings.HasPrefix(text, "from .") {
isRelative = true
modulePath = "."
}
}
// Resolve the base module
var baseModule string
if isRelative {
baseModule = resolveRelativePythonImport(modulePath, relPath, projectName)
} else {
baseModule = resolvePythonModule(modulePath, projectName)
}
// Extract each imported name
for i := uint(0); i < node.NamedChildCount(); i++ {
child := node.NamedChild(i)
if child == nil {
continue
}
switch child.Kind() {
case "dotted_name":
name := parser.NodeText(child, source)
// Skip the module_name itself (first dotted_name is often the source)
if name == modulePath {
continue
}
localName := lastDotSegment(name)
if baseModule != "" {
imports[localName] = baseModule + "." + name
} else {
imports[localName] = name
}
case "aliased_import":
nameNode := child.ChildByFieldName("name")
aliasNode := child.ChildByFieldName("alias")
if nameNode == nil {
continue
}
name := parser.NodeText(nameNode, source)
localName := lastDotSegment(name)
if aliasNode != nil {
localName = parser.NodeText(aliasNode, source)
}
if baseModule != "" {
imports[localName] = baseModule + "." + name
} else {
imports[localName] = name
}
}
}
}
// resolvePythonModule converts a Python module path to a project QN.
// "utils" -> "project.utils", "foo.bar" -> "project.foo.bar"
func resolvePythonModule(modulePath, projectName string) string {
if modulePath == "" {
return projectName
}
return projectName + "." + modulePath
}
// resolveRelativePythonImport resolves relative imports like "from . import X"
// or "from ..utils import X" based on the current file's location.
func resolveRelativePythonImport(modulePath, relPath, projectName string) string {
// Count leading dots for relative depth
dots := 0
for _, ch := range modulePath {
if ch == '.' {
dots++
} else {
break
}
}
remainder := strings.TrimLeft(modulePath, ".")
// Navigate up from the current file's directory
dir := filepath.Dir(relPath)
for i := 1; i < dots; i++ {
dir = filepath.Dir(dir)
}
baseQN := fqn.FolderQN(projectName, dir)
if dir == "." || dir == "" {
baseQN = projectName
}
if remainder != "" {
return baseQN + "." + remainder
}
return baseQN
}
// stripQuotes removes surrounding quotes from a string literal.
func stripQuotes(s string) string {
if len(s) >= 2 {
if (s[0] == '"' && s[len(s)-1] == '"') || (s[0] == '\'' && s[len(s)-1] == '\'') {
return s[1 : len(s)-1]
}
// Handle backtick quotes (Go raw strings)
if s[0] == '`' && s[len(s)-1] == '`' {
return s[1 : len(s)-1]
}
}
return s
}
// lastPathSegment returns the last segment of a /-separated path.
func lastPathSegment(path string) string {
parts := strings.Split(path, "/")
return parts[len(parts)-1]
}
// lastDotSegment returns the last segment of a .-separated name.
func lastDotSegment(name string) string {
parts := strings.Split(name, ".")
return parts[len(parts)-1]
}
internal/pipeline/pipeline.go
+1007
View File
File diff suppressed because it is too large Load Diff
internal/pipeline/pipeline_test.go
+411
View File
@@ -0,0 +1,411 @@
package pipeline
import (
"os"
"path/filepath"
"testing"
"github.com/DeusData/codebase-memory-mcp/internal/store"
)
func setupTestRepo(t *testing.T) (string, func()) {
t.Helper()
dir, err := os.MkdirTemp("", "cgm-test-*")
if err != nil {
t.Fatal(err)
}
cleanup := func() { os.RemoveAll(dir) }
// Create a simple Go project
writeFile(t, filepath.Join(dir, "main.go"), `package main
func main() {
result := Add(1, 2)
_ = result
}
func Add(a, b int) int {
return a + b
}
`)
writeFile(t, filepath.Join(dir, "service", "service.go"), `package service
func ProcessOrder(id string) error {
return nil
}
func SubmitOrder(order interface{}) error {
ProcessOrder("test")
return nil
}
`)
return dir, cleanup
}
func writeFile(t *testing.T, path string, content string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
func TestPipelineRun(t *testing.T) {
repoDir, cleanup := setupTestRepo(t)
defer cleanup()
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
p := New(s, repoDir)
if err := p.Run(); err != nil {
t.Fatalf("Pipeline.Run: %v", err)
}
// Check node counts
nodeCount, _ := s.CountNodes(p.ProjectName)
if nodeCount == 0 {
t.Fatal("expected nodes, got 0")
}
t.Logf("Total nodes: %d", nodeCount)
// Check that Functions were found
funcs, _ := s.FindNodesByLabel(p.ProjectName, "Function")
t.Logf("Functions: %d", len(funcs))
for _, f := range funcs {
t.Logf(" %s (qn=%s, sig=%v)", f.Name, f.QualifiedName, f.Properties["signature"])
}
if len(funcs) < 3 { // main, Add, at minimum
t.Errorf("expected at least 3 functions, got %d", len(funcs))
}
// Check that ProcessOrder exists
found, _ := s.FindNodesByName(p.ProjectName, "ProcessOrder")
if len(found) == 0 {
t.Error("ProcessOrder not found")
}
// Check that Module nodes exist
modules, _ := s.FindNodesByLabel(p.ProjectName, "Module")
if len(modules) < 2 {
t.Errorf("expected at least 2 modules, got %d", len(modules))
}
// Check edges exist
edgeCount, _ := s.CountEdges(p.ProjectName)
t.Logf("Total edges: %d", edgeCount)
if edgeCount == 0 {
t.Error("expected edges, got 0")
}
// Check CALLS edges
// SubmitOrder calls ProcessOrder
sender, _ := s.FindNodesByName(p.ProjectName, "SubmitOrder")
if len(sender) > 0 {
edges, _ := s.FindEdgesBySourceAndType(sender[0].ID, "CALLS")
t.Logf("SubmitOrder CALLS edges: %d", len(edges))
}
}
func TestPipelinePythonProject(t *testing.T) {
dir, err := os.MkdirTemp("", "cgm-py-test-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
writeFile(t, filepath.Join(dir, "main.py"), `
def greet(name):
return f"Hello, {name}"
def process():
result = greet("world")
return result
`)
writeFile(t, filepath.Join(dir, "utils.py"), `
API_URL = "https://example.com/api"
MAX_RETRIES = 3
def fetch_data(url):
pass
class DataProcessor:
def transform(self, data):
return data
`)
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
p := New(s, dir)
if err := p.Run(); err != nil {
t.Fatalf("Pipeline.Run: %v", err)
}
funcs, _ := s.FindNodesByLabel(p.ProjectName, "Function")
t.Logf("Python functions: %d", len(funcs))
if len(funcs) < 3 { // greet, process, fetch_data
t.Errorf("expected at least 3 functions, got %d", len(funcs))
}
classes, _ := s.FindNodesByLabel(p.ProjectName, "Class")
if len(classes) < 1 {
t.Errorf("expected at least 1 class, got %d", len(classes))
}
methods, _ := s.FindNodesByLabel(p.ProjectName, "Method")
if len(methods) < 1 {
t.Errorf("expected at least 1 method, got %d", len(methods))
}
}
// TestGoCrossPackageCallViaImport verifies that a Go function in package A
// that imports package B and calls B.Func() gets a CALLS edge resolved.
func TestGoCrossPackageCallViaImport(t *testing.T) {
dir, err := os.MkdirTemp("", "cgm-go-import-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
// Package "svc" defines ProcessOrder
writeFile(t, filepath.Join(dir, "svc", "handler.go"), `package svc
func ProcessOrder(id string) error {
return nil
}
`)
// Package "main" imports "svc" and calls svc.ProcessOrder
writeFile(t, filepath.Join(dir, "main.go"), `package main
import "example.com/myapp/svc"
func run() {
svc.ProcessOrder("123")
}
`)
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
p := New(s, dir)
if err := p.Run(); err != nil {
t.Fatalf("Pipeline.Run: %v", err)
}
// Verify ProcessOrder exists
targets, _ := s.FindNodesByName(p.ProjectName, "ProcessOrder")
if len(targets) == 0 {
t.Fatal("ProcessOrder not found in store")
}
t.Logf("ProcessOrder QN: %s", targets[0].QualifiedName)
// Verify run() exists
callers, _ := s.FindNodesByName(p.ProjectName, "run")
if len(callers) == 0 {
t.Fatal("run() not found in store")
}
// Check that run() has a CALLS edge to ProcessOrder
edges, _ := s.FindEdgesBySourceAndType(callers[0].ID, "CALLS")
found := false
for _, e := range edges {
if e.TargetID == targets[0].ID {
found = true
break
}
}
if !found {
t.Error("expected CALLS edge from run() to ProcessOrder via import, but none found")
t.Logf("run() CALLS edges: %d", len(edges))
for _, e := range edges {
t.Logf(" target_id=%d", e.TargetID)
}
}
}
// TestPythonCrossModuleCallViaImport verifies that a Python file that does
// "from utils import fetch_data" and calls fetch_data() gets a CALLS edge.
func TestPythonCrossModuleCallViaImport(t *testing.T) {
dir, err := os.MkdirTemp("", "cgm-py-import-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
writeFile(t, filepath.Join(dir, "utils.py"), `
def fetch_data(url):
return {"status": "ok"}
`)
writeFile(t, filepath.Join(dir, "main.py"), `
from utils import fetch_data
def process():
result = fetch_data("https://example.com")
return result
`)
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
p := New(s, dir)
if err := p.Run(); err != nil {
t.Fatalf("Pipeline.Run: %v", err)
}
// Verify fetch_data exists
targets, _ := s.FindNodesByName(p.ProjectName, "fetch_data")
if len(targets) == 0 {
t.Fatal("fetch_data not found in store")
}
t.Logf("fetch_data QN: %s", targets[0].QualifiedName)
// Verify process() exists
callers, _ := s.FindNodesByName(p.ProjectName, "process")
if len(callers) == 0 {
t.Fatal("process() not found in store")
}
// Check that process() has a CALLS edge to fetch_data
edges, _ := s.FindEdgesBySourceAndType(callers[0].ID, "CALLS")
found := false
for _, e := range edges {
if e.TargetID == targets[0].ID {
found = true
break
}
}
if !found {
t.Error("expected CALLS edge from process() to fetch_data via import, but none found")
t.Logf("process() CALLS edges: %d", len(edges))
}
}
// TestPythonMethodDispatchViaTypeInference verifies that type inference allows
// resolving method calls: p = DataProcessor() then p.transform() -> CALLS DataProcessor.transform
func TestPythonMethodDispatchViaTypeInference(t *testing.T) {
dir, err := os.MkdirTemp("", "cgm-py-type-*")
if err != nil {
t.Fatal(err)
}
defer os.RemoveAll(dir)
writeFile(t, filepath.Join(dir, "processor.py"), `
class DataProcessor:
def transform(self, data):
return data.upper()
def validate(self, data):
return len(data) > 0
`)
writeFile(t, filepath.Join(dir, "main.py"), `
from processor import DataProcessor
def run():
p = DataProcessor()
result = p.transform("hello")
return result
`)
s, err := store.OpenMemory()
if err != nil {
t.Fatal(err)
}
defer s.Close()
p := New(s, dir)
if err := p.Run(); err != nil {
t.Fatalf("Pipeline.Run: %v", err)
}
// Verify DataProcessor.transform exists as a Method
methods, _ := s.FindNodesByName(p.ProjectName, "transform")
if len(methods) == 0 {
t.Fatal("transform method not found in store")
}
t.Logf("transform QN: %s", methods[0].QualifiedName)
// Verify run() exists
callers, _ := s.FindNodesByName(p.ProjectName, "run")
if len(callers) == 0 {
t.Fatal("run() not found in store")
}
// Check that run() has a CALLS edge to DataProcessor.transform
edges, _ := s.FindEdgesBySourceAndType(callers[0].ID, "CALLS")
found := false
for _, e := range edges {
if e.TargetID == methods[0].ID {
found = true
break
}
}
if !found {
t.Error("expected CALLS edge from run() to DataProcessor.transform via type inference, but none found")
t.Logf("run() CALLS edges: %d", len(edges))
for _, e := range edges {
t.Logf(" target_id=%d", e.TargetID)
}
}
}
// TestFunctionRegistry tests the registry in isolation.
func TestFunctionRegistry(t *testing.T) {
r := NewFunctionRegistry()
r.Register("Foo", "proj.pkg.Foo", "Function")
r.Register("Bar", "proj.pkg.Bar", "Function")
r.Register("Foo", "proj.other.Foo", "Function")
r.Register("transform", "proj.utils.DataProcessor.transform", "Method")
// FindByName returns all entries
foos := r.FindByName("Foo")
if len(foos) != 2 {
t.Errorf("expected 2 Foo entries, got %d", len(foos))
}
// FindEndingWith
matches := r.FindEndingWith("DataProcessor.transform")
if len(matches) != 1 {
t.Errorf("expected 1 match for DataProcessor.transform, got %d", len(matches))
}
// Resolve same-module
qn := r.Resolve("Foo", "proj.pkg", nil)
if qn != "proj.pkg.Foo" {
t.Errorf("expected proj.pkg.Foo, got %s", qn)
}
// Resolve via import map
imports := map[string]string{"other": "proj.other"}
qn = r.Resolve("other.Foo", "proj.pkg", imports)
if qn != "proj.other.Foo" {
t.Errorf("expected proj.other.Foo, got %s", qn)
}
// Resolve unique name
qn = r.Resolve("Bar", "proj.unrelated", nil)
if qn != "proj.pkg.Bar" {
t.Errorf("expected proj.pkg.Bar, got %s", qn)
}
}
internal/pipeline/resolve.go
+731
View File
@@ -0,0 +1,731 @@
package pipeline
import (
"strings"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
"github.com/DeusData/codebase-memory-mcp/internal/parser"
)
// resolveModuleStrings performs in-memory constant propagation on module-level
// string assignments. It walks the AST top-to-bottom, collects simple string
// literals, then resolves interpolated and concatenated strings using the
// collected symbol table. Returns a map of variable name → resolved string.
//
// Supports: Python f-strings, JS/TS template literals, PHP encapsed strings,
// Scala string interpolation, Rust format!, Go fmt.Sprintf, and string
// concatenation (+ or .) in all languages.
//
// Source files are never modified — resolution is purely in RAM.
func resolveModuleStrings(root *tree_sitter.Node, source []byte, language lang.Language) map[string]string {
symbols := make(map[string]string)
// Walk only top-level children (module-level declarations)
for i := uint(0); i < root.ChildCount(); i++ {
child := root.Child(i)
if child == nil {
continue
}
name, value := resolveAssignment(child, source, language, symbols)
if name != "" && value != "" {
symbols[name] = value
}
}
return symbols
}
// resolveAssignment tries to extract a (name, resolved_value) pair from
// a top-level AST node. Returns ("","") if the node isn't a string assignment.
func resolveAssignment(node *tree_sitter.Node, source []byte, language lang.Language, symbols map[string]string) (string, string) {
switch language {
case lang.Python:
return resolvePython(node, source, symbols)
case lang.Go:
return resolveGo(node, source, symbols)
case lang.JavaScript, lang.TypeScript, lang.TSX:
return resolveJS(node, source, symbols)
case lang.Rust:
return resolveRust(node, source, symbols)
case lang.Java:
return resolveJava(node, source, symbols)
case lang.PHP:
return resolvePHP(node, source, symbols)
case lang.Scala:
return resolveScala(node, source, symbols)
case lang.CPP:
return resolveCPP(node, source, symbols)
case lang.Lua:
return resolveLua(node, source, symbols)
default:
return "", ""
}
}
// --- Python ---
// expression_statement → assignment → (identifier, string|binary_operator)
func resolvePython(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "expression_statement" {
return "", ""
}
assign := findChildByKind(node, "assignment")
if assign == nil {
return "", ""
}
nameNode := assign.ChildByFieldName("left")
valueNode := assign.ChildByFieldName("right")
if nameNode == nil || valueNode == nil || nameNode.Kind() != "identifier" {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- Go ---
// const_declaration → const_spec → (identifier, expression_list)
// var_declaration → var_spec → (identifier, expression_list)
func resolveGo(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
var spec *tree_sitter.Node
switch node.Kind() {
case "const_declaration":
spec = findChildByKind(node, "const_spec")
case "var_declaration":
spec = findChildByKind(node, "var_spec")
default:
return "", ""
}
if spec == nil {
return "", ""
}
nameNode := spec.ChildByFieldName("name")
valueNode := spec.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
return "", ""
}
// value is an expression_list; take the first child
if valueNode.Kind() == "expression_list" && valueNode.ChildCount() > 0 {
valueNode = firstNonTrivialChild(valueNode)
if valueNode == nil {
return "", ""
}
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- JavaScript / TypeScript / TSX ---
// lexical_declaration → variable_declarator → (identifier, value)
func resolveJS(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "lexical_declaration" {
return "", ""
}
decl := findChildByKind(node, "variable_declarator")
if decl == nil {
return "", ""
}
nameNode := decl.ChildByFieldName("name")
valueNode := decl.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- Rust ---
// const_item → (identifier, string_literal|binary_expression|macro_invocation)
// let_declaration → (identifier, value)
func resolveRust(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
switch node.Kind() {
case "const_item", "let_declaration":
// both use field "name" for identifier and "value" for the expression
default:
return "", ""
}
nameNode := node.ChildByFieldName("name")
if nameNode == nil {
// let_declaration uses "pattern" field
nameNode = node.ChildByFieldName("pattern")
}
valueNode := node.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- Java ---
// class_declaration → class_body → field_declaration → variable_declarator
// We need to look inside class bodies for static final fields.
func resolveJava(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "class_declaration" {
return "", ""
}
body := node.ChildByFieldName("body")
if body == nil {
return "", ""
}
// Walk field_declarations inside the class body
for i := uint(0); i < body.ChildCount(); i++ {
child := body.Child(i)
if child == nil || child.Kind() != "field_declaration" {
continue
}
decl := child.ChildByFieldName("declarator")
if decl == nil || decl.Kind() != "variable_declarator" {
continue
}
nameNode := decl.ChildByFieldName("name")
valueNode := decl.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
continue
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
if name != "" && value != "" {
symbols[name] = value
}
}
return "", "" // all collected via symbols map directly
}
// --- PHP ---
// expression_statement → assignment_expression → (variable_name, encapsed_string|binary_expression)
func resolvePHP(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "expression_statement" {
return "", ""
}
assign := findChildByKind(node, "assignment_expression")
if assign == nil {
return "", ""
}
nameNode := assign.ChildByFieldName("left")
valueNode := assign.ChildByFieldName("right")
if nameNode == nil || valueNode == nil {
return "", ""
}
// PHP variable names include $, extract just the name part
name := extractPHPVarName(nameNode, source)
if name == "" {
return "", ""
}
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
func extractPHPVarName(node *tree_sitter.Node, source []byte) string {
if node.Kind() != "variable_name" {
return ""
}
// variable_name has children: $ and name
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child != nil && child.Kind() == "name" {
return parser.NodeText(child, source)
}
}
return ""
}
// --- Scala ---
// val_definition → (identifier, string|interpolated_string_expression|infix_expression)
func resolveScala(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "val_definition" {
return "", ""
}
nameNode := node.ChildByFieldName("pattern")
valueNode := node.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- C++ ---
// preproc_def → identifier + preproc_arg (for #define)
// declaration → init_declarator → identifier + value (for const std::string x = "...")
func resolveCPP(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
switch node.Kind() {
case "preproc_def":
// #define NAME "value"
nameNode := findChildByKind(node, "identifier")
valueNode := findChildByKind(node, "preproc_arg")
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
// preproc_arg contains the raw text — try to extract string content
argText := strings.TrimSpace(parser.NodeText(valueNode, source))
if len(argText) >= 2 && argText[0] == '"' && argText[len(argText)-1] == '"' {
return name, argText[1 : len(argText)-1]
}
// Could be a reference to another #define
if val, ok := symbols[argText]; ok {
return name, val
}
return "", ""
case "declaration":
// const std::string x = "value" or std::string x = base + "/path"
// AST: declaration → init_declarator → declarator(identifier) + value
initDecl := findChildByKind(node, "init_declarator")
if initDecl == nil {
return "", ""
}
nameNode := initDecl.ChildByFieldName("declarator")
valueNode := initDecl.ChildByFieldName("value")
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
return "", ""
}
// --- Lua ---
// variable_declaration → assignment_statement → variable_list + expression_list
// Local: local x = "value" → variable_declaration containing assignment_statement
func resolveLua(node *tree_sitter.Node, source []byte, symbols map[string]string) (string, string) {
if node.Kind() != "variable_declaration" {
return "", ""
}
assign := findChildByKind(node, "assignment_statement")
if assign == nil {
return "", ""
}
varList := findChildByKind(assign, "variable_list")
exprList := findChildByKind(assign, "expression_list")
if varList == nil || exprList == nil {
return "", ""
}
// Take the first variable name and first expression value
nameNode := firstNonTrivialChild(varList)
valueNode := firstNonTrivialChild(exprList)
if nameNode == nil || valueNode == nil {
return "", ""
}
name := parser.NodeText(nameNode, source)
value := resolveStringExpr(valueNode, source, symbols)
return name, value
}
// --- Universal expression resolver ---
// resolveStringExpr resolves a string expression node to its string value.
// Handles: literal strings, interpolated strings, concatenation, fmt.Sprintf, format!.
func resolveStringExpr(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
if node == nil {
return ""
}
kind := node.Kind()
// Simple string literals
if isStringLiteral(kind) {
return extractStringContent(node, source)
}
// Identifiers: look up in symbol table
if kind == "identifier" {
return symbols[parser.NodeText(node, source)]
}
// PHP variable names ($varName): look up by name part (without $)
if kind == "variable_name" {
name := extractPHPVarName(node, source)
return symbols[name]
}
// Python f-strings: string with string_start = f" or f'
if kind == "string" {
start := findChildByKind(node, "string_start")
if start != nil {
startText := parser.NodeText(start, source)
if strings.HasPrefix(startText, "f") || strings.HasPrefix(startText, "F") {
return resolveInterpolatedChildren(node, source, symbols, "interpolation", "string_content")
}
}
// Plain string
return extractStringContent(node, source)
}
// JS/TS template strings
if kind == "template_string" {
return resolveInterpolatedChildren(node, source, symbols, "template_substitution", "string_fragment")
}
// PHP encapsed strings (interpolated)
if kind == "encapsed_string" {
return resolvePHPEncapsed(node, source, symbols)
}
// Scala interpolated strings
if kind == "interpolated_string_expression" {
interpStr := findChildByKind(node, "interpolated_string")
if interpStr != nil {
return resolveScalaInterpolated(interpStr, source, symbols)
}
return ""
}
// String concatenation: binary_expression/binary_operator/infix_expression with +/.
if kind == "binary_expression" || kind == "binary_operator" || kind == "infix_expression" {
return resolveBinaryConcat(node, source, symbols)
}
// Go fmt.Sprintf / Java String.format / Lua string.format / Rust format! macro
if kind == "call_expression" || kind == "function_call" {
return resolveCallExpr(node, source, symbols)
}
if kind == "macro_invocation" {
return resolveRustFormatMacro(node, source, symbols)
}
return ""
}
// resolveInterpolatedChildren resolves a node whose children alternate between
// interpolation nodes (containing variable refs) and literal content nodes.
// Used for Python f-strings and JS/TS template strings.
func resolveInterpolatedChildren(node *tree_sitter.Node, source []byte, symbols map[string]string, interpKind, contentKind string) string {
var b strings.Builder
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child == nil {
continue
}
switch child.Kind() {
case interpKind:
// Find the identifier inside the interpolation
ident := findDescendantByKind(child, "identifier")
if ident != nil {
name := parser.NodeText(ident, source)
if val, ok := symbols[name]; ok {
b.WriteString(val)
} else {
// Unresolvable — emit placeholder
b.WriteString("{}")
}
}
case contentKind:
b.WriteString(parser.NodeText(child, source))
}
}
return b.String()
}
// resolvePHPEncapsed resolves PHP interpolated strings.
// Children: " { variable_name } string_content "
func resolvePHPEncapsed(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
var b strings.Builder
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child == nil {
continue
}
switch child.Kind() {
case "variable_name":
name := extractPHPVarName(child, source)
if val, ok := symbols[name]; ok {
b.WriteString(val)
} else {
b.WriteString("{}")
}
case "string_content":
b.WriteString(parser.NodeText(child, source))
}
}
return b.String()
}
// resolveScalaInterpolated resolves Scala s"..." interpolated strings.
// The Scala tree-sitter grammar does NOT create child nodes for literal text
// between interpolations — we must extract them from byte gaps between children.
func resolveScalaInterpolated(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
var b strings.Builder
nodeStart := node.StartByte()
nodeEnd := node.EndByte()
// Skip opening quote
cursor := nodeStart + 1 // skip "
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child == nil {
continue
}
kind := child.Kind()
// Skip the quote delimiters themselves
if kind == "\"" || parser.NodeText(child, source) == "\"" {
cursor = child.EndByte()
continue
}
// Emit any literal text between cursor and this child
if child.StartByte() > cursor {
b.Write(source[cursor:child.StartByte()])
}
if kind == "interpolation" {
ident := findDescendantByKind(child, "identifier")
if ident != nil {
name := parser.NodeText(ident, source)
if val, ok := symbols[name]; ok {
b.WriteString(val)
} else {
b.WriteString("{}")
}
}
}
cursor = child.EndByte()
}
// Emit trailing literal text before closing quote
if cursor < nodeEnd-1 { // -1 to skip closing "
b.Write(source[cursor : nodeEnd-1])
}
return b.String()
}
// resolveBinaryConcat resolves string concatenation: left + right or left . right (PHP).
func resolveBinaryConcat(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
opNode := node.ChildByFieldName("operator")
if opNode == nil {
return ""
}
op := parser.NodeText(opNode, source)
if op != "+" && op != "." && op != ".." {
return ""
}
left := resolveStringExpr(node.ChildByFieldName("left"), source, symbols)
right := resolveStringExpr(node.ChildByFieldName("right"), source, symbols)
if left == "" && right == "" {
return ""
}
return left + right
}
// resolveCallExpr resolves format-style calls: Go fmt.Sprintf, Lua string.format, Java String.format.
func resolveCallExpr(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
// Try "function" field (Go, Java) or "name" field (Lua function_call)
funcNode := node.ChildByFieldName("function")
if funcNode == nil {
funcNode = node.ChildByFieldName("name")
}
if funcNode == nil {
return ""
}
funcName := parser.NodeText(funcNode, source)
switch funcName {
case "fmt.Sprintf", "String.format", "string.format":
// continue — supported format functions
default:
return ""
}
// Try "arguments" field (Go, Java, Lua)
args := node.ChildByFieldName("arguments")
if args == nil {
return ""
}
// Collect non-punctuation children as arguments
var argNodes []*tree_sitter.Node
for i := uint(0); i < args.ChildCount(); i++ {
child := args.Child(i)
if child == nil {
continue
}
kind := child.Kind()
if kind == "(" || kind == ")" || kind == "," {
continue
}
argNodes = append(argNodes, child)
}
if len(argNodes) == 0 {
return ""
}
// First arg is the format string
fmtStr := extractStringContent(argNodes[0], source)
if fmtStr == "" {
return ""
}
// Substitute %s, %v, %d with resolved argument values
argIdx := 1
var b strings.Builder
for j := 0; j < len(fmtStr); j++ {
if j+1 < len(fmtStr) && fmtStr[j] == '%' && (fmtStr[j+1] == 's' || fmtStr[j+1] == 'v' || fmtStr[j+1] == 'd') {
if argIdx < len(argNodes) {
val := resolveStringExpr(argNodes[argIdx], source, symbols)
b.WriteString(val)
argIdx++
} else {
b.WriteString("{}")
}
j++ // skip the format specifier char
} else {
b.WriteByte(fmtStr[j])
}
}
return b.String()
}
// resolveRustFormatMacro resolves Rust format!("...", args) macros.
func resolveRustFormatMacro(node *tree_sitter.Node, source []byte, symbols map[string]string) string {
macroName := node.ChildByFieldName("macro")
if macroName == nil || parser.NodeText(macroName, source) != "format" {
return ""
}
tokenTree := findChildByKind(node, "token_tree")
if tokenTree == nil {
return ""
}
// Collect children: skip ( ) ,
var parts []*tree_sitter.Node
for i := uint(0); i < tokenTree.ChildCount(); i++ {
child := tokenTree.Child(i)
if child == nil {
continue
}
kind := child.Kind()
if kind == "(" || kind == ")" || kind == "," {
continue
}
parts = append(parts, child)
}
if len(parts) == 0 {
return ""
}
// First part is the format string
fmtStr := extractStringContent(parts[0], source)
if fmtStr == "" {
return ""
}
// Substitute {} with resolved argument values
argIdx := 1
var b strings.Builder
for j := 0; j < len(fmtStr); j++ {
if j+1 < len(fmtStr) && fmtStr[j] == '{' && fmtStr[j+1] == '}' {
if argIdx < len(parts) {
val := resolveStringExpr(parts[argIdx], source, symbols)
b.WriteString(val)
argIdx++
} else {
b.WriteString("{}")
}
j++ // skip }
} else {
b.WriteByte(fmtStr[j])
}
}
return b.String()
}
// --- Helpers ---
func isStringLiteral(kind string) bool {
switch kind {
case "interpreted_string_literal", "raw_string_literal", // Go
"string_literal": // Rust, Java
return true
}
// Note: PHP "encapsed_string" is NOT here because it can contain interpolation.
// It's handled by resolvePHPEncapsed which covers both simple and interpolated cases.
return false
}
// extractStringContent extracts the text content from a string literal node,
// stripping quotes. Works for Go, Rust, Java, JS/TS, PHP, Scala string nodes.
func extractStringContent(node *tree_sitter.Node, source []byte) string {
if node == nil {
return ""
}
// Look for content children (language-specific names for inner text)
contentKinds := map[string]bool{
"string_content": true, // Python, Rust, PHP, Scala
"string_fragment": true, // JS/TS, Java
"interpreted_string_literal_content": true, // Go
}
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child != nil && contentKinds[child.Kind()] {
return parser.NodeText(child, source)
}
}
// Fallback: strip quotes from full text
text := parser.NodeText(node, source)
if len(text) >= 2 {
first, last := text[0], text[len(text)-1]
if (first == '"' && last == '"') || (first == '\'' && last == '\'') || (first == '`' && last == '`') {
return text[1 : len(text)-1]
}
}
return ""
}
func findChildByKind(node *tree_sitter.Node, kind string) *tree_sitter.Node {
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child != nil && child.Kind() == kind {
return child
}
}
return nil
}
func findDescendantByKind(node *tree_sitter.Node, kind string) *tree_sitter.Node {
if node.Kind() == kind {
return node
}
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child == nil {
continue
}
if found := findDescendantByKind(child, kind); found != nil {
return found
}
}
return nil
}
// firstNonTrivialChild returns the first child that isn't punctuation.
func firstNonTrivialChild(node *tree_sitter.Node) *tree_sitter.Node {
trivial := map[string]bool{"(": true, ")": true, ",": true, ";": true}
for i := uint(0); i < node.ChildCount(); i++ {
child := node.Child(i)
if child != nil && !trivial[child.Kind()] {
return child
}
}
return nil
}
internal/pipeline/resolve_test.go
+266
View File
@@ -0,0 +1,266 @@
package pipeline
import (
"testing"
"unsafe"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
tree_sitter_go "github.com/tree-sitter/tree-sitter-go/bindings/go"
tree_sitter_java "github.com/tree-sitter/tree-sitter-java/bindings/go"
tree_sitter_javascript "github.com/tree-sitter/tree-sitter-javascript/bindings/go"
tree_sitter_php "github.com/tree-sitter/tree-sitter-php/bindings/go"
tree_sitter_python "github.com/tree-sitter/tree-sitter-python/bindings/go"
tree_sitter_rust "github.com/tree-sitter/tree-sitter-rust/bindings/go"
tree_sitter_scala "github.com/tree-sitter/tree-sitter-scala/bindings/go"
tree_sitter_cpp "github.com/tree-sitter/tree-sitter-cpp/bindings/go"
tree_sitter_lua "github.com/tree-sitter-grammars/tree-sitter-lua/bindings/go"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
)
func parseSource(t *testing.T, language lang.Language, code string) (*tree_sitter.Tree, []byte) {
t.Helper()
var tsLang *tree_sitter.Language
switch language {
case lang.Python:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_python.Language()))
case lang.Go:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_go.Language()))
case lang.JavaScript:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_javascript.Language()))
case lang.Rust:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_rust.Language()))
case lang.Java:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_java.Language()))
case lang.PHP:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_php.LanguagePHPOnly()))
case lang.Scala:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_scala.Language()))
case lang.CPP:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_cpp.Language()))
case lang.Lua:
tsLang = tree_sitter.NewLanguage(unsafe.Pointer(tree_sitter_lua.Language()))
default:
t.Fatalf("unsupported language: %s", language)
}
p := tree_sitter.NewParser()
defer p.Close()
if err := p.SetLanguage(tsLang); err != nil {
t.Fatal(err)
}
source := []byte(code)
tree := p.Parse(source, nil)
return tree, source
}
func TestResolvePythonFString(t *testing.T) {
code := `BASE_URL = "https://example.com"
URL = f"{BASE_URL}/notify-failure"
CONCAT = BASE_URL + "/api/orders"
`
tree, source := parseSource(t, lang.Python, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Python)
assertSymbol(t, symbols, "BASE_URL", "https://example.com")
assertSymbol(t, symbols, "URL", "https://example.com/notify-failure")
assertSymbol(t, symbols, "CONCAT", "https://example.com/api/orders")
}
func TestResolvePythonChained(t *testing.T) {
// 3-level chaining: A → B → C
code := `HOST = "https://api.example.com"
BASE = f"{HOST}/v1"
ENDPOINT = f"{BASE}/orders"
`
tree, source := parseSource(t, lang.Python, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Python)
assertSymbol(t, symbols, "HOST", "https://api.example.com")
assertSymbol(t, symbols, "BASE", "https://api.example.com/v1")
assertSymbol(t, symbols, "ENDPOINT", "https://api.example.com/v1/orders")
}
func TestResolveGoConcat(t *testing.T) {
code := `package main
const baseURL = "https://example.com"
var fullURL = baseURL + "/api/orders"
`
tree, source := parseSource(t, lang.Go, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Go)
assertSymbol(t, symbols, "baseURL", "https://example.com")
assertSymbol(t, symbols, "fullURL", "https://example.com/api/orders")
}
func TestResolveGoSprintf(t *testing.T) {
code := `package main
const baseURL = "https://example.com"
var url = fmt.Sprintf("%s/api/items", baseURL)
`
tree, source := parseSource(t, lang.Go, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Go)
assertSymbol(t, symbols, "baseURL", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/items")
}
func TestResolveJSTemplate(t *testing.T) {
code := "const baseUrl = \"https://example.com\";\nconst url = `${baseUrl}/api/orders`;\nconst concat = baseUrl + \"/api/orders\";\n"
tree, source := parseSource(t, lang.JavaScript, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.JavaScript)
assertSymbol(t, symbols, "baseUrl", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/orders")
assertSymbol(t, symbols, "concat", "https://example.com/api/orders")
}
func TestResolveRustFormatMacro(t *testing.T) {
code := `const BASE_URL: &str = "https://example.com";
let url = format!("{}/api/orders", BASE_URL);
let concat = String::from(BASE_URL) + "/api/orders";
`
tree, source := parseSource(t, lang.Rust, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Rust)
assertSymbol(t, symbols, "BASE_URL", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/orders")
// concat: String::from(BASE_URL) is a call_expression with "String::from", not "fmt.Sprintf"
// The resolver sees binary_expression: call_expression + string_literal
// call_expression resolves to "" (not fmt.Sprintf), so concat = "" + "/api/orders"
assertSymbol(t, symbols, "concat", "/api/orders")
}
func TestResolveJavaConcat(t *testing.T) {
code := `class Main {
static final String BASE_URL = "https://example.com";
static final String URL = BASE_URL + "/api/orders";
}`
tree, source := parseSource(t, lang.Java, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Java)
assertSymbol(t, symbols, "BASE_URL", "https://example.com")
assertSymbol(t, symbols, "URL", "https://example.com/api/orders")
}
func TestResolvePHPInterpolation(t *testing.T) {
code := `<?php
$baseUrl = "https://example.com";
$url = "{$baseUrl}/api/orders";
$concat = $baseUrl . "/api/orders";
`
tree, source := parseSource(t, lang.PHP, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.PHP)
assertSymbol(t, symbols, "baseUrl", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/orders")
assertSymbol(t, symbols, "concat", "https://example.com/api/orders")
}
func TestResolveScalaInterpolation(t *testing.T) {
code := `val baseUrl = "https://example.com"
val url = s"${baseUrl}/api/orders"
val concat = baseUrl + "/api/orders"
`
tree, source := parseSource(t, lang.Scala, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Scala)
assertSymbol(t, symbols, "baseUrl", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/orders")
assertSymbol(t, symbols, "concat", "https://example.com/api/orders")
}
func TestResolveUnknownVariable(t *testing.T) {
// When a variable can't be resolved, it should emit {}
code := `URL = f"{UNKNOWN_VAR}/api/orders"
`
tree, source := parseSource(t, lang.Python, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Python)
assertSymbol(t, symbols, "URL", "{}/api/orders")
}
func TestResolveNonStringAssignment(t *testing.T) {
// Integer/boolean assignments should not produce entries
code := `MAX_RETRIES = 3
DEBUG = True
NAME = "hello"
`
tree, source := parseSource(t, lang.Python, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Python)
if _, ok := symbols["MAX_RETRIES"]; ok {
t.Error("MAX_RETRIES should not be in symbols")
}
if _, ok := symbols["DEBUG"]; ok {
t.Error("DEBUG should not be in symbols")
}
assertSymbol(t, symbols, "NAME", "hello")
}
func TestResolveCPPDefineAndConcat(t *testing.T) {
code := `#define BASE_URL "https://example.com"
const std::string fullUrl = BASE_URL + "/api/orders";
`
// Note: C++ #define string concat isn't valid C++ (can't + on string literals in preprocessor),
// but tree-sitter parses it structurally and we resolve the intent.
tree, source := parseSource(t, lang.CPP, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.CPP)
assertSymbol(t, symbols, "BASE_URL", "https://example.com")
// The declaration uses binary_expression with + — BASE_URL resolves from symbol table
assertSymbol(t, symbols, "fullUrl", "https://example.com/api/orders")
}
func TestResolveLuaConcatAndFormat(t *testing.T) {
code := `local base_url = "https://example.com"
local url = base_url .. "/api/orders"
local formatted = string.format("%s/api/items", base_url)
`
tree, source := parseSource(t, lang.Lua, code)
defer tree.Close()
symbols := resolveModuleStrings(tree.RootNode(), source, lang.Lua)
assertSymbol(t, symbols, "base_url", "https://example.com")
assertSymbol(t, symbols, "url", "https://example.com/api/orders")
assertSymbol(t, symbols, "formatted", "https://example.com/api/items")
}
func assertSymbol(t *testing.T, symbols map[string]string, name, want string) {
t.Helper()
got, ok := symbols[name]
if !ok {
t.Errorf("symbol %q not found in resolved symbols: %v", name, symbols)
return
}
if got != want {
t.Errorf("symbol %q = %q, want %q", name, got, want)
}
}
internal/pipeline/resolver.go
+209
View File
@@ -0,0 +1,209 @@
package pipeline
import (
"strings"
"sync"
)
// FunctionRegistry indexes all Function, Method, and Class nodes by qualified
// name and simple name for fast call resolution.
type FunctionRegistry struct {
mu sync.RWMutex
// exact maps qualifiedName -> label (Function/Method/Class)
exact map[string]string
// byName maps simpleName -> []qualifiedName for reverse lookup
byName map[string][]string
}
// NewFunctionRegistry creates an empty registry.
func NewFunctionRegistry() *FunctionRegistry {
return &FunctionRegistry{
exact: make(map[string]string),
byName: make(map[string][]string),
}
}
// Register adds a node to the registry.
func (r *FunctionRegistry) Register(name, qualifiedName, nodeLabel string) {
r.mu.Lock()
defer r.mu.Unlock()
r.exact[qualifiedName] = nodeLabel
// Index by simple name (last segment after the final dot)
simple := simpleName(qualifiedName)
// Avoid duplicates in the slice
for _, existing := range r.byName[simple] {
if existing == qualifiedName {
return
}
}
r.byName[simple] = append(r.byName[simple], qualifiedName)
}
// Resolve attempts to find the qualified name of a callee using a prioritized
// resolution strategy:
// 1. Import map lookup
// 2. Same-module match
// 3. Project-wide single match by simple name
// 4. Suffix match with import distance scoring
func (r *FunctionRegistry) Resolve(calleeName, moduleQN string, importMap map[string]string) string {
r.mu.RLock()
defer r.mu.RUnlock()
// Split calleeName for qualified calls like "pkg.Func" or "obj.method"
parts := strings.SplitN(calleeName, ".", 2)
prefix := parts[0]
var suffix string
if len(parts) > 1 {
suffix = parts[1]
}
// Strategy 1: Import map lookup
if importMap != nil {
if resolved, ok := importMap[prefix]; ok {
var candidate string
if suffix != "" {
// Qualified call: pkg.Func -> resolved + "." + Func
candidate = resolved + "." + suffix
} else {
// Direct import: from X import func -> resolved is the full QN
candidate = resolved
}
if _, exists := r.exact[candidate]; exists {
return candidate
}
// If the resolved path is a module, try appending the calleeName
if suffix != "" {
// Also try looking up just the suffix under the resolved module
for qn := range r.exact {
if strings.HasPrefix(qn, resolved+".") && strings.HasSuffix(qn, "."+suffix) {
return qn
}
}
}
}
}
// Strategy 2: Same-module match
sameModule := moduleQN + "." + calleeName
if _, exists := r.exact[sameModule]; exists {
return sameModule
}
// For qualified calls in the same module, try the full calleeName
if suffix != "" {
sameModuleQualified := moduleQN + "." + suffix
if _, exists := r.exact[sameModuleQualified]; exists {
return sameModuleQualified
}
}
// Strategy 3: Project-wide single match by simple name
lookupName := calleeName
if suffix != "" {
lookupName = suffix
}
simple := simpleName(lookupName)
candidates := r.byName[simple]
if len(candidates) == 1 {
return candidates[0]
}
// Strategy 4: Suffix match with import distance scoring
if suffix != "" {
var matches []string
for _, qn := range candidates {
if strings.HasSuffix(qn, "."+calleeName) {
return qn // exact suffix match
}
if strings.HasSuffix(qn, "."+suffix) {
matches = append(matches, qn)
}
}
if len(matches) == 1 {
return matches[0]
}
if len(matches) > 1 {
return bestByImportDistance(matches, moduleQN)
}
}
// For non-qualified calls with multiple candidates, use import distance
if len(candidates) > 1 {
return bestByImportDistance(candidates, moduleQN)
}
return ""
}
// FindByName returns all qualified names with the given simple name.
func (r *FunctionRegistry) FindByName(name string) []string {
r.mu.RLock()
defer r.mu.RUnlock()
result := make([]string, len(r.byName[name]))
copy(result, r.byName[name])
return result
}
// FindEndingWith returns all qualified names ending with ".suffix".
func (r *FunctionRegistry) FindEndingWith(suffix string) []string {
r.mu.RLock()
defer r.mu.RUnlock()
target := "." + suffix
var result []string
for qn := range r.exact {
if strings.HasSuffix(qn, target) {
result = append(result, qn)
}
}
return result
}
// Size returns the number of entries in the registry.
func (r *FunctionRegistry) Size() int {
r.mu.RLock()
defer r.mu.RUnlock()
return len(r.exact)
}
// simpleName extracts the last dot-separated segment.
func simpleName(qn string) string {
if idx := strings.LastIndex(qn, "."); idx >= 0 {
return qn[idx+1:]
}
return qn
}
// bestByImportDistance picks the candidate whose QN shares the longest common
// prefix with the caller's module QN. This approximates "closest in the
// project structure".
func bestByImportDistance(candidates []string, callerModuleQN string) string {
best := ""
bestLen := -1
for _, c := range candidates {
prefixLen := commonPrefixLen(c, callerModuleQN)
if prefixLen > bestLen {
bestLen = prefixLen
best = c
}
}
return best
}
// commonPrefixLen returns the length of the common dot-segment prefix.
func commonPrefixLen(a, b string) int {
aParts := strings.Split(a, ".")
bParts := strings.Split(b, ".")
count := 0
for i := 0; i < len(aParts) && i < len(bParts); i++ {
if aParts[i] != bParts[i] {
break
}
count++
}
return count
}
internal/pipeline/typeinfer.go
+269
View File
@@ -0,0 +1,269 @@
package pipeline
import (
"strings"
tree_sitter "github.com/tree-sitter/go-tree-sitter"
"github.com/DeusData/codebase-memory-mcp/internal/lang"
"github.com/DeusData/codebase-memory-mcp/internal/parser"
)
// TypeMap tracks variable names to their inferred class/type qualified names.
// Key: variable name, Value: class/type QN in the registry.
type TypeMap map[string]string
// inferTypes walks the AST looking for variable assignments where the value
// is a constructor call (class instantiation) and builds a mapping from
// variable name to the class QN. This enables resolving method calls like
// `obj.method()` to `ClassName.method`.
func inferTypes(
root *tree_sitter.Node,
source []byte,
language lang.Language,
registry *FunctionRegistry,
moduleQN string,
importMap map[string]string,
) TypeMap {
types := make(TypeMap)
switch language {
case lang.Python:
inferPythonTypes(root, source, registry, moduleQN, importMap, types)
case lang.Go:
inferGoTypes(root, source, registry, moduleQN, importMap, types)
}
return types
}
// inferPythonTypes handles Python patterns like:
//
// x = ClassName(args)
// x = module.ClassName(args)
func inferPythonTypes(
root *tree_sitter.Node,
source []byte,
registry *FunctionRegistry,
moduleQN string,
importMap map[string]string,
types TypeMap,
) {
parser.Walk(root, func(node *tree_sitter.Node) bool {
// Look for assignment: expression_statement -> assignment
if node.Kind() != "assignment" {
return true
}
leftNode := node.ChildByFieldName("left")
rightNode := node.ChildByFieldName("right")
if leftNode == nil || rightNode == nil {
return false
}
// Left side must be a simple identifier
if leftNode.Kind() != "identifier" {
return false
}
varName := parser.NodeText(leftNode, source)
// Right side must be a call expression
if rightNode.Kind() != "call" {
return false
}
calleeName := extractCalleeForTypeInfer(rightNode, source)
if calleeName == "" {
return false
}
// Resolve the callee to see if it's a class
classQN := resolveAsClass(calleeName, registry, moduleQN, importMap)
if classQN != "" {
types[varName] = classQN
}
return false
})
}
// inferGoTypes handles Go patterns like:
//
// var x = StructName{...} (composite_literal)
// x := StructName{...} (short_var_declaration)
// var x StructName (var_declaration with type)
func inferGoTypes(
root *tree_sitter.Node,
source []byte,
registry *FunctionRegistry,
moduleQN string,
importMap map[string]string,
types TypeMap,
) {
parser.Walk(root, func(node *tree_sitter.Node) bool {
switch node.Kind() {
case "short_var_declaration":
inferGoShortVar(node, source, registry, moduleQN, importMap, types)
return false
case "var_declaration":
inferGoVarDecl(node, source, registry, moduleQN, importMap, types)
return false
}
return true
})
}
// inferGoShortVar handles: x := StructName{} or x := pkg.StructName{}
func inferGoShortVar(
node *tree_sitter.Node,
source []byte,
registry *FunctionRegistry,
moduleQN string,
importMap map[string]string,
types TypeMap,
) {
leftNode := node.ChildByFieldName("left")
rightNode := node.ChildByFieldName("right")
if leftNode == nil || rightNode == nil {
return
}
varName := extractFirstIdentifier(leftNode, source)
if varName == "" {
return
}
// Check if right side is a composite literal (struct initialization)
typeName := extractCompositeLiteralType(rightNode, source)
if typeName == "" {
// Try call expression (constructor pattern: NewFoo())
if rightNode.Kind() == "expression_list" && rightNode.NamedChildCount() > 0 {
firstExpr := rightNode.NamedChild(0)
if firstExpr != nil {
typeName = extractCompositeLiteralType(firstExpr, source)
}
}
if typeName == "" {
return
}
}
classQN := resolveAsClass(typeName, registry, moduleQN, importMap)
if classQN != "" {
types[varName] = classQN
}
}
// inferGoVarDecl handles: var x StructName or var x = StructName{}
func inferGoVarDecl(
node *tree_sitter.Node,
source []byte,
registry *FunctionRegistry,
moduleQN string,
importMap map[string]string,
types TypeMap,
) {
// Walk var_spec children
parser.Walk(node, func(child *tree_sitter.Node) bool {
if child.Kind() != "var_spec" {
return true
}
nameNode := child.ChildByFieldName("name")
typeNode := child.ChildByFieldName("type")
if nameNode == nil {
return false
}
varName := parser.NodeText(nameNode, source)
// If there's an explicit type, use it
if typeNode != nil {
typeName := parser.NodeText(typeNode, source)
// Strip pointer prefix
typeName = strings.TrimPrefix(typeName, "*")
classQN := resolveAsClass(typeName, registry, moduleQN, importMap)
if classQN != "" {
types[varName] = classQN
}
}
return false
})
}
// resolveAsClass checks if a name refers to a Class/Type node in the registry.
func resolveAsClass(name string, registry *FunctionRegistry, moduleQN string, importMap map[string]string) string {
qn := registry.Resolve(name, moduleQN, importMap)
if qn == "" {
return ""
}
registry.mu.RLock()
defer registry.mu.RUnlock()
label, exists := registry.exact[qn]
if !exists {
return ""
}
// Only return if it's a class-like node
switch label {
case "Class", "Type", "Interface", "Enum":
return qn
}
return ""
}
// extractCalleeForTypeInfer extracts the function/class name from a call node.
func extractCalleeForTypeInfer(callNode *tree_sitter.Node, source []byte) string {
funcNode := callNode.ChildByFieldName("function")
if funcNode == nil {
return ""
}
switch funcNode.Kind() {
case "identifier":
return parser.NodeText(funcNode, source)
case "attribute", "selector_expression":
return parser.NodeText(funcNode, source)
}
return ""
}
// extractFirstIdentifier gets the first identifier from an expression list node.
func extractFirstIdentifier(node *tree_sitter.Node, source []byte) string {
if node.Kind() == "identifier" {
return parser.NodeText(node, source)
}
if node.Kind() == "expression_list" && node.NamedChildCount() > 0 {
first := node.NamedChild(0)
if first != nil && first.Kind() == "identifier" {
return parser.NodeText(first, source)
}
}
return ""
}
// extractCompositeLiteralType extracts the type name from a composite literal.
// E.g., "StructName{field: val}" -> "StructName"
func extractCompositeLiteralType(node *tree_sitter.Node, source []byte) string {
if node.Kind() == "expression_list" && node.NamedChildCount() > 0 {
node = node.NamedChild(0)
if node == nil {
return ""
}
}
if node.Kind() != "composite_literal" {
return ""
}
typeNode := node.ChildByFieldName("type")
if typeNode == nil {
return ""
}
typeName := parser.NodeText(typeNode, source)
// Handle pointer types
typeName = strings.TrimPrefix(typeName, "&")
typeName = strings.TrimPrefix(typeName, "*")
return typeName
}
internal/store/edges.go
+96
View File
@@ -0,0 +1,96 @@
package store
import (
"database/sql"
"fmt"
)
// InsertEdge inserts an edge (dedup by source_id, target_id, type).
func (s *Store) InsertEdge(e *Edge) (int64, error) {
res, err := s.db.Exec(`
INSERT INTO edges (project, source_id, target_id, type, properties)
VALUES (?, ?, ?, ?, ?)
ON CONFLICT(source_id, target_id, type) DO UPDATE SET properties=excluded.properties`,
e.Project, e.SourceID, e.TargetID, e.Type, marshalProps(e.Properties))
if err != nil {
return 0, fmt.Errorf("insert edge: %w", err)
}
return res.LastInsertId()
}
// FindEdgesBySource finds all edges from a given source node.
func (s *Store) FindEdgesBySource(sourceID int64) ([]*Edge, error) {
rows, err := s.db.Query(`SELECT id, project, source_id, target_id, type, properties
FROM edges WHERE source_id=?`, sourceID)
if err != nil {
return nil, fmt.Errorf("find edges by source: %w", err)
}
defer rows.Close()
return scanEdges(rows)
}
// FindEdgesByTarget finds all edges to a given target node.
func (s *Store) FindEdgesByTarget(targetID int64) ([]*Edge, error) {
rows, err := s.db.Query(`SELECT id, project, source_id, target_id, type, properties
FROM edges WHERE target_id=?`, targetID)
if err != nil {
return nil, fmt.Errorf("find edges by target: %w", err)
}
defer rows.Close()
return scanEdges(rows)
}
// FindEdgesBySourceAndType finds edges from a source with a specific type.
func (s *Store) FindEdgesBySourceAndType(sourceID int64, edgeType string) ([]*Edge, error) {
rows, err := s.db.Query(`SELECT id, project, source_id, target_id, type, properties
FROM edges WHERE source_id=? AND type=?`, sourceID, edgeType)
if err != nil {
return nil, fmt.Errorf("find edges by source+type: %w", err)
}
defer rows.Close()
return scanEdges(rows)
}
// FindEdgesByTargetAndType finds edges to a target with a specific type.
func (s *Store) FindEdgesByTargetAndType(targetID int64, edgeType string) ([]*Edge, error) {
rows, err := s.db.Query(`SELECT id, project, source_id, target_id, type, properties
FROM edges WHERE target_id=? AND type=?`, targetID, edgeType)
if err != nil {
return nil, fmt.Errorf("find edges by target+type: %w", err)
}
defer rows.Close()
return scanEdges(rows)
}
// CountEdges returns the number of edges in a project.
func (s *Store) CountEdges(project string) (int, error) {
var count int
err := s.db.QueryRow("SELECT COUNT(*) FROM edges WHERE project=?", project).Scan(&count)
return count, err
}
// DeleteEdgesByProject deletes all edges for a project.
func (s *Store) DeleteEdgesByProject(project string) error {
_, err := s.db.Exec("DELETE FROM edges WHERE project=?", project)
return err
}
// DeleteEdgesByType deletes all edges of a given type for a project.
func (s *Store) DeleteEdgesByType(project, edgeType string) error {
_, err := s.db.Exec("DELETE FROM edges WHERE project=? AND type=?", project, edgeType)
return err
}
func scanEdges(rows *sql.Rows) ([]*Edge, error) {
var result []*Edge
for rows.Next() {
var e Edge
var props string
if err := rows.Scan(&e.ID, &e.Project, &e.SourceID, &e.TargetID, &e.Type, &props); err != nil {
return nil, err
}
e.Properties = unmarshalProps(props)
result = append(result, &e)
}
return result, rows.Err()
}
internal/store/nodes.go
+147
View File
@@ -0,0 +1,147 @@
package store
import (
"database/sql"
"fmt"
)
// UpsertNode inserts or replaces a node (dedup by qualified_name).
func (s *Store) UpsertNode(n *Node) (int64, error) {
res, err := s.db.Exec(`
INSERT INTO nodes (project, label, name, qualified_name, file_path, start_line, end_line, properties)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(project, qualified_name) DO UPDATE SET
label=excluded.label, name=excluded.name, file_path=excluded.file_path,
start_line=excluded.start_line, end_line=excluded.end_line, properties=excluded.properties`,
n.Project, n.Label, n.Name, n.QualifiedName, n.FilePath, n.StartLine, n.EndLine, marshalProps(n.Properties))
if err != nil {
return 0, fmt.Errorf("upsert node: %w", err)
}
id, err := res.LastInsertId()
if err != nil {
return 0, err
}
// On conflict, LastInsertId may return 0; query the actual id
if id == 0 {
err = s.db.QueryRow("SELECT id FROM nodes WHERE project=? AND qualified_name=?", n.Project, n.QualifiedName).Scan(&id)
if err != nil {
return 0, fmt.Errorf("get node id: %w", err)
}
}
return id, nil
}
// FindNodeByID finds a node by its primary key ID.
func (s *Store) FindNodeByID(id int64) (*Node, error) {
row := s.db.QueryRow(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE id=?`, id)
return scanNode(row)
}
// FindNodeByQN finds a node by project and qualified name.
func (s *Store) FindNodeByQN(project, qualifiedName string) (*Node, error) {
row := s.db.QueryRow(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE project=? AND qualified_name=?`, project, qualifiedName)
return scanNode(row)
}
// FindNodesByName finds nodes by project and name.
func (s *Store) FindNodesByName(project, name string) ([]*Node, error) {
rows, err := s.db.Query(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE project=? AND name=?`, project, name)
if err != nil {
return nil, fmt.Errorf("find by name: %w", err)
}
defer rows.Close()
return scanNodes(rows)
}
// FindNodesByLabel finds all nodes with a given label in a project.
func (s *Store) FindNodesByLabel(project, label string) ([]*Node, error) {
rows, err := s.db.Query(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE project=? AND label=?`, project, label)
if err != nil {
return nil, fmt.Errorf("find by label: %w", err)
}
defer rows.Close()
return scanNodes(rows)
}
// FindNodesByFile finds all nodes in a given file.
func (s *Store) FindNodesByFile(project, filePath string) ([]*Node, error) {
rows, err := s.db.Query(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE project=? AND file_path=?`, project, filePath)
if err != nil {
return nil, fmt.Errorf("find by file: %w", err)
}
defer rows.Close()
return scanNodes(rows)
}
// CountNodes returns the number of nodes in a project.
func (s *Store) CountNodes(project string) (int, error) {
var count int
err := s.db.QueryRow("SELECT COUNT(*) FROM nodes WHERE project=?", project).Scan(&count)
return count, err
}
// DeleteNodesByProject deletes all nodes for a project.
func (s *Store) DeleteNodesByProject(project string) error {
_, err := s.db.Exec("DELETE FROM nodes WHERE project=?", project)
return err
}
// DeleteNodesByFile deletes all nodes for a specific file in a project.
func (s *Store) DeleteNodesByFile(project, filePath string) error {
_, err := s.db.Exec("DELETE FROM nodes WHERE project=? AND file_path=?", project, filePath)
return err
}
// DeleteNodesByLabel deletes all nodes with a given label in a project.
func (s *Store) DeleteNodesByLabel(project, label string) error {
_, err := s.db.Exec("DELETE FROM nodes WHERE project=? AND label=?", project, label)
return err
}
// AllNodes returns all nodes for a project.
func (s *Store) AllNodes(project string) ([]*Node, error) {
rows, err := s.db.Query(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE project=?`, project)
if err != nil {
return nil, err
}
defer rows.Close()
return scanNodes(rows)
}
type scanner interface {
Scan(dest ...any) error
}
func scanNode(row scanner) (*Node, error) {
var n Node
var props string
err := row.Scan(&n.ID, &n.Project, &n.Label, &n.Name, &n.QualifiedName, &n.FilePath, &n.StartLine, &n.EndLine, &props)
if err != nil {
if err == sql.ErrNoRows {
return nil, nil
}
return nil, err
}
n.Properties = unmarshalProps(props)
return &n, nil
}
func scanNodes(rows *sql.Rows) ([]*Node, error) {
var result []*Node
for rows.Next() {
var n Node
var props string
if err := rows.Scan(&n.ID, &n.Project, &n.Label, &n.Name, &n.QualifiedName, &n.FilePath, &n.StartLine, &n.EndLine, &props); err != nil {
return nil, err
}
n.Properties = unmarshalProps(props)
result = append(result, &n)
}
return result, rows.Err()
}
internal/store/projects.go
+100
View File
@@ -0,0 +1,100 @@
package store
import "fmt"
// Project represents an indexed project.
type Project struct {
Name string
IndexedAt string
RootPath string
}
// UpsertProject creates or updates a project record.
func (s *Store) UpsertProject(name, rootPath string) error {
_, err := s.db.Exec(`
INSERT INTO projects (name, indexed_at, root_path) VALUES (?, ?, ?)
ON CONFLICT(name) DO UPDATE SET indexed_at=excluded.indexed_at, root_path=excluded.root_path`,
name, Now(), rootPath)
return err
}
// GetProject returns a project by name.
func (s *Store) GetProject(name string) (*Project, error) {
var p Project
err := s.db.QueryRow("SELECT name, indexed_at, root_path FROM projects WHERE name=?", name).
Scan(&p.Name, &p.IndexedAt, &p.RootPath)
if err != nil {
return nil, err
}
return &p, nil
}
// ListProjects returns all indexed projects.
func (s *Store) ListProjects() ([]*Project, error) {
rows, err := s.db.Query("SELECT name, indexed_at, root_path FROM projects ORDER BY name")
if err != nil {
return nil, err
}
defer rows.Close()
var result []*Project
for rows.Next() {
var p Project
if err := rows.Scan(&p.Name, &p.IndexedAt, &p.RootPath); err != nil {
return nil, err
}
result = append(result, &p)
}
return result, rows.Err()
}
// DeleteProject deletes a project and all associated data (CASCADE).
func (s *Store) DeleteProject(name string) error {
_, err := s.db.Exec("DELETE FROM projects WHERE name=?", name)
return err
}
// FileHash represents a stored file content hash for incremental reindex.
type FileHash struct {
Project string
RelPath string
SHA256 string
}
// UpsertFileHash stores a file's content hash.
func (s *Store) UpsertFileHash(project, relPath, sha256 string) error {
_, err := s.db.Exec(`
INSERT INTO file_hashes (project, rel_path, sha256) VALUES (?, ?, ?)
ON CONFLICT(project, rel_path) DO UPDATE SET sha256=excluded.sha256`,
project, relPath, sha256)
return err
}
// GetFileHashes returns all file hashes for a project.
func (s *Store) GetFileHashes(project string) (map[string]string, error) {
rows, err := s.db.Query("SELECT rel_path, sha256 FROM file_hashes WHERE project=?", project)
if err != nil {
return nil, fmt.Errorf("get file hashes: %w", err)
}
defer rows.Close()
result := make(map[string]string)
for rows.Next() {
var path, hash string
if err := rows.Scan(&path, &hash); err != nil {
return nil, err
}
result[path] = hash
}
return result, rows.Err()
}
// DeleteFileHash deletes a single file hash entry.
func (s *Store) DeleteFileHash(project, relPath string) error {
_, err := s.db.Exec("DELETE FROM file_hashes WHERE project=? AND rel_path=?", project, relPath)
return err
}
// DeleteFileHashes deletes all file hashes for a project.
func (s *Store) DeleteFileHashes(project string) error {
_, err := s.db.Exec("DELETE FROM file_hashes WHERE project=?", project)
return err
}
internal/store/schema.go
+134
View File
@@ -0,0 +1,134 @@
package store
import "fmt"
// SchemaInfo contains graph schema statistics.
type SchemaInfo struct {
NodeLabels []LabelCount `json:"node_labels"`
RelationshipTypes []TypeCount `json:"relationship_types"`
RelationshipPatterns []string `json:"relationship_patterns"`
SampleFunctionNames []string `json:"sample_function_names"`
SampleClassNames []string `json:"sample_class_names"`
SampleQualifiedNames []string `json:"sample_qualified_names"`
}
// LabelCount is a label with its count.
type LabelCount struct {
Label string `json:"label"`
Count int `json:"count"`
}
// TypeCount is a relationship type with its count.
type TypeCount struct {
Type string `json:"type"`
Count int `json:"count"`
}
// GetSchema returns graph schema statistics for a project.
func (s *Store) GetSchema(project string) (*SchemaInfo, error) {
info := &SchemaInfo{}
// Node label counts
rows, err := s.db.Query("SELECT label, COUNT(*) as cnt FROM nodes WHERE project=? GROUP BY label ORDER BY cnt DESC", project)
if err != nil {
return nil, fmt.Errorf("schema labels: %w", err)
}
defer rows.Close()
for rows.Next() {
var lc LabelCount
if err := rows.Scan(&lc.Label, &lc.Count); err != nil {
return nil, err
}
info.NodeLabels = append(info.NodeLabels, lc)
}
if err := rows.Err(); err != nil {
return nil, err
}
// Edge type counts
rows2, err := s.db.Query("SELECT type, COUNT(*) as cnt FROM edges WHERE project=? GROUP BY type ORDER BY cnt DESC", project)
if err != nil {
return nil, fmt.Errorf("schema edge types: %w", err)
}
defer rows2.Close()
for rows2.Next() {
var tc TypeCount
if err := rows2.Scan(&tc.Type, &tc.Count); err != nil {
return nil, err
}
info.RelationshipTypes = append(info.RelationshipTypes, tc)
}
if err := rows2.Err(); err != nil {
return nil, err
}
// Relationship patterns: (src_label)-[type]->(tgt_label) with counts
rows3, err := s.db.Query(`
SELECT sn.label, e.type, tn.label, COUNT(*) as cnt
FROM edges e
JOIN nodes sn ON e.source_id = sn.id
JOIN nodes tn ON e.target_id = tn.id
WHERE e.project=?
GROUP BY sn.label, e.type, tn.label
ORDER BY cnt DESC
LIMIT 25`, project)
if err != nil {
return nil, fmt.Errorf("schema patterns: %w", err)
}
defer rows3.Close()
for rows3.Next() {
var src, rel, tgt string
var cnt int
if err := rows3.Scan(&src, &rel, &tgt, &cnt); err != nil {
return nil, err
}
info.RelationshipPatterns = append(info.RelationshipPatterns, fmt.Sprintf("(:%s)-[:%s]->(:%s) [%dx]", src, rel, tgt, cnt))
}
if err := rows3.Err(); err != nil {
return nil, err
}
// Sample function names
rows4, err := s.db.Query("SELECT name FROM nodes WHERE project=? AND label='Function' ORDER BY name LIMIT 30", project)
if err != nil {
return nil, fmt.Errorf("schema sample funcs: %w", err)
}
defer rows4.Close()
for rows4.Next() {
var name string
if err := rows4.Scan(&name); err != nil {
return nil, err
}
info.SampleFunctionNames = append(info.SampleFunctionNames, name)
}
// Sample class names
rows5, err := s.db.Query("SELECT name FROM nodes WHERE project=? AND label='Class' ORDER BY name LIMIT 20", project)
if err != nil {
return nil, fmt.Errorf("schema sample classes: %w", err)
}
defer rows5.Close()
for rows5.Next() {
var name string
if err := rows5.Scan(&name); err != nil {
return nil, err
}
info.SampleClassNames = append(info.SampleClassNames, name)
}
// Sample qualified names
rows6, err := s.db.Query("SELECT qualified_name FROM nodes WHERE project=? LIMIT 5", project)
if err != nil {
return nil, fmt.Errorf("schema sample qns: %w", err)
}
defer rows6.Close()
for rows6.Next() {
var qn string
if err := rows6.Scan(&qn); err != nil {
return nil, err
}
info.SampleQualifiedNames = append(info.SampleQualifiedNames, qn)
}
return info, nil
}
internal/store/search.go
+205
View File
@@ -0,0 +1,205 @@
package store
import (
"fmt"
"regexp"
"strings"
)
// SearchParams defines structured search parameters.
type SearchParams struct {
Project string
Label string
NamePattern string
FilePattern string
Relationship string
Direction string // "inbound", "outbound", "any"
MinDegree int
MaxDegree int
Limit int
ExcludeEntryPoints bool // when true, exclude nodes with is_entry_point=true
}
// SearchResult is a node with edge degree info.
type SearchResult struct {
Node *Node
InDegree int
OutDegree int
ConnectedNames []string
}
// Search executes a parameterized search query.
func (s *Store) Search(params SearchParams) ([]*SearchResult, error) {
if params.Limit <= 0 {
params.Limit = 50
}
if params.Limit > 200 {
params.Limit = 200
}
// Build the query dynamically with parameterized values
var conditions []string
var args []any
conditions = append(conditions, "n.project = ?")
args = append(args, params.Project)
if params.Label != "" {
conditions = append(conditions, "n.label = ?")
args = append(args, params.Label)
}
if params.FilePattern != "" {
// Convert glob to SQL LIKE pattern
likePattern := globToLike(params.FilePattern)
conditions = append(conditions, "n.file_path LIKE ?")
args = append(args, likePattern)
}
where := strings.Join(conditions, " AND ")
// When Go-side filtering is needed (regex, degree), fetch more rows from SQL
// and apply the user limit after filtering.
hasDegreeFilter := params.MinDegree >= 0 || params.MaxDegree >= 0
var sqlLimit int
if params.NamePattern != "" || hasDegreeFilter {
sqlLimit = 10000 // fetch enough rows for Go-side filtering
} else {
sqlLimit = params.Limit
}
query := fmt.Sprintf(`
SELECT n.id, n.project, n.label, n.name, n.qualified_name, n.file_path, n.start_line, n.end_line, n.properties
FROM nodes n
WHERE %s
LIMIT ?`, where)
args = append(args, sqlLimit)
rows, err := s.db.Query(query, args...)
if err != nil {
return nil, fmt.Errorf("search: %w", err)
}
defer rows.Close()
var nodes []*Node
for rows.Next() {
var n Node
var props string
if err := rows.Scan(&n.ID, &n.Project, &n.Label, &n.Name, &n.QualifiedName, &n.FilePath, &n.StartLine, &n.EndLine, &props); err != nil {
return nil, err
}
n.Properties = unmarshalProps(props)
nodes = append(nodes, &n)
}
if err := rows.Err(); err != nil {
return nil, err
}
// Apply name pattern filter in Go (regex)
if params.NamePattern != "" {
nodes, err = filterByNamePattern(nodes, params.NamePattern)
if err != nil {
return nil, err
}
}
// Apply limit after name filtering (but not when degree filters are active,
// since degree filtering happens in the loop below with its own limit check)
if !hasDegreeFilter && len(nodes) > params.Limit {
nodes = nodes[:params.Limit]
}
// Build results with degree info
var results []*SearchResult
for _, n := range nodes {
sr := &SearchResult{Node: n}
// Count degrees
if params.Relationship != "" {
var inCount, outCount int
s.db.QueryRow("SELECT COUNT(*) FROM edges WHERE target_id=? AND type=?", n.ID, params.Relationship).Scan(&inCount)
s.db.QueryRow("SELECT COUNT(*) FROM edges WHERE source_id=? AND type=?", n.ID, params.Relationship).Scan(&outCount)
sr.InDegree = inCount
sr.OutDegree = outCount
} else {
s.db.QueryRow("SELECT COUNT(*) FROM edges WHERE target_id=?", n.ID).Scan(&sr.InDegree)
s.db.QueryRow("SELECT COUNT(*) FROM edges WHERE source_id=?", n.ID).Scan(&sr.OutDegree)
}
// Apply degree filters (-1 means "not set")
degree := sr.InDegree
if params.Direction == "outbound" {
degree = sr.OutDegree
}
if params.MinDegree >= 0 && degree < params.MinDegree {
continue
}
if params.MaxDegree >= 0 && degree > params.MaxDegree {
continue
}
// Exclude entry points from dead code results
if params.ExcludeEntryPoints && isEntryPoint(n) {
continue
}
// Get connected node names (limit to 10 for display)
connRows, connErr := s.db.Query(`
SELECT DISTINCT n2.name FROM edges e
JOIN nodes n2 ON (e.target_id = n2.id OR e.source_id = n2.id)
WHERE (e.source_id = ? OR e.target_id = ?) AND n2.id != ?
LIMIT 10`, n.ID, n.ID, n.ID)
if connErr == nil {
for connRows.Next() {
var name string
connRows.Scan(&name)
sr.ConnectedNames = append(sr.ConnectedNames, name)
}
connRows.Close()
}
results = append(results, sr)
if len(results) >= params.Limit {
break
}
}
return results, nil
}
// globToLike converts a glob pattern to SQL LIKE pattern.
func globToLike(pattern string) string {
// Replace ** with % and * with %
result := strings.ReplaceAll(pattern, "**", "%")
result = strings.ReplaceAll(result, "*", "%")
result = strings.ReplaceAll(result, "?", "_")
return result
}
// isEntryPoint returns true if a node has is_entry_point=true in its properties.
func isEntryPoint(n *Node) bool {
if n.Properties == nil {
return false
}
ep, ok := n.Properties["is_entry_point"]
if !ok {
return false
}
b, ok := ep.(bool)
return ok && b
}
// filterByNamePattern filters nodes by a regex name pattern.
func filterByNamePattern(nodes []*Node, pattern string) ([]*Node, error) {
re, err := regexp.Compile(pattern)
if err != nil {
return nil, fmt.Errorf("invalid name pattern: %w", err)
}
var filtered []*Node
for _, n := range nodes {
if re.MatchString(n.Name) || re.MatchString(n.QualifiedName) {
filtered = append(filtered, n)
}
}
return filtered, nil
}
internal/store/store.go
+181
View File
@@ -0,0 +1,181 @@
package store
import (
"database/sql"
"encoding/json"
"fmt"
"os"
"path/filepath"
"time"
_ "modernc.org/sqlite"
)
// Store wraps a SQLite connection for graph storage.
type Store struct {
db *sql.DB
dbPath string
}
// Node represents a graph node stored in SQLite.
type Node struct {
ID int64
Project string
Label string
Name string
QualifiedName string
FilePath string
StartLine int
EndLine int
Properties map[string]any
}
// Edge represents a graph edge stored in SQLite.
type Edge struct {
ID int64
Project string
SourceID int64
TargetID int64
Type string
Properties map[string]any
}
// cacheDir returns the default cache directory for databases.
func cacheDir() (string, error) {
home, err := os.UserHomeDir()
if err != nil {
return "", fmt.Errorf("home dir: %w", err)
}
dir := filepath.Join(home, ".cache", "codebase-memory-mcp")
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", fmt.Errorf("mkdir cache: %w", err)
}
return dir, nil
}
// Open opens or creates a SQLite database for the given project.
func Open(project string) (*Store, error) {
dir, err := cacheDir()
if err != nil {
return nil, err
}
dbPath := filepath.Join(dir, project+".db")
return OpenPath(dbPath)
}
// OpenPath opens a SQLite database at the given path.
func OpenPath(dbPath string) (*Store, error) {
db, err := sql.Open("sqlite", dbPath+"?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)&_pragma=foreign_keys(ON)")
if err != nil {
return nil, fmt.Errorf("open db: %w", err)
}
s := &Store{db: db, dbPath: dbPath}
if err := s.initSchema(); err != nil {
db.Close()
return nil, fmt.Errorf("init schema: %w", err)
}
return s, nil
}
// OpenMemory opens an in-memory SQLite database (for testing).
func OpenMemory() (*Store, error) {
db, err := sql.Open("sqlite", ":memory:?_pragma=foreign_keys(ON)")
if err != nil {
return nil, fmt.Errorf("open memory db: %w", err)
}
s := &Store{db: db, dbPath: ":memory:"}
if err := s.initSchema(); err != nil {
db.Close()
return nil, fmt.Errorf("init schema: %w", err)
}
return s, nil
}
// Close closes the database connection.
func (s *Store) Close() error {
return s.db.Close()
}
// DB returns the underlying sql.DB (for advanced queries).
func (s *Store) DB() *sql.DB {
return s.db
}
func (s *Store) initSchema() error {
schema := `
CREATE TABLE IF NOT EXISTS projects (
name TEXT PRIMARY KEY,
indexed_at TEXT NOT NULL,
root_path TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS file_hashes (
project TEXT NOT NULL REFERENCES projects(name) ON DELETE CASCADE,
rel_path TEXT NOT NULL,
sha256 TEXT NOT NULL,
PRIMARY KEY (project, rel_path)
);
CREATE TABLE IF NOT EXISTS nodes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project TEXT NOT NULL REFERENCES projects(name) ON DELETE CASCADE,
label TEXT NOT NULL,
name TEXT NOT NULL,
qualified_name TEXT NOT NULL,
file_path TEXT DEFAULT '',
start_line INTEGER DEFAULT 0,
end_line INTEGER DEFAULT 0,
properties TEXT DEFAULT '{}',
UNIQUE(project, qualified_name)
);
CREATE INDEX IF NOT EXISTS idx_nodes_label ON nodes(project, label);
CREATE INDEX IF NOT EXISTS idx_nodes_name ON nodes(project, name);
CREATE INDEX IF NOT EXISTS idx_nodes_file ON nodes(project, file_path);
CREATE TABLE IF NOT EXISTS edges (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project TEXT NOT NULL REFERENCES projects(name) ON DELETE CASCADE,
source_id INTEGER NOT NULL REFERENCES nodes(id) ON DELETE CASCADE,
target_id INTEGER NOT NULL REFERENCES nodes(id) ON DELETE CASCADE,
type TEXT NOT NULL,
properties TEXT DEFAULT '{}',
UNIQUE(source_id, target_id, type)
);
CREATE INDEX IF NOT EXISTS idx_edges_source ON edges(source_id, type);
CREATE INDEX IF NOT EXISTS idx_edges_target ON edges(target_id, type);
CREATE INDEX IF NOT EXISTS idx_edges_type ON edges(project, type);
`
_, err := s.db.Exec(schema)
return err
}
// marshalProps serializes properties to JSON.
func marshalProps(props map[string]any) string {
if props == nil {
return "{}"
}
b, err := json.Marshal(props)
if err != nil {
return "{}"
}
return string(b)
}
// unmarshalProps deserializes JSON properties.
func unmarshalProps(data string) map[string]any {
if data == "" {
return map[string]any{}
}
var m map[string]any
if err := json.Unmarshal([]byte(data), &m); err != nil {
return map[string]any{}
}
return m
}
// Now returns the current time in ISO 8601 format.
func Now() string {
return time.Now().UTC().Format(time.RFC3339)
}
internal/store/store_test.go
+279
View File
@@ -0,0 +1,279 @@
package store
import "testing"
func TestOpenMemory(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
}
func TestNodeCRUD(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
// Create project first
if err := s.UpsertProject("test", "/tmp/test"); err != nil {
t.Fatalf("UpsertProject: %v", err)
}
// Insert node
n := &Node{
Project: "test",
Label: "Function",
Name: "Foo",
QualifiedName: "test.main.Foo",
FilePath: "main.go",
StartLine: 10,
EndLine: 20,
Properties: map[string]any{"signature": "func Foo(x int) error"},
}
id, err := s.UpsertNode(n)
if err != nil {
t.Fatalf("UpsertNode: %v", err)
}
if id == 0 {
t.Fatal("expected non-zero id")
}
// Find by QN
found, err := s.FindNodeByQN("test", "test.main.Foo")
if err != nil {
t.Fatalf("FindNodeByQN: %v", err)
}
if found == nil {
t.Fatal("expected node, got nil")
}
if found.Name != "Foo" {
t.Errorf("expected Foo, got %s", found.Name)
}
if found.Properties["signature"] != "func Foo(x int) error" {
t.Errorf("unexpected signature: %v", found.Properties["signature"])
}
// Find by name
nodes, err := s.FindNodesByName("test", "Foo")
if err != nil {
t.Fatalf("FindNodesByName: %v", err)
}
if len(nodes) != 1 {
t.Fatalf("expected 1 node, got %d", len(nodes))
}
// Count
count, err := s.CountNodes("test")
if err != nil {
t.Fatalf("CountNodes: %v", err)
}
if count != 1 {
t.Errorf("expected 1, got %d", count)
}
}
func TestNodeDedup(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
if err := s.UpsertProject("test", "/tmp/test"); err != nil {
t.Fatalf("UpsertProject: %v", err)
}
// Insert same qualified_name twice — should update, not duplicate
n1 := &Node{Project: "test", Label: "Function", Name: "Foo", QualifiedName: "test.main.Foo"}
n2 := &Node{Project: "test", Label: "Function", Name: "Foo", QualifiedName: "test.main.Foo", Properties: map[string]any{"updated": true}}
s.UpsertNode(n1)
s.UpsertNode(n2)
count, _ := s.CountNodes("test")
if count != 1 {
t.Errorf("expected 1 node after dedup, got %d", count)
}
// Verify it was updated
found, _ := s.FindNodeByQN("test", "test.main.Foo")
if found.Properties["updated"] != true {
t.Error("expected updated property")
}
}
func TestEdgeCRUD(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
if err := s.UpsertProject("test", "/tmp/test"); err != nil {
t.Fatalf("UpsertProject: %v", err)
}
// Create two nodes
id1, _ := s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "A", QualifiedName: "test.A"})
id2, _ := s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "B", QualifiedName: "test.B"})
// Insert edge
_, err = s.InsertEdge(&Edge{Project: "test", SourceID: id1, TargetID: id2, Type: "CALLS"})
if err != nil {
t.Fatalf("InsertEdge: %v", err)
}
// Find by source
edges, err := s.FindEdgesBySource(id1)
if err != nil {
t.Fatalf("FindEdgesBySource: %v", err)
}
if len(edges) != 1 {
t.Fatalf("expected 1 edge, got %d", len(edges))
}
if edges[0].Type != "CALLS" {
t.Errorf("expected CALLS, got %s", edges[0].Type)
}
// Count
count, _ := s.CountEdges("test")
if count != 1 {
t.Errorf("expected 1, got %d", count)
}
}
func TestCascadeDelete(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
// Create project with nodes and edges
s.UpsertProject("test", "/tmp/test")
id1, _ := s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "A", QualifiedName: "test.A"})
id2, _ := s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "B", QualifiedName: "test.B"})
s.InsertEdge(&Edge{Project: "test", SourceID: id1, TargetID: id2, Type: "CALLS"})
// Delete project — should cascade
if err := s.DeleteProject("test"); err != nil {
t.Fatalf("DeleteProject: %v", err)
}
nodes, _ := s.CountNodes("test")
edges, _ := s.CountEdges("test")
if nodes != 0 {
t.Errorf("expected 0 nodes after cascade, got %d", nodes)
}
if edges != 0 {
t.Errorf("expected 0 edges after cascade, got %d", edges)
}
}
func TestProjectCRUD(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
// Create
if err := s.UpsertProject("myproject", "/home/user/myproject"); err != nil {
t.Fatalf("UpsertProject: %v", err)
}
// Get
p, err := s.GetProject("myproject")
if err != nil {
t.Fatalf("GetProject: %v", err)
}
if p.Name != "myproject" {
t.Errorf("expected myproject, got %s", p.Name)
}
if p.RootPath != "/home/user/myproject" {
t.Errorf("unexpected root: %s", p.RootPath)
}
// List
projects, err := s.ListProjects()
if err != nil {
t.Fatalf("ListProjects: %v", err)
}
if len(projects) != 1 {
t.Fatalf("expected 1 project, got %d", len(projects))
}
}
func TestFileHashes(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
s.UpsertProject("test", "/tmp/test")
// Upsert
if err := s.UpsertFileHash("test", "main.go", "abc123"); err != nil {
t.Fatalf("UpsertFileHash: %v", err)
}
// Get
hashes, err := s.GetFileHashes("test")
if err != nil {
t.Fatalf("GetFileHashes: %v", err)
}
if hashes["main.go"] != "abc123" {
t.Errorf("expected abc123, got %s", hashes["main.go"])
}
// Update
s.UpsertFileHash("test", "main.go", "def456")
hashes, _ = s.GetFileHashes("test")
if hashes["main.go"] != "def456" {
t.Errorf("expected def456, got %s", hashes["main.go"])
}
}
func TestSearch(t *testing.T) {
s, err := OpenMemory()
if err != nil {
t.Fatalf("OpenMemory: %v", err)
}
defer s.Close()
s.UpsertProject("test", "/tmp/test")
s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "SubmitOrder", QualifiedName: "test.main.SubmitOrder", FilePath: "main.go"})
s.UpsertNode(&Node{Project: "test", Label: "Function", Name: "ProcessOrder", QualifiedName: "test.service.ProcessOrder", FilePath: "service.go"})
s.UpsertNode(&Node{Project: "test", Label: "Class", Name: "OrderService", QualifiedName: "test.service.OrderService", FilePath: "service.go"})
// Search by label
results, err := s.Search(SearchParams{Project: "test", Label: "Function"})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(results) != 2 {
t.Errorf("expected 2 functions, got %d", len(results))
}
// Search by name pattern
results, err = s.Search(SearchParams{Project: "test", NamePattern: ".*Submit.*"})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(results) != 1 {
t.Errorf("expected 1 match, got %d", len(results))
}
// Search by file pattern
results, err = s.Search(SearchParams{Project: "test", FilePattern: "service*"})
if err != nil {
t.Fatalf("Search: %v", err)
}
if len(results) != 2 {
t.Errorf("expected 2 nodes in service.go, got %d", len(results))
}
}
internal/store/traverse.go
+103
View File
@@ -0,0 +1,103 @@
package store
// TraverseResult holds BFS traversal results.
type TraverseResult struct {
Root *Node
Visited []*NodeHop
Edges []EdgeInfo
}
// NodeHop is a node with its BFS hop distance.
type NodeHop struct {
Node *Node
Hop int
}
// EdgeInfo is a simplified edge for output.
type EdgeInfo struct {
FromName string
ToName string
Type string
}
// BFS performs breadth-first traversal following edges of given types.
// direction: "outbound" follows source->target, "inbound" follows target->source.
// maxDepth caps the BFS depth, maxResults caps total visited nodes.
func (s *Store) BFS(startNodeID int64, direction string, edgeTypes []string, maxDepth, maxResults int) (*TraverseResult, error) {
if maxDepth <= 0 {
maxDepth = 3
}
if maxResults <= 0 {
maxResults = 200
}
result := &TraverseResult{}
visited := make(map[int64]int) // nodeID -> hop
visited[startNodeID] = 0
type queueItem struct {
nodeID int64
hop int
}
queue := []queueItem{{startNodeID, 0}}
for len(queue) > 0 && len(result.Visited) < maxResults {
item := queue[0]
queue = queue[1:]
if item.hop >= maxDepth {
continue
}
// Get edges from this node
var edges []*Edge
for _, et := range edgeTypes {
var found []*Edge
var err error
if direction == "outbound" {
found, err = s.FindEdgesBySourceAndType(item.nodeID, et)
} else {
found, err = s.FindEdgesByTargetAndType(item.nodeID, et)
}
if err != nil {
return nil, err
}
edges = append(edges, found...)
}
for _, e := range edges {
var nextID int64
if direction == "outbound" {
nextID = e.TargetID
} else {
nextID = e.SourceID
}
if _, seen := visited[nextID]; !seen {
visited[nextID] = item.hop + 1
row := s.db.QueryRow(`SELECT id, project, label, name, qualified_name, file_path, start_line, end_line, properties
FROM nodes WHERE id=?`, nextID)
nextNode, err := scanNode(row)
if err != nil || nextNode == nil {
continue
}
result.Visited = append(result.Visited, &NodeHop{Node: nextNode, Hop: item.hop + 1})
queue = append(queue, queueItem{nextID, item.hop + 1})
if len(result.Visited) >= maxResults {
break
}
}
// Record edge info
var fromName, toName string
s.db.QueryRow("SELECT name FROM nodes WHERE id=?", e.SourceID).Scan(&fromName)
s.db.QueryRow("SELECT name FROM nodes WHERE id=?", e.TargetID).Scan(&toName)
result.Edges = append(result.Edges, EdgeInfo{FromName: fromName, ToName: toName, Type: e.Type})
}
}
return result, nil
}
internal/tools/code_search.go
+164
View File
@@ -0,0 +1,164 @@
package tools
import (
"bufio"
"context"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
type codeMatch struct {
File string `json:"file"`
Line int `json:"line"`
Content string `json:"content"`
}
func (s *Server) handleSearchCode(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
pattern := getStringArg(args, "pattern")
if pattern == "" {
return errResult("pattern is required"), nil
}
fileGlob := getStringArg(args, "file_pattern")
maxResults := getIntArg(args, "max_results", 50)
if maxResults > 200 {
maxResults = 200
}
isRegex := false
if v, ok := args["regex"]; ok {
if b, ok := v.(bool); ok {
isRegex = b
}
}
// Resolve project root
root, err := s.resolveProjectRoot()
if err != nil {
return errResult(fmt.Sprintf("resolve root: %v", err)), nil
}
// Compile regex or prepare literal search
var re *regexp.Regexp
if isRegex {
re, err = regexp.Compile(pattern)
if err != nil {
return errResult(fmt.Sprintf("invalid regex: %v", err)), nil
}
}
// Get indexed file paths from the store
projects, _ := s.store.ListProjects()
var filePaths []string
for _, p := range projects {
files, _ := s.store.FindNodesByLabel(p.Name, "File")
for _, f := range files {
if f.FilePath == "" {
continue
}
if fileGlob != "" {
matched, _ := filepath.Match(fileGlob, filepath.Base(f.FilePath))
// Also try against the full relative path using double-star simulation
if !matched {
matched = globMatch(fileGlob, f.FilePath)
}
if !matched {
continue
}
}
filePaths = append(filePaths, f.FilePath)
}
}
var matches []codeMatch
for _, relPath := range filePaths {
if len(matches) >= maxResults {
break
}
absPath := filepath.Join(root, relPath)
fileMatches := searchFile(absPath, relPath, pattern, re, isRegex, maxResults-len(matches))
matches = append(matches, fileMatches...)
}
return jsonResult(map[string]any{
"pattern": pattern,
"total": len(matches),
"truncated": len(matches) >= maxResults,
"matches": matches,
"files_count": len(filePaths),
}), nil
}
func searchFile(absPath, relPath, pattern string, re *regexp.Regexp, isRegex bool, limit int) []codeMatch {
f, err := os.Open(absPath)
if err != nil {
return nil
}
defer f.Close()
var matches []codeMatch
scanner := bufio.NewScanner(f)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
lineNum := 0
for scanner.Scan() {
lineNum++
line := scanner.Text()
var found bool
if isRegex {
found = re.MatchString(line)
} else {
found = strings.Contains(line, pattern)
}
if found {
content := strings.TrimSpace(line)
if len(content) > 200 {
content = content[:200] + "..."
}
matches = append(matches, codeMatch{
File: relPath,
Line: lineNum,
Content: content,
})
if len(matches) >= limit {
break
}
}
}
return matches
}
// globMatch does a simple glob match supporting ** patterns.
func globMatch(pattern, path string) bool {
if strings.Contains(pattern, "**") {
// Split pattern on **
parts := strings.SplitN(pattern, "**", 2)
prefix := strings.TrimRight(parts[0], "/")
suffix := strings.TrimLeft(parts[1], "/")
if prefix != "" && !strings.HasPrefix(path, prefix) {
return false
}
if suffix != "" {
matched, _ := filepath.Match(suffix, filepath.Base(path))
return matched
}
return true
}
matched, _ := filepath.Match(pattern, path)
return matched
}
internal/tools/file.go
+199
View File
@@ -0,0 +1,199 @@
package tools
import (
"bufio"
"context"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleReadFile(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
filePath := getStringArg(args, "path")
if filePath == "" {
return errResult("path is required"), nil
}
startLine := getIntArg(args, "start_line", 0)
endLine := getIntArg(args, "end_line", 0)
// Resolve relative path against project root
absPath := filePath
if !filepath.IsAbs(filePath) {
root, rootErr := s.resolveProjectRoot()
if rootErr != nil {
return errResult(fmt.Sprintf("resolve root: %v", rootErr)), nil
}
absPath = filepath.Join(root, filePath)
}
// Check file exists and is not a directory
info, err := os.Stat(absPath)
if err != nil {
return errResult(fmt.Sprintf("file not found: %s", absPath)), nil
}
if info.IsDir() {
return errResult("path is a directory, use list_directory instead"), nil
}
// Cap file size at 500KB
if info.Size() > 500*1024 {
return errResult(fmt.Sprintf("file too large (%d bytes, max 500KB). Use start_line/end_line to read a portion", info.Size())), nil
}
f, err := os.Open(absPath)
if err != nil {
return errResult(fmt.Sprintf("open: %v", err)), nil
}
defer f.Close()
var lines []string
scanner := bufio.NewScanner(f)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024) // 1MB line buffer
lineNum := 0
for scanner.Scan() {
lineNum++
if startLine > 0 && lineNum < startLine {
continue
}
if endLine > 0 && lineNum > endLine {
break
}
line := scanner.Text()
if len(line) > 500 {
line = line[:500] + "..."
}
lines = append(lines, fmt.Sprintf("%4d | %s", lineNum, line))
}
if err := scanner.Err(); err != nil {
return errResult(fmt.Sprintf("read: %v", err)), nil
}
result := map[string]any{
"path": absPath,
"total_lines": lineNum,
"content": strings.Join(lines, "\n"),
}
if startLine > 0 || endLine > 0 {
result["range"] = fmt.Sprintf("%d-%d", startLine, endLine)
}
return jsonResult(result), nil
}
func (s *Server) handleListDirectory(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
dirPath := getStringArg(args, "path")
globPattern := getStringArg(args, "pattern")
// Resolve relative path against project root
absPath := dirPath
if dirPath == "" || !filepath.IsAbs(dirPath) {
root, rootErr := s.resolveProjectRoot()
if rootErr != nil {
return errResult(fmt.Sprintf("resolve root: %v", rootErr)), nil
}
if dirPath == "" {
absPath = root
} else {
absPath = filepath.Join(root, dirPath)
}
}
info, err := os.Stat(absPath)
if err != nil {
return errResult(fmt.Sprintf("path not found: %s", absPath)), nil
}
if !info.IsDir() {
return errResult("path is a file, use read_file instead"), nil
}
type entry struct {
Name string `json:"name"`
Path string `json:"path"`
IsDir bool `json:"is_dir"`
Size int64 `json:"size,omitempty"`
}
var entries []entry
if globPattern != "" {
// Glob matching within the directory
matches, globErr := filepath.Glob(filepath.Join(absPath, globPattern))
if globErr != nil {
return errResult(fmt.Sprintf("glob: %v", globErr)), nil
}
for _, m := range matches {
fi, statErr := os.Stat(m)
if statErr != nil {
continue
}
relPath, _ := filepath.Rel(absPath, m)
e := entry{
Name: relPath,
Path: m,
IsDir: fi.IsDir(),
}
if !fi.IsDir() {
e.Size = fi.Size()
}
entries = append(entries, e)
}
} else {
// List immediate children
dirEntries, readErr := os.ReadDir(absPath)
if readErr != nil {
return errResult(fmt.Sprintf("read dir: %v", readErr)), nil
}
for _, de := range dirEntries {
// Skip hidden files/dirs
if strings.HasPrefix(de.Name(), ".") {
continue
}
fi, statErr := de.Info()
if statErr != nil {
continue
}
e := entry{
Name: de.Name(),
Path: filepath.Join(absPath, de.Name()),
IsDir: de.IsDir(),
}
if !de.IsDir() {
e.Size = fi.Size()
}
entries = append(entries, e)
}
}
return jsonResult(map[string]any{
"directory": absPath,
"count": len(entries),
"entries": entries,
}), nil
}
// resolveProjectRoot finds the first indexed project root path.
func (s *Server) resolveProjectRoot() (string, error) {
projects, err := s.store.ListProjects()
if err != nil {
return "", err
}
if len(projects) == 0 {
return "", fmt.Errorf("no projects indexed")
}
return projects[0].RootPath, nil
}
internal/tools/index.go
+54
View File
@@ -0,0 +1,54 @@
package tools
import (
"context"
"fmt"
"path/filepath"
"github.com/DeusData/codebase-memory-mcp/internal/pipeline"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleIndexRepository(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
repoPath := getStringArg(args, "repo_path")
if repoPath == "" {
return errResult("repo_path is required"), nil
}
// Resolve to absolute path
absPath, err := filepath.Abs(repoPath)
if err != nil {
return errResult(fmt.Sprintf("invalid path: %v", err)), nil
}
projectName := filepath.Base(absPath)
// Run the indexing pipeline
p := pipeline.New(s.store, absPath)
if err := p.Run(); err != nil {
return errResult(fmt.Sprintf("indexing failed: %v", err)), nil
}
// Gather stats
nodeCount, _ := s.store.CountNodes(projectName)
edgeCount, _ := s.store.CountEdges(projectName)
proj, _ := s.store.GetProject(projectName)
indexedAt := store.Now()
if proj != nil {
indexedAt = proj.IndexedAt
}
return jsonResult(map[string]any{
"project": projectName,
"nodes": nodeCount,
"edges": edgeCount,
"indexed_at": indexedAt,
}), nil
}
internal/tools/projects.go
+65
View File
@@ -0,0 +1,65 @@
package tools
import (
"context"
"fmt"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleListProjects(_ context.Context, _ *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
projects, err := s.store.ListProjects()
if err != nil {
return errResult(fmt.Sprintf("list projects: %v", err)), nil
}
type projectInfo struct {
Name string `json:"name"`
RootPath string `json:"root_path"`
IndexedAt string `json:"indexed_at"`
Nodes int `json:"nodes"`
Edges int `json:"edges"`
}
result := make([]projectInfo, 0, len(projects))
for _, p := range projects {
nc, _ := s.store.CountNodes(p.Name)
ec, _ := s.store.CountEdges(p.Name)
result = append(result, projectInfo{
Name: p.Name,
RootPath: p.RootPath,
IndexedAt: p.IndexedAt,
Nodes: nc,
Edges: ec,
})
}
return jsonResult(result), nil
}
func (s *Server) handleDeleteProject(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
name := getStringArg(args, "project_name")
if name == "" {
return errResult("project_name is required"), nil
}
// Verify project exists
proj, err := s.store.GetProject(name)
if err != nil || proj == nil {
return errResult(fmt.Sprintf("project not found: %s", name)), nil
}
if err := s.store.DeleteProject(name); err != nil {
return errResult(fmt.Sprintf("delete failed: %v", err)), nil
}
return jsonResult(map[string]any{
"deleted": name,
"status": "ok",
}), nil
}
internal/tools/query.go
+33
View File
@@ -0,0 +1,33 @@
package tools
import (
"context"
"fmt"
"github.com/DeusData/codebase-memory-mcp/internal/cypher"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleQueryGraph(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
query := getStringArg(args, "query")
if query == "" {
return errResult("missing required 'query' parameter"), nil
}
exec := &cypher.Executor{Store: s.store}
result, err := exec.Execute(query)
if err != nil {
return errResult(fmt.Sprintf("query error: %v", err)), nil
}
return jsonResult(map[string]any{
"columns": result.Columns,
"rows": result.Rows,
"total": len(result.Rows),
}), nil
}
internal/tools/schema.go
+44
View File
@@ -0,0 +1,44 @@
package tools
import (
"context"
"fmt"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleGetGraphSchema(_ context.Context, _ *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
projects, err := s.store.ListProjects()
if err != nil {
return errResult(fmt.Sprintf("list projects: %v", err)), nil
}
if len(projects) == 0 {
return jsonResult(map[string]any{
"message": "no projects indexed",
"projects": []any{},
}), nil
}
type projectSchema struct {
Project string `json:"project"`
Schema *store.SchemaInfo `json:"schema"`
}
var schemas []projectSchema
for _, p := range projects {
schema, schemaErr := s.store.GetSchema(p.Name)
if schemaErr != nil {
continue
}
schemas = append(schemas, projectSchema{
Project: p.Name,
Schema: schema,
})
}
return jsonResult(map[string]any{
"projects": schemas,
}), nil
}
internal/tools/search.go
+83
View File
@@ -0,0 +1,83 @@
package tools
import (
"context"
"fmt"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleSearchGraph(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
params := store.SearchParams{
Label: getStringArg(args, "label"),
NamePattern: getStringArg(args, "name_pattern"),
FilePattern: getStringArg(args, "file_pattern"),
Relationship: getStringArg(args, "relationship"),
Direction: getStringArg(args, "direction"),
MinDegree: getIntArg(args, "min_degree", -1),
MaxDegree: getIntArg(args, "max_degree", -1),
Limit: getIntArg(args, "limit", 50),
ExcludeEntryPoints: getBoolArg(args, "exclude_entry_points"),
}
projects, err := s.store.ListProjects()
if err != nil {
return errResult(fmt.Sprintf("list projects: %v", err)), nil
}
if len(projects) == 0 {
return jsonResult(map[string]any{
"message": "no projects indexed",
"results": []any{},
}), nil
}
// Search across all projects, collect results
type resultEntry struct {
Project string `json:"project"`
Name string `json:"name"`
QualifiedName string `json:"qualified_name"`
Label string `json:"label"`
FilePath string `json:"file_path"`
StartLine int `json:"start_line"`
EndLine int `json:"end_line"`
InDegree int `json:"in_degree"`
OutDegree int `json:"out_degree"`
ConnectedNames []string `json:"connected_names,omitempty"`
}
var allResults []resultEntry
for _, p := range projects {
params.Project = p.Name
results, searchErr := s.store.Search(params)
if searchErr != nil {
continue
}
for _, r := range results {
entry := resultEntry{
Project: p.Name,
Name: r.Node.Name,
QualifiedName: r.Node.QualifiedName,
Label: r.Node.Label,
FilePath: r.Node.FilePath,
StartLine: r.Node.StartLine,
EndLine: r.Node.EndLine,
InDegree: r.InDegree,
OutDegree: r.OutDegree,
ConnectedNames: r.ConnectedNames,
}
allResults = append(allResults, entry)
}
}
return jsonResult(map[string]any{
"total": len(allResults),
"results": allResults,
}), nil
}
internal/tools/snippet.go
+93
View File
@@ -0,0 +1,93 @@
package tools
import (
"bufio"
"context"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleGetCodeSnippet(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
qn := getStringArg(args, "qualified_name")
if qn == "" {
return errResult("qualified_name is required"), nil
}
// Find the node across all projects
node, project, err := s.findNodeByQNAcrossProjects(qn)
if err != nil {
return errResult(fmt.Sprintf("node not found: %s", qn)), nil
}
if node.FilePath == "" {
return errResult("node has no file path"), nil
}
if node.StartLine == 0 || node.EndLine == 0 {
return errResult("node has no line range"), nil
}
// Resolve file path against the project's root path
proj, projErr := s.store.GetProject(project)
if projErr != nil {
return errResult(fmt.Sprintf("project not found: %s", project)), nil
}
absPath := filepath.Join(proj.RootPath, node.FilePath)
// Read the source file and extract lines
source, readErr := readLines(absPath, node.StartLine, node.EndLine)
if readErr != nil {
return errResult(fmt.Sprintf("read file: %v", readErr)), nil
}
return jsonResult(map[string]any{
"qualified_name": node.QualifiedName,
"name": node.Name,
"label": node.Label,
"file_path": absPath,
"start_line": node.StartLine,
"end_line": node.EndLine,
"source": source,
}), nil
}
// readLines reads specific lines from a file, returning them with line numbers.
func readLines(path string, startLine, endLine int) (string, error) {
f, err := os.Open(path)
if err != nil {
return "", fmt.Errorf("open: %w", err)
}
defer f.Close()
var sb strings.Builder
scanner := bufio.NewScanner(f)
lineNum := 0
for scanner.Scan() {
lineNum++
if lineNum > endLine {
break
}
if lineNum >= startLine {
fmt.Fprintf(&sb, "%4d | %s\n", lineNum, scanner.Text())
}
}
if err := scanner.Err(); err != nil {
return "", fmt.Errorf("scan: %w", err)
}
if sb.Len() == 0 {
return "", fmt.Errorf("no lines found in range %d-%d (file has %d lines)", startLine, endLine, lineNum)
}
return sb.String(), nil
}
internal/tools/tools.go
+369
View File
@@ -0,0 +1,369 @@
package tools
import (
"encoding/json"
"fmt"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
// Server wraps the MCP server with tool handlers.
type Server struct {
mcp *mcp.Server
store *store.Store
}
// NewServer creates a new MCP server with all tools registered.
func NewServer(s *store.Store) *Server {
srv := &Server{
store: s,
mcp: mcp.NewServer(
&mcp.Implementation{
Name: "codebase-memory-mcp",
Version: "0.1.0",
},
nil,
),
}
srv.registerTools()
return srv
}
// MCPServer returns the underlying MCP server.
func (s *Server) MCPServer() *mcp.Server {
return s.mcp
}
func (s *Server) registerTools() {
// 1. index_repository
s.mcp.AddTool(&mcp.Tool{
Name: "index_repository",
Description: "Index a repository into the code graph. Parses source files, extracts functions/classes/modules, resolves call relationships, and stores the graph for querying. Supports incremental reindex via content hashing.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"repo_path": {
"type": "string",
"description": "Absolute path to the repository to index. If omitted, uses the configured project root."
}
}
}`),
}, s.handleIndexRepository)
// 2. trace_call_path
s.mcp.AddTool(&mcp.Tool{
Name: "trace_call_path",
Description: "Trace call paths from/to a function using BFS traversal. Returns the root function with signature and module constants, hop-by-hop callees/callers, and call edges with type (CALLS or HTTP_CALLS). Use for understanding call chains and data flow.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"function_name": {
"type": "string",
"description": "Name of the function to trace (e.g. 'ProcessOrder')"
},
"depth": {
"type": "integer",
"description": "Maximum BFS depth (1-5, default 3)"
},
"direction": {
"type": "string",
"description": "Traversal direction: 'outbound' (what it calls), 'inbound' (what calls it), or 'both'",
"enum": ["outbound", "inbound", "both"]
}
},
"required": ["function_name"]
}`),
}, s.handleTraceCallPath)
// 3. get_graph_schema
s.mcp.AddTool(&mcp.Tool{
Name: "get_graph_schema",
Description: "Return the schema of the indexed code graph: node label counts, edge type counts, relationship patterns (e.g. Function-CALLS->Function), and sample function/class names. Use to understand what's in the graph before querying.",
InputSchema: json.RawMessage(`{"type": "object"}`),
}, s.handleGetGraphSchema)
// 4. get_code_snippet
s.mcp.AddTool(&mcp.Tool{
Name: "get_code_snippet",
Description: "Retrieve source code for a function/class by qualified name. Reads directly from disk using the stored file path and line range. Returns the source code with line numbers.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"qualified_name": {
"type": "string",
"description": "Fully qualified name of the node (e.g. 'myproject.cmd.server.main.HandleRequest')"
}
},
"required": ["qualified_name"]
}`),
}, s.handleGetCodeSnippet)
// 5. search_graph
s.mcp.AddTool(&mcp.Tool{
Name: "search_graph",
Description: "Search the code graph with structured filters. Replaces raw Cypher queries with safe, parameterized search. Supports filtering by node label, name pattern (regex), file pattern (glob), relationship type, direction, and degree (fan-in/fan-out). Returns matching nodes with properties and edge counts.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"label": {
"type": "string",
"description": "Node label filter: Function, Class, Module, Method, Interface, Enum, Type, File, Package, Folder"
},
"name_pattern": {
"type": "string",
"description": "Regex pattern for node name (e.g. '.*Handler', 'Send.*')"
},
"file_pattern": {
"type": "string",
"description": "Glob pattern for file path (e.g. '**/order-service/**')"
},
"relationship": {
"type": "string",
"description": "Filter by relationship type: CALLS, HTTP_CALLS, IMPORTS, DEFINES, etc."
},
"direction": {
"type": "string",
"description": "Edge direction for degree filters: 'inbound', 'outbound', or 'any'",
"enum": ["inbound", "outbound", "any"]
},
"min_degree": {
"type": "integer",
"description": "Minimum edge count (e.g. 10 for high fan-out functions)"
},
"max_degree": {
"type": "integer",
"description": "Maximum edge count (e.g. 0 for dead code detection)"
},
"exclude_entry_points": {
"type": "boolean",
"description": "Exclude entry points (route handlers, main(), framework-registered functions) from results. Use with max_degree=0 for accurate dead code detection."
},
"limit": {
"type": "integer",
"description": "Max results (default 50, max 200)"
}
}
}`),
}, s.handleSearchGraph)
// 6. list_projects
s.mcp.AddTool(&mcp.Tool{
Name: "list_projects",
Description: "List all indexed projects with their indexed_at timestamp, root path, and node/edge counts.",
InputSchema: json.RawMessage(`{"type": "object"}`),
}, s.handleListProjects)
// 7. delete_project
s.mcp.AddTool(&mcp.Tool{
Name: "delete_project",
Description: "Delete an indexed project and all its graph data (nodes, edges, file hashes). This action is irreversible.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"project_name": {
"type": "string",
"description": "Name of the project to delete"
}
},
"required": ["project_name"]
}`),
}, s.handleDeleteProject)
// 8. read_file
s.mcp.AddTool(&mcp.Tool{
Name: "read_file",
Description: "Read any file from the indexed project. Supports line range selection for large files. Use for reading config files (Dockerfile, go.mod, requirements.txt), source code, or any text file.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "File path (absolute, or relative to project root)"
},
"start_line": {
"type": "integer",
"description": "Start reading from this line (1-based, optional)"
},
"end_line": {
"type": "integer",
"description": "Stop reading at this line (inclusive, optional)"
}
},
"required": ["path"]
}`),
}, s.handleReadFile)
// 9. list_directory
s.mcp.AddTool(&mcp.Tool{
Name: "list_directory",
Description: "List files and subdirectories in a directory. Supports glob patterns for filtering. Use for exploring project structure.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"path": {
"type": "string",
"description": "Directory path (absolute, or relative to project root). Empty for project root."
},
"pattern": {
"type": "string",
"description": "Glob pattern to filter entries (e.g. '*.go', '*.py')"
}
}
}`),
}, s.handleListDirectory)
// 10. search_code
s.mcp.AddTool(&mcp.Tool{
Name: "search_code",
Description: "Search for text patterns within source code files. Like grep/ripgrep but scoped to indexed project files. Returns matching lines with file paths and line numbers. Use for finding string literals, error messages, TODO comments, or any text within function bodies.",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"pattern": {
"type": "string",
"description": "Text to search for (literal string, or regex if regex=true)"
},
"file_pattern": {
"type": "string",
"description": "Glob pattern to filter files (e.g. '*.go', '*.py')"
},
"regex": {
"type": "boolean",
"description": "Treat pattern as a regular expression (default: false)"
},
"max_results": {
"type": "integer",
"description": "Maximum number of matches to return (default 50, max 200)"
}
},
"required": ["pattern"]
}`),
}, s.handleSearchCode)
// 11. query_graph
s.mcp.AddTool(&mcp.Tool{
Name: "query_graph",
Description: "Execute a Cypher-like graph query. Supports MATCH patterns with node labels, relationship types, variable-length paths, WHERE filters (=, =~, CONTAINS, STARTS WITH, >, <), and RETURN with COUNT/ORDER BY/LIMIT/DISTINCT. Read-only (no CREATE/DELETE).",
InputSchema: json.RawMessage(`{
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Cypher query, e.g. MATCH (f:Function)-[:CALLS]->(g:Function) WHERE f.name = 'main' RETURN g.name, g.qualified_name LIMIT 20"
}
},
"required": ["query"]
}`),
}, s.handleQueryGraph)
}
// jsonResult marshals data to JSON and returns as tool result.
func jsonResult(data any) *mcp.CallToolResult {
b, err := json.MarshalIndent(data, "", " ")
if err != nil {
return errResult("json marshal err=" + err.Error())
}
return &mcp.CallToolResult{
Content: []mcp.Content{
&mcp.TextContent{Text: string(b)},
},
}
}
// errResult returns a tool result indicating an error.
func errResult(msg string) *mcp.CallToolResult {
return &mcp.CallToolResult{
Content: []mcp.Content{
&mcp.TextContent{Text: msg},
},
IsError: true,
}
}
// parseArgs unmarshals the raw JSON arguments into a map.
func parseArgs(req *mcp.CallToolRequest) (map[string]any, error) {
if req.Params.Arguments == nil || len(req.Params.Arguments) == 0 {
return map[string]any{}, nil
}
var m map[string]any
if err := json.Unmarshal(req.Params.Arguments, &m); err != nil {
return nil, fmt.Errorf("invalid arguments: %w", err)
}
return m, nil
}
// getStringArg extracts a string argument from parsed args.
func getStringArg(args map[string]any, key string) string {
v, ok := args[key]
if !ok {
return ""
}
s, ok := v.(string)
if !ok {
return ""
}
return s
}
// getIntArg extracts an integer argument with a default value.
func getIntArg(args map[string]any, key string, defaultVal int) int {
v, ok := args[key]
if !ok {
return defaultVal
}
f, ok := v.(float64) // JSON numbers decode as float64
if !ok {
return defaultVal
}
return int(f)
}
// getBoolArg extracts a boolean argument from parsed args.
func getBoolArg(args map[string]any, key string) bool {
v, ok := args[key]
if !ok {
return false
}
b, ok := v.(bool)
if !ok {
return false
}
return b
}
// findNodeAcrossProjects searches all projects for a node by simple name.
func (s *Server) findNodeAcrossProjects(name string) (*store.Node, string, error) {
projects, err := s.store.ListProjects()
if err != nil {
return nil, "", fmt.Errorf("list projects: %w", err)
}
for _, p := range projects {
nodes, findErr := s.store.FindNodesByName(p.Name, name)
if findErr != nil {
continue
}
if len(nodes) > 0 {
return nodes[0], p.Name, nil
}
}
return nil, "", fmt.Errorf("node not found: %s", name)
}
// findNodeByQNAcrossProjects searches all projects for a node by qualified name.
func (s *Server) findNodeByQNAcrossProjects(qn string) (*store.Node, string, error) {
projects, err := s.store.ListProjects()
if err != nil {
return nil, "", fmt.Errorf("list projects: %w", err)
}
for _, p := range projects {
node, findErr := s.store.FindNodeByQN(p.Name, qn)
if findErr != nil {
continue
}
if node != nil {
return node, p.Name, nil
}
}
return nil, "", fmt.Errorf("node not found: %s", qn)
}
internal/tools/trace.go
+176
View File
@@ -0,0 +1,176 @@
package tools
import (
"context"
"fmt"
"github.com/DeusData/codebase-memory-mcp/internal/store"
"github.com/modelcontextprotocol/go-sdk/mcp"
)
func (s *Server) handleTraceCallPath(_ context.Context, req *mcp.CallToolRequest) (*mcp.CallToolResult, error) {
args, err := parseArgs(req)
if err != nil {
return errResult(err.Error()), nil
}
funcName := getStringArg(args, "function_name")
if funcName == "" {
return errResult("function_name is required"), nil
}
depth := getIntArg(args, "depth", 3)
if depth < 1 {
depth = 1
}
if depth > 5 {
depth = 5
}
direction := getStringArg(args, "direction")
if direction == "" {
direction = "outbound"
}
// Find the function node across all projects
rootNode, project, err := s.findNodeAcrossProjects(funcName)
if err != nil {
return errResult(fmt.Sprintf("function not found: %s", funcName)), nil
}
edgeTypes := []string{"CALLS", "HTTP_CALLS"}
// Build root info
root := buildNodeInfo(rootNode)
// Get module info (constants) by finding the module that defines this function
moduleInfo := s.getModuleInfo(rootNode, project)
// Run BFS
var allVisited []*store.NodeHop
var allEdges []store.EdgeInfo
if direction == "both" {
// Run outbound + inbound separately, merge
outResult, outErr := s.store.BFS(rootNode.ID, "outbound", edgeTypes, depth, 200)
if outErr == nil {
allVisited = append(allVisited, outResult.Visited...)
allEdges = append(allEdges, outResult.Edges...)
}
inResult, inErr := s.store.BFS(rootNode.ID, "inbound", edgeTypes, depth, 200)
if inErr == nil {
allVisited = append(allVisited, inResult.Visited...)
allEdges = append(allEdges, inResult.Edges...)
}
} else {
result, bfsErr := s.store.BFS(rootNode.ID, direction, edgeTypes, depth, 200)
if bfsErr != nil {
return errResult(fmt.Sprintf("bfs err: %v", bfsErr)), nil
}
allVisited = result.Visited
allEdges = result.Edges
}
// Group visited nodes by hop
hops := buildHops(allVisited)
// Build edge list
edges := buildEdgeList(allEdges)
// Get indexed_at from project
proj, _ := s.store.GetProject(project)
indexedAt := ""
if proj != nil {
indexedAt = proj.IndexedAt
}
return jsonResult(map[string]any{
"root": root,
"module": moduleInfo,
"hops": hops,
"edges": edges,
"indexed_at": indexedAt,
"total_results": len(allVisited),
}), nil
}
func buildNodeInfo(n *store.Node) map[string]any {
info := map[string]any{
"name": n.Name,
"qualified_name": n.QualifiedName,
"label": n.Label,
"file_path": n.FilePath,
"start_line": n.StartLine,
"end_line": n.EndLine,
}
if sig, ok := n.Properties["signature"]; ok {
info["signature"] = sig
}
if rt, ok := n.Properties["return_type"]; ok {
info["return_type"] = rt
}
return info
}
func (s *Server) getModuleInfo(funcNode *store.Node, project string) map[string]any {
if funcNode.FilePath == "" {
return map[string]any{}
}
// Find module nodes in the same file
modules, err := s.store.FindNodesByLabel(project, "Module")
if err != nil {
return map[string]any{}
}
for _, m := range modules {
if m.FilePath == funcNode.FilePath {
info := map[string]any{"name": m.Name}
if constants, ok := m.Properties["constants"]; ok {
info["constants"] = constants
}
return info
}
}
return map[string]any{}
}
type hopEntry struct {
Hop int `json:"hop"`
Nodes []map[string]any `json:"nodes"`
}
func buildHops(visited []*store.NodeHop) []hopEntry {
hopMap := map[int][]map[string]any{}
for _, nh := range visited {
info := map[string]any{
"name": nh.Node.Name,
"qualified_name": nh.Node.QualifiedName,
"label": nh.Node.Label,
}
if sig, ok := nh.Node.Properties["signature"]; ok {
info["signature"] = sig
}
hopMap[nh.Hop] = append(hopMap[nh.Hop], info)
}
var hops []hopEntry
for h := 1; h <= len(hopMap); h++ {
if nodes, ok := hopMap[h]; ok {
hops = append(hops, hopEntry{Hop: h, Nodes: nodes})
}
}
return hops
}
func buildEdgeList(edges []store.EdgeInfo) []map[string]any {
result := make([]map[string]any, 0, len(edges))
for _, e := range edges {
result = append(result, map[string]any{
"from": e.FromName,
"to": e.ToName,
"type": e.Type,
})
}
return result
}