338 Commits

Author SHA1 Message Date
Martin Vogel c00723e5c8 Require project param on all MCP tool calls
Make project a required parameter for all query tools (search_graph,
query_graph, trace_call_path, get_code_snippet, get_graph_schema,
get_architecture, search_code, index_status, detect_changes,
manage_adr, ingest_traces). Removes implicit fallback to session
project or last-opened store.

When project is missing or not found, return error with list of
available indexed projects so agents can self-correct.

Rename delete_project param from project_name to project for
consistency. Fix smoke test trace_call_path depth param name
(max_depth -> depth).
2026-03-24 13:38:25 +01:00
Martin Vogel f24475ca4a Add simulated binary replacement to smoke tests (Phase 6e)
Tests the update command's replacement flow without network calls:
copy binary as "downloaded" version, unlink-then-replace the
"installed" version, verify replaced binary runs --version.
Also tests read-only binary replacement edge case (#114).
2026-03-23 22:02:51 +01:00
Martin Vogel 20814e8b27 Add smoke tests to Docker test infra + CI pipeline
Expand smoke-test.sh with Phase 5 (MCP stdio transport), Phase 6
(CLI subcommands: install/uninstall/update --dry-run), and Phase 7
(MCP advanced tool calls: search_code v2, get_code_snippet).

Add smoke/smoke-amd64 services to Docker compose that build then
run all 7 smoke test phases. Include in run.sh full/all flows.

Add python3-minimal to Dockerfile for smoke test JSON parsing.
Fix Phase 4a shutdown test to use portable background+kill pattern
instead of `timeout` (not available on macOS).

Add --dry-run, --standard, --ui flags to update command. Fix
clang-tidy readability-implicit-bool-conversion in dry_run ternary.
2026-03-23 21:56:33 +01:00
Martin Vogel 85da0908f6 Add MCP stdio transport smoke tests (Phase 5)
Tests the actual agent handshake path that Claude Code, OpenCode, etc. use:
- 5a-d: initialize → notifications/initialized → tools/list via bare JSONL
- 5e: Full tool call round-trip (index + search) via JSON-RPC
- 5f: Content-Length framing (OpenCode compatibility)

Uses portable background process + kill instead of timeout command
(not available on all macOS configurations).
2026-03-23 18:33:27 +01:00
Martin Vogel de322d98a3 Merge branch 'main' into fix/mcp-stdio-buffering 2026-03-23 14:20:15 +01:00
Martin Vogel c38853f999 Remove Go-era cruft, update THIRD_PARTY.md for pure C project
- Delete cmd/codebase-memory-mcp/assets/ — skills and codex instructions
  are embedded as C string literals in src/cli/cli.c since v0.5.0
- Move BENCHMARK.md to docs/ to reduce root clutter
- Fix setup scripts: replace stale go build with scripts/build.sh
- THIRD_PARTY.md: replace Go module dependencies with vendored C
  libraries (sqlite3, mimalloc, mongoose, yyjson, xxhash, tre),
  fix CGo reference to static compilation
2026-03-22 01:09:12 +01:00
Martin Vogel 0888726073 Add security defense tests + eliminate system() command injection
TDD: 31 security tests covering shell injection prevention, SQLite
authorizer (ATTACH/DETACH blocked), SQL injection via Cypher, path
containment, and shell-free subprocess execution.

- Add cbm_exec_no_shell() in compat_fs: fork+execvp (POSIX), _spawnvp
  (Windows) — executes commands without shell interpretation
- Replace system() with cbm_exec_no_shell() for unzip extraction and
  version verification in update command — eliminates CodeQL
  command-line-injection alerts
- CodeQL: switch to build-mode manual for 100% source file coverage
- CodeQL gate: fix race condition between scan completion and alert API
  propagation (60s settle + double-check polling)
- Dismiss TOCTOU in pass_envscan.c (benign read-only directory walk)
2026-03-21 23:02:34 +01:00
Martin Vogel aa2b60be7d Fix 29 CodeQL alerts: command injection, snprintf overflow, TOCTOU races
- Command injection (CRITICAL): validate shell args before system() in
  update command's unzip and version-check calls
- TOCTOU cli.c: use open(O_CREAT, 0755) + fdopen() to set permissions
  atomically instead of fopen() + chmod() after close
- TOCTOU pass_envscan.c: open file first, then fstat() on fd to check
  size, eliminating stat-then-open race window
- Overflowing snprintf (11 locations): clamp offset after each append
  to prevent unsigned underflow on truncation in cypher.c, store.c,
  http_server.c, test_c_lsp.c
- Add CBM_SNPRINTF_APPEND macro in str_util.h for future safe appends
- CodeQL: remove pull_request trigger (only scan push to main)
- CodeQL gate: increase timeout from 30 to 45 minutes
- Add fuzz testing script (random JSON-RPC + Cypher mutations)
- 12 Scorecard governance alerts dismissed (not code vulnerabilities)
2026-03-21 22:18:31 +01:00
Shane McCarron 5ca5f3291f fix(mcp): address QA round 1 findings
- Use O_NONBLOCK + clearerr() in Phase 2 fgetc probe to preserve the
  60s idle eviction timeout when both kernel fd and FILE* buffer are
  empty (fgetc on a blocking fd would otherwise block indefinitely,
  bypassing Phase 3 poll timeout and preventing cbm_mcp_server_evict_idle)
- Add #include <fcntl.h> for fcntl()/O_NONBLOCK
- Fix comment: "two-phase" → "three-phase" (implementation has 3 phases)
- Improve Python integration test: verify id:1 (initialize) and id:2
  (tools/list) response IDs are both present, not just "tools" substring

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-21 12:37:28 -05:00
Shane McCarron 16552befdd test(mcp): add rapid-init integration test for poll/getline buffering fix 2026-03-21 12:03:52 -05:00
Martin Vogel dd30132e2d Pin Actions to SHA + enforce via security audit + Dependabot
- All GitHub Actions pinned to immutable commit SHAs (prevents
  tag-poisoning attacks like tj-actions/changed-files incident)
- Security audit (Layer 1) now blocks unpinned Actions in CI
- Dependabot configured to auto-propose SHA updates weekly
- Pre-commit hooks tracked in scripts/hooks/ for contributors
- Time-bomb detection + MCP file read audit added to Layer 1
2026-03-21 16:57:17 +01:00
Martin Vogel c371e9f07d Add time-bomb + MCP file read audit, tracked pre-commit hooks
Security audit (Layer 1) new checks:
- Time-bomb scan: flags time()/sleep()/clock() near dangerous calls
- MCP file read audit: tracks fopen/fread count in mcp.c against
  expected max (detects data exfiltration through tool responses)

Pre-commit hooks now tracked in scripts/hooks/:
- Contributors activate with: git config core.hooksPath scripts/hooks
- Runs: lint → security audit → build + test
- setup.sh already configures this automatically
2026-03-21 16:43:48 +01:00
Martin Vogel f7315b9bc5 Robust UI audit: skip inline URL scan on dist/, keep structural checks
Minified JS in dist/ contains hundreds of string-constant URLs from
bundled npm libraries (React error pages, W3C namespace URIs, CDN
references, OSS credits) that are never fetched at runtime. Scanning
them produced an endless stream of false positives requiring per-URL
allowlisting.

Fix: split the A1 URL check by directory:
- src/ (our code): strict — any external URL is blocked
- dist/ (npm build output): skip inline URL scan entirely

Structural checks (A2-A6) still apply to dist/:
- External <script>/<link> loads in HTML
- Tracking/analytics identifiers
- Iframes, eval/Function, external WebSockets
2026-03-20 20:56:52 +01:00
Martin Vogel a9382ce110 Fix UI audit: allow truncated CDN URL in dist/ scan 2026-03-20 20:52:58 +01:00
Martin Vogel 93d332d1e0 Fix security audit false positives in CI
- UI audit (L6): allowlist bundled framework URLs in dist/ (React error
  URLs, W3C namespace URIs, Three.js credits, Google Fonts, Tailwind).
  These are embedded by npm deps during Vite build, not our code.
- Binary strings (L2): skip URLs shorter than 15 chars — Windows binary
  has byte sequences that strings(1) interprets as "https://H9" etc.
- Allow Google Fonts <link> in HTML (loaded by index.html for Inter/
  JetBrains Mono fonts).
2026-03-20 18:47:55 +01:00
Martin Vogel cd1417427c Add 8-layer security test suite + hardening
Code-level defenses:
- cbm_validate_shell_arg(): reject shell metacharacters before popen/system
- SQLite authorizer: block ATTACH/DETACH at engine level
- CORS localhost-only origin reflection (replaces wildcard *)
- Path containment: realpath() check in get_code_snippet
- process-kill restricted to server-spawned PIDs
- SHA256 checksum verification in update command

Security audit scripts (8 layers):
- L1: Static allow-list for dangerous calls + URLs
- L2: Binary string audit (URLs, payloads, credentials)
- L3: Network egress monitoring via strace (Linux)
- L4: Install output path + content validation
- L5: Smoke test hardening (clean shutdown, residual procs)
- L6: Graph UI audit (external domains, CORS, binding)
- L7: MCP robustness (23 adversarial JSON-RPC payloads)
- L8: Vendored integrity (checksums + dangerous call scan)

CI: parallel security-static job (no build needed), binary
layers in smoke jobs per-platform. Cleanup of test fixture
dirs in clean.sh + .gitignore.
2026-03-20 18:19:15 +01:00
Martin Vogel 0f3c772014 Fix Windows smoke: convert MSYS2 POSIX paths for native binary
MSYS2 mktemp returns /tmp/tmp.XXX but the Windows binary can't resolve
POSIX paths. Use cygpath -m to convert to C:/msys64/tmp/tmp.XXX which
works in both MSYS2 shell and Windows file APIs.
2026-03-19 12:29:54 +01:00
Martin Vogel a71a65f68e Fix smoke tests: CLANG64 runtime for Windows, debug stderr on error
1. smoke-windows: Use CLANG64 (not UCRT64) so runtime matches the
   build environment. Binary built with CLANG64 needs CLANG64 DLLs.
2. smoke-unix: Add fail-fast: false so one platform failure doesn't
   cancel all 7 other smoke jobs.
3. smoke-test.sh: Capture and display stderr on indexing failure to
   help debug smoke-unix errors.
4. release.yml: Same CLANG64 smoke fix.
2026-03-19 11:31:12 +01:00
Martin Vogel 0e9a47177b Fix Windows build: add zip pkg, portable embed (od replaces xxd)
1. Add zip to MSYS2 build packages (both dry-run and release).
2. embed-frontend.sh: replace xxd with od+sed for hex dump — xxd
   requires vim which isn't in MSYS2 CLANG64 by default.
3. embed-frontend.sh: flip IS_MACOS→IS_LINUX — Linux gets ld -r -b
   binary (ELF-only), everything else (macOS, Windows) gets the
   portable C-array approach via od.
4. release.yml: same Windows archive fix (msys2 shell + .exe detect).
2026-03-19 10:59:14 +01:00
Martin Vogel 89dd3bb7c6 Add skip_tests/skip_builds flags, fix build-stage failures
1. dry-run.yml: Add skip_tests and skip_builds workflow_dispatch inputs
   for faster iteration. Build jobs use always() to run even when tests
   are skipped.

2. Windows archive: Switch from PowerShell Copy-Item to MSYS2 shell
   with .exe extension detection. MSYS2 Clang produces .exe suffix
   that Copy-Item didn't account for.

3. Linux embed: Use absolute path for ld output when cd'd into dist
   directory. Relative path build/c/embedded/*.o was invalid from
   inside graph-ui/dist/.

4. release.yml: Same Windows archive fix as dry-run.
2026-03-19 10:43:32 +01:00
Martin Vogel 0f83f08653 Auto-detect WIN32_LIBS in Makefile, remove shell quoting entirely 2026-03-19 00:00:19 +01:00
Martin Vogel b5f14f115d Fix WIN32_LIBS quoting via env vars, add skip_lint to dry-run
- Use env: block instead of shell quoting for WIN32_LIBS in CI
- Forward WIN32_LIBS from environment in test.sh and build.sh
- Add skip_lint input to dry-run workflow for faster iteration
2026-03-18 23:35:35 +01:00
Martin Vogel 30505ec596 Switch Windows CI to CLANG64 for ASan/UBSan support
MinGW GCC does not ship ASan runtime on Windows. MSYS2 CLANG64
provides full sanitizer support via compiler-rt. All platforms
now run tests with ASan + UBSan — no exceptions.
2026-03-18 21:21:30 +01:00
Martin Vogel c8bdf924e5 Fix production build errors, add build step to Docker test infra
- Fix GCC -O2 warnings: calloc for cf (maybe-uninitialized),
  suppress stringop-truncation/alloc-size-larger-than (false positives),
  guard louvain against negative node_count
- Default compiler: cc on macOS, gcc on Linux/Windows (fixes gcc-14
  not found on MSYS2)
- Docker test infra now runs BOTH test (ASan) and build (-O2 -Werror)
2026-03-18 17:15:46 +01:00
Martin Vogel 8f225a37dd Use cc (Apple Clang) on macOS, gcc on Linux — unified CI/local parity
- Add scripts/env.sh: detects true hardware arch (even under Rosetta),
  verifies compiler supports target arch, adds arch prefix on macOS
- macOS uses cc/c++ everywhere (tests + builds) — ASan works natively
- Linux uses gcc/g++ everywhere — full sanitizer support
- Remove broken CC ?= gcc-14 from Makefile (GNU Make built-in overrides it)
- Scripts accept --arch flag and CC=/CXX= overrides
2026-03-18 13:08:33 +01:00
Martin Vogel fd1640ec29 Add build scripts, fix GCC warnings, unify CI workflows
- Add scripts/clean.sh, test.sh, build.sh, lint.sh as single source
  of truth for local and CI builds (always clean, always parallel)
- Fix -Wmissing-braces: replace {{0}} with memset for struct arrays
- Fix -Wrestrict: use intermediate buffer for same-struct snprintf
- Update dry-run.yml and release.yml to use the new scripts
2026-03-18 12:31:53 +01:00
Martin Vogel ce96ecc085 Add comprehensive smoke test: version + index + search + trace + folder integrity 2026-03-17 22:25:26 +01:00
Martin Vogel 18fa9979ee Replace vmem with mimalloc global allocator, add extraction-phase prescan, fix __init__.py QN collision
Memory management:
- Vendor mimalloc v2.1.9 as global allocator (MI_OVERRIDE=1 in prod)
- New mem.h/mem.c: RSS-based budget tracking via mi_process_info()
- Remove vmem.c/vmem.h (mmap-based budget tracking)
- Remove slab tier2 bump allocator (~300 LOC); >64B goes to mimalloc
- Slab tier1 pages from malloc (= mimalloc) instead of vmem
- Arena blocks from malloc instead of vmem
- Budget raised from 35% to 50% RAM (no more untracked C++ heap)

Extraction-phase prescan (eliminates disk re-reads):
- HTTP call sites: keyword check + URL extraction during extraction
- HTTP routes: decorator + source-based extraction during extraction
- Config file refs: regex scan during extraction
- httplinks: 41.8s → 13ms on Linux kernel (3,212x faster)
- configlink: 41.4s → 0.8s on Linux kernel (54x faster)
- Linux kernel fast-mode total: 2m38s → 1m18s

Bug fixes:
- __init__.py Module QN no longer collides with Folder QN
- index.ts same fix for JS/TS packages
- 13 regression tests for QN collision at FQN + extraction layers
- search_graph/search_code default limit raised from 10 to 500k
- Resolve all clang-tidy, cppcheck, and clang-format warnings

Repo cleanup:
- tree-sitter-form, tree-sitter-magma moved to tools/
- .gitignore: build/, node_modules/, graph-ui/dist/, TEST_PLAN.md
2026-03-17 22:05:57 +01:00
Martin Vogel 6f0910278b Remove all Go code — project is now pure C
Delete Go MCP server, CGo bridges, Go pipeline, store, tools, watcher,
cypher engine, language specs, and all Go tests. Replace Go pre-commit
hook with C-only linting (clang-tidy, cppcheck, clang-format, tests).
Tree-sitter extraction engine (internal/cbm/*.c) retained for C pipeline.
2026-03-15 20:32:19 +01:00
Martin Vogel 94a9a92f83 Add C-native pipeline with blocking clang-tidy, cppcheck, clang-format linting
Complete C implementation of the indexing pipeline (src/), parallel worker
pool, graph buffer with merge support, SQLite writer, and 1893-test suite.

Linter setup: zero warnings from clang-tidy (all checks enabled, no NOLINT
suppressions), cppcheck, and clang-format. All issues fixed at source —
proper headers for external linkage, GROW_ARRAY macro restructured to
eliminate type parameter, null-deref paths guarded, named intermediates
for suspicious-argument checks.
2026-03-15 19:26:12 +01:00
Martin Vogel bd6bbc2ab1 Add Go LSP hybrid type resolver, config store, and supporting infrastructure
- Go LSP cross-file type resolution engine (internal/cbm/lsp/) with C-based
  type registry, scope tracking, and Go stdlib data (30K+ definitions)
- lsp_bridge.go: CrossFileDef struct and RunGoLSPCrossFile CGo bridge
- config.go: persistent ConfigStore (SQLite) for auto_index, mem_limit settings
- CLI config subcommand for get/set/list/delete operations
- Store: edge batch upsert, project config columns, goleak test harness
- Pipeline: prefetch parallelism, resolver improvements, memleak tests
- CI: dry-run workflow improvements, vendor-grammar.sh enhancements
2026-03-12 13:26:13 +01:00
Chitral Verma d248e7348f Fix binary move command in setup script (#16) 2026-03-06 21:19:38 +01:00
Martin Vogel 5b25f885d7 Add smoke-test.sh for CI binary validation 2026-03-06 18:27:51 +01:00
Martin Vogel 05a35ac9b1 v0.5.1: Fix C/C++/CUDA/GLSL function extraction, add config linker and tests
- Fix extract_defs.c declarator chain traversal for C-family function_definition
  nodes (off-by-one in field name length, missing depth traversal)
- Add resolve_func_name() support for CommonLisp defun, Makefile rule, VimScript,
  Julia, Elm value_declaration, Groovy field_function methods
- Add config linker pipeline: 3 strategies linking config keys to code symbols,
  dependency manifest entries to imports, and config file path references
- Add regression tests (~150 cases across 50+ languages), language failure tests,
  and config extraction tests
- Refactor discover.go, configures.go, configlink_strategies.go to reduce
  cognitive complexity (extract helpers for gocognit compliance)
- Fix Louvain community detection: use map lookup instead of O(N) scan
2026-03-06 17:50:18 +01:00
Martin Vogel 59d40d4035 v0.5: CBM tree-sitter engine, 59 vendored language grammars
- Replace Go-binding tree-sitter with vendored C grammars compiled via
  CGo (internal/cbm/); eliminates all Go module/linker fragility
- Add 32 new languages: Clojure, CMake, COBOL, Common Lisp, CUDA, Elm,
  Emacs Lisp, Fortran, F#, GLSL, GraphQL, INI, JSON, Julia, Makefile,
  Markdown, Meson, Nix, Protobuf, Svelte, Verilog, Vim Script, Vue, XML
  (plus C, C#, Erlang, SQL, YAML quality improvements)
- Adaptive worker concurrency and mmap prefetch for faster indexing
- Graph buffer: batched SQLite writes for 3-5x throughput improvement
- Fix cancellation test: use 50ms timeout (pipeline now indexes 212-file
  Erlang repo in ~400ms, well under the old 2s deadline)
- Fix isTestFunction: add "Test" prefix for C#, "test" prefix for Scala
2026-03-05 22:39:01 +01:00
Martin Vogel 02bc3b7352 Release v0.3.0: 25 languages, install/update CLI, skills, benchmark
Language expansion (13 → 25):
  Add Ruby, C, Bash, Zig, Elixir, Haskell, OCaml, HTML, CSS, YAML,
  TOML, HCL with tree-sitter grammars and language specs.
  Restore Erlang and SQL with extraction fixes.

CLI install/update/uninstall commands:
  Auto-detect Claude Code and Codex CLI, register MCP server,
  install task-specific skills, self-update with SHA-256 verification.

Embedded skills (4):
  exploring, tracing, quality, reference — auto-trigger in Claude Code
  for graph-first code discovery.

Pipeline improvements:
  Docstring extraction, enrichment pass for params/returns/complexity,
  graceful context cancellation, improved test detection.

35-language benchmark (BENCHMARK.md):
  12 questions × 35 repos, 91.8% overall score, Linux kernel stress
  test (20K nodes, zero timeouts). Replaces old benchmark artifacts.
2026-03-01 18:01:52 +01:00
Martin Vogel b86738acfb Fix setup.sh: Apple Silicon detection under Rosetta and tmpdir cleanup
- detect_platform() now checks hw.optional.arm64 sysctl on macOS to
  pick the arm64 binary even when bash runs under Rosetta (uname -m
  reports x86_64 in that case)
- Move tmpdir to a global CLEANUP_DIR so the EXIT trap does not hit
  an unbound variable error from set -u after the function returns
2026-02-25 10:19:32 +01:00
Martin Vogel 3c6ad62ed0 Add cross-platform setup scripts and CI release workflow
- GitHub Actions workflow builds native binaries for 5 platforms:
  darwin-arm64, darwin-amd64, linux-amd64, linux-arm64, windows-amd64
- Windows build uses MSYS2 UCRT64 toolchain for CGO compatibility
- setup.sh: macOS/Linux installer (binary download or --from-source)
- setup-windows.ps1: Windows installer (native .exe or -FromSource via WSL)
- Both scripts offer interactive Claude Code MCP auto-configuration
- Release includes checksums.txt with SHA-256 hashes
- PS5.1-compatible JSON handling (no -AsHashtable dependency)
2026-02-25 10:07:45 +01:00