rust-kernel
6 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Colby McHenry
|
c5eebe6beb |
feat(kernel): R1 scaffold — napi-rs extraction kernel, buffer contract, routing + fallback, grammar-parity CI
Phase 0 of the Rust extraction-kernel migration (docs/design/ rust-kernel-migration-plan.md, now checked in with §3a recording the shipped state): - codegraph-kernel/ napi-rs crate: extractFile(path, content, language) → five flat buffers (meta/nodes/edges/refs/arena), one JS boundary crossing per file. Node ids computed Rust-side, byte-identical to generateNodeId (pinned by test vector). Reserved per-node metrics slot for the Arc 3.2 code-metrics work. - Generic .scm-driven emitter (@def.<kind>/@name/@ref.<kind> captures, byte-range scope stack → ::-joined qualified names, contains edges, refs attributed to the innermost enclosing symbol). Seed TS/JS queries are smoke-level; R2 replaces them with the full port. - Routing seam in extractFromSource with per-file wasm fallback. DEFAULT_ROUTED is empty — no behavior change until a language passes its equivalence gate (R3). Dev opt-in: CODEGRAPH_KERNEL_LANGS. Kill switch: CODEGRAPH_KERNEL=0. Loader verifies ABI + kind tables before routing; EDGE_KINDS became a runtime array because kind order is now wire contract. - Grammar-source parity: vendored TS/TSX/JS wasm grammars built from the exact crate revisions (tree-sitter-typescript v0.23.2, tree-sitter-javascript v0.25.0, checked-in parser.c, ts-cli 0.25.10) — the tree-sitter-wasms builds were 2023-era, which the new kernel-grammar-parity test caught on day one. Production TS/JS parsing gets 2.5 years of grammar fixes; full suite green (2456 tests). - Build/release wiring: scripts/build-kernel.sh + npm run build:kernel; release.yml kernel prebuild matrix (continue-on-error — the kernel is optional everywhere, bundles fall back to the wasm path); bundles stage lib/kernel/codegraph-kernel.node; release job runs the kernel suites with CODEGRAPH_KERNEL_EXPECT=1. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby Mchenry
|
ad5300a601 |
fix(ui): show synthesis as a 'Linking dynamic dispatch' phase; mute node:sqlite warning spam (#1299)
Two first-run UX bugs surfaced by indexing a real 1,342-file C repo: 1. After 'Resolving refs' hit 100%, the ~40 dynamic-dispatch synthesis passes ran with no progress surface, so the bar sat frozen at 100% long enough to read as a hang (the C fn-pointer pass alone can hold for a while on C-heavy repos). Synthesis now reports per-pass progress through a new 'linking' IndexProgress phase, rendered as 'Linking dynamic dispatch'. The step total is pinned by a test to the synthesizer's actual __mark() count so adding a pass without bumping it fails loudly. 2. node:sqlite's ExperimentalWarning is emitted once per THREAD, so the main process plus every parse worker printed it mid-index, interleaved with the progress UI. All launch paths now pass --disable-warning=ExperimentalWarning: both bundle launchers, the Windows npm-shim invocation, and the CLI self-relaunch (NODE_RUNTIME_FLAGS, deliberately excluded from the re-exec gate so an older installed launcher never triggers a pointless re-exec, and version-gated off nodes older than the flag). Verified end-to-end on the same repo: zero warnings, live linking bar, byte-identical graph (50,520 nodes / 148,232 edges). Full suite green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Colby Mchenry
|
c9f8c0ebaf |
fix(mcp): reap the server when its launcher is killed during startup (#1185) (#1199)
An MCP host that kills the launcher chain within the server's first ~100ms while keeping the stdio pipes open (config probe, cancelled request, startup timeout; Rust hosts that kill a child without dropping its stdio handles) left the server orphaned: it booted already reparented to init, so the PPID watchdog's "ppid changed" baseline was captured as 1 and could never fire, and stdin never EOF'd. The process lingered — idle, ~30MB — until the host itself exited, accumulating one per abandoned launch (the pile-up reported in #1185). Reproduced on released 1.2.0/macOS: SIGKILL the launcher at +50ms → permanent orphan; at +150ms the old late baseline had already run and reaped it. Three-part fix: - Capture process.ppid at the earliest line of the CLI entry (early-ppid.ts) and use it as every watchdog baseline, shrinking the blind window to the few ms before our first JS runs. - Thread the real host pid down the bundled path: the npm shim and the standalone sh launcher set CODEGRAPH_HOST_PPID (an outer launcher's value wins), so the watchdog polls the host directly. Previously only the --liftoff-only relaunch set it, leaving the entire npm/standalone install base with hostPpid=null. - Never-initialized backstop (startup-handshake.ts): a serve --mcp that receives no MCP traffic for CODEGRAPH_STARTUP_HANDSHAKE_TIMEOUT_MS (default 15min, 0 disables) shuts down — the catch-all for a kill landing in the residual pre-JS window. Disarmed on the first byte, so a quiet-but-live session is never touched. Also scrub CODEGRAPH_HOST_PPID from the detached daemon's env — it has no host, and a stale pid must not leak into anything it spawns. Validated end-to-end on the built bundle: the +50ms early-kill orphan is now reaped while the host still holds the pipes open, and all six normal lifecycle paths (clean close, SIGTERM/SIGKILL child, host exit/SIGKILL, fd-holding adversarial host) stay clean. New coverage in startup-handshake.test.ts, mcp-startup-orphan.test.ts, and npm-shim.test.ts. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Colby Mchenry
|
e5d633075c |
fix: prevent V8 turboshaft WASM Zone OOM during indexing (#298, #293) (#322)
Large multi-language indexes crashed with `Fatal process out of memory: Zone` on Node 22/24 (including the bundled runtime) — V8's turboshaft optimizing WASM compiler exhausts its per-compilation Zone arena while compiling tree-sitter grammars on a background thread, even with tens of GB free (the Zone is a V8-internal arena, not the JS heap). Run node with V8 `--liftoff-only`, which keeps grammar compilation on the Liftoff baseline and never reaches the optimizing tier. Delivered via the bundled launcher + a one-shot CLI re-exec guard for all other launch paths. Empirically only `--liftoff-only` stops it (`--no-wasm-tier-up` / `--no-wasm-dynamic-tiering` do not), and it must be on node's command line (setFlagsFromString / worker execArgv / NODE_OPTIONS all fail). Reproduced the exact crash with the real indexer on Node 24.16 against a 2,880-file / 18-language repo and confirmed the fix eliminates it; full suite + 7 new tests pass. Bumps to 0.9.4. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Colby McHenry
|
2759afa57b |
fix(dist): resolve symlinks in the bundle launcher (curl install was broken)
install.sh symlinks ~/.local/bin/codegraph -> the bundle launcher, but the launcher derived its dir from $0, which is the symlink path — so it looked for `node` next to the symlink and failed with `exec: .../node: not found`. Follow the symlink chain to the real bundle dir first. (npm was unaffected — the shim invokes the launcher by absolute path.) Verified via the symlinked install path in a clean no-Node Linux container. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Colby Mchenry
|
ac52fd76c0 |
Self-contained distribution: bundle Node + node:sqlite, drop better-sqlite3/wasm (closes #238) (#282)
* fix(db): eliminate concurrent-read "database is locked"; add node:sqlite backend (#238) WAL + busy_timeout were already enabled, so the issue's suggested fix was a no-op. The real causes, addressed here: - busy_timeout is now set first (before journal_mode) and lowered 120s -> 5s, so open-time pragmas wait out a lock instead of hanging for two minutes. - getCodeGraph no longer opens a second connection to the default project when a tool passes its own projectPath (the in-process lock amplifier). - The wasm fallback (no WAL) gets a bounded read-retry on SQLITE_BUSY. - New: node:sqlite backend, preferred over wasm, so installs whose native better-sqlite3 build fails land on a real-WAL backend instead of no-WAL wasm. - codegraph status / codegraph_status now report the effective journal mode, so a lock report is triageable (wal vs delete). - CLI hard-blocks Node < 20 to actually enforce the engines floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(db)!: node:sqlite is the sole backend; drop better-sqlite3 + wasm Now that distribution will bundle a Node 24 runtime, node:sqlite (real SQLite with WAL + FTS5) is always available. Collapse the three-backend adapter to node:sqlite only and remove the machinery the other two needed: - Remove better-sqlite3 (optionalDependency) and node-sqlite3-wasm (dependency). - Remove WasmDatabaseAdapter, the named->positional param translation, the SQLITE_BUSY read-retry, the wasm fallback banner, the backend env override, and the native/node-sqlite/wasm selection chain. - createDatabase now opens node:sqlite directly, with a clear error pointing at the bundled release / Node 22.5+ when the module is absent. - NodeSqliteAdapter.close() is idempotent and pragma() supports { simple }, to match the better-sqlite3 behavior callers relied on. - status (CLI + MCP) reports the single node:sqlite backend; journal-mode diagnostics and the getCodeGraph single-connection fix are retained. - Tests repointed off better-sqlite3 onto node:sqlite. Net -1044 lines. Running from source now requires Node 22.5+. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): self-contained bundle prototype (vendored Node + install channels) Phase 3 of the node:sqlite migration: ship a vendored Node runtime so CodeGraph runs with no system Node and no native build (node:sqlite is built in). - scripts/build-bundle.sh: build a per-platform archive (official Node + dist + prod deps + launcher). Same recipe per platform; pins Node v24.16.0. - install.sh: curl|sh installer (no Node required) — detects os/arch, pulls the archive from Releases, symlinks onto PATH; re-run to upgrade, --uninstall to remove. The VPS/SSH path. - scripts/npm-shim.js: thin launcher for the npm channel — resolves the per-platform optionalDependency bundle and execs it, so `npm i -g` keeps working and the real work runs on the bundled Node regardless of the user's. - BUNDLING.md: distribution design + release-pipeline TODO (CI matrix, platform packages, code signing, brew, retiring the Node-version gate). Validated end-to-end: darwin-arm64 and linux-x64 bundles both run init + index + status (Backend: node:sqlite, Journal: wal) + FTS query with NO system Node — linux-x64 verified in a clean ubuntu:24.04 amd64 container. Release archives are gitignored; CI will produce and upload them. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): add Windows PowerShell installer (install.ps1) The `irm … | iex` one-liner for Windows, mirroring install.sh: detect arch, pull the matching bundle from Releases, extract to %LOCALAPPDATA%\codegraph, add it to user PATH. Re-run to upgrade. (Windows bundle production in build-bundle.sh is still TODO.) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): release workflow + npm packaging; README/CHANGELOG for bundled distro - .github/workflows/release.yml: manually-triggered (workflow_dispatch) release matrix. Builds a self-contained bundle per platform on its own runner (darwin-arm64/x64, linux-x64/arm64), publishes a GitHub Release with all archives, and publishes the npm thin-installer (shim + per-platform packages). Windows targets are TODO (build-bundle.sh is unix-only). - scripts/pack-npm.sh: assemble the npm packages from built bundles — per-platform packages tagged os/cpu + the main shim package with them as optionalDependencies (esbuild pattern). Proven locally: npm-install the tarballs, run via the shim, resolves the bundle and runs on the bundled Node 24 (node:sqlite / WAL). - README: install section now leads with the no-Node one-liners (curl|sh, irm|iex) then npm/npx; "bundled · none required" badge. - CHANGELOG: standout headline for the self-contained release, plus Added/Changed/ Removed for the install channels, node:sqlite backend, and dropped deps. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dist): Windows bundles + single-trigger release workflow - build-bundle.sh: add win32-x64 / win32-arm64 targets — download Node's Windows zip, bundle node.exe + a .cmd launcher, output a .zip. Verified structurally (PE32+ node.exe, CRLF .cmd, portable node_modules). Since there are no native addons, any target builds on any OS, so the whole matrix builds on one runner. - pack-npm.sh: handle .zip bundles and win32 packages (os: win32, node.exe). - release.yml: simplified to your spec — manual trigger reads the version from package.json, builds all platform bundles, creates the GitHub Release with notes pulled from CHANGELOG.md, and publishes the npm shim + platform packages. - BUNDLING.md: Windows + build-anywhere notes; release pipeline documented. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |