Files
colbymchenry--codegraph/__tests__/concurrent-locking.test.ts
Colby Mchenry ac52fd76c0 Self-contained distribution: bundle Node + node:sqlite, drop better-sqlite3/wasm (closes #238) (#282)
* fix(db): eliminate concurrent-read "database is locked"; add node:sqlite backend (#238)

WAL + busy_timeout were already enabled, so the issue's suggested fix was a
no-op. The real causes, addressed here:

- busy_timeout is now set first (before journal_mode) and lowered 120s -> 5s,
  so open-time pragmas wait out a lock instead of hanging for two minutes.
- getCodeGraph no longer opens a second connection to the default project when
  a tool passes its own projectPath (the in-process lock amplifier).
- The wasm fallback (no WAL) gets a bounded read-retry on SQLITE_BUSY.
- New: node:sqlite backend, preferred over wasm, so installs whose native
  better-sqlite3 build fails land on a real-WAL backend instead of no-WAL wasm.
- codegraph status / codegraph_status now report the effective journal mode, so
  a lock report is triageable (wal vs delete).
- CLI hard-blocks Node < 20 to actually enforce the engines floor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(db)!: node:sqlite is the sole backend; drop better-sqlite3 + wasm

Now that distribution will bundle a Node 24 runtime, node:sqlite (real SQLite
with WAL + FTS5) is always available. Collapse the three-backend adapter to
node:sqlite only and remove the machinery the other two needed:

- Remove better-sqlite3 (optionalDependency) and node-sqlite3-wasm (dependency).
- Remove WasmDatabaseAdapter, the named->positional param translation, the
  SQLITE_BUSY read-retry, the wasm fallback banner, the backend env override,
  and the native/node-sqlite/wasm selection chain.
- createDatabase now opens node:sqlite directly, with a clear error pointing at
  the bundled release / Node 22.5+ when the module is absent.
- NodeSqliteAdapter.close() is idempotent and pragma() supports { simple }, to
  match the better-sqlite3 behavior callers relied on.
- status (CLI + MCP) reports the single node:sqlite backend; journal-mode
  diagnostics and the getCodeGraph single-connection fix are retained.
- Tests repointed off better-sqlite3 onto node:sqlite.

Net -1044 lines. Running from source now requires Node 22.5+.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dist): self-contained bundle prototype (vendored Node + install channels)

Phase 3 of the node:sqlite migration: ship a vendored Node runtime so CodeGraph
runs with no system Node and no native build (node:sqlite is built in).

- scripts/build-bundle.sh: build a per-platform archive (official Node + dist +
  prod deps + launcher). Same recipe per platform; pins Node v24.16.0.
- install.sh: curl|sh installer (no Node required) — detects os/arch, pulls the
  archive from Releases, symlinks onto PATH; re-run to upgrade, --uninstall to
  remove. The VPS/SSH path.
- scripts/npm-shim.js: thin launcher for the npm channel — resolves the
  per-platform optionalDependency bundle and execs it, so `npm i -g` keeps
  working and the real work runs on the bundled Node regardless of the user's.
- BUNDLING.md: distribution design + release-pipeline TODO (CI matrix, platform
  packages, code signing, brew, retiring the Node-version gate).

Validated end-to-end: darwin-arm64 and linux-x64 bundles both run init + index +
status (Backend: node:sqlite, Journal: wal) + FTS query with NO system Node —
linux-x64 verified in a clean ubuntu:24.04 amd64 container. Release archives are
gitignored; CI will produce and upload them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dist): add Windows PowerShell installer (install.ps1)

The `irm … | iex` one-liner for Windows, mirroring install.sh: detect arch,
pull the matching bundle from Releases, extract to %LOCALAPPDATA%\codegraph,
add it to user PATH. Re-run to upgrade. (Windows bundle production in
build-bundle.sh is still TODO.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dist): release workflow + npm packaging; README/CHANGELOG for bundled distro

- .github/workflows/release.yml: manually-triggered (workflow_dispatch) release
  matrix. Builds a self-contained bundle per platform on its own runner
  (darwin-arm64/x64, linux-x64/arm64), publishes a GitHub Release with all
  archives, and publishes the npm thin-installer (shim + per-platform packages).
  Windows targets are TODO (build-bundle.sh is unix-only).
- scripts/pack-npm.sh: assemble the npm packages from built bundles — per-platform
  packages tagged os/cpu + the main shim package with them as optionalDependencies
  (esbuild pattern). Proven locally: npm-install the tarballs, run via the shim,
  resolves the bundle and runs on the bundled Node 24 (node:sqlite / WAL).
- README: install section now leads with the no-Node one-liners (curl|sh, irm|iex)
  then npm/npx; "bundled · none required" badge.
- CHANGELOG: standout headline for the self-contained release, plus Added/Changed/
  Removed for the install channels, node:sqlite backend, and dropped deps.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(dist): Windows bundles + single-trigger release workflow

- build-bundle.sh: add win32-x64 / win32-arm64 targets — download Node's Windows
  zip, bundle node.exe + a .cmd launcher, output a .zip. Verified structurally
  (PE32+ node.exe, CRLF .cmd, portable node_modules). Since there are no native
  addons, any target builds on any OS, so the whole matrix builds on one runner.
- pack-npm.sh: handle .zip bundles and win32 packages (os: win32, node.exe).
- release.yml: simplified to your spec — manual trigger reads the version from
  package.json, builds all platform bundles, creates the GitHub Release with notes
  pulled from CHANGELOG.md, and publishes the npm shim + platform packages.
- BUNDLING.md: Windows + build-anywhere notes; release pipeline documented.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-21 15:48:04 -05:00

153 lines
5.7 KiB
TypeScript

/**
* Issue #238 — "database is locked" on concurrent MCP tool calls.
*
* With node:sqlite (real WAL) as the backend, the fixes that remain relevant:
* 1. busy_timeout is a bounded few-second wait (not a 2-minute hang) and WAL is
* active — so a reader never blocks on a concurrent writer.
* 2. The MCP ToolHandler reuses the default instance when a tool passes a
* projectPath pointing at the default project, instead of opening a SECOND
* connection to the same DB.
*/
import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src';
import { ToolHandler } from '../src/mcp/tools';
import { DatabaseConnection } from '../src/db';
/** Normalize a PRAGMA read across return shapes (array | object | scalar). */
function pragmaValue(raw: unknown, key: string): unknown {
const row = Array.isArray(raw) ? raw[0] : raw;
if (row !== null && typeof row === 'object') return (row as Record<string, unknown>)[key];
return row;
}
describe('issue #238 — connection PRAGMAs (#1)', () => {
let dir: string;
let conn: DatabaseConnection;
beforeAll(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg238-pragma-'));
conn = DatabaseConnection.initialize(path.join(dir, 'codegraph.db'));
});
afterAll(() => {
conn.close();
fs.rmSync(dir, { recursive: true, force: true });
});
it('uses a bounded busy_timeout, not the old 2-minute hang', () => {
const ms = Number(pragmaValue(conn.getDb().pragma('busy_timeout'), 'timeout'));
expect(ms).toBeGreaterThan(0);
expect(ms).toBeLessThanOrEqual(30000); // far below the old 120000
});
it('runs in WAL mode — the mode that lets readers proceed during a write', () => {
const mode = String(pragmaValue(conn.getDb().pragma('journal_mode'), 'journal_mode')).toLowerCase();
expect(mode).toBe('wal');
});
it('getJournalMode() surfaces the effective mode for status triage', () => {
expect(conn.getJournalMode()).toBe('wal');
});
});
describe('issue #238 — WAL lets a reader proceed during a writer', () => {
let dir: string;
beforeAll(() => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg238-wal-'));
});
afterAll(() => {
fs.rmSync(dir, { recursive: true, force: true });
});
it('a read on a 2nd connection succeeds while a writer holds the lock', () => {
const dbPath = path.join(dir, 'codegraph.db');
const writer = DatabaseConnection.initialize(dbPath);
// The property only holds under WAL; skip if the filesystem couldn't enable it.
if (writer.getJournalMode() !== 'wal') {
writer.close();
return;
}
const reader = DatabaseConnection.open(dbPath);
try {
writer.getDb().prepare('BEGIN EXCLUSIVE').run(); // hard write lock, held open
const t0 = Date.now();
const row = reader.getDb().prepare('SELECT COUNT(*) AS c FROM nodes').get() as { c: number };
const waited = Date.now() - t0;
expect(row.c).toBe(0);
expect(waited).toBeLessThan(1000); // proceeds immediately, no busy wait
} finally {
try { writer.getDb().prepare('COMMIT').run(); } catch { /* ignore */ }
reader.close();
writer.close();
}
});
});
describe('issue #238 — ToolHandler reuses the default instance (#2)', () => {
let dir: string;
let cg: CodeGraph;
let root: string;
let handler: ToolHandler;
beforeAll(async () => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg238-tools-'));
fs.writeFileSync(path.join(dir, 'a.ts'), 'export function helper(): number { return 1; }\n');
fs.writeFileSync(
path.join(dir, 'b.ts'),
"import { helper } from './a';\nexport function main(): number { return helper(); }\n"
);
cg = await CodeGraph.init(dir, { index: true });
root = cg.getProjectRoot();
handler = new ToolHandler(cg);
});
afterAll(() => {
cg.close();
fs.rmSync(dir, { recursive: true, force: true });
});
it('getCodeGraph(defaultRoot) returns the default instance, not a new connection', () => {
const openSpy = vi.spyOn(CodeGraph, 'openSync');
try {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const resolved = (handler as any).getCodeGraph(root);
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const nested = (handler as any).getCodeGraph(path.join(root, 'does', 'not', 'exist'));
expect(resolved).toBe(cg);
expect(nested).toBe(cg); // a sub-path resolves up to the same default project
expect(openSpy).not.toHaveBeenCalled(); // no second connection opened
} finally {
openSpy.mockRestore();
}
});
it('concurrent read tool calls (mixed projectPath) all succeed without "database is locked"', async () => {
const openSpy = vi.spyOn(CodeGraph, 'openSync');
try {
const calls: Promise<{ content: Array<{ text: string }>; isError?: boolean }>[] = [
handler.execute('codegraph_search', { query: 'helper' }),
handler.execute('codegraph_search', { query: 'helper', projectPath: root }),
handler.execute('codegraph_callers', { symbol: 'helper', projectPath: root }),
handler.execute('codegraph_callees', { symbol: 'main' }),
handler.execute('codegraph_files', { projectPath: root }),
handler.execute('codegraph_status', { projectPath: root }),
];
const results = await Promise.all(calls);
for (const r of results) {
expect(r.isError).not.toBe(true);
expect(r.content[0]?.text ?? '').not.toMatch(/database is locked/i);
}
// Passing the default project's own path must not open a second connection.
expect(openSpy).not.toHaveBeenCalled();
} finally {
openSpy.mockRestore();
}
});
});