feat(kernel): R3 — TS/JS equivalence gate passed, kernel default-on

Gate evidence (docs/design/rust-kernel-migration-plan.md §4b):

- Graph parity, byte-identical (stronger than the §5 ≤0.5% bar): full
  codegraph-init dump-diffs kernel-vs-wasm on express (13,712 rows),
  excalidraw (89,898), and vscode (2,378,238 rows) — identical bytes.
  Python control repo (flask) identical + timing unchanged. The parity
  harness is now ORDER-sensitive (emission order drives rowids, which
  drive resolution order) and dumps come from the new
  scripts/dump-graph.mjs (natural keys, no rowids/timestamps).

- The one real find, caught by the vscode tier: tree-sitter error
  RECOVERY is encoding-dependent — byte-identical grammar sources and
  the same core (0.25.10) recover erroring files differently under
  UTF-8 (native) vs UTF-16 (web-tree-sitter) parsing; proven by
  reproducing the wasm tree with a native UTF-16 parse. Policy: the
  kernel defers any file whose tree has_error() to the wasm extractor
  (silent 'defer:' signal, per file) — parity by construction on
  erroring files (incidence 0-0.42% across the gate repos), and the
  harness fails if deferrals exceed 10% so a broken kernel can't hide
  behind the fallback.

- Retrieval invariants: canonical excalidraw flow (mutateElement →
  renderStaticScene) connects end-to-end on the kernel-indexed graph;
  synthesized-edge families present. Agent A/B is vacuous under
  byte-identical DBs (same justification as #1320-#1322).

- Perf: vscode init 105.4s → 82.1s (1.28×) on an 11-core Mac;
  excalidraw on a 2-CPU/6GB Linux container (the CI-runner envelope)
  6.2-7.1s → 4.3-4.8s (~1.5×). Linux arm64 in-container build: all 22
  kernel tests green under CODEGRAPH_KERNEL_EXPECT=1. Windows VM leg
  deferred (VM stopped; prlctl start needs Parallels Pro) — benign: a
  missing .node falls back to wasm, and the release matrix builds and
  gates the win32 prebuilds.

- Full suite: 2,465 tests pass WITH default-on routing, so the entire
  extraction corpus now exercises the kernel for TS/JS wherever a
  .node is staged.

DEFAULT_ROUTED = {typescript, tsx, javascript, jsx}. Override:
CODEGRAPH_KERNEL_LANGS (replaces the set) / CODEGRAPH_KERNEL=0 (kill).
Changelog entry added under [Unreleased].

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

.dockerignore

@@ -5,3 +5,6 @@ dist
 .kommandr
 docs
 assets
+codegraph-kernel/target
+codegraph-kernel/prebuilds
+release

CHANGELOG.md

@@ -11,6 +11,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### New Features
 
+- Indexing TypeScript, TSX, JavaScript, and JSX projects is faster: parsing and symbol extraction now run in a native engine when a prebuilt binary is available for your platform (release bundles include one), producing exactly the same graph — verified byte-for-byte against the previous engine on real repositories, from small libraries up to vscode-scale codebases. The speedup is largest on resource-constrained machines like CI runners. No setup needed: platforms without the native binary, and individual files with syntax errors, automatically use the previous engine, and `CODEGRAPH_KERNEL=0` turns the native path off entirely.
 - Reference resolution now runs in parallel on large projects. When a project has enough pending references to make it worthwhile (roughly 150k+, typical for big Java/Kotlin/Spring codebases), resolution fans out across worker threads while results are applied in the exact order the single-threaded path would have used — the graph comes out byte-for-byte identical, about twice as fast end-to-end on a 4,000-file Java project in our testing. Small projects keep the single-threaded path automatically (the fan-out costs more than it saves there). Set `CODEGRAPH_NO_PARALLEL_RESOLVE=1` to disable, or `CODEGRAPH_PARALLEL_RESOLVE_MIN=<count>` to tune when it engages.
 - Indexing large projects got another sizeable speedup — about a quarter less wall-clock on the same 4,000-file Java project, with the graph still byte-for-byte identical. Two changes: the database no longer interleaves expensive checkpoint housekeeping into the middle of resolution on a fresh index (it's folded once at the end instead), and while one batch's results are being written out, the worker threads are already resolving the next batch instead of sitting idle.
 - The dynamic-dispatch analysis that runs at the end of indexing (callback, event, and framework wiring) now runs its passes in parallel on large projects, cutting that stage roughly in half there — and a pass that crashes now retries safely instead of failing the whole index, which also makes very large codebases that previously died in this stage more likely to index to completion. Graphs remain byte-for-byte identical.

__tests__/kernel-scaffold.test.ts

@@ -72,9 +72,16 @@ describe.skipIf(!kernelBuilt)('kernel scaffold', () => {
     expect(info.languages).toContain('javascript');
   });
 
-  it('no language routes to the kernel by default (R1: wasm path unchanged)', () => {
+  it('TS/JS family routes to the kernel by default (R3 default-on); others stay wasm', () => {
+    for (const lang of ['typescript', 'tsx', 'javascript', 'jsx'] as const) {
+      expect(kernelRoutes(lang), lang).toBe(true);
+    }
+    expect(kernelRoutes('python')).toBe(false);
+    expect(tryKernelExtract('src/a.py', 'def f():\n  pass\n', 'python')).toBeNull();
+    // CODEGRAPH_KERNEL_LANGS REPLACES the default set when present.
+    process.env.CODEGRAPH_KERNEL_LANGS = 'tsx';
     expect(kernelRoutes('typescript')).toBe(false);
-    expect(tryKernelExtract('src/a.ts', 'function f() {}', 'typescript')).toBeNull();
+    expect(kernelRoutes('tsx')).toBe(true);
   });
 
   describe('with typescript routed (CODEGRAPH_KERNEL_LANGS)', () => {
@@ -170,12 +177,14 @@ describe.skipIf(!kernelBuilt)('kernel scaffold', () => {
       await loadGrammarsForLanguages(['typescript']);
     });
 
-    it('unrouted language flows through the wasm extractor unchanged', () => {
-      // `const f = () => 1` yields a function node on the wasm path; the seed
-      // kernel query deliberately doesn't extract it — so its presence proves
-      // which path ran.
+    it('kill switch routes through the wasm extractor unchanged', () => {
+      process.env.CODEGRAPH_KERNEL = '0';
       const result = extractFromSource('src/a.ts', 'export const f = () => 1;\n', 'typescript');
       expect(result.nodes.some((n) => n.kind === 'function' && n.name === 'f')).toBe(true);
+      delete process.env.CODEGRAPH_KERNEL;
+      // Default-routed path produces the same node (R2 parity).
+      const viaKernel = extractFromSource('src/a.ts', 'export const f = () => 1;\n', 'typescript');
+      expect(viaKernel.nodes.some((n) => n.kind === 'function' && n.name === 'f')).toBe(true);
     });
 
     it('routed language takes the kernel and falls back per file on kernel absence', () => {

__tests__/kernel-tsjs-parity.test.ts

@@ -110,6 +110,21 @@ describe.skipIf(!kernelBuilt)('kernel TS/JS extraction parity', () => {
     assertParity(rel, fs.readFileSync(file, 'utf8'), 'typescript');
   });
 
+  it('files with parse errors defer to the wasm extractor (recovery is encoding-dependent)', () => {
+    // tree-sitter error RECOVERY differs between UTF-8 (native) and UTF-16
+    // (web-tree-sitter) parsing — same grammar, same core version — so the
+    // kernel defers any erroring file to keep routing graph-neutral.
+    const broken = 'export function f( {\n  return }} 12 (\n';
+    process.env.CODEGRAPH_KERNEL_LANGS = 'all';
+    delete process.env.CODEGRAPH_KERNEL;
+    expect(tryKernelExtract('src/broken.ts', broken, 'typescript')).toBeNull();
+    // The seam still serves the file — through the wasm path.
+    process.env.CODEGRAPH_KERNEL = '0';
+    const viaWasm = extractFromSource('src/broken.ts', broken, 'typescript');
+    delete process.env.CODEGRAPH_KERNEL;
+    expect(viaWasm.nodes.some((n) => n.kind === 'file')).toBe(true);
+  });
+
   it('typescript fixture parsed as plain typescript variant', () => {
     // Same content through the non-tsx grammar exercises the typescript
     // (vs tsx) LangSpec pairing.

codegraph-kernel/src/tsjs/mod.rs

@@ -177,6 +177,19 @@ pub fn extract(file_path: &str, source: &str, language: &str) -> Result<EmitOut,
         .parse(source, None)
         .ok_or_else(|| "parser returned null tree".to_string())?;
 
+    // Files with parse ERRORS defer to the wasm extractor (the `defer:` prefix
+    // tells the TS side this is expected routing, not a malfunction). Reason:
+    // tree-sitter's error RECOVERY — same grammar, same core version — resolves
+    // differently under UTF-8 (native) vs UTF-16 (web-tree-sitter) parsing, so
+    // an erroring file's tree can differ between the paths (proven on vscode:
+    // `readonly import('x').T[]` recovered with the ERROR inside vs outside the
+    // type annotation). Erroring files are rare (0-0.42% across express/
+    // excalidraw/vscode) and per-file wasm fallback keeps routing graph-neutral
+    // by construction; clean files — 99.6%+ — stay on the fast path.
+    if tree.root_node().has_error() {
+        return Err("defer: parse tree contains errors — wasm recovery is canonical".to_string());
+    }
+
     let mut w = Walker {
         src: source,
         file_path,

docs/design/rust-kernel-migration-plan.md

@@ -19,8 +19,11 @@ Work top to bottom; each step has a section below with the detail.
       (353 files) + excalidraw (643 files) + torture fixtures; extraction 2.6× single-thread.
       R3's gate (large repo, retrieval invariants, agent A/B, Linux/Windows) still gates
       default-on.
-- [ ] **R3. Run TS/JS through the equivalence gate** — graph parity, retrieval
+- [x] **R3. Run TS/JS through the equivalence gate** — graph parity, retrieval
       invariants, agent A/B, perf + control repo. Ship behind the env flag, then default-on. (§5)
+      — **passed + DEFAULT-ON 2026-07-16, see §4b.** One deferred leg: Windows-VM run
+      (VM stopped, `prlctl start` needs Parallels Pro — benign: no .node ⇒ wasm fallback;
+      the release matrix builds + gates win32 prebuilds).
 - [ ] **R4. Port Java** → re-run the dubbo benchmark → the cbm-parity headline. (§4, §6)
 - [ ] **R5. Port Python, Go.** (§4)
 - [ ] **R6. Kernel-scale re-validation** in the cg1212 container (expect parse 6m → ~2m). (§6)
@@ -172,6 +175,51 @@ to wasm is the universal fallback. Zero-native-build-on-install stays true.
 - **Not yet done (R3 gate):** large-repo parity (vscode-class), full-repo dump-diff
   through the DB, retrieval invariants, agent A/B, Linux docker + Windows VM parity
   runs, control-repo perf. Routing stays opt-in (`CODEGRAPH_KERNEL_LANGS`) until then.
+  **→ Done same day, §4b.**
+
+### 4b. R3 — gate PASSED, TS/JS DEFAULT-ON (2026-07-16)
+
+Evidence (tools: `scripts/kernel-parity.mjs` now ORDER-sensitive — identical multisets
+in a different emission order would shift rowids and change resolution — and
+`scripts/dump-graph.mjs`, natural-key full-DB dumps):
+
+1. **Graph parity — byte-identical, not ≤0.5%:** full `init` dump-diff kernel-vs-wasm:
+   express (13,712 rows), excalidraw (89,898), **vscode (2,378,238 rows)** — all
+   byte-identical. Control repo (flask, Python) byte-identical + timing unchanged.
+   Extraction-level order-sensitive sweeps: repo 352/354 (+2 deferred), express
+   141/141, excalidraw 643/643, vscode 12,055/12,106 (+51 deferred), 0 diffs.
+2. **The one real find — encoding-dependent error recovery:** same grammar bytes
+   (sha-verified parser.c/scanner.h), same tree-sitter core (0.25.10), but error
+   RECOVERY on files with parse errors differs between UTF-8 (native) and UTF-16
+   (web-tree-sitter) parsing — proven by parsing the divergent vscode file natively
+   in UTF-16, which reproduced the wasm tree exactly. Incidence: 0% (express) /
+   0.31% (excalidraw) / 0.42% (vscode) of files. **Policy: the kernel defers any
+   file whose tree `has_error()` to the wasm extractor** (`defer:` signal, silent,
+   per-file) — parity by construction on erroring files, 99.6%+ keep the fast path,
+   and the harness fails if deferrals exceed 10% (a broken kernel can't hide).
+3. **Retrieval invariants:** kernel-indexed excalidraw — `mutateElement →
+   renderStaticScene` connects end-to-end via explore (callback + react-render +
+   jsx hops shown); synthesized-edge families present (408 jsx-render / 46
+   react-render / 14 interface-impl / 1 callback); byte-identical DB ⇒ counts equal
+   by construction.
+4. **Agent A/B:** byte-identical DBs make the A/B vacuous (identical graph, identical
+   MCP server) — same justification as the #1320–#1322 perf PRs, which shipped on the
+   dump-diff gate. Not burned.
+5. **Perf:** vscode init 105.4s → 82.1s (**1.28×**) on the 11-core Mac; excalidraw on
+   a 2-CPU/6GB Linux container (the CI-runner envelope) 6.2–7.1s → 4.3–4.8s
+   (**~1.5×**, n=2 interleaved); Mac excalidraw ≈ neutral-to-slightly-better (parse
+   already a small pool-parallelized slice at 11 cores). Control unchanged.
+6. **Platforms:** Linux (arm64 bookworm container, in-container cargo build): all 22
+   kernel tests green under `CODEGRAPH_KERNEL_EXPECT=1`. **Windows VM: deferred** —
+   VM stopped and `prlctl start` needs Parallels Pro; benign because a missing/broken
+   `.node` falls back to wasm, and the release workflow builds + gates win32
+   prebuilds. Run the kernel suites on the VM when it's next up.
+7. **Suite:** 2,465 tests pass WITH default-on routing — the entire extraction test
+   corpus now exercises the kernel for TS/JS on machines with a staged `.node`.
+
+Default routing: `DEFAULT_ROUTED = {typescript, tsx, javascript, jsx}` in
+`src/extraction/kernel/index.ts`. `CODEGRAPH_KERNEL_LANGS` REPLACES the set;
+`CODEGRAPH_KERNEL=0` kills. Changelog entry added under [Unreleased].
 
 ## 4. Per-language tracker
 
@@ -190,7 +238,7 @@ parity before porting the language.
 
 | Language(s) | Today | Tier | Grammar source | Migration notes / known traps | Status |
 |---|---|---|---|---|---|
-| typescript, tsx, javascript, jsx | `languages/typescript.ts`, `javascript.ts` + shared branches | T1 | crates.io | First target. Value-reference edges (#895/#897) and component recognition (#841 forwardRef/memo/styled) must survive — they're extraction-side. Largest test surface; gate is strictest here. **PORTED (§4a) — value-refs, component recognition, fn-refs, stores all byte-parity; awaiting the R3 gate before default-on.** | ◐ |
+| typescript, tsx, javascript, jsx | `languages/typescript.ts`, `javascript.ts` + shared branches | T1 | crates.io | First target. Value-reference edges (#895/#897) and component recognition (#841 forwardRef/memo/styled) must survive — they're extraction-side. Largest test surface; gate is strictest here. **PORTED + GATE PASSED + DEFAULT-ON (§4a/§4b); erroring files defer to wasm per-file.** | ✅ |
 | java | `languages/java.ts` | T1 | crates.io | Second target; unlocks the dubbo-parity claim. Lombok member synthesis (#912) is a NODE synthesizer hook in extraction (`synthesizeMembers`) — port or keep as TS post-pass. | ☐ |
 | python | `languages/python.ts` | T1 | crates.io | Third. Decorator extraction feeds framework route detection — parity required. | ☐ |
 | go | `languages/go.ts` | T1 | crates.io | Third (tie). Value-reference edges ship here too (#897). | ☐ |

scripts/dump-graph.mjs

@@ -0,0 +1,57 @@
+#!/usr/bin/env node
+/**
+ * Dump a .codegraph/codegraph.db graph by NATURAL KEYS (no rowids, no
+ * timestamps), sorted — two dumps diff clean iff the graphs are semantically
+ * identical. The byte-identical gate used by every perf/kernel PR:
+ *
+ *   node scripts/dump-graph.mjs <repo-or-db> > a.dump
+ *   node scripts/dump-graph.mjs <repo-or-db> > b.dump
+ *   diff a.dump b.dump
+ *
+ * Volatile fields excluded: nodes.updated_at, files.modified_at/indexed_at/
+ * content_hash+size (environment-dependent), edges.id / unresolved_refs.id
+ * (insertion rowids), and unresolved_refs.status (resolution bookkeeping —
+ * kept, actually: status is deterministic given the same input; excluded only
+ * if it proves flaky. We keep status.)
+ */
+
+import { DatabaseSync } from 'node:sqlite';
+import * as fs from 'node:fs';
+import * as path from 'node:path';
+
+const arg = process.argv[2];
+if (!arg) {
+  console.error('usage: dump-graph.mjs <repo-root-or-db-path>');
+  process.exit(2);
+}
+let dbPath = arg;
+if (fs.statSync(arg).isDirectory()) {
+  dbPath = path.join(arg, '.codegraph', 'codegraph.db');
+}
+const db = new DatabaseSync(dbPath, { readOnly: true });
+
+function dump(title, sql) {
+  const rows = db.prepare(sql).all();
+  const lines = rows.map((r) => JSON.stringify(r)).sort();
+  process.stdout.write(`== ${title} (${lines.length})\n`);
+  for (const l of lines) process.stdout.write(l + '\n');
+}
+
+dump(
+  'nodes',
+  `SELECT id, kind, name, qualified_name, file_path, language, start_line, end_line,
+          start_column, end_column, docstring, signature, visibility, is_exported,
+          is_async, is_static, is_abstract, decorators, type_parameters, return_type
+   FROM nodes`
+);
+dump(
+  'edges',
+  `SELECT source, target, kind, metadata, line, col, provenance FROM edges`
+);
+dump(
+  'refs',
+  `SELECT from_node_id, reference_name, reference_kind, line, col, candidates,
+          file_path, language, status, name_tail
+   FROM unresolved_refs`
+);
+dump('files', `SELECT path, language, node_count FROM files`);

scripts/kernel-parity.mjs

@@ -150,7 +150,7 @@ function report(category, sample) {
 
 let filesWithDiffs = 0;
 let filesOk = 0;
-let kernelFailed = 0;
+let deferred = 0;
 let totals = { nodes: 0, edges: 0, refs: 0 };
 
 process.env.CODEGRAPH_KERNEL_LANGS = 'all';
@@ -162,8 +162,11 @@ for (const { file, lang } of files) {
   delete process.env.CODEGRAPH_KERNEL; // kernel path on
   const kres = kernel.tryKernelExtract(rel, source, lang);
   if (!kres) {
-    kernelFailed++;
-    report('kernel-extract-failed', rel);
+    // Expected: files with parse errors defer to wasm (parity by
+    // construction — both arms run the same extractor). Counted, and
+    // guarded below so a broken kernel can't silently defer everything.
+    deferred++;
+    report('kernel-deferred', rel);
     continue;
   }
   process.env.CODEGRAPH_KERNEL = '0'; // wasm path
@@ -192,6 +195,19 @@ for (const { file, lang } of files) {
       const o = JSON.parse(x);
       report(`${table}:extra-in-kernel:${o.kind ?? ''}`, `${rel}: ${x}`);
     }
+    // ORDER matters too: identical multisets in a different emission order
+    // change DB rowids, and resolution iterates refs in rowid order — the
+    // full-index dump-diff would surface it as a downstream mystery. Catch it
+    // here instead.
+    if (onlyA.length === 0 && onlyB.length === 0) {
+      for (let i = 0; i < wasm.length; i++) {
+        if (wasm[i] !== kern[i]) {
+          fileHasDiff = true;
+          report(`${table}:order-mismatch`, `${rel}: index ${i}: wasm=${wasm[i]} kernel=${kern[i]}`);
+          break;
+        }
+      }
+    }
   }
   if (fileHasDiff) {
     filesWithDiffs++;
@@ -202,7 +218,7 @@ for (const { file, lang } of files) {
 }
 
 console.log(`\n=== kernel parity: ${filesOk}/${files.length} files byte-parity` +
-  ` (${filesWithDiffs} with diffs, ${kernelFailed} kernel-failed)` +
+  ` (${filesWithDiffs} with diffs, ${deferred} deferred-to-wasm)` +
   ` | wasm totals: ${totals.nodes} nodes / ${totals.edges} edges / ${totals.refs} refs ===\n`);
 
 const sorted = [...buckets.entries()].sort((a, b) => b[1].count - a[1].count);
@@ -211,4 +227,11 @@ for (const [cat, { count, samples }] of sorted) {
   for (const s of samples) console.log(`    ${s.length > 400 ? s.slice(0, 400) + '…' : s}`);
 }
 
-process.exit(filesWithDiffs > 0 || kernelFailed > 0 ? 1 : 0);
+// Deferrals are per-file parse-error routing (expected, rare). A high rate
+// means the kernel is broken and hiding behind the fallback — fail loudly.
+const deferralRate = deferred / files.length;
+if (deferralRate > 0.1) {
+  console.error(`deferral rate ${(deferralRate * 100).toFixed(1)}% exceeds 10% — kernel likely broken`);
+  process.exit(1);
+}
+process.exit(filesWithDiffs > 0 ? 1 : 0);

src/extraction/kernel/index.ts

@@ -7,9 +7,11 @@
  * everything else stays on the wasm path forever if need be. Rollback per
  * language = removing it from DEFAULT_ROUTED (or CODEGRAPH_KERNEL=0 for all).
  *
- * R1 status: NO language is default-routed yet. Development/testing opt-in:
- *   CODEGRAPH_KERNEL_LANGS=typescript,tsx   (or "all" for every kernel-capable
- *   language). R3 flips TS/JS into DEFAULT_ROUTED once the gate passes.
+ * Routing status: TypeScript/TSX/JavaScript/JSX are default-routed (R3 gate
+ * passed 2026-07-16 — full-index dumps byte-identical on express/excalidraw/
+ * vscode, control repo unchanged; see the migration plan §4a). Override with
+ *   CODEGRAPH_KERNEL_LANGS=<langs|all>  (replaces the default set), or
+ *   CODEGRAPH_KERNEL=0                  (kill switch, everything → wasm).
  */
 
 import type { ExtractionResult, Language } from '../../types';
@@ -22,8 +24,16 @@ export { decodeExtractBuffers } from './decode';
 /**
  * Languages routed to the kernel by default (gate-passed only — see the
  * per-language tracker in docs/design/rust-kernel-migration-plan.md §4).
+ * Per-file safety valve regardless of routing: a file whose parse tree
+ * contains ERRORS defers to the wasm extractor (error recovery differs
+ * between UTF-8 and UTF-16 parsing — wasm's recovery is canonical).
  */
-const DEFAULT_ROUTED: ReadonlySet<Language> = new Set<Language>([]);
+const DEFAULT_ROUTED: ReadonlySet<Language> = new Set<Language>([
+  'typescript',
+  'tsx',
+  'javascript',
+  'jsx',
+]);
 
 /**
  * Per-language TS post-pass over the decoded result — the escape hatch for
@@ -82,12 +92,15 @@ export function tryKernelExtract(
     result.durationMs = Date.now() - t0;
     return result;
   } catch (err) {
+    const message = err instanceof Error ? err.message : String(err);
+    // `defer:` is the kernel's expected-routing signal (files with parse
+    // errors take the wasm path — its error RECOVERY is the canonical one;
+    // recovery differs between UTF-8 and UTF-16 parsing). Silent by design.
+    if (message.includes('defer:')) return null;
     if (!warned.has(language)) {
       warned.add(language);
       process.stderr.write(
-        `[codegraph-kernel] ${language} extraction failed (${
-          err instanceof Error ? err.message : String(err)
-        }) — falling back to the wasm path\n`
+        `[codegraph-kernel] ${language} extraction failed (${message}) — falling back to the wasm path\n`
       );
     }
     return null;