github-featured/colbymchenry--codegraph
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

Merge main into fix/union-declarations-not-indexed

Browse Source 
Resolves the CHANGELOG conflict — main and this branch each prepended a
bullet to [Unreleased] > Fixes; both are kept. Everything else auto-merged,
including src/mcp/tools.ts, which main reworked heavily for the explore
allocation/displacement work (CG-28/31/36/38) while this branch added the
`union` kind to its container sets.

Verified on the merged tree with the native kernel built: 3070 passed,
9 skipped, 0 failed.
This commit is contained in:
Colby McHenry
2026-08-07 21:26:45 -05:00
parent 5b0c4b8b93 222f82b9a5
commit d289bf84d3
105 changed files with 15911 additions and 190 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+14
View File
@@ -15,20 +15,34 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- `codegraph_explore` no longer re-sends source it already returned earlier in the same conversation. A file it has already shown you comes back as a short pointer — the path, the symbols and the exact line range, with confirmation that the file hasn't changed since — and the space that frees is spent on code you haven't seen yet, so a follow-up call covers new ground instead of repeating the last one. If a file was edited in between, its source is always shown again in full. Set `CODEGRAPH_EXPLORE_DEDUP=0` to turn this off.
- When an agent connects over MCP, CodeGraph now states up front that it indexes 30+ languages — TypeScript/JavaScript, Python, Go, Rust, Java, C#, C/C++, PHP, Ruby, Swift, Kotlin, and more — so agents no longer assume a language isn't supported and skip the graph. (#671)
- GitHub Copilot is now a supported agent: `codegraph install` can configure Copilot Chat in VS Code (`copilot-vscode`), the GitHub Copilot CLI (`copilot-cli`), and the Copilot plugin in JetBrains IDEs (`copilot-jetbrains`). Installed Copilot surfaces are auto-detected like every other agent, existing MCP server entries in their config files are preserved, and `codegraph uninstall` reverses the setup cleanly. Restart VS Code or your JetBrains IDE after installing so Copilot picks up the server.
### Fixes
- C, C++, Objective-C and Rust unions are now indexed as first-class `union` nodes. A `union` declaration previously produced no symbol at all, so it never appeared in search or `codegraph_explore`, and anything attached to it disappeared with it — in Rust, every `impl SomeTrait for MyUnion` lost its edge, the methods from that impl were left pointing at a type the graph did not contain, and asking which types implement a trait quietly skipped the union ones. A union-shaped dispatch table in C now resolves its function pointers like a struct-shaped one. A `typedef union { … } Name;` in C keeps the typedef's name and is no longer mistaken for a plain type alias. Thanks @ctype-lab. Re-index after upgrading to pick up unions in existing projects. (#1515)
- A long-lived index no longer drifts away from what a fresh `codegraph index` would produce. When a file gained or lost a symbol, references to that name in files the sync never touched kept pointing at the definition that was correct before the change, and — because nothing distinguished two same-named definitions — the winner could come down to the order files happened to be written, which differs between a full index and a sync. On this project's own repository, replaying 80 commits through `sync` left 5.7% of connections wrong; it is now 1.3%, and the wrong-answers-still-being-asserted half drops by 99.7%. Since call edges are what flow questions follow and what `codegraph_explore` ranks files by, this quietly degraded answers as an index aged, with nothing to indicate it. Syncing is unchanged in speed, and an edit that only changes a function's body does no extra work at all. Set `CODEGRAPH_NO_REBIND=1` to opt out.
- `codegraph_explore` now concentrates its answer on the code that actually answers your question instead of spreading it across files that merely share a word with it, so more of the answer arrives in a single call. Thanks @LeDuyViet for the detailed measurements and reproduction. (#1500)
- Files only weakly related to your question now come back as a name, symbol and line number instead of spending the answer on their source — name one of them in a follow-up `codegraph_explore` to get it back in full. (#1500)
- A generated CRUD or protobuf layer no longer crowds out the hand-written code sitting beside it: generated files are now recognized by the `// Code generated by … DO NOT EDIT.` style banner written at the top of the file, not just by a filename that looks generated. Re-index after upgrading to pick up the new detection. (#1500)
- Test and spec files in a repository's top-level `test/` or `spec/` directory are now recognized as such, so they no longer take room from the code you asked about. (#1500)
- Generated type-declaration files that announce themselves with a "Generated by … by running …" banner — Cloudflare Wrangler's `worker-configuration.d.ts` is the common one — are now recognized as generated. Previously a file like that could take most of a `codegraph_explore` answer on nothing more than a few common words, pushing the hand-written code you asked about out of the response entirely. Re-index after upgrading to pick up the new detection.
- A hand-written type-declaration file — an ambient `.d.ts` of global shims, vendored typings, module augmentation — no longer takes over a `codegraph_explore` answer about how something works. Files like these declare common names (`Body`, `Message`, `ImageMetadata`) and nothing else, so a plainly-worded question could match one strongly enough that it ranked first and crowded the actual handler out of the answer. They are now ranked lower for questions about behaviour, and are still listed by name so one follow-up call fetches them. Asking about a type by name still returns its declaration first, and a shared types module the rest of your code imports is unaffected.
- A CodeGraph process that gets force-killed — by the stuck-process watchdog, a crash, or the OS — no longer leaves the database's write-ahead log behind to grow without bound. Previously each killed session stacked more data onto the same log file and nothing ever shrank it, which on machines where sessions were killed regularly could quietly eat tens of gigabytes of disk. The log is now capped, and any oversized leftover is reclaimed automatically the next time the project is opened. Thanks @tiendungdev for the exceptional Windows report that pinned this down. (#1431)
- The background server's watchdog no longer kills a healthy server that is just waiting on a slow disk: like indexing already does, it now checks whether the database files are still making progress before concluding the process is stuck. Fewer spurious kills also means fewer leftover write-ahead logs. (#1431)
- `codegraph status` now shows the write-ahead log's size next to the database size and warns when killed sessions have left it oversized, and every line in the background server's log now carries a timestamp so kills and restarts can be placed in time. (#1431)
- On Windows, the Claude Code prompt hook written by `codegraph install` failed with "command not found" when hooks run through Git Bash, which needs the `.cmd` extension to find the launcher. The installer now writes the platform-correct command, and re-running `codegraph install` (or `codegraph upgrade`) repairs an existing install in place. (#1466)
- Python classes used as values — `return SomeSerializer` from a factory method, `handler = SomeClass` aliases, registry dicts and lists, and classes passed as arguments — now produce reference edges in the graph. Previously these idioms were invisible, so on Django and Django REST Framework projects, asking for a serializer's callers or the impact of editing it missed the views that actually use it. Re-index after upgrading to pick up the new edges. (#1478)
- When a file changed on disk after its last index sync, `codegraph_node` and `codegraph_explore` could return a different symbol's code under the requested name — current file bytes cut at outdated line positions — while presenting it as verbatim, trustworthy source. This hit hardest on projects queried through `projectPath` (for example, sub-projects of a monorepo), which have no live file watcher to flag pending edits. Both tools now verify each file against the index before showing sliced code: an out-of-date file is either shown whole with its full current source, or its code is withheld with a clear "changed on disk" notice — never served as a wrong slice. A fresh re-index restores normal output automatically. Thanks @inth3shadows for the thorough report and verification passes. (#1474)
- A file built around one very long function no longer takes the whole `codegraph_explore` answer for itself — or disappears from it. Previously such a file was shown in full however big it was, which used up the room every file after it needed, and when the function was larger than the entire response the file was dropped without a word. These files now come back as a bounded window on whole lines — the signature and the top of the body, plus the call site when the call path runs through it — with the rest one follow-up `codegraph_explore` away.
- `codegraph_explore` no longer lets the first file in an answer spend the room set aside for the files below it, so the rest of the answer still arrives. Previously a large file near the top could quietly use up everything left, and the files ranked under it — each already judged relevant enough to include — were dropped with no source at all; on one question only one of six made it into the answer. Every file now keeps what it was given, and a question that really is about one file still concentrates on that file.
- When a `codegraph_explore` answer runs right up against its size limit, it now drops the trailing notes rather than a whole file's source. Previously the last file was cut even though trimming the notes alone would have fit, so a file that had already been read, ranked and rendered was thrown away at the last moment. Across a range of real projects this returns one more file and up to 20% more source per call.
- Every file `codegraph_explore` decides to include now actually arrives. A file shown in full could still spend room set aside for files below it — the fix above covered files shown as excerpts but not files shown whole — and the answer's own size bookkeeping under-counted each file's heading, so the answer ran past its limit and a fully prepared file was discarded at the end. A file that no longer fits whole is now shown as excerpts instead of vanishing, and one that overshoots by a little is trimmed to fit rather than dropped.
- The list of files an answer could not cover — the "explore these names for their source" pointers — is no longer thrown away when the answer is full. It is now budgeted for and trimmed to fit, so a full answer still tells you what it left out and which names to ask for next, instead of ending with no pointers at all.
- When a `codegraph_explore` answer shows a file as excerpts, a large excerpt that no longer fit was dropped entirely instead of being shortened. If the file's first excerpt happened to be a trivial one — an import block, a one-line helper next to the code you asked about — the excerpt carrying the actual answer was the one thrown away, and the file came back with a quarter of the room it had been given. On real projects that meant the top-ranked file delivered a fraction of its share while a far less relevant file took the rest. Excerpts are now shortened to fit, whole method by whole method, and only dropped when what is left is too small to hold anything readable.
- When you name a symbol in a `codegraph_explore` query, its definition now actually comes back. Two cases previously lost it. If the symbols you named don't call one another — sibling functions inside the same factory or module are the everyday example — CodeGraph stopped treating them as symbols you had asked for, and answered with whatever sat at the top of their file instead; on one 1,400-line file that meant a same-stem `QueuedMessage` interface on line 70 came back while the `queueMessage` function on line 1087 did not. And when an answer had to be trimmed to fit, it was trimmed from the bottom of the file down, so a symbol near the end of a long file was always the first thing cut. Trimming now protects the definitions you named wherever they sit in the file.
- The blast-radius section of `codegraph_explore` flagged "no covering tests found" whenever no test called a symbol directly — falsely branding helpers that tests exercise through their callers as untested (about 40% of flagged symbols in a measured sample). The check now follows caller chains up to 3 hops and reports indirect coverage as "tested via callers"; when nothing is found it states exactly what was checked instead of an unconditional warning. Thanks @inth3shadows for measuring the false-positive rate. (#1475)
## [1.5.0] - 2026-07-21
README.md
+9 -5
View File
@@ -6,7 +6,7 @@ Already installed? Run `codegraph upgrade`
Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates.
### Supercharge Claude Code, Cursor, Codex, OpenCode, Hermes Agent, Gemini, Antigravity, and Kiro with Semantic Code Intelligence
### Supercharge Claude Code, Cursor, Codex, OpenCode, Hermes Agent, Gemini, Antigravity, Kiro, and GitHub Copilot with Semantic Code Intelligence
**The fastest complete code graph · surgical context · built for how agents actually work · 100% local**
@@ -35,6 +35,7 @@ Follow [@getcodegraph](https://x.com/getcodegraph) on X for updates.
[![Gemini](https://img.shields.io/badge/Gemini-supported-blueviolet.svg)](#supported-agents)
[![Antigravity](https://img.shields.io/badge/Antigravity-supported-blueviolet.svg)](#supported-agents)
[![Kiro](https://img.shields.io/badge/Kiro-supported-blueviolet.svg)](#supported-agents)
[![GitHub Copilot](https://img.shields.io/badge/GitHub_Copilot-supported-blueviolet.svg)](#supported-agents)
<br>
@@ -104,7 +105,7 @@ In a **new terminal**, run the installer to connect CodeGraph to the agents you
codegraph install
```
<sub>Detects and auto-configures Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, and Kiro — wiring the CodeGraph MCP server into each. **This is the step that connects CodeGraph to your agent;** installing the CLI in step 1 does not do it on its own. It only wires up your agent — it does **not** index any code; building each project's graph is the separate `codegraph init` in step 3. (Shortcut: `npx @colbymchenry/codegraph` downloads and runs this in one go.)</sub>
<sub>Detects and auto-configures Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, Kiro, and GitHub Copilot (VS Code, Copilot CLI, JetBrains IDEs) — wiring the CodeGraph MCP server into each. **This is the step that connects CodeGraph to your agent;** installing the CLI in step 1 does not do it on its own. It only wires up your agent — it does **not** index any code; building each project's graph is the separate `codegraph init` in step 3. (Shortcut: `npx @colbymchenry/codegraph` downloads and runs this in one go.)</sub>
### 3. Initialize each project
@@ -375,7 +376,7 @@ npx @colbymchenry/codegraph
```
The installer will:
- Ask which agent(s) to configure — auto-detects installed ones from: **Claude Code**, **Cursor**, **Codex CLI**, **opencode**, **Hermes Agent**, **Gemini CLI**, **Antigravity IDE**, **Kiro**
- Ask which agent(s) to configure — auto-detects installed ones from: **Claude Code**, **Cursor**, **Codex CLI**, **opencode**, **Hermes Agent**, **Gemini CLI**, **Antigravity IDE**, **Kiro**, **GitHub Copilot** (VS Code, Copilot CLI, JetBrains IDEs)
- Prompt to install `codegraph` on your PATH (so agents can launch the MCP server)
- Ask whether configs apply to all your projects or just this one
- Write each chosen agent's MCP server config, plus a small marker-fenced CodeGraph section in the agent's instructions file (`CLAUDE.md` / `AGENTS.md` / `GEMINI.md`) — that's how subagents and non-MCP agents learn the `codegraph explore` command, since the MCP server's own guidance only reaches the main agent. Removed cleanly by `codegraph uninstall`.
@@ -389,7 +390,9 @@ The installer **wires up your agents only — it does not index your code.** Aft
codegraph install --yes # auto-detect agents, install global
codegraph install --target=cursor,claude --yes # explicit target list
codegraph install --target=auto --location=local # detected agents, project-local
codegraph install --target=copilot-vscode,copilot-cli,copilot-jetbrains --yes # GitHub Copilot everywhere
codegraph install --print-config codex # print snippet, no file writes
codegraph install --print-config copilot-vscode # same, for Copilot in VS Code
```
| Flag | Values | Default |
@@ -402,7 +405,7 @@ codegraph install --print-config codex # print snippet, no file wr
### 2. Restart Your Agent
Restart your agent (Claude Code / Cursor / Codex CLI / opencode / Hermes Agent / Gemini CLI / Antigravity IDE / Kiro) for the MCP server to load.
Restart your agent (Claude Code / Cursor / Codex CLI / opencode / Hermes Agent / Gemini CLI / Antigravity IDE / Kiro / VS Code, the Copilot CLI, or your JetBrains IDE for GitHub Copilot) for the MCP server to load.
### 3. Initialize Projects
@@ -760,6 +763,7 @@ is written):
- **Gemini CLI**
- **Antigravity IDE**
- **Kiro**
- **GitHub Copilot** — Copilot Chat in VS Code (`copilot-vscode`), the Copilot CLI (`copilot-cli`), and the Copilot plugin in JetBrains IDEs (`copilot-jetbrains`)
## Supported Languages
@@ -858,7 +862,7 @@ MIT
<div align="center">
**Made for AI coding agents — Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, and Kiro**
**Made for AI coding agents — Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, Kiro, and GitHub Copilot**
[Report Bug](https://github.com/colbymchenry/codegraph/issues) · [Request Feature](https://github.com/colbymchenry/codegraph/issues)
__tests__/explore-cluster-starvation.test.ts
+170
View File
@@ -0,0 +1,170 @@
/**
* Regression gate for CLUSTER-LEVEL STARVATION inside one file (task CG-36).
*
* A file's ranked clusters used to be all-or-nothing past the first one: the
* top-ranked cluster was taken (shrunk to fit if it had to be), and every
* cluster below it was rendered whole and then either fit the remainder or was
* dropped entirely. On a file whose top-ranked cluster is TRIVIAL that discards
* the answer — django's `db/models/sql/query.py` kept a 22-line glue cluster and
* dropped the 624-line `Query` body beneath it, spending 1,923 of a 7,947
* reservation, and okhttp's `RealInterceptorChain.kt` did the same behind its
* import header.
*
* What makes it hard to see is that the response stays FULL: the unspent
* reservation carries forward exactly as designed, so a lower-scoring file takes
* the bytes and every envelope-share measure still looks healthy. The gate is
* therefore per-file spend, not share.
*
* Two fixtures, pulling in opposite directions — read them together:
*
* - `starved-cluster-ts` is the defect. Its answer-bearing cluster must be
* SHRUNK into whatever the trivial cluster left, not dropped.
* - `dense-header-ts` is the Session.swift shape that cluster ranking puts
* importance ahead of density FOR. Its query's methods sit ~200 lines under
* a dense property list, and they must keep winning the budget. Any future
* rework of selection or shrinking has to satisfy both.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
interface Run {
dir: string;
cg: CodeGraph;
response: string;
report: ExploreDiagnosticReport;
}
/** Copy a fixture tree to a temp dir, index it, and run one explore call. */
async function runFixture(fixture: string, query: string): Promise<Run> {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg36-'));
fs.cpSync(path.join(__dirname, 'fixtures', fixture), dir, { recursive: true });
fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
const cg = CodeGraph.initSync(dir);
await cg.indexAll();
const sidecar = path.join(dir, 'explore-diag.jsonl');
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
let response: string;
try {
response = (await new ToolHandler(cg).execute('codegraph_explore', { query }))
.content?.[0]?.text ?? '';
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
return { dir, cg, response, report: JSON.parse(written[written.length - 1]!) };
}
function teardown(run: Run | undefined): void {
if (!run) return;
run.cg.destroy();
if (fs.existsSync(run.dir)) fs.rmSync(run.dir, { recursive: true, force: true });
}
describe('CG-36 — a trivial cluster must not starve the answer-bearing one', () => {
const TARGET = 'src/pipeline/chain.ts';
const QUERY = 'how does a request travel from sendRequest to the socket';
let run: Run;
let target: ExploreDiagnosticReport['files'][number];
beforeAll(async () => {
run = await runFixture('starved-cluster-ts', QUERY);
target = run.report.files.find((f) => f.path === TARGET)!;
}, 120_000);
afterAll(() => teardown(run));
describe('fixture shape — if this rots, the gate below means nothing', () => {
it('renders through the cluster path, with the answer past the trivial helper', () => {
expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
expect(target.render).toBe('clusters');
// The helper the entry point calls directly, and the class it does not.
const nodes = run.cg.getNodesInFile(TARGET);
const helper = nodes.find((n) => n.name === 'describeChain')!;
const proceed = nodes.find((n) => n.name === 'proceed')!;
expect(helper).toBeDefined();
expect(proceed).toBeDefined();
// Far enough apart to cluster separately at any gap threshold we ship.
expect(proceed.startLine - helper.endLine).toBeGreaterThan(20);
});
it('reserves the file the largest share, so an unspent share is a defect', () => {
expect(target.allowance ?? 0).toBeGreaterThan(4000);
const others = run.report.files.filter((f) => f.path !== TARGET);
for (const f of others) expect(f.allowance ?? 0).toBeLessThan(target.allowance!);
});
});
describe('the gate', () => {
it('spends most of the reservation it was given', () => {
// 28.8% on the CG-24 epic tip, 131% (its reservation plus carry-forward
// slack it can now actually use) with the fix. The bar is deliberately
// well below both so ordinary budget movement does not fail the suite.
expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
});
it('delivers the flow the query asked about, not just the helper beside it', () => {
// Both ends of the in-file flow, in the cluster that used to be dropped.
expect(run.response).toContain('async proceed(request: PipelineRequest)');
expect(run.response).toContain('private async writeAndRead(request: PipelineRequest)');
});
it('keeps the response inside the hard ceiling', () => {
expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
});
});
});
describe('CG-36 — a dense declaration block must not bury the query\'s methods', () => {
const TARGET = 'src/net/session.ts';
const QUERY = 'how does perform create a URLRequest and start the task';
let run: Run;
let target: ExploreDiagnosticReport['files'][number];
beforeAll(async () => {
run = await runFixture('dense-header-ts', QUERY);
target = run.report.files.find((f) => f.path === TARGET)!;
}, 120_000);
afterAll(() => teardown(run));
describe('fixture shape — if this rots, the gate below means nothing', () => {
it('has a dense low-importance header and the named methods far below it', () => {
expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
expect(target.render).toBe('clusters');
const nodes = run.cg.getNodesInFile(TARGET);
const perform = nodes.find((n) => n.name === 'perform')!;
expect(perform).toBeDefined();
// The header block: many adjacent declarations above the first named
// method, which is what makes it the densest region of the file.
const above = nodes.filter((n) => n.endLine < perform.startLine
&& (n.kind === 'property' || n.kind === 'field' || n.kind === 'method'));
expect(above.length).toBeGreaterThan(20);
expect(perform.startLine).toBeGreaterThan(150);
});
});
describe('the gate', () => {
it('delivers all three methods the query named', () => {
expect(run.response).toContain('async perform(url: string, method: string');
expect(run.response).toContain('didCreateURLRequest(request: URLRequest)');
expect(run.response).toContain('task(request: URLRequest, identifier: number)');
});
it('spends the file\'s reservation on them', () => {
expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
});
it('keeps the response inside the hard ceiling', () => {
expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
});
});
});
__tests__/explore-declaration-only.test.ts
+207
View File
@@ -0,0 +1,207 @@
/**
* Regression gate for DECLARATION-ONLY files in explore ranking (task CG-28).
*
* A file that holds nothing but type declarations — an ambient `.d.ts`, vendored
* typings, a `types.ts` of pure interfaces — cannot answer a FLOW question: no
* bodies, no call edges, no behaviour. But the identifiers it declares are
* exactly the generic ones a prose question uses (`Body`, `Message`,
* `ImageMetadata`, `ReadableStream`), so on term overlap it out-scored the
* implementation and took the envelope. Measured on this fixture before the fix:
* rank #1 and 51% of delivered source on a prose flow query.
*
* CG-25 already covers the file that STARTED this — a Wrangler
* `worker-configuration.d.ts`, which announces itself with a generated banner.
* `docs/benchmarks/explore-declaration-only-cg28.md` has that measurement; the
* banner alone is worth 15–46 points of envelope share. What it does not cover
* is a declaration file with no banner at all, which is what this fixture's
* `platform-shims.d.ts` is, and what the damping in `rankPenalty` addresses.
*
* Two claims, and BOTH have to hold — the counter-case is why the penalty is
* guarded rather than flat:
*
* 1. a prose flow query must not let a declaration-only file outrank the
* implementation files that answer it;
* 2. a query genuinely ABOUT a declared type must still reach the declaration
* at full weight.
*
* The suppression the issue explicitly forbids is also pinned: a damped file is
* still a candidate and still named in the response, so one follow-up explore
* fetches it.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'ambient-decls-ts');
/** Declaration-only, hand-written, NO generated banner — the surviving gap. */
const HANDWRITTEN_DECL = 'types/platform-shims.d.ts';
/** Declaration-only WITH a Wrangler banner — the CG-25 control in the same run. */
const GENERATED_DECL = 'types/worker-configuration.d.ts';
/** Declaration-only but IMPORTED by the storage layer — must never be damped. */
const SHARED_TYPES = 'src/storage/types.ts';
/** Prose, naming no symbol — the query shape that let the original file in. */
const FLOW_QUERY =
'how does an upload request stream the file body to storage and record image metadata';
/** Prose that DOES name a declared type — the counter-case. */
const TYPE_QUERY = 'what does the UploadStorage interface declare for putting an object';
describe('CG-28 — a declaration-only file does not outrank implementation on a flow query', () => {
let testDir: string;
let cg: CodeGraph;
let sidecar: string;
/** One explore call; returns its diagnostic report plus the response text. */
const explore = async (query: string): Promise<{ report: ExploreDiagnosticReport; text: string }> => {
fs.rmSync(sidecar, { force: true });
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
let text: string;
try {
text = (await new ToolHandler(cg).execute('codegraph_explore', { query })).content?.[0]?.text ?? '';
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
return { report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport, text };
};
const fileOf = (report: ExploreDiagnosticReport, p: string): ExploreDiagnosticFile | undefined =>
report.files.find((f) => f.path === p);
let flow: { report: ExploreDiagnosticReport; text: string };
let typed: { report: ExploreDiagnosticReport; text: string };
beforeAll(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg28-'));
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
sidecar = path.join(testDir, 'explore-diag.jsonl');
cg = CodeGraph.initSync(testDir);
await cg.indexAll();
flow = await explore(FLOW_QUERY);
typed = await explore(TYPE_QUERY);
}, 120_000);
afterAll(() => {
if (cg) cg.destroy();
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
describe('fixture shape — if this rots, the gate below means nothing', () => {
it('holds two declaration-only files that differ only in the banner', () => {
for (const p of [HANDWRITTEN_DECL, GENERATED_DECL]) {
const nodes = cg.getNodesInFile(p).filter((n) => n.kind !== 'file' && n.kind !== 'import');
expect(nodes.length, `${p} declares nothing`).toBeGreaterThan(10);
// Every symbol type-level, nothing with a body — the structural test the
// penalty keys on. A `function`/`class` creeping in would silently exempt
// the file and make every assertion below vacuous.
expect(nodes.every((n) => n.kind === 'interface' || n.kind === 'type_alias'), `${p} has a non-type symbol`).toBe(true);
}
// Only one of them announces itself, so the CG-25 penalty is the ONLY
// difference between the two — that is what makes them comparable.
expect(cg.getFile(GENERATED_DECL)?.generated).toBe(true);
expect(cg.getFile(HANDWRITTEN_DECL)?.generated).toBeFalsy();
});
it('holds a pure-type module the code IMPORTS, as the safety control', () => {
// Identical to the ambient files on kinds and bodies; different only in
// that the storage layer is typed by it. This is the shape the penalty
// must NOT catch — a `types.ts` the codebase depends on is part of the
// structure of any answer about that code.
const nodes = cg.getNodesInFile(SHARED_TYPES).filter((n) => n.kind !== 'file' && n.kind !== 'import');
expect(nodes.length).toBeGreaterThan(0);
expect(nodes.every((n) => n.kind === 'interface' || n.kind === 'type_alias')).toBe(true);
expect(cg.getFile(SHARED_TYPES)?.generated).toBeFalsy();
});
it('holds implementation files that DO answer the flow question', () => {
for (const p of ['src/routes/upload.ts', 'src/storage/stream.ts', 'src/storage/metadata.ts']) {
expect(cg.getNodesInFile(p).some((n) => n.kind === 'function'), `${p} has no functions`).toBe(true);
}
});
});
describe('the gate — a prose flow query', () => {
it('damps the un-bannered declaration file rather than letting it rank free', () => {
const rec = fileOf(flow.report, HANDWRITTEN_DECL);
expect(rec, 'the declaration file is not even a candidate — fixture drifted').toBeDefined();
expect(rec!.ambientDeclaration).toBe(true);
expect(rec!.penalty).toBeLessThan(1);
});
it('does not let it outrank the implementation files', () => {
const decl = fileOf(flow.report, HANDWRITTEN_DECL)!;
const impl = flow.report.files.filter((f) => f.path.startsWith('src/') && f.finalChars > 0);
expect(impl.length, 'no implementation file delivered anything').toBeGreaterThanOrEqual(2);
// Measured before the fix: the declaration file was rank #1 with score 53
// against the best implementation file's 34. The bar is that at least one
// implementation file now ranks above it — ordinary budget movement must
// not fail the suite, but the inversion coming back must.
expect(impl.some((f) => f.rank < decl.rank), 'declaration file still ranks first').toBe(true);
});
it('still names it in the response, so one follow-up call fetches it', () => {
// The issue forbids suppression: a damped file must remain reachable.
expect(flow.text).toContain(HANDWRITTEN_DECL);
});
it('leaves the implementation files at full weight', () => {
for (const f of flow.report.files.filter((x) => x.path.startsWith('src/'))) {
expect(f.ambientDeclaration, `${f.path} was misread as an ambient declaration`).toBe(false);
expect(f.penalty).toBe(1);
}
});
it('does not damp a pure-type module the codebase imports', () => {
// The condition that keeps this narrow enough to be safe. Without it the
// same rule demotes `displacement-ts`'s pipeline `types.ts` — pure
// interfaces, but 13 inbound imports — and breaks the CG-31 gate.
const rec = flow.report.files.find((f) => f.path === SHARED_TYPES);
if (rec) {
expect(rec.ambientDeclaration, `${SHARED_TYPES} was flagged ambient`).toBe(false);
expect(rec.penalty).toBe(1);
}
// Independent of whether this query ranked it: the predicate itself must
// separate the two shapes.
const isAmbient = cg.ambientDeclarationFilePredicate([SHARED_TYPES, HANDWRITTEN_DECL]);
expect(isAmbient(SHARED_TYPES)).toBe(false);
expect(isAmbient(HANDWRITTEN_DECL)).toBe(true);
});
});
describe('the counter-case — a query that NAMES a declared type', () => {
it('reaches the declaration at full weight, undamped', () => {
const rec = fileOf(typed.report, HANDWRITTEN_DECL);
expect(rec, 'the named type\'s file is not a candidate').toBeDefined();
expect(rec!.ambientDeclaration).toBe(true);
// Detected as declaration-only, but EXEMPT — the query asked for it.
expect(rec!.penalty).toBe(1);
});
it('ranks it first and delivers its source', () => {
const rec = fileOf(typed.report, HANDWRITTEN_DECL)!;
expect(rec.rank).toBe(1);
expect(rec.finalChars).toBeGreaterThan(0);
});
});
describe('the two penalties do not stack', () => {
it('charges a generated declaration file once, at the stronger rate', () => {
// A file that is BOTH generated and declaration-only has ONE property two
// signals happen to see. Penalising twice (0.3 * 0.5 = 0.15) is how a file
// gets cliffed out of answers where it is genuinely relevant.
const rec = flow.report.files.find((f) => f.generated && f.ambientDeclaration);
if (!rec) return; // not a candidate for this query — nothing to assert
expect(rec.penalty).toBeGreaterThanOrEqual(0.3);
});
});
});
__tests__/explore-displacement-guard.test.ts
+245
View File
@@ -0,0 +1,245 @@
/**
* Regression fixture for CG-31 — a clustered render may not spend a reservation
* still owed to a file the loop has not reached.
*
* The allocator hands every admitted file a reservation (CG-12), and the render
* loop then walks the files in rank order. Carry-forward slack lets a file spend
* what the files ABOVE it left on the table, which is right; what was missing is
* the other half — nothing was held back for the files BELOW it. The whole-file
* BUY arm has always refused that trade (`owedBelow`, `tools.ts`); the cluster
* path had no equivalent, so `fileBudget`/`SPINE_CEILING` read what was left
* before the hard ceiling rather than what was still promised, and the first
* oversize file could take the response.
*
* `__tests__/fixtures/displacement-ts/` reproduces it. Four pipeline stages
* compete for one envelope; the first, `ingest.ts`, is a single ~20K function —
* one cluster member far bigger than any reservation it can earn — so it takes
* the bounded overshoot CG-30 left it. The fixture is padded to >500 indexed
* files on purpose: the displacement only exists on the 24K tier, where the
* reservations plus the response preamble genuinely saturate the hard ceiling.
*
* Measured against the pre-fix build (CG-30 landed, CG-31 not):
*
* ingest.ts 9,301 chars emitted on a 6,289 spendable — then dropped whole
* by the final ceiling, so it cost the response and delivered 0
* types.ts skipped `budget-whole-file`
* sink.ts skipped `budget-whole-file`
* delivered 3 of 6 admitted files, 14,908-char envelope
*
* With the guard: 6 of 6, 22,066-char envelope, and `ingest.ts` bounded to the
* 4,913 that were actually still free.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'displacement-ts');
/**
* Padding modules, written into the temp copy rather than checked in. The
* output tier is chosen by INDEXED FILE COUNT, and the displacement this test
* pins only exists at >=500 files (24K envelope against a 24.4K render ceiling
* that also has to hold the response preamble). Below that the ceiling has
* enough slack to absorb an overshoot and the bug is invisible.
*/
const FILLER_FILES = 520;
/** A symbol bag spanning all four stages — they compete for one envelope. */
const QUERY = 'ingestRecords normalizeRecords enrichRecords publishRecords';
/** One symbol, one file — the concentration case the guard must not flatten. */
const PRECISE_QUERY = 'ingestRecords';
/** The giant: one ~20K function, the file that used to take the response. */
const GIANT = 'src/pipeline/ingest.ts';
/** Ranked below the giant and dropped by it pre-fix. */
const STARVED = ['src/pipeline/types.ts', 'src/pipeline/sink.ts'];
interface Probe {
response: string;
report: ExploreDiagnosticReport;
bytes: Map<string, number>;
}
describe('CG-31 — the cluster path holds back what is still owed below it', () => {
let testDir: string;
let cg: CodeGraph;
let spread: Probe;
let precise: Probe;
const fileOf = (probe: Probe, p: string): ExploreDiagnosticFile => {
const rec = probe.report.files.find((f) => f.path === p);
if (!rec) throw new Error(`${p} absent from the diagnostic report`);
return rec;
};
/** Admitted = the allocator reserved bytes for it. */
const admitted = (probe: Probe): ExploreDiagnosticFile[] =>
probe.report.files.filter((f) => (f.allowance ?? 0) > 0);
beforeAll(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg31-'));
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
const filler = path.join(testDir, 'src', 'generated');
fs.mkdirSync(filler, { recursive: true });
for (let i = 0; i < FILLER_FILES; i++) {
// Deterministic, unrelated to the query — these pad the file count, they
// must never rank.
fs.writeFileSync(
path.join(filler, `unit${i}.ts`),
`export const seed${i} = ${i};\n`
+ `export function widget${i}(n: number): number {\n return n * ${i + 1} + seed${i};\n}\n`,
);
}
cg = CodeGraph.initSync(testDir);
await cg.indexAll();
// The per-file bounds are only observable through the diagnostic sidecar.
const sidecar = path.join(testDir, 'explore-diag.jsonl');
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
const run = async (handler: ToolHandler, query: string): Promise<Probe> => {
const result = await handler.execute('codegraph_explore', { query });
const response = result.content?.[0]?.text ?? '';
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
return {
response,
report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport,
bytes: attributeSourceBytes(response),
};
};
try {
const handler = new ToolHandler(cg);
spread = await run(handler, QUERY);
precise = await run(handler, PRECISE_QUERY);
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
}, 180_000);
afterAll(() => {
if (cg) cg.destroy();
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
// ── Fixture shape — if these rot, the gate below means nothing ─────────────
describe('fixture shape', () => {
it('sits on the 24K tier, where the reservations saturate the ceiling', () => {
expect(cg.getStats().fileCount).toBeGreaterThanOrEqual(500);
expect(spread.report.budget.maxOutputChars).toBe(24000);
});
it('admits every stage file, so there is something to displace', () => {
const paths = admitted(spread).map((f) => f.path);
expect(paths).toContain(GIANT);
for (const p of STARVED) expect(paths).toContain(p);
expect(paths.length).toBeGreaterThanOrEqual(5);
});
it('renders the giant through the CLUSTER path, over its reservation', () => {
const rec = fileOf(spread, GIANT);
expect(rec.render).toBe('clusters');
// One member bigger than anything it can earn beside its siblings — the
// shape that makes the bounded overshoot fire at all.
const source = fs.readFileSync(path.join(testDir, GIANT), 'utf-8');
expect(source.length).toBeGreaterThan((rec.spendable ?? 0) * 2);
// And the guard actually bit — a vacuous pass here would hide a
// regression. Measured against the bounded overshoot a cluster's top
// member may otherwise take (1.5x, CG-30), which is what it refused.
expect(rec.funded).not.toBeNull();
expect(rec.funded!).toBeLessThan(Math.round(rec.spendable! * 1.5));
});
});
// ── The gate ──────────────────────────────────────────────────────────────
describe('displacement refusal', () => {
it('CG-31 GATE: no clustered file emits past what was still free to spend', () => {
for (const probe of [spread, precise]) {
const over = probe.report.files
.filter((f) => f.render === 'clusters' && f.funded !== null)
// +1 for the render loop's own rounding on the windowed cut.
.filter((f) => f.emittedChars > f.funded! + 1)
.map((f) => `${f.path}: ${f.emittedChars} of ${f.funded}`);
expect(over).toEqual([]);
}
});
it('CG-31 GATE: every admitted file below the top one is delivered', () => {
// Pre-fix: 3 of 6 — `ingest.ts` overshot, was itself cut by the final
// ceiling, and took `types.ts` + `sink.ts` down with it.
for (const rec of admitted(spread)) {
expect(rec.skipped, `${rec.path} skipped`).toBeNull();
expect(spread.bytes.get(rec.path) ?? 0, `${rec.path} bytes`).toBeGreaterThan(0);
}
for (const p of STARVED) expect(spread.bytes.get(p) ?? 0).toBeGreaterThan(0);
});
it('the guard is symmetric — it is about ORDER, not rank', () => {
// Nothing here protects rank #1 specifically: the LAST admitted file, the
// only one with no reservation owed below it, is delivered too.
const files = admitted(spread);
const last = files[files.length - 1]!;
expect(last.skipped).toBeNull();
expect(spread.bytes.get(last.path) ?? 0).toBeGreaterThan(0);
// And the last file is never itself cut by the guard — nothing is owed
// below it, so `funded` may not sit under its own reservation.
expect(last.funded!).toBeGreaterThanOrEqual(Math.min(last.allowance!, last.emittedChars));
});
it('a kept promise is not a displacement — no file is cut below its reservation', () => {
for (const probe of [spread, precise]) {
for (const rec of admitted(probe)) {
if (rec.funded === null) continue;
expect(rec.funded, rec.path).toBeGreaterThanOrEqual(
Math.min(rec.allowance!, rec.emittedChars));
}
}
});
it('nothing is lost to the hard ceiling — the epilogue is cut before a section', () => {
// A section thrown away by the final truncation is the same starvation
// arriving after the guard has done its work: the bytes were held back
// for that file and then nobody received them.
for (const probe of [spread, precise]) {
expect(probe.report.files.filter((f) => f.render === 'dropped')).toEqual([]);
}
});
it('keeps the response inside the hard ceiling', () => {
for (const probe of [spread, precise]) {
expect(probe.report.envelope.chars).toBeLessThanOrEqual(probe.report.budget.hardCeiling);
}
});
});
// ── The thing the guard must NOT become ───────────────────────────────────
describe('concentration survives', () => {
it('a precise symbol query still puts the most source in the named file', () => {
const mine = precise.bytes.get(GIANT) ?? 0;
const others = [...precise.bytes.entries()].filter(([p]) => p !== GIANT);
expect(mine).toBeGreaterThan(0);
for (const [p, n] of others) {
expect(mine, `${GIANT} vs ${p}`).toBeGreaterThan(n);
}
// Not a forced even split: the named file takes a clear plurality.
const total = [...precise.bytes.values()].reduce((s, n) => s + n, 0);
expect(mine / total).toBeGreaterThan(1 / precise.bytes.size);
});
it('the named file still outspends what it would get from an even split', () => {
const rec = fileOf(precise, GIANT);
const even = precise.report.budget.maxOutputChars / admitted(precise).length;
expect(rec.emittedChars).toBeGreaterThan(even);
});
});
});
__tests__/explore-factory-closure.test.ts
+157
View File
@@ -0,0 +1,157 @@
/**
* Regression gate for the FACTORY-CLOSURE file shape (task CG-27).
*
* A `createFoo()` that returns an object of closures spans almost all of its
* file, so its indexed range is an ENVELOPE around every symbol the query
* actually wants. Svelte 5 rune stores, React custom-hook modules, IIFE
* module-pattern JS and Zustand's `create((set, get) => ({ … }))` are all
* written this way, so it is a shape rather than a one-repo quirk.
*
* CG-27 asked whether the >50%-of-file envelope drop — which fires for `class`,
* `struct`, `interface` and friends but not for `function`/`method` — should be
* extended to cover it. **Measured, it should not**, and the issue was closed as
* obsolete: `docs/benchmarks/explore-factory-closure-cg27.md` has the numbers.
* Two independent mechanisms already absorb the shape:
*
* - `shrinkCluster` orders members by (importance desc, SIZE ASC) and refuses
* any member that overruns the cap once something is kept, so a file-spanning
* member is only ever selected when it is the sole member of the top
* importance tier;
* - when it IS selected, CG-30 windows it on whole lines rather than emitting
* it whole, so the file still delivers bounded, readable source.
*
* Dropping the range instead SPLITS the file into several clusters, and only the
* first-chosen cluster may be shrunk — measured, a trivial 7-line cluster won the
* density tiebreak and the answer-bearing cluster was dropped whole, taking the
* rank-#1 file from 7,539 chars and 7 of 11 inner definitions to 397 and none.
*
* So this file pins the OUTCOME, not the mechanism: whatever future work does to
* clustering, a factory-closure file must keep delivering the closures inside it
* — that is what stops the agent Reading the file back.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'factory-closure-ts');
/** The factory file, and the closure factory whose body is nearly all of it. */
const TARGET = 'src/stores/dashboard-store.ts';
const FACTORY = 'createDashboardStore';
/** Prose the way a newcomer asks it, naming two of the closures inside. */
const QUERY = 'how does the dashboard store refresh its metrics and apply a filter';
describe('CG-27 — a factory-closure file delivers the closures inside it', () => {
let testDir: string;
let cg: CodeGraph;
let response: string;
let report: ExploreDiagnosticReport;
/** Source lines of TARGET the response actually carried. */
let delivered: Set<number>;
let sourceLines: string[];
beforeAll(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg27-'));
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
cg = CodeGraph.initSync(testDir);
await cg.indexAll();
const sidecar = path.join(testDir, 'explore-diag.jsonl');
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
try {
response = (await new ToolHandler(cg).execute('codegraph_explore', { query: QUERY }))
.content?.[0]?.text ?? '';
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
report = JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport;
// A line counts as delivered only when the response numbers it AND the text
// matches that source line — a line number quoted in prose must not count.
sourceLines = fs.readFileSync(path.join(testDir, TARGET), 'utf-8').split('\n');
delivered = new Set();
for (const line of response.split('\n')) {
const m = /^(\d+)\t(.*)$/.exec(line);
if (!m) continue;
const n = Number(m[1]);
if (n >= 1 && n <= sourceLines.length && sourceLines[n - 1] === m[2]) delivered.add(n);
}
}, 120_000);
afterAll(() => {
if (cg) cg.destroy();
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
/** The closures defined inside the factory, straight from the index. */
const innerClosures = () => {
const nodes = cg.getNodesInFile(TARGET);
const factory = nodes.find((n) => n.name === FACTORY)!;
return nodes.filter((n) => (n.kind === 'function' || n.kind === 'method')
&& n.name !== FACTORY
&& n.startLine > factory.startLine && n.endLine <= factory.endLine);
};
describe('fixture shape — if this rots, the gate below means nothing', () => {
it('holds one symbol spanning most of the file, with closures inside it', () => {
const factory = cg.getNodesInFile(TARGET).find((n) => n.name === FACTORY);
expect(factory, `${TARGET} has no ${FACTORY} node`).toBeDefined();
// The envelope condition the >50% drop tests for — and `function`, the kind
// that drop does not cover.
expect(factory!.kind).toBe('function');
expect(factory!.endLine - factory!.startLine + 1)
.toBeGreaterThan(sourceLines.length * 0.5);
expect(innerClosures().length).toBeGreaterThanOrEqual(8);
});
it('is too long to ship whole, so it renders through the cluster path', () => {
// Past WHOLE_FILE_MAX_LINES (220 for a non-central file): the whole-file
// grace and buy arms cannot claim it, so the envelope actually matters.
expect(sourceLines.length).toBeGreaterThan(220);
expect(report.files.find((f) => f.path === TARGET)?.render).toBe('clusters');
});
});
describe('the gate', () => {
it('delivers the closures the query named, not just the factory head', () => {
const inner = innerClosures();
for (const name of ['refreshMetrics', 'applyFilter']) {
const node = inner.find((n) => n.name === name)!;
expect(node, `${name} is not an inner closure any more`).toBeDefined();
expect(delivered.has(node.startLine), `${name} definition line not delivered`).toBe(true);
}
});
it('delivers most of the closures, spread across the file', () => {
const inner = innerClosures();
const hit = inner.filter((n) => delivered.has(n.startLine));
// Measured on the `feature/CG-24` tip: 7 of 11. The bar is half, so ordinary
// budget movement does not fail the suite, but losing the closures does.
expect(hit.length).toBeGreaterThanOrEqual(Math.ceil(inner.length / 2));
// Not one contiguous head window off the top of the factory: the whole
// point is that selection reaches symbols deep in the body.
const last = inner[inner.length - 1]!;
const deepest = Math.max(...hit.map((n) => n.startLine));
expect(deepest).toBeGreaterThan((last.startLine + inner[0]!.startLine) / 2);
});
it('never renders an empty section for the file', () => {
const rec = report.files.find((f) => f.path === TARGET)!;
expect(rec.emittedChars).toBeGreaterThan(0);
expect(delivered.size).toBeGreaterThan(20);
});
it('keeps the response inside the hard ceiling', () => {
expect(report.envelope.chars).toBeLessThanOrEqual(report.budget.hardCeiling);
});
});
});
__tests__/explore-named-symbol-render.test.ts
+199
View File
@@ -0,0 +1,199 @@
/**
* Standing gate for THE GUARANTEE (task CG-38): if the agent names a symbol and
* that symbol's file is admitted to the response, the symbol's DEFINITION renders.
*
* This is the measurement the CG-24 epic never had. Its probes all score the
* response in aggregate — envelope share, per-file spend, source totals, file
* counts — and every one of them is green on a response that returns 25K of
* source from the right file and still omits the function the agent asked for by
* name. That is what CG-38 was: on a 1,414-line Svelte store, `queueMessage`
* (L1087) and `flushQueuedMessages` (L1102) never rendered even though their file
* won rank #1 with 67% of the envelope; the agent got the same-stem
* `QueuedMessage` INTERFACE at L70 and had to Read the file to find the
* functions. Longstanding, not an epic regression — the controlled bisect (index
* held fixed, engine varied across every epic merge point) found it at every
* build including pre-epic.
*
* Two independent causes, and the fixture below fails on either:
*
* 1. `buildFlowFromNamedSymbols` returned EMPTY — throwing away the NAMED-SYMBOL
* IDENTITY along with the narrative — whenever the named symbols happened not
* to form a call chain. Two sibling closures in one factory produce no chain,
* no synthesized hop and no dispatch boundary, so both defs lost the
* importance-9 rank that the named-def injection exists to give them.
* 2. The ceiling trim cut in SOURCE ORDER, so whatever survived the shrink at
* the END of a large file was always the first thing dropped.
*
* The fixture mirrors the reported file's geometry deliberately: a decoy
* same-stem interface at L70, a factory closure at L104 spanning ~92% of the file
* (so every symbol merges into ONE cluster), the target functions past L1000, and
* a 2,500-line generated `.d.ts` for the ranker to penalise.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
const FIXTURE = 'tail-render-ts';
const TARGET = 'src/lib/session-store.ts';
let dir: string;
let cg: CodeGraph;
/** Every `<n>\t<text>` line number the response actually sent. */
function renderedLines(response: string): Set<number> {
const out = new Set<number>();
for (const m of response.matchAll(/^(\d+)\t/gm)) out.add(Number(m[1]));
return out;
}
async function explore(query: string): Promise<string> {
const res = await new ToolHandler(cg).execute('codegraph_explore', { query });
return res.content?.[0]?.text ?? '';
}
function defLineOf(name: string): number {
const node = cg.getNodesByName(name).find((n) => n.filePath === TARGET && n.startLine > 0);
expect(node, `${name} is not indexed in ${TARGET}`).toBeDefined();
return node!.startLine;
}
beforeAll(async () => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg38-'));
fs.cpSync(path.join(__dirname, 'fixtures', FIXTURE), dir, { recursive: true });
fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
cg = CodeGraph.initSync(dir);
await cg.indexAll();
}, 180_000);
afterAll(() => {
cg?.destroy();
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
});
describe('CG-38 fixture shape — if this rots, the gate below means nothing', () => {
it('puts the target functions past L1000 of a ~1,400-line file', () => {
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
expect(lines.length).toBeGreaterThan(1300);
expect(defLineOf('queueMessage')).toBeGreaterThan(1000);
expect(defLineOf('flushQueuedMessages')).toBeGreaterThan(1000);
});
it('wraps them in a closure spanning most of the file, so they all cluster as one', () => {
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
const factory = cg.getNodesByName('createSessionStore')
.find((n) => n.filePath === TARGET)!;
expect(factory).toBeDefined();
expect(factory.endLine - factory.startLine + 1).toBeGreaterThan(lines.length * 0.5);
});
it('carries the same-stem decoy near the top', () => {
const decoy = cg.getNodesByName('QueuedMessage').find((n) => n.filePath === TARGET)!;
expect(decoy).toBeDefined();
expect(decoy.kind).toBe('interface');
expect(decoy.startLine).toBeLessThan(100);
});
it('carries a generated declaration file for the ranker to penalise', () => {
const dts = path.join(dir, 'types/worker-configuration.d.ts');
expect(fs.existsSync(dts)).toBe(true);
expect(fs.readFileSync(dts, 'utf-8').split('\n').length).toBeGreaterThan(2000);
});
it('neither target calls the other — that absence is what produced no flow', () => {
const queue = cg.getNodesByName('queueMessage').find((n) => n.filePath === TARGET)!;
const flush = cg.getNodesByName('flushQueuedMessages').find((n) => n.filePath === TARGET)!;
const between = [...cg.getCallees(queue.id), ...cg.getCallees(flush.id)]
.filter(({ node }) => node.id === queue.id || node.id === flush.id);
expect(between).toHaveLength(0);
});
});
describe('CG-38 — an agent-named symbol renders its definition', () => {
/**
* Both reported query shapes. They fail for different reasons — the symbol bag
* never built a flow at all, the prose question built one and then lost the
* tail to the ceiling trim — so a fix for one does not imply the other.
*/
const CASES: Array<{ shape: string; query: string; symbols: string[] }> = [
{
shape: 'symbol bag',
query: 'queueMessage flushQueuedMessages',
symbols: ['queueMessage', 'flushQueuedMessages'],
},
{
shape: 'prose question',
query: 'how does queueMessage hand its entries to flushQueuedMessages',
symbols: ['queueMessage', 'flushQueuedMessages'],
},
{
shape: 'three siblings, with the decoy interface competing',
query: 'explain queueMessage, removeQueuedMessage and flushQueuedMessages',
symbols: ['queueMessage', 'removeQueuedMessage', 'flushQueuedMessages'],
},
];
for (const { shape, query, symbols } of CASES) {
it(`renders every named definition — ${shape}`, async () => {
const response = await explore(query);
const lines = renderedLines(response);
for (const name of symbols) {
const line = defLineOf(name);
// The NAME alone proves nothing: it appears in the section header's
// symbol list and at call sites whether or not the body was sent. Only
// the definition LINE being among the rendered lines counts.
expect(lines.has(line), `${name} (${TARGET}:${line}) did not render for "${query}"`)
.toBe(true);
}
}, 120_000);
}
it('never steers the agent to Read', async () => {
const response = await explore('queueMessage flushQueuedMessages');
expect(response).not.toMatch(/\buse Read\b|\bRead the file\b/i);
}, 120_000);
});
describe('CG-38 — a penalty on one file cannot shrink an unrelated file\'s render', () => {
/**
* The issue's sharpest lead: on an index where the generated `.d.ts` was NOT
* flagged, the target file rendered ~581 lines including both symbols; on an
* index where it WAS flagged, the same engine rendered 12. `rankPenalty` scales
* `fileGraphScore`, which moves the relevance gate (6% of max) and so reshuffles
* the admitted set — a demotion of one file must not cost an unrelated
* top-ranked file its source.
*
* Flipping `files.generated` on that one row holds the INDEX constant and
* attributes any delta to the ranker alone (the CG-25 method).
*/
const DTS = 'types/worker-configuration.d.ts';
const QUERY = 'queueMessage flushQueuedMessages';
it('renders the same named definitions with the .d.ts flagged and unflagged', async () => {
const setGenerated = (value: number) => {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const db = (cg as any).db?.getDatabase?.() ?? (cg as any).db?.db;
db.prepare('UPDATE files SET generated = ? WHERE path = ?').run(value, DTS);
};
const linesFor = async () => renderedLines(await explore(QUERY));
const flagged = await linesFor();
setGenerated(0);
try {
const unflagged = await linesFor();
for (const name of ['queueMessage', 'flushQueuedMessages']) {
const line = defLineOf(name);
expect(flagged.has(line), `${name} missing with the .d.ts FLAGGED`).toBe(true);
expect(unflagged.has(line), `${name} missing with the .d.ts UNFLAGGED`).toBe(true);
}
// The guarantee is about the named defs, not byte equality — the penalty is
// supposed to move bytes around. What it must never do is cost the
// top-ranked file the source the agent asked for.
expect(unflagged.size).toBeGreaterThan(0);
} finally {
setGenerated(1);
}
}, 180_000);
});
__tests__/explore-oversize-member.test.ts
+179
View File
@@ -0,0 +1,179 @@
/**
* Regression fixture for CG-30 — a cluster's top member may not overshoot the
* file's budget without bound.
*
* `shrinkCluster` keeps the highest-importance member of an oversize cluster
* WHOLE, deliberately: an empty file section sends the agent to Read, which is
* the outcome explore exists to prevent. What it lacked was a bound. On the
* originating repo one file emitted 22,376 chars against a 9,181-char
* reservation — 2.44x — past both the per-file budget and the spine ceiling,
* because its top member alone was that big. The overshoot is what collapses
* `headroom` for every file ranked below it (CG-31), and it has a second face:
* a member too big for the whole response ceiling makes the file drop out
* entirely rather than render short.
*
* `__tests__/fixtures/oversize-member-ts/` reproduces both permanently. Three
* report builders compete for one envelope, each a single long function far
* bigger than any reservation it can earn beside its siblings. Measured against
* the pre-fix build, this fixture produced:
*
* monthly.ts 12,391 chars emitted on a 3,334 budget (3.7x)
* quarterly.ts dropped entirely — no headroom left (the CG-31 half)
*
* The gate below is that both are now bounded AND delivered: the bound cuts the
* overshoot, and cutting the overshoot is what buys back the starved file.
*
* Measured against `spendable`, not `reserved`: the render paths bound
* themselves by the reservation PLUS whatever slack the files above left on the
* table, so a file legitimately spending inherited slack is not an overshoot.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'oversize-member-ts');
/** A symbol bag spanning the three builders — the sibling files compete. */
const QUERY = 'buildMonthlyReport buildWeeklyReport buildQuarterlyReport formatReportRow persistReport';
/** The giant: one ~24K function, far past the whole-response ceiling. */
const GIANT = 'src/report/monthly.ts';
/** Mid-size: one ~11K function — the file the giant's overshoot used to starve. */
const STARVED = 'src/report/quarterly.ts';
/** The bound: 1.5x, the same multiple the spine ceiling already draws. */
const OVERSHOOT_FACTOR = 1.5;
describe('CG-30 — an oversize cluster member is bounded, not unbounded', () => {
let testDir: string;
let cg: CodeGraph;
let response: string;
let report: ExploreDiagnosticReport;
let bytes: Map<string, number>;
const fileOf = (p: string): ExploreDiagnosticFile => {
const rec = report.files.find((f) => f.path === p);
if (!rec) throw new Error(`${p} absent from the diagnostic report`);
return rec;
};
/** What the render paths actually bound themselves by. */
const budgetOf = (rec: ExploreDiagnosticFile): number => rec.spendable ?? rec.allowance ?? 0;
beforeAll(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg30-'));
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
cg = CodeGraph.initSync(testDir);
await cg.indexAll();
// The per-file budget is only observable through the diagnostic sidecar, and
// the whole gate is "emitted vs what the file was allowed to spend".
const sidecar = path.join(testDir, 'explore-diag.jsonl');
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
try {
const handler = new ToolHandler(cg);
const result = await handler.execute('codegraph_explore', { query: QUERY });
response = result.content?.[0]?.text ?? '';
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
report = JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport;
bytes = attributeSourceBytes(response);
}, 120_000);
afterAll(() => {
if (cg) cg.destroy();
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
// ── Fixture shape — if these rot, the gate below means nothing ─────────────
describe('fixture shape', () => {
it('holds single members far bigger than any budget they can earn', () => {
for (const file of [GIANT, STARVED]) {
const source = fs.readFileSync(path.join(testDir, file), 'utf-8');
const top = cg.getNodesInFile(file)
.filter((n) => n.kind === 'function')
.sort((a, b) => (b.endLine - b.startLine) - (a.endLine - a.startLine))[0];
expect(top, `${file} has no function node`).toBeDefined();
// One symbol, most of the file — the "top member alone is oversize" shape.
expect(top!.endLine - top!.startLine).toBeGreaterThan(180);
expect(source.length).toBeGreaterThan(budgetOf(fileOf(file)) * 2);
}
});
it('is too long to ship whole, so both render through the cluster path', () => {
for (const file of [GIANT, STARVED]) {
const lineCount = fs.readFileSync(path.join(testDir, file), 'utf-8').split('\n').length;
// Past WHOLE_FILE_MAX_LINES (220 for a non-central file), so the
// whole-file paths — grace and buy — cannot claim it.
expect(lineCount, file).toBeGreaterThan(220);
expect(fileOf(file).render, file).toBe('clusters');
}
});
});
// ── The gate ──────────────────────────────────────────────────────────────
describe('bounded overshoot', () => {
it('CG-30 GATE: the giant no longer emits a multiple of its budget', () => {
const rec = fileOf(GIANT);
// Pre-fix this file emitted 12,391 on a 3,334 budget (3.7x).
expect(rec.emittedChars).toBeLessThanOrEqual(
Math.round(budgetOf(rec) * OVERSHOOT_FACTOR) + 1);
});
it('CG-30 GATE: no clustered file emits past 1.5x what it may spend', () => {
const over = report.files
.filter((f) => f.render === 'clusters' && budgetOf(f) > 0)
.filter((f) => f.emittedChars > Math.round(budgetOf(f) * OVERSHOOT_FACTOR) + 1)
.map((f) => `${f.path}: ${f.emittedChars} of ${budgetOf(f)}`);
expect(over).toEqual([]);
});
it('CG-31: the file the overshoot used to starve is delivered', () => {
// Pre-fix: dropped with skip reason `budget-clusters` — the giant above it
// had already spent the headroom this file needed.
expect(fileOf(STARVED).skipped).toBeNull();
expect(bytes.get(STARVED) ?? 0).toBeGreaterThan(0);
});
it('never emits an empty section — the invariant the old rule protected', () => {
for (const rec of report.files) {
if (rec.render !== 'clusters') continue;
expect(rec.emittedChars, rec.path).toBeGreaterThan(0);
}
// And the windowed file still leads with the symbol the query named.
expect(response).toContain('export function buildMonthlyReport');
});
it('cuts on whole lines — a body is never sliced mid-line', () => {
const source = fs.readFileSync(path.join(testDir, GIANT), 'utf-8').split('\n');
const numbered = response
.split('\n')
.map((l) => /^(\d+)\t(.*)$/.exec(l))
.filter((m): m is RegExpExecArray => m !== null)
.filter((m) => Number(m[1]) >= 1 && Number(m[1]) <= source.length);
const matching = numbered.filter((m) => source[Number(m[1]) - 1] === m[2]);
// Every line the response numbers for this file is that whole source line.
expect(matching.length).toBeGreaterThan(20);
});
it('reports the cut rather than presenting a window as the whole file', () => {
expect(fileOf(GIANT).clipped).toBe(true);
});
it('keeps the response inside the hard ceiling', () => {
expect(report.envelope.chars).toBeLessThanOrEqual(report.budget.hardCeiling);
});
});
});
__tests__/explore-reservation-invariant.test.ts
+265
View File
@@ -0,0 +1,265 @@
/**
* Regression fixture for CG-26 — the end-to-end reservation invariant.
*
* Every admitted file receives at least its reservation before any file draws
* on carry-forward slack.
*
* CG-30 bounded how far an oversize cluster member may overshoot and CG-31 gave
* the cluster path a displacement guard. This pins the invariant they jointly
* satisfy across EVERY render path — cluster, whole-file grace, whole-file BUY —
* and in BOTH directions: the top-ranked file when the files below it overspend,
* and an admitted lower-ranked file when the top one does.
*
* Two things CG-26 fixed are pinned here because nothing else can see them:
*
* - The whole-file arms were fit-tested against raw room before the ceiling,
* never against what was still owed below. A grace-sized file could take a
* pending file's reservation on its way to the ceiling; okhttp's
* `CallServerInterceptor.kt` shipped 8,499 chars on a 5,964 funded ceiling
* and the rank-6 file below it delivered nothing.
* - Every section was charged a flat 200 chars of overhead while a real header
* runs 300–500. The loop believed it had room it did not have (okhttp
* rendered 26,601 chars against a 24,400 ceiling), so the final truncation
* threw a fully-rendered section away — the same starvation, arriving after
* the guard had done its work.
*
* Shares the `displacement-ts` fixture: four pipeline stages competing for one
* envelope, the first a single ~20K function, padded past 500 indexed files so
* the response sits on the 24K tier where reservations genuinely saturate the
* ceiling.
*/
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import { attributeSourceBytes } from '../src/mcp/explore-diagnostics';
import type { ExploreDiagnosticReport, ExploreDiagnosticFile } from '../src/mcp/explore-diagnostics';
const FIXTURE_SRC = path.join(__dirname, 'fixtures', 'displacement-ts');
const FILLER_FILES = 520;
/** The giant: one ~20K function. Ranks #1 under the spread query. */
const GIANT = 'src/pipeline/ingest.ts';
/**
* Three shapes, so the invariant is tested from both sides:
* spread — every stage named; the giant ranks #1 and overspends downwards.
* tail — the stages BELOW the giant named; something small ranks #1 while
* the giant competes from underneath. This is the direction CG-31's
* fixture could not reach.
* precise — one symbol. The concentration case the guard must not flatten.
*/
const QUERIES = {
spread: 'ingestRecords normalizeRecords enrichRecords publishRecords',
tail: 'publishRecords sinkRecord PipelineRecord ingestRecords',
precise: 'ingestRecords',
} as const;
type Shape = keyof typeof QUERIES;
interface Probe {
response: string;
report: ExploreDiagnosticReport;
bytes: Map<string, number>;
}
describe('CG-26 — no admitted file is starved, on any render path', () => {
let testDir: string;
let cg: CodeGraph;
const probes = {} as Record<Shape, Probe>;
/** Admitted = the allocator reserved bytes for it. */
const admitted = (probe: Probe): ExploreDiagnosticFile[] =>
probe.report.files.filter((f) => (f.allowance ?? 0) > 0);
const all = (): Probe[] => Object.values(probes);
beforeAll(async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg26-'));
fs.cpSync(FIXTURE_SRC, testDir, { recursive: true });
fs.rmSync(path.join(testDir, '.codegraph'), { recursive: true, force: true });
const filler = path.join(testDir, 'src', 'generated');
fs.mkdirSync(filler, { recursive: true });
for (let i = 0; i < FILLER_FILES; i++) {
fs.writeFileSync(
path.join(filler, `unit${i}.ts`),
`export const seed${i} = ${i};\n`
+ `export function widget${i}(n: number): number {\n return n * ${i + 1} + seed${i};\n}\n`,
);
}
cg = CodeGraph.initSync(testDir);
await cg.indexAll();
const sidecar = path.join(testDir, 'explore-diag.jsonl');
const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
try {
const handler = new ToolHandler(cg);
for (const [shape, query] of Object.entries(QUERIES) as [Shape, string][]) {
const result = await handler.execute('codegraph_explore', { query });
const response = result.content?.[0]?.text ?? '';
const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
probes[shape] = {
response,
report: JSON.parse(written[written.length - 1]!) as ExploreDiagnosticReport,
bytes: attributeSourceBytes(response),
};
}
} finally {
if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
}
}, 180_000);
afterAll(() => {
if (cg) cg.destroy();
if (testDir && fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
// ── Fixture shape — if these rot, the gates below mean nothing ─────────────
describe('fixture shape', () => {
it('sits on the 24K tier, where the reservations saturate the ceiling', () => {
expect(cg.getStats().fileCount).toBeGreaterThanOrEqual(500);
for (const probe of all()) expect(probe.report.budget.maxOutputChars).toBe(24000);
});
it('exercises both directions — the giant ranks #1 in one shape and lower in another', () => {
// Which shape puts it where is the ranker's business and may move; that
// it lands on BOTH sides across the three is what makes the gates below
// test the invariant rather than one arrangement of it.
const ranks = all().map((p) => p.report.files.find((f) => f.path === GIANT)?.rank ?? -1);
expect(ranks).toContain(1);
expect(ranks.some((r) => r > 1)).toBe(true);
});
it('exercises both render paths — something ships whole, something clusters', () => {
const modes = new Set(all().flatMap((p) => p.report.files.map((f) => f.render)));
expect(modes).toContain('clusters');
expect(modes).toContain('whole');
});
});
// ── The invariant ─────────────────────────────────────────────────────────
describe('the reservation invariant', () => {
it('CG-26 GATE: no file on ANY render path emits past what was still free', () => {
// CG-31 pinned this for `clusters` only. The whole-file arms were fit-
// tested against `renderCeiling - totalChars`, which is everyone's room,
// not this file's — so a whole render could spend a reservation the loop
// had already promised further down.
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
const over = probe.report.files
.filter((f) => f.render !== null && f.render !== 'dropped' && f.funded !== null)
// +1 for the render loop's own rounding on a windowed cut.
.filter((f) => f.emittedChars > f.funded! + 1)
.map((f) => `${shape}/${f.path}: ${f.emittedChars} emitted of ${f.funded} funded (${f.render})`);
expect(over).toEqual([]);
}
});
it('CG-26 GATE: every admitted file is delivered, whatever its rank', () => {
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
for (const rec of admitted(probe)) {
expect(rec.skipped, `${shape}/${rec.path} skipped`).toBeNull();
expect(probe.bytes.get(rec.path) ?? 0, `${shape}/${rec.path} bytes`).toBeGreaterThan(0);
}
}
});
it('CG-26 GATE: the rank-#1 file gets its reservation even when a file below overspends', () => {
// The direction CG-31's fixture could not reach: under `tail` the giant
// ranks below a small file and draws far past its own reservation from
// carry-forward slack. Rank #1 must still receive what it was promised
// (or its whole file, if that is less).
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
const top = admitted(probe).sort((a, b) => a.rank - b.rank)[0];
if (!top) continue;
const onDisk = fs.statSync(path.join(testDir, top.path)).size;
expect(probe.bytes.get(top.path) ?? 0, `${shape}/${top.path}`)
.toBeGreaterThanOrEqual(Math.min(top.allowance!, onDisk) * 0.9);
}
});
it('and the gate above is not vacuous — a lower-ranked file does overspend', () => {
const overspenders = (probe: Probe) => admitted(probe)
.filter((f) => f.rank > 1 && f.emittedChars > f.allowance!);
expect(overspenders(probes.tail).length).toBeGreaterThan(0);
});
});
// ── What the ceiling must no longer do ────────────────────────────────────
describe('the hard ceiling never throws a rendered section away', () => {
it('the render loop spends what it counts — nothing is allocated past the ceiling', () => {
// Sections used to be charged a flat 200 chars against a header that runs
// 300–500, so the loop over-filled and the final truncation dropped whole
// sections. `allocatedChars` is the pre-truncation length: it staying
// under the ceiling IS the accounting being exact.
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
expect(probe.report.envelope.allocatedChars, shape)
.toBeLessThanOrEqual(probe.report.budget.hardCeiling);
expect(probe.report.envelope.truncated, shape).toBe(false);
}
});
it('no file is rendered and then dropped', () => {
for (const probe of all()) {
expect(probe.report.files.filter((f) => f.render === 'dropped')).toEqual([]);
}
});
it('keeps the response inside the hard ceiling', () => {
for (const probe of all()) {
expect(probe.report.envelope.chars).toBeLessThanOrEqual(probe.report.budget.hardCeiling);
}
});
});
// ── The epilogue is budgeted, not discarded ───────────────────────────────
describe('the epilogue the loop budgeted for is the epilogue it emits', () => {
it('a response that withheld files still says so, and says to explore not Read', () => {
// The flat 600-char margin was neither the epilogue's size nor a bound on
// it, so a saturated response shipped with no pointer list and no
// reminders at all. Whatever else is traded away, the agent must be told
// an uncovered area exists and that another explore reaches it.
for (const [shape, probe] of Object.entries(probes) as [Shape, Probe][]) {
const withheld = probe.report.files.some(
(f) => f.render === null || (probe.bytes.get(f.path) ?? 0) === 0);
if (!withheld) continue;
expect(
/Not shown above|omitted for size|codegraph_explore/.test(probe.response),
`${shape} withheld files without saying where to look`,
).toBe(true);
}
});
it('never steers the agent to Read', () => {
for (const probe of all()) {
expect(/use (the )?Read|fall back to Read(?!ing those files)/i.test(probe.response)).toBe(false);
}
});
});
// ── The thing the invariant must NOT become ───────────────────────────────
describe('concentration survives', () => {
it('a precise symbol query still puts the most source in the named file', () => {
const mine = probes.precise.bytes.get(GIANT) ?? 0;
expect(mine).toBeGreaterThan(0);
for (const [p, n] of probes.precise.bytes) {
if (p === GIANT) continue;
expect(mine, `${GIANT} vs ${p}`).toBeGreaterThan(n);
}
});
it('is not an even split — the named file outspends its equal share', () => {
const rec = probes.precise.report.files.find((f) => f.path === GIANT)!;
const even = probes.precise.report.budget.maxOutputChars / admitted(probes.precise).length;
expect(rec.emittedChars).toBeGreaterThan(even);
});
});
});
__tests__/fixtures/ambient-decls-ts/package.json
+7
View File
@@ -0,0 +1,7 @@
{
"name": "ambient-decls-ts-fixture",
"private": true,
"version": "0.0.0",
"type": "module",
"description": "CG-28 fixture — declaration-only files competing with implementation for one explore envelope."
}
__tests__/fixtures/ambient-decls-ts/src/lib/bucket.ts
+46
View File
@@ -0,0 +1,46 @@
export interface BucketObject {
key: string;
body: ReadableStream<Uint8Array>;
size: number;
}
export interface Bucket {
put(
key: string,
value: ReadableStream<Uint8Array>,
options?: { httpMetadata?: { contentType?: string } },
): Promise<void>;
get(key: string): Promise<BucketObject | null>;
}
export interface MetadataStore {
put(id: string, value: string): Promise<void>;
get(id: string): Promise<string | null>;
}
const objects = new Map<string, BucketObject>();
const rows = new Map<string, string>();
/** The object-storage binding. */
export function openBucket(): Bucket {
return {
async put(key, value) {
objects.set(key, { key, body: value, size: 0 });
},
async get(key) {
return objects.get(key) ?? null;
},
};
}
/** The metadata key-value binding. */
export function openMetadataStore(): MetadataStore {
return {
async put(id, value) {
rows.set(id, value);
},
async get(id) {
return rows.get(id) ?? null;
},
};
}
__tests__/fixtures/ambient-decls-ts/src/lib/queue.ts
+37
View File
@@ -0,0 +1,37 @@
export interface UploadMessageBody {
key: string;
metadataId: string;
contentType: string;
}
/**
* Publish the follow-up message for a stored upload. Batched so a burst of
* uploads does not open one producer call per object.
*/
export async function enqueueUploadMessage(body: UploadMessageBody): Promise<void> {
const queue = openUploadQueue();
await queue.send(body, { contentType: 'json' });
}
/** Consumer side: process a batch of upload messages. */
export async function consumeUploadBatch(messages: UploadMessageBody[]): Promise<number> {
let handled = 0;
for (const message of messages) {
if (!message.key) continue;
handled += 1;
}
return handled;
}
interface UploadQueue {
send(body: UploadMessageBody, options: { contentType: string }): Promise<void>;
}
/** The binding lookup, isolated so tests can swap it. */
export function openUploadQueue(): UploadQueue {
return {
async send() {
/* binding provided by the runtime */
},
};
}
__tests__/fixtures/ambient-decls-ts/src/lib/request.ts
+44
View File
@@ -0,0 +1,44 @@
export interface ParsedUpload {
ok: true;
key: string;
body: ReadableStream<Uint8Array>;
contentType: string;
width: number;
height: number;
format: string;
}
export interface ParseFailure {
ok: false;
error: string;
}
/**
* Pull the object key, declared dimensions and the raw body stream off an
* upload request. Never buffers the body — the stream is handed straight to
* the storage layer.
*/
export async function parseUploadRequest(
request: Request,
): Promise<ParsedUpload | ParseFailure> {
const url = new URL(request.url);
const key = url.searchParams.get('key');
if (!key) return { ok: false, error: 'missing key' };
if (!request.body) return { ok: false, error: 'missing body' };
return {
ok: true,
key,
body: request.body as ReadableStream<Uint8Array>,
contentType: request.headers.get('content-type') ?? 'application/octet-stream',
width: numberParam(url, 'width'),
height: numberParam(url, 'height'),
format: url.searchParams.get('format') ?? 'jpeg',
};
}
function numberParam(url: URL, name: string): number {
const raw = url.searchParams.get(name);
const parsed = raw ? Number.parseInt(raw, 10) : 0;
return Number.isFinite(parsed) ? parsed : 0;
}
__tests__/fixtures/ambient-decls-ts/src/routes/upload.ts
+66
View File
@@ -0,0 +1,66 @@
import { streamBodyToStorage } from '../storage/stream.js';
import { recordImageMetadata } from '../storage/metadata.js';
import { enqueueUploadMessage } from '../lib/queue.js';
import { parseUploadRequest } from '../lib/request.js';
export interface UploadResult {
key: string;
bytes: number;
contentType: string;
}
/**
* Entry point for an upload request: parse it, stream the body into object
* storage, record the image metadata, then queue the follow-up work.
*/
export async function handleUploadRequest(request: Request): Promise<Response> {
const parsed = await parseUploadRequest(request);
if (!parsed.ok) {
return new Response(JSON.stringify({ error: parsed.error }), { status: 400 });
}
const stored = await streamBodyToStorage(parsed.body, parsed.key, parsed.contentType);
const metadata = await recordImageMetadata(stored.key, {
width: parsed.width,
height: parsed.height,
format: parsed.format,
bytes: stored.bytes,
});
await enqueueUploadMessage({
key: stored.key,
metadataId: metadata.id,
contentType: stored.contentType,
});
return new Response(JSON.stringify(summarizeUpload(stored, metadata.id)), {
status: 201,
headers: { 'content-type': 'application/json' },
});
}
/** Shape the client sees back after a successful upload. */
export function summarizeUpload(stored: UploadResult, metadataId: string) {
return {
key: stored.key,
bytes: stored.bytes,
contentType: stored.contentType,
metadataId,
};
}
/** Reject uploads whose declared size exceeds the per-account ceiling. */
export function isWithinUploadLimit(bytes: number, limit: number): boolean {
if (!Number.isFinite(bytes) || bytes < 0) return false;
return bytes <= limit;
}
/** Delete-side counterpart, kept here so the route module is not a one-liner. */
export async function handleDeleteRequest(request: Request, key: string): Promise<Response> {
const parsed = await parseUploadRequest(request);
if (!parsed.ok) {
return new Response(JSON.stringify({ error: parsed.error }), { status: 400 });
}
await enqueueUploadMessage({ key, metadataId: '', contentType: 'application/x-delete' });
return new Response(null, { status: 204 });
}
__tests__/fixtures/ambient-decls-ts/src/storage/metadata.ts
+54
View File
@@ -0,0 +1,54 @@
import { openMetadataStore } from '../lib/bucket.js';
export interface ImageMetadataInput {
width: number;
height: number;
format: string;
bytes: number;
}
export interface ImageMetadataRecord extends ImageMetadataInput {
id: string;
key: string;
recordedAt: number;
}
/**
* Record the image metadata for a stored object. Writes go to the metadata
* store keyed by object key; the returned record carries the id the queue
* message references.
*/
export async function recordImageMetadata(
key: string,
input: ImageMetadataInput,
): Promise<ImageMetadataRecord> {
const store = openMetadataStore();
const record: ImageMetadataRecord = {
...input,
id: metadataIdFor(key, input),
key,
recordedAt: 0,
};
await store.put(record.id, JSON.stringify(record));
return record;
}
/** Deterministic id so a retried upload records the same metadata row. */
export function metadataIdFor(key: string, input: ImageMetadataInput): string {
return `${key}:${input.format}:${input.width}x${input.height}`;
}
/** Read a metadata record back for the download and listing paths. */
export async function loadImageMetadata(id: string): Promise<ImageMetadataRecord | null> {
const store = openMetadataStore();
const raw = await store.get(id);
return raw ? (JSON.parse(raw) as ImageMetadataRecord) : null;
}
/** Normalize a client-declared format string to the canonical set. */
export function normalizeFormat(format: string): string {
const lowered = format.trim().toLowerCase();
if (lowered === 'jpg') return 'jpeg';
if (lowered === 'tif') return 'tiff';
return lowered;
}
__tests__/fixtures/ambient-decls-ts/src/storage/stream.ts
+86
View File
@@ -0,0 +1,86 @@
import { openBucket } from '../lib/bucket.js';
import type { StorageFailure, UploadTelemetry } from './types.js';
export interface StoredObject {
key: string;
bytes: number;
contentType: string;
}
/**
* Stream a request body into object storage without buffering it in memory.
* The body is piped through a counting transform so the byte total is known
* by the time the put resolves.
*/
export async function streamBodyToStorage(
body: ReadableStream<Uint8Array>,
key: string,
contentType: string,
): Promise<StoredObject> {
const bucket = openBucket();
const counter = createByteCounter();
const piped = body.pipeThrough(counter.transform, { preventClose: false });
await bucket.put(key, piped, { httpMetadata: { contentType } });
return { key, bytes: counter.total(), contentType };
}
/**
* A transform stream that counts the bytes flowing through it. Separated from
* the pipe above so the byte total can be read after the stream settles.
*/
export function createByteCounter() {
let total = 0;
const transform = new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
total += chunk.byteLength;
controller.enqueue(chunk);
},
});
return { transform, total: () => total };
}
/**
* Read a stored object back out of the bucket as a stream, for the download
* path. Mirrors the upload side so both directions live in one module.
*/
export async function readObjectStream(key: string): Promise<ReadableStream<Uint8Array> | null> {
const bucket = openBucket();
const object = await bucket.get(key);
if (!object) return null;
return object.body;
}
/** Timing/retry record for one stored object, handed to the metrics sink. */
export function telemetryFor(stored: StoredObject, durationMs: number): UploadTelemetry {
return { key: stored.key, bytes: stored.bytes, durationMs, retries: 0 };
}
/** Describe a failed stage so the caller can report it without re-deriving it. */
export function storageFailure(
key: string,
stage: StorageFailure['stage'],
message: string,
): StorageFailure {
return { key, stage, message };
}
/** Cap a stream at `limit` bytes, erroring out rather than storing an overrun. */
export function limitStream(
source: ReadableStream<Uint8Array>,
limit: number,
): ReadableStream<Uint8Array> {
let seen = 0;
const guard = new TransformStream<Uint8Array, Uint8Array>({
transform(chunk, controller) {
seen += chunk.byteLength;
if (seen > limit) {
controller.error(new Error(`upload exceeded ${limit} bytes`));
return;
}
controller.enqueue(chunk);
},
});
return source.pipeThrough(guard);
}
__tests__/fixtures/ambient-decls-ts/src/storage/types.ts
+18
View File
@@ -0,0 +1,18 @@
/**
* Shared shapes for the storage layer. Declaration-only like the ambient files
* under `types/` — but the modules that answer a flow question are typed BY it,
* so it is part of that answer's structure rather than a global shim.
*/
export interface UploadTelemetry {
key: string;
bytes: number;
durationMs: number;
retries: number;
}
export interface StorageFailure {
key: string;
stage: 'parse' | 'stream' | 'metadata' | 'queue';
message: string;
}
__tests__/fixtures/ambient-decls-ts/types/platform-shims.d.ts
Vendored
+212
View File
@@ -0,0 +1,212 @@
// Hand-maintained ambient declarations for the parts of the platform our
// runtime exposes but the published typings do not cover yet. Edit freely —
// nothing regenerates this file. Kept alongside the app so module augmentation
// and the global shims live in one place.
declare global {
interface UploadStorage {
put(
key: string,
body: ReadableStream<Uint8Array>,
options?: UploadPutOptions,
): Promise<StoredUploadObject>;
get(key: string): Promise<StoredUploadObject | null>;
head(key: string): Promise<StoredUploadHead | null>;
delete(key: string | string[]): Promise<void>;
list(options?: UploadListOptions): Promise<UploadListResult>;
}
interface StoredUploadObject {
readonly key: string;
readonly size: number;
readonly etag: string;
readonly uploaded: Date;
readonly body: ReadableStream<Uint8Array>;
readonly contentType: string;
readonly metadata?: ImageMetadataShim;
arrayBuffer(): Promise<ArrayBuffer>;
text(): Promise<string>;
json<T>(): Promise<T>;
}
interface StoredUploadHead {
readonly key: string;
readonly size: number;
readonly etag: string;
readonly uploaded: Date;
readonly contentType: string;
}
interface UploadPutOptions {
contentType?: string;
cacheControl?: string;
customMetadata?: Record<string, string>;
checksum?: string;
storageClass?: 'standard' | 'infrequent';
}
interface UploadListOptions {
prefix?: string;
cursor?: string;
limit?: number;
delimiter?: string;
include?: ('metadata' | 'contentType')[];
}
interface UploadListResult {
objects: StoredUploadHead[];
truncated: boolean;
cursor?: string;
prefixes: string[];
}
interface ImageMetadataShim {
format: string;
fileSize: number;
width: number;
height: number;
orientation?: number;
colorSpace?: string;
}
interface MetadataRowShim {
id: string;
key: string;
recordedAt: number;
format: string;
bytes: number;
width: number;
height: number;
}
interface MetadataStoreShim {
put(id: string, value: string, options?: MetadataPutOptions): Promise<void>;
get(id: string): Promise<string | null>;
getWithMetadata<T>(id: string): Promise<{ value: string | null; metadata: T | null }>;
delete(id: string): Promise<void>;
list(options?: MetadataListOptions): Promise<MetadataListResult>;
}
interface MetadataPutOptions {
expiration?: number;
expirationTtl?: number;
metadata?: unknown;
}
interface MetadataListOptions {
prefix?: string | null;
cursor?: string | null;
limit?: number;
}
interface MetadataListResult {
keys: { name: string; expiration?: number }[];
list_complete: boolean;
cursor?: string;
}
interface UploadQueueShim<Body = unknown> {
send(body: Body, options?: UploadSendOptions): Promise<void>;
sendBatch(bodies: Iterable<UploadSendRequest<Body>>): Promise<void>;
}
interface UploadSendOptions {
contentType?: UploadContentType;
delaySeconds?: number;
}
type UploadContentType = 'text' | 'bytes' | 'json' | 'v8';
interface UploadSendRequest<Body = unknown> {
body: Body;
options?: UploadSendOptions;
}
interface UploadMessageShim<Body = unknown> {
readonly id: string;
readonly timestamp: Date;
readonly body: Body;
readonly attempts: number;
retry(options?: UploadRetryOptions): void;
ack(): void;
}
interface UploadRetryOptions {
delaySeconds?: number;
}
interface UploadMessageBatch<Body = unknown> {
readonly messages: readonly UploadMessageShim<Body>[];
readonly queue: string;
retryAll(options?: UploadRetryOptions): void;
ackAll(): void;
}
interface StreamPipeOptionsShim {
preventClose?: boolean;
preventAbort?: boolean;
preventCancel?: boolean;
signal?: AbortSignal;
}
interface ByteCounterShim {
readonly transform: TransformStream<Uint8Array, Uint8Array>;
total(): number;
}
interface StreamLimitShim {
readonly limit: number;
readonly seen: number;
exceeded(): boolean;
}
interface RequestBodyShim {
readonly body: ReadableStream<Uint8Array> | null;
readonly bodyUsed: boolean;
readonly headers: Headers;
readonly url: string;
arrayBuffer(): Promise<ArrayBuffer>;
formData(): Promise<FormData>;
blob(): Promise<Blob>;
}
interface ParsedUploadShim {
key: string;
contentType: string;
width: number;
height: number;
format: string;
}
interface ImageTransformerShim {
transform(transform: ImageTransformShim): ImageTransformerShim;
output(options: ImageOutputShim): Promise<ImageResultShim>;
}
interface ImageTransformShim {
width?: number;
height?: number;
fit?: 'scale-down' | 'contain' | 'cover' | 'crop' | 'pad';
rotate?: number;
}
interface ImageOutputShim {
format?: string;
quality?: number;
background?: string;
}
interface ImageResultShim {
contentType(): string;
image(): ReadableStream<Uint8Array>;
response(): Response;
}
interface UploadEnvShim {
UPLOADS: UploadStorage;
METADATA: MetadataStoreShim;
UPLOAD_QUEUE: UploadQueueShim<unknown>;
}
}
export {};
__tests__/fixtures/ambient-decls-ts/types/worker-configuration.d.ts
Vendored
+271
View File
@@ -0,0 +1,271 @@
// Generated by Wrangler by running `wrangler types` (hash: 4f1c8ad2b90e)
// Runtime types generated with workerd@1.20260701.0 2026-07-01 nodejs_compat
declare namespace Cloudflare {
interface Env {
UPLOADS: R2Bucket;
METADATA: KVNamespace;
UPLOAD_QUEUE: Queue<UploadMessageBody>;
IMAGES: ImagesBinding;
}
}
interface UploadMessageBody {
key: string;
metadataId: string;
contentType: string;
}
interface R2Bucket {
head(key: string): Promise<R2Object | null>;
get(key: string, options?: R2GetOptions): Promise<R2ObjectBody | null>;
put(
key: string,
value: ReadableStream | ArrayBuffer | string | null,
options?: R2PutOptions,
): Promise<R2Object>;
delete(keys: string | string[]): Promise<void>;
list(options?: R2ListOptions): Promise<R2Objects>;
createMultipartUpload(key: string, options?: R2MultipartOptions): Promise<R2MultipartUpload>;
resumeMultipartUpload(key: string, uploadId: string): R2MultipartUpload;
}
interface R2Object {
readonly key: string;
readonly version: string;
readonly size: number;
readonly etag: string;
readonly httpEtag: string;
readonly checksums: R2Checksums;
readonly uploaded: Date;
readonly httpMetadata?: R2HTTPMetadata;
readonly customMetadata?: Record<string, string>;
readonly range?: R2Range;
readonly storageClass: string;
writeHttpMetadata(headers: Headers): void;
}
interface R2ObjectBody extends R2Object {
get body(): ReadableStream;
get bodyUsed(): boolean;
arrayBuffer(): Promise<ArrayBuffer>;
text(): Promise<string>;
json<T>(): Promise<T>;
blob(): Promise<Blob>;
bytes(): Promise<Uint8Array>;
}
interface R2GetOptions {
onlyIf?: R2Conditional | Headers;
range?: R2Range;
ssecKey?: ArrayBuffer | string;
}
interface R2PutOptions {
onlyIf?: R2Conditional | Headers;
httpMetadata?: R2HTTPMetadata | Headers;
customMetadata?: Record<string, string>;
md5?: ArrayBuffer | string;
sha1?: ArrayBuffer | string;
sha256?: ArrayBuffer | string;
storageClass?: string;
ssecKey?: ArrayBuffer | string;
}
interface R2ListOptions {
limit?: number;
prefix?: string;
cursor?: string;
delimiter?: string;
startAfter?: string;
include?: ('httpMetadata' | 'customMetadata')[];
}
interface R2Objects {
objects: R2Object[];
truncated: boolean;
cursor?: string;
delimitedPrefixes: string[];
}
interface R2MultipartOptions {
httpMetadata?: R2HTTPMetadata | Headers;
customMetadata?: Record<string, string>;
storageClass?: string;
}
interface R2MultipartUpload {
readonly key: string;
readonly uploadId: string;
uploadPart(
partNumber: number,
value: ReadableStream | ArrayBuffer | string | Blob,
): Promise<R2UploadedPart>;
abort(): Promise<void>;
complete(uploadedParts: R2UploadedPart[]): Promise<R2Object>;
}
interface R2UploadedPart {
partNumber: number;
etag: string;
}
interface R2HTTPMetadata {
contentType?: string;
contentLanguage?: string;
contentDisposition?: string;
contentEncoding?: string;
cacheControl?: string;
cacheExpiry?: Date;
}
interface R2Checksums {
readonly md5?: ArrayBuffer;
readonly sha1?: ArrayBuffer;
readonly sha256?: ArrayBuffer;
toJSON(): R2StringChecksums;
}
interface R2StringChecksums {
md5?: string;
sha1?: string;
sha256?: string;
}
interface R2Conditional {
etagMatches?: string;
etagDoesNotMatch?: string;
uploadedBefore?: Date;
uploadedAfter?: Date;
secondsGranularity?: boolean;
}
interface R2Range {
offset?: number;
length?: number;
suffix?: number;
}
interface KVNamespace<Key extends string = string> {
get(key: Key, options?: Partial<KVNamespaceGetOptions<undefined>>): Promise<string | null>;
getWithMetadata<Metadata = unknown>(
key: Key,
options?: Partial<KVNamespaceGetOptions<undefined>>,
): Promise<KVNamespaceGetWithMetadataResult<string, Metadata>>;
put(
key: Key,
value: string | ArrayBuffer | ArrayBufferView | ReadableStream,
options?: KVNamespacePutOptions,
): Promise<void>;
delete(key: Key): Promise<void>;
list<Metadata = unknown>(
options?: KVNamespaceListOptions,
): Promise<KVNamespaceListResult<Metadata, Key>>;
}
interface KVNamespaceGetOptions<Type> {
type: Type;
cacheTtl?: number;
}
interface KVNamespacePutOptions {
expiration?: number;
expirationTtl?: number;
metadata?: unknown | null;
}
interface KVNamespaceListOptions {
limit?: number;
prefix?: string | null;
cursor?: string | null;
}
interface KVNamespaceListResult<Metadata, Key extends string = string> {
keys: KVNamespaceListKey<Metadata, Key>[];
list_complete: boolean;
cursor?: string;
}
interface KVNamespaceListKey<Metadata, Key extends string = string> {
name: Key;
expiration?: number;
metadata?: Metadata;
}
interface KVNamespaceGetWithMetadataResult<Value, Metadata> {
value: Value | null;
metadata: Metadata | null;
cacheStatus: string | null;
}
interface Queue<Body = unknown> {
send(message: Body, options?: QueueSendOptions): Promise<void>;
sendBatch(messages: Iterable<MessageSendRequest<Body>>): Promise<void>;
}
interface QueueSendOptions {
contentType?: QueueContentType;
delaySeconds?: number;
}
type QueueContentType = 'text' | 'bytes' | 'json' | 'v8';
interface MessageSendRequest<Body = unknown> {
body: Body;
options?: QueueSendOptions;
}
interface Message<Body = unknown> {
readonly id: string;
readonly timestamp: Date;
readonly body: Body;
readonly attempts: number;
retry(options?: QueueRetryOptions): void;
ack(): void;
}
interface QueueRetryOptions {
delaySeconds?: number;
}
interface MessageBatch<Body = unknown> {
readonly messages: readonly Message<Body>[];
readonly queue: string;
retryAll(options?: QueueRetryOptions): void;
ackAll(): void;
}
interface ImagesBinding {
info(stream: ReadableStream<Uint8Array>): Promise<ImageMetadata>;
input(stream: ReadableStream<Uint8Array>): ImageTransformer;
}
interface ImageMetadata {
format: string;
fileSize: number;
width: number;
height: number;
}
interface ImageTransformer {
transform(transform: ImageTransform): ImageTransformer;
output(options: ImageOutputOptions): Promise<ImageTransformationResult>;
}
interface ImageTransform {
width?: number;
height?: number;
fit?: 'scale-down' | 'contain' | 'cover' | 'crop' | 'pad';
rotate?: number;
}
interface ImageOutputOptions {
format?: string;
quality?: number;
background?: string;
}
interface ImageTransformationResult {
contentType(): string;
image(): ReadableStream<Uint8Array>;
response(): Response;
}
__tests__/fixtures/dense-header-ts/package.json
+6
View File
@@ -0,0 +1,6 @@
{
"name": "dense-header-fixture",
"private": true,
"version": "0.0.0",
"type": "module"
}
__tests__/fixtures/dense-header-ts/src/core/queue.ts
+23
View File
@@ -0,0 +1,23 @@
import type { URLSessionTask } from './types';
export class RequestQueue {
private readonly waiting: URLSessionTask[] = [];
private running = 0;
enqueue(task: URLSessionTask, limit: number): void {
if (this.running < limit) {
this.running += 1;
return;
}
this.waiting.push(task);
}
release(): URLSessionTask | undefined {
this.running = Math.max(0, this.running - 1);
return this.waiting.shift();
}
get depth(): number {
return this.waiting.length;
}
}
__tests__/fixtures/dense-header-ts/src/core/request-builder.ts
+27
View File
@@ -0,0 +1,27 @@
import type { CachePolicy, URLRequest } from './types';
export function buildURLRequest(options: {
url: string;
method: string;
body?: Uint8Array;
headers: Record<string, string>;
timeout: number;
cachePolicy: CachePolicy;
}): URLRequest {
const headers = { ...options.headers };
if (options.body && !headers['content-length']) {
headers['content-length'] = String(options.body.length);
}
return {
url: normalize(options.url),
method: options.method.toUpperCase(),
headers,
body: options.body,
timeout: options.timeout,
cachePolicy: options.cachePolicy,
};
}
function normalize(url: string): string {
return url.endsWith('/') && url.split('/').length > 4 ? url.slice(0, -1) : url;
}
__tests__/fixtures/dense-header-ts/src/core/task-factory.ts
+23
View File
@@ -0,0 +1,23 @@
import type { RequestDelegate, TaskResponse, URLRequest, URLSessionTask } from './types';
export function makeTask(options: {
identifier: number;
request: URLRequest;
delegate: RequestDelegate;
allowsCellularAccess: boolean;
waitsForConnectivity: boolean;
resourceTimeout: number;
}): URLSessionTask {
const handlers: Array<(response: TaskResponse) => void> = [];
return {
identifier: options.identifier,
request: options.request,
state: 'initialized',
cancel() { this.state = 'cancelled'; },
onComplete(handler) { handlers.push(handler); },
};
}
export function resumeTask(task: URLSessionTask): void {
task.state = 'resumed';
}
__tests__/fixtures/dense-header-ts/src/core/types.ts
+42
View File
@@ -0,0 +1,42 @@
export type CachePolicy = 'useProtocolCachePolicy' | 'reloadIgnoringLocalCacheData' | 'returnCacheDataElseLoad';
export type RequestState = 'initialized' | 'resumed' | 'suspended' | 'cancelled' | 'finished';
export interface URLRequest {
url: string;
method: string;
headers: Record<string, string>;
body?: Uint8Array;
timeout: number;
cachePolicy: CachePolicy;
}
export interface TaskResponse {
status: number;
headers: Record<string, string>;
body: Uint8Array;
}
export interface URLSessionTask {
identifier: number;
request: URLRequest;
state: RequestState;
cancel(): void;
onComplete(handler: (response: TaskResponse) => void): void;
}
export interface Adapter { adapt(request: URLRequest): URLRequest; }
export interface Serializer { serialize(value: unknown): Uint8Array; }
export interface Validator { validate(response: TaskResponse): { ok: boolean; reason?: string }; }
export interface Retrier { shouldRetry(response: TaskResponse, verdict: { ok: boolean }): boolean; }
export interface RedirectHandler { resolve(location: string, original: URLRequest): { url: string; method: string; body?: Uint8Array } | null; }
export interface TrustEvaluator { evaluate(host: string): boolean; }
export interface Credential { apply(request: URLRequest): URLRequest; }
export interface Interceptor { name: string; adapt(request: URLRequest, session: unknown): Promise<URLRequest>; }
export interface RequestDelegate { willSend(request: URLRequest): void; }
export interface EventMonitor {
didAdaptRequest(request: URLRequest, interceptor: string): void;
didCreateTask(task: URLSessionTask, request: URLRequest): void;
didResumeTask(task: URLSessionTask): void;
didRetryTask(task: URLSessionTask, previousIdentifier: number): void;
didCompleteTask(task: URLSessionTask, response: TaskResponse): void;
}
__tests__/fixtures/dense-header-ts/src/index.ts
+3
View File
@@ -0,0 +1,3 @@
export { Session } from './net/session';
export { RequestQueue } from './core/queue';
export { buildURLRequest } from './core/request-builder';
__tests__/fixtures/dense-header-ts/src/net/session.ts
+285
View File
@@ -0,0 +1,285 @@
import type {
Adapter,
CachePolicy,
Credential,
EventMonitor,
Interceptor,
RedirectHandler,
RequestDelegate,
RequestState,
Retrier,
Serializer,
TrustEvaluator,
URLRequest,
URLSessionTask,
Validator,
} from '../core/types';
import { buildURLRequest } from '../core/request-builder';
import { makeTask, resumeTask } from '../core/task-factory';
import { RequestQueue } from '../core/queue';
/**
* The shape density-first ranking exists for: a class whose top-of-file header
* is a long, tightly-packed property list — dozens of adjacent declarations,
* each individually trivial — while the methods a flow question actually asks
* about live hundreds of lines below it.
*
* Ranked by density alone the header wins the file's whole budget and the
* methods are buried. The ranking puts importance first for exactly this
* reason, and density only breaks ties inside one importance tier.
*/
export class Session {
readonly identifier: string;
readonly adapter: Adapter;
readonly serializer: Serializer;
readonly validator: Validator;
readonly retrier: Retrier;
readonly redirectHandler: RedirectHandler;
readonly trustEvaluator: TrustEvaluator;
readonly eventMonitor: EventMonitor;
readonly cachePolicy: CachePolicy;
readonly credential: Credential | null;
readonly interceptors: Interceptor[];
readonly delegate: RequestDelegate;
readonly queue: RequestQueue;
readonly startRequestsImmediately: boolean;
readonly maximumConnectionsPerHost: number;
readonly timeoutIntervalForRequest: number;
readonly timeoutIntervalForResource: number;
readonly allowsCellularAccess: boolean;
readonly waitsForConnectivity: boolean;
readonly httpShouldUsePipelining: boolean;
readonly httpShouldSetCookies: boolean;
readonly httpMaximumConnectionsPerHost: number;
readonly sessionConfigurationName: string;
readonly requestState: RequestState;
readonly defaultHeaders: Record<string, string>;
readonly userAgent: string;
readonly acceptEncoding: string;
readonly acceptLanguage: string;
private taskCounter = 0;
private active = new Map<number, URLSessionTask>();
constructor(options: Partial<Session> & { identifier: string }) {
this.identifier = options.identifier;
this.adapter = options.adapter!;
this.serializer = options.serializer!;
this.validator = options.validator!;
this.retrier = options.retrier!;
this.redirectHandler = options.redirectHandler!;
this.trustEvaluator = options.trustEvaluator!;
this.eventMonitor = options.eventMonitor!;
this.cachePolicy = options.cachePolicy ?? 'useProtocolCachePolicy';
this.credential = options.credential ?? null;
this.interceptors = options.interceptors ?? [];
this.delegate = options.delegate!;
this.queue = options.queue ?? new RequestQueue();
this.startRequestsImmediately = options.startRequestsImmediately ?? true;
this.maximumConnectionsPerHost = options.maximumConnectionsPerHost ?? 6;
this.timeoutIntervalForRequest = options.timeoutIntervalForRequest ?? 60;
this.timeoutIntervalForResource = options.timeoutIntervalForResource ?? 604800;
this.allowsCellularAccess = options.allowsCellularAccess ?? true;
this.waitsForConnectivity = options.waitsForConnectivity ?? false;
this.httpShouldUsePipelining = options.httpShouldUsePipelining ?? false;
this.httpShouldSetCookies = options.httpShouldSetCookies ?? true;
this.httpMaximumConnectionsPerHost = options.httpMaximumConnectionsPerHost ?? 6;
this.sessionConfigurationName = options.sessionConfigurationName ?? 'default';
this.requestState = options.requestState ?? 'initialized';
this.defaultHeaders = options.defaultHeaders ?? {};
this.userAgent = options.userAgent ?? 'session/1.0';
this.acceptEncoding = options.acceptEncoding ?? 'br;q=1.0, gzip;q=0.9';
this.acceptLanguage = options.acceptLanguage ?? 'en;q=1.0';
}
// -- configuration accessors ----------------------------------------------
// Individually trivial, adjacent, and dense. On the density tiebreak alone
// this block outranks anything with a body worth reading.
get isBackground(): boolean {
return this.sessionConfigurationName === 'background';
}
get connectionLimit(): number {
return Math.min(this.maximumConnectionsPerHost, this.httpMaximumConnectionsPerHost);
}
get headerDefaults(): Record<string, string> {
return { ...this.defaultHeaders, 'user-agent': this.userAgent };
}
get acceptHeaders(): Record<string, string> {
return { 'accept-encoding': this.acceptEncoding, 'accept-language': this.acceptLanguage };
}
get activeCount(): number {
return this.active.size;
}
get isIdle(): boolean {
return this.active.size === 0;
}
get nextIdentifier(): number {
return this.taskCounter + 1;
}
get description(): string {
return `Session(${this.identifier}, ${this.sessionConfigurationName})`;
}
cancelAll(): void {
for (const task of this.active.values()) task.cancel();
this.active.clear();
}
taskFor(identifier: number): URLSessionTask | undefined {
return this.active.get(identifier);
}
headers(): Record<string, string> {
return { ...this.headerDefaults, ...this.acceptHeaders };
}
withUserAgent(userAgent: string): Session {
return new Session({ ...this, identifier: this.identifier, userAgent });
}
withTimeout(seconds: number): Session {
return new Session({ ...this, identifier: this.identifier, timeoutIntervalForRequest: seconds });
}
withInterceptor(interceptor: Interceptor): Session {
return new Session({
...this,
identifier: this.identifier,
interceptors: [...this.interceptors, interceptor],
});
}
withCredential(credential: Credential): Session {
return new Session({ ...this, identifier: this.identifier, credential });
}
withCachePolicy(cachePolicy: CachePolicy): Session {
return new Session({ ...this, identifier: this.identifier, cachePolicy });
}
withQueue(queue: RequestQueue): Session {
return new Session({ ...this, identifier: this.identifier, queue });
}
withAdapter(adapter: Adapter): Session {
return new Session({ ...this, identifier: this.identifier, adapter });
}
withValidator(validator: Validator): Session {
return new Session({ ...this, identifier: this.identifier, validator });
}
withRetrier(retrier: Retrier): Session {
return new Session({ ...this, identifier: this.identifier, retrier });
}
withMonitor(eventMonitor: EventMonitor): Session {
return new Session({ ...this, identifier: this.identifier, eventMonitor });
}
// -- the flow ---------------------------------------------------------------
//
// The methods below are what a "how does a request get built and sent" question
// is about, and they sit hundreds of lines under the header block.
/**
* Turn a convenience call into a URLRequest, hand it to the adapter chain and
* start the resulting task. The entry point of the whole flow.
*/
async perform(url: string, method: string, body?: Uint8Array): Promise<URLSessionTask> {
const initial = buildURLRequest({
url,
method,
body,
headers: this.headers(),
timeout: this.timeoutIntervalForRequest,
cachePolicy: this.cachePolicy,
});
const adapted = await this.adapt(initial);
return this.didCreateURLRequest(adapted);
}
/**
* Every interceptor gets a chance to rewrite the request before it becomes a
* task. Runs in registration order, and a thrown error aborts the whole call.
*/
private async adapt(request: URLRequest): Promise<URLRequest> {
let current = request;
for (const interceptor of this.interceptors) {
current = await interceptor.adapt(current, this);
this.eventMonitor.didAdaptRequest(current, interceptor.name);
}
if (this.credential) current = this.credential.apply(current);
return current;
}
/**
* The adapted request is final: build the task around it, register it and —
* unless the session was told to wait — resume it immediately.
*/
didCreateURLRequest(request: URLRequest): URLSessionTask {
this.taskCounter += 1;
const identifier = this.taskCounter;
const created = this.task(request, identifier);
this.active.set(identifier, created);
this.eventMonitor.didCreateTask(created, request);
if (this.startRequestsImmediately) this.resume(created);
return created;
}
/**
* Build the URLSessionTask for a request. Split out from
* `didCreateURLRequest` because retries rebuild the task without going back
* through the adapter chain.
*/
task(request: URLRequest, identifier: number): URLSessionTask {
const created = makeTask({
identifier,
request,
delegate: this.delegate,
allowsCellularAccess: this.allowsCellularAccess,
waitsForConnectivity: this.waitsForConnectivity,
resourceTimeout: this.timeoutIntervalForResource,
});
created.onComplete((response) => {
this.active.delete(identifier);
const verdict = this.validator.validate(response);
if (!verdict.ok && this.retrier.shouldRetry(response, verdict)) {
this.retry(request, identifier);
return;
}
this.eventMonitor.didCompleteTask(created, response);
});
return created;
}
/** Put a built task on the queue and start it. */
resume(task: URLSessionTask): void {
this.queue.enqueue(task, this.connectionLimit);
resumeTask(task);
this.eventMonitor.didResumeTask(task);
}
/** Rebuild and restart a task the retrier asked for. */
private retry(request: URLRequest, previousIdentifier: number): void {
this.taskCounter += 1;
const retried = this.task(request, this.taskCounter);
this.active.set(this.taskCounter, retried);
this.eventMonitor.didRetryTask(retried, previousIdentifier);
this.resume(retried);
}
/** Follow a redirect by adapting and re-performing the new location. */
async follow(response: { location: string }, original: URLRequest): Promise<URLSessionTask> {
const target = this.redirectHandler.resolve(response.location, original);
if (!target) throw new Error(`redirect to ${response.location} refused`);
return this.perform(target.url, target.method, target.body);
}
}
__tests__/fixtures/displacement-ts/package.json
+6
View File
@@ -0,0 +1,6 @@
{
"name": "displacement-fixture",
"version": "1.0.0",
"private": true,
"type": "module"
}
__tests__/fixtures/displacement-ts/src/index.ts
+10
View File
@@ -0,0 +1,10 @@
import { ingestRecords } from './pipeline/ingest';
import { normalizeRecords } from './pipeline/normalize';
import { enrichRecords } from './pipeline/enrich';
import { publishRecords } from './pipeline/publish';
import type { PipelineOptions, PipelineRecord, RawRecord } from './pipeline/types';
/** Run one batch through every pipeline stage, in order. */
export function runPipeline(batch: RawRecord[], options: PipelineOptions): PipelineRecord[] {
return publishRecords(enrichRecords(normalizeRecords(ingestRecords(batch, options), options), options), options);
}
__tests__/fixtures/displacement-ts/src/pipeline/enrich.ts
+127
View File
@@ -0,0 +1,127 @@
import { writeBatch } from './sink';
import type { PipelineOptions, PipelineRecord } from './types';
/** Enrich every record in a batch. */
export function enrichRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
const out: PipelineRecord[] = [];
for (const record of records) {
const tags = [...record.tags];
const warnings = [...record.warnings];
let value = record.value;
// 1. segment
{
const hit = tags.find((t) => t.startsWith('segment:'));
if (hit === undefined) {
if (options.strict) warnings.push('segment: missing after enrich');
} else {
value = weightFacet(value, hit.length);
tags.push('segment.enri');
}
}
// 2. referrer
{
const hit = tags.find((t) => t.startsWith('referrer:'));
if (hit === undefined) {
if (options.strict) warnings.push('referrer: missing after enrich');
} else {
value = blendFacet(value, hit.length);
tags.push('referrer.enri');
}
}
// 3. experiment
{
const hit = tags.find((t) => t.startsWith('experiment:'));
if (hit === undefined) {
if (options.strict) warnings.push('experiment: missing after enrich');
} else {
value = weightFacet(value, hit.length);
tags.push('experiment.enri');
}
}
// 4. subscription
{
const hit = tags.find((t) => t.startsWith('subscription:'));
if (hit === undefined) {
if (options.strict) warnings.push('subscription: missing after enrich');
} else {
value = blendFacet(value, hit.length);
tags.push('subscription.enri');
}
}
// 5. entitlement
{
const hit = tags.find((t) => t.startsWith('entitlement:'));
if (hit === undefined) {
if (options.strict) warnings.push('entitlement: missing after enrich');
} else {
value = weightFacet(value, hit.length);
tags.push('entitlement.enri');
}
}
// 6. invoice
{
const hit = tags.find((t) => t.startsWith('invoice:'));
if (hit === undefined) {
if (options.strict) warnings.push('invoice: missing after enrich');
} else {
value = blendFacet(value, hit.length);
tags.push('invoice.enri');
}
}
// 7. refund
{
const hit = tags.find((t) => t.startsWith('refund:'));
if (hit === undefined) {
if (options.strict) warnings.push('refund: missing after enrich');
} else {
value = weightFacet(value, hit.length);
tags.push('refund.enri');
}
}
// 8. dispute
{
const hit = tags.find((t) => t.startsWith('dispute:'));
if (hit === undefined) {
if (options.strict) warnings.push('dispute: missing after enrich');
} else {
value = blendFacet(value, hit.length);
tags.push('dispute.enri');
}
}
// 9. payout
{
const hit = tags.find((t) => t.startsWith('payout:'));
if (hit === undefined) {
if (options.strict) warnings.push('payout: missing after enrich');
} else {
value = weightFacet(value, hit.length);
tags.push('payout.enri');
}
}
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
}
writeBatch('enrichRecords', out);
return out;
}
/** weightFacet — a small deterministic helper. */
export function weightFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
/** blendFacet — a small deterministic helper. */
export function blendFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
__tests__/fixtures/displacement-ts/src/pipeline/ingest.ts
+541
View File
@@ -0,0 +1,541 @@
import { scaleFacet, clampFacet } from './normalize';
import { writeBatch } from './sink';
import type { PipelineOptions, PipelineRecord, RawRecord } from './types';
/**
* Ingest one batch of raw records.
*
* Every facet is unpacked in its own block so an on-call engineer can read the
* ingest end-to-end in one place. The shape is deliberately flat: this single
* function is the whole stage, which is exactly the shape that makes it the
* biggest cluster member in the file.
*/
export function ingestRecords(batch: RawRecord[], options: PipelineOptions): PipelineRecord[] {
const out: PipelineRecord[] = [];
for (const record of batch) {
const tags: string[] = [];
const warnings: string[] = [];
let value = 0;
// 1. identity — normalise the identity facet of the record.
{
const raw = record.payload['identity'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('identity: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('identity:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('identity: not scalable — ' + text.slice(0, 16));
}
}
}
// 2. geography — normalise the geography facet of the record.
{
const raw = record.payload['geography'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('geography: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('geography:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('geography: not scalable — ' + text.slice(0, 16));
}
}
}
// 3. currency — normalise the currency facet of the record.
{
const raw = record.payload['currency'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('currency: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('currency:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('currency: not scalable — ' + text.slice(0, 16));
}
}
}
// 4. timestamp — normalise the timestamp facet of the record.
{
const raw = record.payload['timestamp'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('timestamp: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('timestamp:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('timestamp: not scalable — ' + text.slice(0, 16));
}
}
}
// 5. channel — normalise the channel facet of the record.
{
const raw = record.payload['channel'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('channel: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('channel:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('channel: not scalable — ' + text.slice(0, 16));
}
}
}
// 6. campaign — normalise the campaign facet of the record.
{
const raw = record.payload['campaign'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('campaign: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('campaign:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('campaign: not scalable — ' + text.slice(0, 16));
}
}
}
// 7. device — normalise the device facet of the record.
{
const raw = record.payload['device'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('device: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('device:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('device: not scalable — ' + text.slice(0, 16));
}
}
}
// 8. locale — normalise the locale facet of the record.
{
const raw = record.payload['locale'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('locale: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('locale:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('locale: not scalable — ' + text.slice(0, 16));
}
}
}
// 9. consent — normalise the consent facet of the record.
{
const raw = record.payload['consent'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('consent: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('consent:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('consent: not scalable — ' + text.slice(0, 16));
}
}
}
// 10. segment — normalise the segment facet of the record.
{
const raw = record.payload['segment'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('segment: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('segment:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('segment: not scalable — ' + text.slice(0, 16));
}
}
}
// 11. referrer — normalise the referrer facet of the record.
{
const raw = record.payload['referrer'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('referrer: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('referrer:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('referrer: not scalable — ' + text.slice(0, 16));
}
}
}
// 12. experiment — normalise the experiment facet of the record.
{
const raw = record.payload['experiment'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('experiment: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('experiment:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('experiment: not scalable — ' + text.slice(0, 16));
}
}
}
// 13. subscription — normalise the subscription facet of the record.
{
const raw = record.payload['subscription'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('subscription: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('subscription:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('subscription: not scalable — ' + text.slice(0, 16));
}
}
}
// 14. entitlement — normalise the entitlement facet of the record.
{
const raw = record.payload['entitlement'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('entitlement: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('entitlement:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('entitlement: not scalable — ' + text.slice(0, 16));
}
}
}
// 15. invoice — normalise the invoice facet of the record.
{
const raw = record.payload['invoice'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('invoice: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('invoice:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('invoice: not scalable — ' + text.slice(0, 16));
}
}
}
// 16. refund — normalise the refund facet of the record.
{
const raw = record.payload['refund'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('refund: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('refund:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('refund: not scalable — ' + text.slice(0, 16));
}
}
}
// 17. dispute — normalise the dispute facet of the record.
{
const raw = record.payload['dispute'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('dispute: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('dispute:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('dispute: not scalable — ' + text.slice(0, 16));
}
}
}
// 18. payout — normalise the payout facet of the record.
{
const raw = record.payload['payout'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('payout: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('payout:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('payout: not scalable — ' + text.slice(0, 16));
}
}
}
// 19. shipment — normalise the shipment facet of the record.
{
const raw = record.payload['shipment'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('shipment: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('shipment:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('shipment: not scalable — ' + text.slice(0, 16));
}
}
}
// 20. inventory — normalise the inventory facet of the record.
{
const raw = record.payload['inventory'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('inventory: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('inventory:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('inventory: not scalable — ' + text.slice(0, 16));
}
}
}
// 21. warehouse — normalise the warehouse facet of the record.
{
const raw = record.payload['warehouse'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('warehouse: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('warehouse:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('warehouse: not scalable — ' + text.slice(0, 16));
}
}
}
// 22. carrier — normalise the carrier facet of the record.
{
const raw = record.payload['carrier'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('carrier: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('carrier:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('carrier: not scalable — ' + text.slice(0, 16));
}
}
}
// 23. customs — normalise the customs facet of the record.
{
const raw = record.payload['customs'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('customs: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('customs:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('customs: not scalable — ' + text.slice(0, 16));
}
}
}
// 24. tariff — normalise the tariff facet of the record.
{
const raw = record.payload['tariff'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('tariff: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('tariff:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('tariff: not scalable — ' + text.slice(0, 16));
}
}
}
// 25. sensor — normalise the sensor facet of the record.
{
const raw = record.payload['sensor'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('sensor: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('sensor:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('sensor: not scalable — ' + text.slice(0, 16));
}
}
}
// 26. firmware — normalise the firmware facet of the record.
{
const raw = record.payload['firmware'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('firmware: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('firmware:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('firmware: not scalable — ' + text.slice(0, 16));
}
}
}
// 27. telemetry — normalise the telemetry facet of the record.
{
const raw = record.payload['telemetry'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('telemetry: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('telemetry:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('telemetry: not scalable — ' + text.slice(0, 16));
}
}
}
// 28. battery — normalise the battery facet of the record.
{
const raw = record.payload['battery'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('battery: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('battery:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('battery: not scalable — ' + text.slice(0, 16));
}
}
}
// 29. network — normalise the network facet of the record.
{
const raw = record.payload['network'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('network: empty, dropped');
} else {
const scaled = scaleFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('network:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('network: not scalable — ' + text.slice(0, 16));
}
}
}
// 30. roaming — normalise the roaming facet of the record.
{
const raw = record.payload['roaming'];
const text = typeof raw === 'string' ? raw.trim() : raw === null ? '' : String(raw);
if (text.length === 0 && options.dropEmpty) {
warnings.push('roaming: empty, dropped');
} else {
const scaled = clampFacet(text.length, options.maxTags);
if (Number.isFinite(scaled) && scaled !== 0) {
tags.push('roaming:' + text.slice(0, 24));
value += scaled;
} else if (options.strict) {
warnings.push('roaming: not scalable — ' + text.slice(0, 16));
}
}
}
out.push({
id: record.id,
source: record.source,
kind: options.defaultKind,
value,
tags: tags.slice(0, options.maxTags),
warnings,
});
}
writeBatch('ingest', out);
return out;
}
__tests__/fixtures/displacement-ts/src/pipeline/normalize.ts
+127
View File
@@ -0,0 +1,127 @@
import { writeBatch } from './sink';
import type { PipelineOptions, PipelineRecord } from './types';
/** Normalize every record in a batch. */
export function normalizeRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
const out: PipelineRecord[] = [];
for (const record of records) {
const tags = [...record.tags];
const warnings = [...record.warnings];
let value = record.value;
// 1. identity
{
const hit = tags.find((t) => t.startsWith('identity:'));
if (hit === undefined) {
if (options.strict) warnings.push('identity: missing after normalize');
} else {
value = scaleFacet(value, hit.length);
tags.push('identity.norm');
}
}
// 2. geography
{
const hit = tags.find((t) => t.startsWith('geography:'));
if (hit === undefined) {
if (options.strict) warnings.push('geography: missing after normalize');
} else {
value = clampFacet(value, hit.length);
tags.push('geography.norm');
}
}
// 3. currency
{
const hit = tags.find((t) => t.startsWith('currency:'));
if (hit === undefined) {
if (options.strict) warnings.push('currency: missing after normalize');
} else {
value = scaleFacet(value, hit.length);
tags.push('currency.norm');
}
}
// 4. timestamp
{
const hit = tags.find((t) => t.startsWith('timestamp:'));
if (hit === undefined) {
if (options.strict) warnings.push('timestamp: missing after normalize');
} else {
value = clampFacet(value, hit.length);
tags.push('timestamp.norm');
}
}
// 5. channel
{
const hit = tags.find((t) => t.startsWith('channel:'));
if (hit === undefined) {
if (options.strict) warnings.push('channel: missing after normalize');
} else {
value = scaleFacet(value, hit.length);
tags.push('channel.norm');
}
}
// 6. campaign
{
const hit = tags.find((t) => t.startsWith('campaign:'));
if (hit === undefined) {
if (options.strict) warnings.push('campaign: missing after normalize');
} else {
value = clampFacet(value, hit.length);
tags.push('campaign.norm');
}
}
// 7. device
{
const hit = tags.find((t) => t.startsWith('device:'));
if (hit === undefined) {
if (options.strict) warnings.push('device: missing after normalize');
} else {
value = scaleFacet(value, hit.length);
tags.push('device.norm');
}
}
// 8. locale
{
const hit = tags.find((t) => t.startsWith('locale:'));
if (hit === undefined) {
if (options.strict) warnings.push('locale: missing after normalize');
} else {
value = clampFacet(value, hit.length);
tags.push('locale.norm');
}
}
// 9. consent
{
const hit = tags.find((t) => t.startsWith('consent:'));
if (hit === undefined) {
if (options.strict) warnings.push('consent: missing after normalize');
} else {
value = scaleFacet(value, hit.length);
tags.push('consent.norm');
}
}
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
}
writeBatch('normalizeRecords', out);
return out;
}
/** scaleFacet — a small deterministic helper. */
export function scaleFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
/** clampFacet — a small deterministic helper. */
export function clampFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
__tests__/fixtures/displacement-ts/src/pipeline/publish.ts
+127
View File
@@ -0,0 +1,127 @@
import { writeBatch } from './sink';
import type { PipelineOptions, PipelineRecord } from './types';
/** Publish every record in a batch. */
export function publishRecords(records: PipelineRecord[], options: PipelineOptions): PipelineRecord[] {
const out: PipelineRecord[] = [];
for (const record of records) {
const tags = [...record.tags];
const warnings = [...record.warnings];
let value = record.value;
// 1. shipment
{
const hit = tags.find((t) => t.startsWith('shipment:'));
if (hit === undefined) {
if (options.strict) warnings.push('shipment: missing after publish');
} else {
value = rankFacet(value, hit.length);
tags.push('shipment.publ');
}
}
// 2. inventory
{
const hit = tags.find((t) => t.startsWith('inventory:'));
if (hit === undefined) {
if (options.strict) warnings.push('inventory: missing after publish');
} else {
value = sealFacet(value, hit.length);
tags.push('inventory.publ');
}
}
// 3. warehouse
{
const hit = tags.find((t) => t.startsWith('warehouse:'));
if (hit === undefined) {
if (options.strict) warnings.push('warehouse: missing after publish');
} else {
value = rankFacet(value, hit.length);
tags.push('warehouse.publ');
}
}
// 4. carrier
{
const hit = tags.find((t) => t.startsWith('carrier:'));
if (hit === undefined) {
if (options.strict) warnings.push('carrier: missing after publish');
} else {
value = sealFacet(value, hit.length);
tags.push('carrier.publ');
}
}
// 5. customs
{
const hit = tags.find((t) => t.startsWith('customs:'));
if (hit === undefined) {
if (options.strict) warnings.push('customs: missing after publish');
} else {
value = rankFacet(value, hit.length);
tags.push('customs.publ');
}
}
// 6. tariff
{
const hit = tags.find((t) => t.startsWith('tariff:'));
if (hit === undefined) {
if (options.strict) warnings.push('tariff: missing after publish');
} else {
value = sealFacet(value, hit.length);
tags.push('tariff.publ');
}
}
// 7. sensor
{
const hit = tags.find((t) => t.startsWith('sensor:'));
if (hit === undefined) {
if (options.strict) warnings.push('sensor: missing after publish');
} else {
value = rankFacet(value, hit.length);
tags.push('sensor.publ');
}
}
// 8. firmware
{
const hit = tags.find((t) => t.startsWith('firmware:'));
if (hit === undefined) {
if (options.strict) warnings.push('firmware: missing after publish');
} else {
value = sealFacet(value, hit.length);
tags.push('firmware.publ');
}
}
// 9. telemetry
{
const hit = tags.find((t) => t.startsWith('telemetry:'));
if (hit === undefined) {
if (options.strict) warnings.push('telemetry: missing after publish');
} else {
value = rankFacet(value, hit.length);
tags.push('telemetry.publ');
}
}
out.push({ ...record, value, tags: tags.slice(0, options.maxTags), warnings });
}
writeBatch('publishRecords', out);
return out;
}
/** rankFacet — a small deterministic helper. */
export function rankFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
/** sealFacet — a small deterministic helper. */
export function sealFacet(base: number, width: number): number {
const scaled = base + width * 3 - (width % 7);
return scaled < 0 ? 0 : scaled;
}
__tests__/fixtures/displacement-ts/src/pipeline/sink.ts
+18
View File
@@ -0,0 +1,18 @@
import type { PipelineRecord } from './types';
const sink = new Map<string, PipelineRecord[]>();
/** Hand a finished batch to the downstream sink. */
export function writeBatch(batchId: string, records: PipelineRecord[]): void {
sink.set(batchId, records);
}
/** Read a batch back out of the sink. */
export function readBatch(batchId: string): PipelineRecord[] {
return sink.get(batchId) ?? [];
}
/** Forget a batch. */
export function dropBatch(batchId: string): void {
sink.delete(batchId);
}
__tests__/fixtures/displacement-ts/src/pipeline/types.ts
+25
View File
@@ -0,0 +1,25 @@
/** One raw record as it arrives from the upstream feed. */
export interface RawRecord {
id: string;
source: string;
payload: Record<string, string | number | null>;
receivedAt: number;
}
/** A record after the pipeline has cleaned and annotated it. */
export interface PipelineRecord {
id: string;
source: string;
kind: string;
value: number;
tags: string[];
warnings: string[];
}
/** Per-run knobs shared by every pipeline stage. */
export interface PipelineOptions {
strict: boolean;
dropEmpty: boolean;
defaultKind: string;
maxTags: number;
}
__tests__/fixtures/factory-closure-ts/package.json
+5
View File
@@ -0,0 +1,5 @@
{
"name": "factory-closure-ts",
"version": "0.0.0",
"private": true
}
__tests__/fixtures/factory-closure-ts/src/index.ts
+23
View File
@@ -0,0 +1,23 @@
import { createDashboardStore } from './stores/dashboard-store';
import { createAlertsStore } from './stores/alerts-store';
import { mountPanel } from './ui/panel';
import { parseFilterText } from './services/filter-parser';
import { refreshMetricCache } from './services/metric-service';
import type { StoreDeps } from './stores/types';
/** Wire a dashboard: build both stores, mount the panel, boot it. */
export async function startDashboard(deps: StoreDeps, baseUrl: string, dashboardId: string) {
const store = createDashboardStore(deps, baseUrl);
const alerts = createAlertsStore(deps, baseUrl);
const panel = mountPanel(store, dashboardId);
await panel.boot();
await alerts.refreshAlerts(dashboardId);
return { store, alerts, panel };
}
/** Apply the filter bar's text to the dashboard store. */
export function searchDashboard(store: ReturnType<typeof createDashboardStore>, text: string) {
return store.applyFilter(parseFilterText(text));
}
export { refreshMetricCache };
__tests__/fixtures/factory-closure-ts/src/lib/http.ts
+25
View File
@@ -0,0 +1,25 @@
/** Minimal fetch helpers the dashboard store depends on. */
export interface RequestOptions {
retries: number;
timeoutMs: number;
}
export const defaultRequestOptions: RequestOptions = { retries: 2, timeoutMs: 5_000 };
/** Build a query string from a plain record, skipping empty values. */
export function toQueryString(params: Record<string, string | number | undefined>): string {
const parts: string[] = [];
for (const [key, value] of Object.entries(params)) {
if (value === undefined || value === '') continue;
parts.push(`${encodeURIComponent(key)}=${encodeURIComponent(String(value))}`);
}
return parts.length > 0 ? `?${parts.join('&')}` : '';
}
/** Join a base path and a resource path without doubling the separator. */
export function joinPath(base: string, resource: string): string {
if (base.endsWith('/') && resource.startsWith('/')) return base + resource.slice(1);
if (!base.endsWith('/') && !resource.startsWith('/')) return `${base}/${resource}`;
return base + resource;
}
__tests__/fixtures/factory-closure-ts/src/lib/metrics.ts
+37
View File
@@ -0,0 +1,37 @@
import type { MetricSample } from '../stores/types';
/** Statistics helpers shared by the store and the panel. */
export function meanOf(samples: readonly MetricSample[]): number {
if (samples.length === 0) return 0;
let total = 0;
for (const sample of samples) total += sample.value;
return total / samples.length;
}
export function medianOf(samples: readonly MetricSample[]): number {
if (samples.length === 0) return 0;
const values = samples.map((s) => s.value).sort((a, b) => a - b);
const mid = Math.floor(values.length / 2);
return values.length % 2 === 0 ? (values[mid - 1]! + values[mid]!) / 2 : values[mid]!;
}
export function rateOfChange(samples: readonly MetricSample[]): number {
if (samples.length < 2) return 0;
const ordered = samples.slice().sort((a, b) => a.at - b.at);
const first = ordered[0]!;
const last = ordered[ordered.length - 1]!;
const elapsed = last.at - first.at;
return elapsed > 0 ? (last.value - first.value) / elapsed : 0;
}
export function bucketByHour(samples: readonly MetricSample[]): Map<number, MetricSample[]> {
const buckets = new Map<number, MetricSample[]>();
for (const sample of samples) {
const hour = Math.floor(sample.at / 3_600_000);
const bucket = buckets.get(hour);
if (bucket) bucket.push(sample);
else buckets.set(hour, [sample]);
}
return buckets;
}
__tests__/fixtures/factory-closure-ts/src/services/filter-parser.ts
+62
View File
@@ -0,0 +1,62 @@
import type { FilterSpec } from '../stores/types';
/** Parse the dashboard's filter bar text into filter specs. */
const OPERATORS: Record<string, FilterSpec['op']> = {
':': 'eq',
'~': 'contains',
'>': 'gt',
'<': 'lt',
};
/** `title~sales kind:chart column>3` → three specs. */
export function parseFilterText(text: string): FilterSpec[] {
const specs: FilterSpec[] = [];
for (const token of tokenize(text)) {
const spec = parseToken(token);
if (spec) specs.push(spec);
}
return specs;
}
/** Split on whitespace, honouring double-quoted values. */
export function tokenize(text: string): string[] {
const tokens: string[] = [];
let current = '';
let quoted = false;
for (const ch of text) {
if (ch === '"') { quoted = !quoted; continue; }
if (!quoted && /\s/.test(ch)) {
if (current.length > 0) { tokens.push(current); current = ''; }
continue;
}
current += ch;
}
if (current.length > 0) tokens.push(current);
return tokens;
}
/** One `field<op>value` token, or null when it does not parse. */
export function parseToken(token: string): FilterSpec | null {
for (const [symbol, op] of Object.entries(OPERATORS)) {
const at = token.indexOf(symbol);
if (at <= 0) continue;
const field = token.slice(0, at).trim();
const value = token.slice(at + symbol.length).trim();
if (field.length === 0 || value.length === 0) return null;
return { field, op, value };
}
return null;
}
/** Render specs back to filter-bar text — the round trip the URL uses. */
export function formatFilterText(specs: readonly FilterSpec[]): string {
const symbolFor = (op: FilterSpec['op']): string =>
Object.entries(OPERATORS).find(([, candidate]) => candidate === op)?.[0] ?? ':';
return specs
.map((spec) => {
const value = /\s/.test(spec.value) ? `"${spec.value}"` : spec.value;
return `${spec.field}${symbolFor(spec.op)}${value}`;
})
.join(' ');
}
__tests__/fixtures/factory-closure-ts/src/services/metric-service.ts
+101
View File
@@ -0,0 +1,101 @@
import type { FilterSpec, MetricSample, Widget } from '../stores/types';
import { bucketByHour, meanOf, rateOfChange } from '../lib/metrics';
/**
* Stateless metric helpers — the server-shaped half of the same domain. These
* are ordinary top-level functions, not closures, so they are the control the
* factory-closure file is measured against.
*/
const STALE_AFTER_MS = 15 * 60 * 1000;
/** Refresh a cached metric map in place, returning the widgets that changed. */
export function refreshMetricCache(
cache: Map<string, MetricSample[]>,
incoming: readonly MetricSample[],
now: number,
): string[] {
const touched = new Set<string>();
for (const sample of incoming) {
if (typeof sample.value !== 'number' || Number.isNaN(sample.value)) continue;
const bucket = cache.get(sample.widgetId);
if (bucket) bucket.push(sample);
else cache.set(sample.widgetId, [sample]);
touched.add(sample.widgetId);
}
for (const [widgetId, bucket] of cache) {
const fresh = bucket.filter((s) => now - s.at <= STALE_AFTER_MS);
if (fresh.length !== bucket.length) {
cache.set(widgetId, fresh);
touched.add(widgetId);
}
}
return [...touched].sort();
}
/** Apply a filter spec set to raw samples rather than to widgets. */
export function filterMetrics(
samples: readonly MetricSample[],
specs: readonly FilterSpec[],
): MetricSample[] {
if (specs.length === 0) return samples.slice();
return samples.filter((sample) => specs.every((spec) => {
const field = spec.field === 'unit'
? sample.unit
: spec.field === 'widget'
? sample.widgetId
: String(sample.value);
switch (spec.op) {
case 'eq': return field === spec.value;
case 'contains': return field.includes(spec.value);
case 'gt': return Number(field) > Number(spec.value);
case 'lt': return Number(field) < Number(spec.value);
default: return false;
}
}));
}
/** Per-widget rollup used by the server-rendered summary card. */
export function rollupByWidget(
samples: readonly MetricSample[],
widgets: readonly Widget[],
): Array<{ widgetId: string; title: string; mean: number; slope: number; hours: number }> {
const titles = new Map(widgets.map((w) => [w.id, w.title]));
const grouped = new Map<string, MetricSample[]>();
for (const sample of samples) {
const bucket = grouped.get(sample.widgetId);
if (bucket) bucket.push(sample);
else grouped.set(sample.widgetId, [sample]);
}
const out: Array<{ widgetId: string; title: string; mean: number; slope: number; hours: number }> = [];
for (const [widgetId, bucket] of grouped) {
out.push({
widgetId,
title: titles.get(widgetId) ?? '(unknown)',
mean: meanOf(bucket),
slope: rateOfChange(bucket),
hours: bucketByHour(bucket).size,
});
}
out.sort((a, b) => b.mean - a.mean);
return out;
}
/** Which widgets have not reported inside the staleness window. */
export function staleWidgets(
samples: readonly MetricSample[],
widgets: readonly Widget[],
now: number,
): string[] {
const newest = new Map<string, number>();
for (const sample of samples) {
const seen = newest.get(sample.widgetId) ?? 0;
if (sample.at > seen) newest.set(sample.widgetId, sample.at);
}
return widgets
.filter((w) => !w.hidden)
.filter((w) => now - (newest.get(w.id) ?? 0) > STALE_AFTER_MS)
.map((w) => w.id)
.sort();
}
__tests__/fixtures/factory-closure-ts/src/stores/alerts-store.ts
+140
View File
@@ -0,0 +1,140 @@
import type { FilterSpec, StoreDeps } from './types';
import { joinPath, toQueryString } from '../lib/http';
const ALERT_ENDPOINT = '/api/dashboard/alerts';
export interface Alert {
id: string;
widgetId: string;
severity: 'info' | 'warn' | 'critical';
message: string;
raisedAt: number;
acknowledgedAt: number | null;
}
/**
* The alerts store — the dashboard's second factory closure. Same shape as the
* metric store: every operation is a closure over private state.
*/
export function createAlertsStore(deps: StoreDeps, baseUrl: string) {
let alerts: Alert[] = [];
let filters: FilterSpec[] = [];
let mutedWidgets = new Set<string>();
let lastRefreshedAt = 0;
/** Pull the current alert set and merge acknowledgements the user made locally. */
async function refreshAlerts(dashboardId: string): Promise<Alert[]> {
const url = joinPath(baseUrl, ALERT_ENDPOINT) + toQueryString({ dashboard: dashboardId });
let payload: unknown;
try {
payload = await deps.fetchJson(url);
} catch (error) {
deps.log(`refreshAlerts failed: ${error instanceof Error ? error.message : String(error)}`);
return alerts;
}
if (!Array.isArray(payload)) {
deps.log('refreshAlerts got a non-array payload');
return alerts;
}
const acknowledged = new Map(
alerts.filter((a) => a.acknowledgedAt !== null).map((a) => [a.id, a.acknowledgedAt]),
);
const merged: Alert[] = [];
for (const raw of payload as Alert[]) {
if (typeof raw.id !== 'string' || raw.id.length === 0) continue;
merged.push({
...raw,
acknowledgedAt: acknowledged.get(raw.id) ?? raw.acknowledgedAt ?? null,
});
}
merged.sort((a, b) => b.raisedAt - a.raisedAt);
alerts = merged;
lastRefreshedAt = deps.now();
return alerts;
}
/** Filter the alert list the same way the metric store filters widgets. */
function applyAlertFilter(specs: readonly FilterSpec[]): Alert[] {
filters = specs.slice();
if (filters.length === 0) return alerts;
const fieldOf = (alert: Alert, field: string): string => {
switch (field) {
case 'severity': return alert.severity;
case 'widget': return alert.widgetId;
case 'message': return alert.message;
default: return '';
}
};
return alerts.filter((alert) => filters.every((spec) => {
const value = fieldOf(alert, spec.field);
switch (spec.op) {
case 'eq': return value.toLowerCase() === spec.value.toLowerCase();
case 'contains': return value.toLowerCase().includes(spec.value.toLowerCase());
case 'gt': return value > spec.value;
case 'lt': return value < spec.value;
default: return false;
}
}));
}
/** Mark an alert acknowledged locally; the next refresh preserves it. */
function acknowledge(alertId: string): boolean {
const target = alerts.find((a) => a.id === alertId);
if (!target || target.acknowledgedAt !== null) return false;
target.acknowledgedAt = deps.now();
deps.log(`acknowledged ${alertId}`);
return true;
}
/** Silence a widget's alerts without dropping them from the buffer. */
function muteWidget(widgetId: string): void {
mutedWidgets.add(widgetId);
deps.log(`muted ${widgetId} (${mutedWidgets.size} muted)`);
}
function unmuteWidget(widgetId: string): boolean {
return mutedWidgets.delete(widgetId);
}
/** The alerts the dashboard should actually show right now. */
function visibleAlerts(): Alert[] {
return applyAlertFilter(filters)
.filter((a) => !mutedWidgets.has(a.widgetId))
.filter((a) => a.acknowledgedAt === null);
}
/** Counts per severity, for the badge on the alerts tab. */
function countBySeverity(): Record<Alert['severity'], number> {
const counts: Record<Alert['severity'], number> = { info: 0, warn: 0, critical: 0 };
for (const alert of visibleAlerts()) counts[alert.severity] += 1;
return counts;
}
function reset(): void {
alerts = [];
filters = [];
mutedWidgets = new Set();
lastRefreshedAt = 0;
}
function snapshot() {
return { alerts: visibleAlerts(), counts: countBySeverity(), lastRefreshedAt };
}
return {
refreshAlerts,
applyAlertFilter,
acknowledge,
muteWidget,
unmuteWidget,
visibleAlerts,
countBySeverity,
reset,
snapshot,
};
}
export type AlertsStore = ReturnType<typeof createAlertsStore>;
__tests__/fixtures/factory-closure-ts/src/stores/dashboard-store.ts
+384
View File
@@ -0,0 +1,384 @@
import type { FilterSpec, MetricSample, StoreDeps, Widget } from './types';
import { defaultRequestOptions, joinPath, toQueryString } from '../lib/http';
const WIDGET_ENDPOINT = '/api/dashboard/widgets';
const METRIC_ENDPOINT = '/api/dashboard/metrics';
const SAMPLE_RETENTION_MS = 6 * 60 * 60 * 1000;
const MAX_SAMPLES_PER_WIDGET = 720;
const COLUMN_COUNT = 12;
/**
* The dashboard store: one factory closure holding every operation the
* dashboard performs. Callers get an object of closures; nothing inside is
* exported on its own.
*/
export function createDashboardStore(deps: StoreDeps, baseUrl: string) {
let widgets: Widget[] = [];
let samples: MetricSample[] = [];
let activeFilters: FilterSpec[] = [];
let lastSyncedAt = 0;
let loading = false;
let lastError: string | null = null;
const listeners = new Set<(snapshot: ReturnType<typeof snapshot>) => void>();
function snapshot() {
return {
widgets: widgets.filter((w) => !w.hidden),
sampleCount: samples.length,
filters: activeFilters.slice(),
lastSyncedAt,
loading,
lastError,
};
}
/**
* Fetch the widget set for the current user and merge it into local state,
* preserving any layout the user has moved since the last sync.
*/
async function loadWidgets(dashboardId: string, includeHidden = false): Promise<Widget[]> {
loading = true;
lastError = null;
const url = joinPath(baseUrl, WIDGET_ENDPOINT) + toQueryString({
dashboard: dashboardId,
hidden: includeHidden ? '1' : undefined,
});
let attempt = 0;
let payload: unknown = null;
while (attempt <= defaultRequestOptions.retries) {
try {
payload = await deps.fetchJson(url);
break;
} catch (error) {
attempt += 1;
if (attempt > defaultRequestOptions.retries) {
lastError = error instanceof Error ? error.message : String(error);
loading = false;
deps.log(`loadWidgets failed after ${attempt} attempts: ${lastError}`);
notify();
return widgets;
}
deps.log(`loadWidgets retry ${attempt} for ${dashboardId}`);
}
}
const incoming = Array.isArray(payload) ? (payload as Widget[]) : [];
const byId = new Map(widgets.map((w) => [w.id, w]));
const merged: Widget[] = [];
for (const next of incoming) {
const existing = byId.get(next.id);
if (!existing) {
merged.push({ ...next });
continue;
}
// Server owns identity and content; the client owns placement.
merged.push({
...next,
column: existing.column,
row: existing.row,
span: existing.span,
hidden: existing.hidden,
});
byId.delete(next.id);
}
for (const orphan of byId.values()) {
deps.log(`widget ${orphan.id} no longer exists on the server`);
}
widgets = merged;
lastSyncedAt = deps.now();
loading = false;
notify();
return widgets;
}
/**
* Pull fresh metric samples for every visible widget, append them to the
* rolling buffer, and drop anything past the retention window.
*/
async function refreshMetrics(windowMs = SAMPLE_RETENTION_MS): Promise<MetricSample[]> {
if (widgets.length === 0) {
deps.log('refreshMetrics called with no widgets loaded');
return samples;
}
loading = true;
const visible = widgets.filter((w) => !w.hidden);
const collected: MetricSample[] = [];
for (const widget of visible) {
const url = joinPath(baseUrl, METRIC_ENDPOINT) + toQueryString({
widget: widget.id,
since: deps.now() - windowMs,
});
let payload: unknown;
try {
payload = await deps.fetchJson(url);
} catch (error) {
lastError = error instanceof Error ? error.message : String(error);
deps.log(`refreshMetrics failed for ${widget.id}: ${lastError}`);
continue;
}
if (!Array.isArray(payload)) {
deps.log(`refreshMetrics got a non-array payload for ${widget.id}`);
continue;
}
for (const raw of payload as MetricSample[]) {
if (typeof raw.value !== 'number' || Number.isNaN(raw.value)) continue;
if (typeof raw.at !== 'number' || raw.at <= 0) continue;
collected.push({
widgetId: widget.id,
at: raw.at,
value: raw.value,
unit: raw.unit ?? 'count',
});
}
}
const cutoff = deps.now() - windowMs;
const kept = samples.filter((s) => s.at >= cutoff);
samples = kept.concat(collected);
pruneSamples(MAX_SAMPLES_PER_WIDGET);
lastSyncedAt = deps.now();
loading = false;
notify();
return samples;
}
/**
* Replace the active filter set and recompute which widgets stay visible.
* A widget survives when every filter matches one of its fields.
*/
function applyFilter(specs: readonly FilterSpec[]): Widget[] {
activeFilters = specs.slice();
if (activeFilters.length === 0) {
widgets = widgets.map((w) => ({ ...w, hidden: false }));
notify();
return widgets;
}
const matches = (widget: Widget, spec: FilterSpec): boolean => {
const field = spec.field === 'title'
? widget.title
: spec.field === 'kind'
? widget.kind
: spec.field === 'column'
? String(widget.column)
: '';
switch (spec.op) {
case 'eq':
return field.toLowerCase() === spec.value.toLowerCase();
case 'contains':
return field.toLowerCase().includes(spec.value.toLowerCase());
case 'gt':
return Number(field) > Number(spec.value);
case 'lt':
return Number(field) < Number(spec.value);
default:
return false;
}
};
let hiddenCount = 0;
widgets = widgets.map((widget) => {
const visible = activeFilters.every((spec) => matches(widget, spec));
if (!visible) hiddenCount += 1;
return { ...widget, hidden: !visible };
});
deps.log(`applyFilter hid ${hiddenCount} of ${widgets.length} widgets`);
notify();
return widgets;
}
/**
* Render the current sample buffer as CSV, one row per sample, ordered by
* widget then timestamp so a diff between two exports stays readable.
*/
function exportCsv(separator = ','): string {
const header = ['widget', 'title', 'at', 'value', 'unit'].join(separator);
if (samples.length === 0) return header;
const titles = new Map(widgets.map((w) => [w.id, w.title]));
const ordered = samples.slice().sort((a, b) => {
if (a.widgetId !== b.widgetId) return a.widgetId < b.widgetId ? -1 : 1;
return a.at - b.at;
});
const escape = (value: string): string => {
if (!value.includes(separator) && !value.includes('"') && !value.includes('\n')) return value;
return `"${value.replace(/"/g, '""')}"`;
};
const rows = ordered.map((sample) => [
escape(sample.widgetId),
escape(titles.get(sample.widgetId) ?? '(unknown)'),
String(sample.at),
String(sample.value),
escape(sample.unit),
].join(separator));
return [header, ...rows].join('\n');
}
/**
* Pack widgets back into a dense grid after a move or a hide, so the layout
* never leaves a hole a user has to scroll past.
*/
function reconcileLayout(columnCount = COLUMN_COUNT): Widget[] {
const visible = widgets.filter((w) => !w.hidden);
const hidden = widgets.filter((w) => w.hidden);
const ordered = visible.slice().sort((a, b) => {
if (a.row !== b.row) return a.row - b.row;
return a.column - b.column;
});
const rowWidth = new Map<number, number>();
const placed: Widget[] = [];
for (const widget of ordered) {
const span = Math.max(1, Math.min(widget.span, columnCount));
let row = 0;
let column = 0;
for (;;) {
const used = rowWidth.get(row) ?? 0;
if (used + span <= columnCount) {
column = used;
rowWidth.set(row, used + span);
break;
}
row += 1;
}
placed.push({ ...widget, row, column, span });
}
let trailing = placed.length > 0 ? Math.max(...placed.map((w) => w.row)) + 1 : 0;
for (const widget of hidden) {
placed.push({ ...widget, row: trailing, column: 0 });
trailing += 1;
}
widgets = placed;
notify();
return widgets;
}
/**
* Cap the rolling buffer per widget, keeping the newest samples. Called after
* every refresh so memory stays bounded on a long-lived dashboard.
*/
function pruneSamples(perWidget = MAX_SAMPLES_PER_WIDGET): number {
if (samples.length === 0) return 0;
const grouped = new Map<string, MetricSample[]>();
for (const sample of samples) {
const bucket = grouped.get(sample.widgetId);
if (bucket) bucket.push(sample);
else grouped.set(sample.widgetId, [sample]);
}
let dropped = 0;
const kept: MetricSample[] = [];
for (const [, bucket] of grouped) {
bucket.sort((a, b) => a.at - b.at);
if (bucket.length > perWidget) {
dropped += bucket.length - perWidget;
kept.push(...bucket.slice(bucket.length - perWidget));
} else {
kept.push(...bucket);
}
}
kept.sort((a, b) => a.at - b.at);
samples = kept;
if (dropped > 0) deps.log(`pruneSamples dropped ${dropped} samples`);
return dropped;
}
/**
* Reduce the buffer to one aggregate per widget — the numbers the summary
* strip at the top of the dashboard renders.
*/
function summarize(): Array<{ widgetId: string; title: string; min: number; max: number; mean: number; count: number }> {
const titles = new Map(widgets.map((w) => [w.id, w.title]));
const grouped = new Map<string, MetricSample[]>();
for (const sample of samples) {
const bucket = grouped.get(sample.widgetId);
if (bucket) bucket.push(sample);
else grouped.set(sample.widgetId, [sample]);
}
const out: Array<{ widgetId: string; title: string; min: number; max: number; mean: number; count: number }> = [];
for (const [widgetId, bucket] of grouped) {
let min = Number.POSITIVE_INFINITY;
let max = Number.NEGATIVE_INFINITY;
let total = 0;
for (const sample of bucket) {
if (sample.value < min) min = sample.value;
if (sample.value > max) max = sample.value;
total += sample.value;
}
out.push({
widgetId,
title: titles.get(widgetId) ?? '(unknown)',
min: bucket.length > 0 ? min : 0,
max: bucket.length > 0 ? max : 0,
mean: bucket.length > 0 ? total / bucket.length : 0,
count: bucket.length,
});
}
out.sort((a, b) => b.count - a.count || (a.title < b.title ? -1 : 1));
return out;
}
/** Register a listener and get an unsubscribe back. */
function subscribe(listener: (snapshot: ReturnType<typeof snapshot>) => void): () => void {
listeners.add(listener);
listener(snapshot());
return () => {
listeners.delete(listener);
};
}
function notify(): void {
const current = snapshot();
for (const listener of listeners) {
try {
listener(current);
} catch (error) {
deps.log(`dashboard listener threw: ${error instanceof Error ? error.message : String(error)}`);
}
}
}
/** Drop every sample and widget — used when the user switches dashboards. */
function reset(): void {
widgets = [];
samples = [];
activeFilters = [];
lastSyncedAt = 0;
lastError = null;
loading = false;
notify();
}
return {
loadWidgets,
refreshMetrics,
applyFilter,
exportCsv,
reconcileLayout,
pruneSamples,
summarize,
subscribe,
reset,
snapshot,
};
}
export type DashboardStore = ReturnType<typeof createDashboardStore>;
/** One-line description of a store's state, for the debug panel. */
export function describeStore(store: DashboardStore): string {
const state = store.snapshot();
return `${state.widgets.length} widgets · ${state.sampleCount} samples · synced ${state.lastSyncedAt}`;
}
__tests__/fixtures/factory-closure-ts/src/stores/session-store.ts
+148
View File
@@ -0,0 +1,148 @@
import type { StoreDeps } from './types';
import { joinPath, toQueryString } from '../lib/http';
/**
* The session store — a factory closure and NOTHING else at file scope. No
* companion type alias, no tail helper, no exported constants: every other
* symbol in this file lives inside the closure. That shape matters, because it
* is the one where the enclosing range is the only top-importance symbol the
* file can offer a query.
*/
export function createSessionStore(deps: StoreDeps, baseUrl: string) {
const SESSION_ENDPOINT = '/api/session';
const REFRESH_SKEW_MS = 30_000;
let token: string | null = null;
let expiresAt = 0;
let profile: { id: string; email: string; roles: string[] } | null = null;
let refreshing: Promise<string | null> | null = null;
const auditLog: Array<{ at: number; event: string }> = [];
function record(event: string): void {
auditLog.push({ at: deps.now(), event });
if (auditLog.length > 200) auditLog.splice(0, auditLog.length - 200);
}
/** Exchange credentials for a session token and cache the profile. */
async function signIn(email: string, password: string): Promise<boolean> {
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ email });
let payload: unknown;
try {
payload = await deps.fetchJson(url);
} catch (error) {
record(`signIn failed: ${error instanceof Error ? error.message : String(error)}`);
return false;
}
if (typeof payload !== 'object' || payload === null) {
record('signIn got a non-object payload');
return false;
}
const body = payload as { token?: string; expiresAt?: number; profile?: typeof profile };
if (typeof body.token !== 'string' || body.token.length === 0) {
record('signIn payload carried no token');
return false;
}
void password;
token = body.token;
expiresAt = typeof body.expiresAt === 'number' ? body.expiresAt : deps.now() + 3_600_000;
profile = body.profile ?? null;
record(`signIn ok for ${email}`);
return true;
}
/** Drop every trace of the session, locally and on the server. */
async function signOut(): Promise<void> {
if (token === null) return;
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ action: 'revoke' });
try {
await deps.fetchJson(url);
} catch (error) {
record(`signOut revoke failed: ${error instanceof Error ? error.message : String(error)}`);
}
token = null;
expiresAt = 0;
profile = null;
refreshing = null;
record('signOut complete');
}
/**
* Renew the token before it expires. Concurrent callers share one in-flight
* request so a burst of requests cannot start a refresh storm.
*/
async function refreshToken(): Promise<string | null> {
if (token === null) return null;
if (refreshing !== null) return refreshing;
refreshing = (async () => {
const url = joinPath(baseUrl, SESSION_ENDPOINT) + toQueryString({ action: 'refresh' });
try {
const payload = await deps.fetchJson(url);
const body = payload as { token?: string; expiresAt?: number };
if (typeof body?.token === 'string' && body.token.length > 0) {
token = body.token;
expiresAt = typeof body.expiresAt === 'number' ? body.expiresAt : deps.now() + 3_600_000;
record('refreshToken renewed the session');
return token;
}
record('refreshToken payload carried no token');
return null;
} catch (error) {
record(`refreshToken failed: ${error instanceof Error ? error.message : String(error)}`);
return null;
} finally {
refreshing = null;
}
})();
return refreshing;
}
/** The token to send with a request, renewing it first when it is close to expiry. */
async function authorize(): Promise<string | null> {
if (token === null) return null;
if (deps.now() + REFRESH_SKEW_MS < expiresAt) return token;
return refreshToken();
}
/** Does the signed-in user hold every one of these roles? */
function hasRoles(...required: string[]): boolean {
if (profile === null) return false;
const held = new Set(profile.roles);
for (const role of required) {
if (!held.has(role)) return false;
}
return true;
}
/** Seconds left on the session, floored at zero. */
function secondsRemaining(): number {
if (token === null) return 0;
return Math.max(0, Math.floor((expiresAt - deps.now()) / 1000));
}
/** The last N audit entries, newest first — what the account page renders. */
function recentActivity(limit = 20): Array<{ at: number; event: string }> {
return auditLog.slice(-limit).reverse();
}
function snapshot() {
return {
signedIn: token !== null,
email: profile?.email ?? null,
roles: profile?.roles ?? [],
secondsRemaining: secondsRemaining(),
};
}
return {
signIn,
signOut,
refreshToken,
authorize,
hasRoles,
secondsRemaining,
recentActivity,
snapshot,
};
}
__tests__/fixtures/factory-closure-ts/src/stores/types.ts
+28
View File
@@ -0,0 +1,28 @@
export interface Widget {
id: string;
kind: 'chart' | 'table' | 'stat';
title: string;
column: number;
row: number;
span: number;
hidden: boolean;
}
export interface MetricSample {
widgetId: string;
at: number;
value: number;
unit: string;
}
export interface FilterSpec {
field: string;
op: 'eq' | 'gt' | 'lt' | 'contains';
value: string;
}
export interface StoreDeps {
fetchJson: (url: string) => Promise<unknown>;
now: () => number;
log: (message: string) => void;
}
__tests__/fixtures/factory-closure-ts/src/ui/panel.ts
+43
View File
@@ -0,0 +1,43 @@
import type { DashboardStore } from '../stores/dashboard-store';
import type { FilterSpec } from '../stores/types';
import { medianOf } from '../lib/metrics';
/** The dashboard panel — the only consumer of the store's closures. */
export function mountPanel(store: DashboardStore, dashboardId: string) {
let disposed = false;
const unsubscribe = store.subscribe((state) => {
if (disposed) return;
render(state.widgets.length, state.sampleCount, state.loading);
});
async function boot(): Promise<void> {
await store.loadWidgets(dashboardId);
await store.refreshMetrics();
store.reconcileLayout();
}
function search(text: string): void {
const specs: FilterSpec[] = text.trim().length === 0
? []
: [{ field: 'title', op: 'contains', value: text.trim() }];
store.applyFilter(specs);
}
function download(): string {
return store.exportCsv();
}
function render(widgetCount: number, sampleCount: number, loading: boolean): void {
void widgetCount;
void sampleCount;
void loading;
}
function dispose(): void {
disposed = true;
unsubscribe();
}
return { boot, search, download, dispose, median: medianOf };
}
__tests__/fixtures/oversize-member-ts/package.json
+6
View File
@@ -0,0 +1,6 @@
{
"name": "oversize-member-fixture",
"version": "1.0.0",
"private": true,
"type": "module"
}
__tests__/fixtures/oversize-member-ts/src/index.ts
+14
View File
@@ -0,0 +1,14 @@
import { buildMonthlyReport } from './report/monthly';
import { buildWeeklyReport } from './report/weekly';
import { buildQuarterlyReport } from './report/quarterly';
import { formatReportRows } from './report/format';
import type { Ledger, ReportOptions } from './report/types';
/** Run every report for a ledger and render them. */
export function runReports(ledger: Ledger, options: ReportOptions): string {
return [
formatReportRows(buildMonthlyReport(ledger, options)),
formatReportRows(buildWeeklyReport(ledger, options)),
formatReportRows(buildQuarterlyReport(ledger, options)),
].join('\n\n');
}
__tests__/fixtures/oversize-member-ts/src/report/format.ts
+22
View File
@@ -0,0 +1,22 @@
import type { ReportRow } from './types';
/** Format one category total as a report row. */
export function formatReportRow(category: string, amountCents: number, currency: string): ReportRow {
return {
category,
amount: formatAmount(amountCents),
currency,
};
}
/** Render cents as a fixed-point amount. */
export function formatAmount(amountCents: number): string {
const sign = amountCents < 0 ? '-' : '';
const abs = Math.abs(amountCents);
return `${sign}${Math.floor(abs / 100)}.${String(abs % 100).padStart(2, '0')}`;
}
/** Render a set of rows as plain text. */
export function formatReportRows(rows: ReportRow[]): string {
return rows.map((row) => `${row.category}\t${row.amount} ${row.currency}`).join('\n');
}
__tests__/fixtures/oversize-member-ts/src/report/monthly.ts
+509
View File
@@ -0,0 +1,509 @@
import { formatReportRow } from './format';
import { persistReport } from './store';
import type { Ledger, ReportOptions, ReportRow } from './types';
/**
* Build the monthly report for one ledger.
*
* Every expense category is accrued in its own block so the finance team can
* read the month end-to-end in one place; the shape is deliberately flat.
*/
export function buildMonthlyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
const rows: ReportRow[] = [];
const totals = new Map<string, number>();
// 1. payroll — accrue the payroll component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'payroll');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('payroll', adjusted, options.currency));
totals.set('payroll', adjusted);
}
}
// 2. benefits — accrue the benefits component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'benefits');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('benefits', adjusted, options.currency));
totals.set('benefits', adjusted);
}
}
// 3. travel — accrue the travel component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'travel');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('travel', adjusted, options.currency));
totals.set('travel', adjusted);
}
}
// 4. equipment — accrue the equipment component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'equipment');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('equipment', adjusted, options.currency));
totals.set('equipment', adjusted);
}
}
// 5. software — accrue the software component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'software');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('software', adjusted, options.currency));
totals.set('software', adjusted);
}
}
// 6. contractors — accrue the contractors component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'contractors');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('contractors', adjusted, options.currency));
totals.set('contractors', adjusted);
}
}
// 7. marketing — accrue the marketing component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'marketing');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('marketing', adjusted, options.currency));
totals.set('marketing', adjusted);
}
}
// 8. training — accrue the training component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'training');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('training', adjusted, options.currency));
totals.set('training', adjusted);
}
}
// 9. utilities — accrue the utilities component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'utilities');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('utilities', adjusted, options.currency));
totals.set('utilities', adjusted);
}
}
// 10. rent — accrue the rent component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'rent');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('rent', adjusted, options.currency));
totals.set('rent', adjusted);
}
}
// 11. insurance — accrue the insurance component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'insurance');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('insurance', adjusted, options.currency));
totals.set('insurance', adjusted);
}
}
// 12. legal — accrue the legal component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'legal');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('legal', adjusted, options.currency));
totals.set('legal', adjusted);
}
}
// 13. shipping — accrue the shipping component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'shipping');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('shipping', adjusted, options.currency));
totals.set('shipping', adjusted);
}
}
// 14. hosting — accrue the hosting component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'hosting');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('hosting', adjusted, options.currency));
totals.set('hosting', adjusted);
}
}
// 15. support — accrue the support component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'support');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('support', adjusted, options.currency));
totals.set('support', adjusted);
}
}
// 16. recruiting — accrue the recruiting component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'recruiting');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('recruiting', adjusted, options.currency));
totals.set('recruiting', adjusted);
}
}
// 17. licenses — accrue the licenses component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'licenses');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('licenses', adjusted, options.currency));
totals.set('licenses', adjusted);
}
}
// 18. taxes — accrue the taxes component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'taxes');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('taxes', adjusted, options.currency));
totals.set('taxes', adjusted);
}
}
// 19. refunds — accrue the refunds component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'refunds');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('refunds', adjusted, options.currency));
totals.set('refunds', adjusted);
}
}
// 20. discounts — accrue the discounts component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'discounts');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('discounts', adjusted, options.currency));
totals.set('discounts', adjusted);
}
}
// 21. interest — accrue the interest component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'interest');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('interest', adjusted, options.currency));
totals.set('interest', adjusted);
}
}
// 22. depreciation — accrue the depreciation component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'depreciation');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('depreciation', adjusted, options.currency));
totals.set('depreciation', adjusted);
}
}
// 23. maintenance — accrue the maintenance component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'maintenance');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('maintenance', adjusted, options.currency));
totals.set('maintenance', adjusted);
}
}
// 24. subscriptions — accrue the subscriptions component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'subscriptions');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('subscriptions', adjusted, options.currency));
totals.set('subscriptions', adjusted);
}
}
// 25. hardware — accrue the hardware component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'hardware');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('hardware', adjusted, options.currency));
totals.set('hardware', adjusted);
}
}
// 26. catering — accrue the catering component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'catering');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('catering', adjusted, options.currency));
totals.set('catering', adjusted);
}
}
// 27. conferences — accrue the conferences component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'conferences');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('conferences', adjusted, options.currency));
totals.set('conferences', adjusted);
}
}
// 28. advertising — accrue the advertising component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'advertising');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('advertising', adjusted, options.currency));
totals.set('advertising', adjusted);
}
}
// 29. research — accrue the research component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'research');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('research', adjusted, options.currency));
totals.set('research', adjusted);
}
}
// 30. logistics — accrue the logistics component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'logistics');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('logistics', adjusted, options.currency));
totals.set('logistics', adjusted);
}
}
// 31. warranty — accrue the warranty component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'warranty');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('warranty', adjusted, options.currency));
totals.set('warranty', adjusted);
}
}
// 32. penalties — accrue the penalties component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'penalties');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('penalties', adjusted, options.currency));
totals.set('penalties', adjusted);
}
}
// 33. bonuses — accrue the bonuses component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'bonuses');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('bonuses', adjusted, options.currency));
totals.set('bonuses', adjusted);
}
}
// 34. commissions — accrue the commissions component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'commissions');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('commissions', adjusted, options.currency));
totals.set('commissions', adjusted);
}
}
// 35. relocation — accrue the relocation component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'relocation');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('relocation', adjusted, options.currency));
totals.set('relocation', adjusted);
}
}
// 36. tooling — accrue the tooling component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'tooling');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('tooling', adjusted, options.currency));
totals.set('tooling', adjusted);
}
}
// 37. audit — accrue the audit component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'audit');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('audit', adjusted, options.currency));
totals.set('audit', adjusted);
}
}
// 38. compliance — accrue the compliance component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'compliance');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('compliance', adjusted, options.currency));
totals.set('compliance', adjusted);
}
}
// 39. storage — accrue the storage component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'storage');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('storage', adjusted, options.currency));
totals.set('storage', adjusted);
}
}
// 40. bandwidth — accrue the bandwidth component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'bandwidth');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('bandwidth', adjusted, options.currency));
totals.set('bandwidth', adjusted);
}
}
const grandTotal = [...totals.values()].reduce((sum, value) => sum + value, 0);
rows.push(formatReportRow('total', grandTotal, options.currency));
persistReport(ledger.periodId, rows);
return rows;
}
/** Header line for a rendered monthly report. */
export function monthlyReportHeader(ledger: Ledger, options: ReportOptions): string {
return `Monthly report ${ledger.periodId} (${options.currency})`;
}
/** Footer line for a rendered monthly report. */
export function monthlyReportFooter(rows: ReportRow[]): string {
return `${rows.length} categories reported`;
}
__tests__/fixtures/oversize-member-ts/src/report/quarterly.ts
+235
View File
@@ -0,0 +1,235 @@
import { formatReportRow } from './format';
import { persistReport } from './store';
import type { Ledger, ReportOptions, ReportRow } from './types';
/** Build the quarterly report for one ledger. */
export function buildQuarterlyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
const rows: ReportRow[] = [];
const totals = new Map<string, number>();
// 1. insurance — accrue the insurance component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'insurance');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('insurance', adjusted, options.currency));
totals.set('insurance', adjusted);
}
}
// 2. legal — accrue the legal component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'legal');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('legal', adjusted, options.currency));
totals.set('legal', adjusted);
}
}
// 3. shipping — accrue the shipping component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'shipping');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('shipping', adjusted, options.currency));
totals.set('shipping', adjusted);
}
}
// 4. hosting — accrue the hosting component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'hosting');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('hosting', adjusted, options.currency));
totals.set('hosting', adjusted);
}
}
// 5. support — accrue the support component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'support');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('support', adjusted, options.currency));
totals.set('support', adjusted);
}
}
// 6. recruiting — accrue the recruiting component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'recruiting');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('recruiting', adjusted, options.currency));
totals.set('recruiting', adjusted);
}
}
// 7. licenses — accrue the licenses component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'licenses');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('licenses', adjusted, options.currency));
totals.set('licenses', adjusted);
}
}
// 8. taxes — accrue the taxes component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'taxes');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('taxes', adjusted, options.currency));
totals.set('taxes', adjusted);
}
}
// 9. refunds — accrue the refunds component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'refunds');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('refunds', adjusted, options.currency));
totals.set('refunds', adjusted);
}
}
// 10. discounts — accrue the discounts component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'discounts');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('discounts', adjusted, options.currency));
totals.set('discounts', adjusted);
}
}
// 11. interest — accrue the interest component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'interest');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('interest', adjusted, options.currency));
totals.set('interest', adjusted);
}
}
// 12. depreciation — accrue the depreciation component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'depreciation');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('depreciation', adjusted, options.currency));
totals.set('depreciation', adjusted);
}
}
// 13. maintenance — accrue the maintenance component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'maintenance');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('maintenance', adjusted, options.currency));
totals.set('maintenance', adjusted);
}
}
// 14. subscriptions — accrue the subscriptions component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'subscriptions');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('subscriptions', adjusted, options.currency));
totals.set('subscriptions', adjusted);
}
}
// 15. hardware — accrue the hardware component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'hardware');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('hardware', adjusted, options.currency));
totals.set('hardware', adjusted);
}
}
// 16. catering — accrue the catering component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'catering');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('catering', adjusted, options.currency));
totals.set('catering', adjusted);
}
}
// 17. conferences — accrue the conferences component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'conferences');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('conferences', adjusted, options.currency));
totals.set('conferences', adjusted);
}
}
// 18. advertising — accrue the advertising component of the month.
{
const bucket = ledger.entries.filter((entry) => entry.category === 'advertising');
const gross = bucket.reduce((sum, entry) => sum + entry.amountCents, 0);
const pending = bucket.filter((entry) => entry.pending).reduce((s, e) => s + e.amountCents, 0);
const adjusted = options.includePending ? gross : gross - pending;
if (adjusted !== 0 || options.includeEmptyCategories) {
rows.push(formatReportRow('advertising', adjusted, options.currency));
totals.set('advertising', adjusted);
}
}
const grandTotal = [...totals.values()].reduce((sum, value) => sum + value, 0);
rows.push(formatReportRow('total', grandTotal, options.currency));
persistReport(ledger.periodId, rows);
return rows;
}
/** Header line for a rendered quarterly report. */
export function buildQuarterlyReportHeader(ledger: Ledger, options: ReportOptions): string {
return `quarterly report ${ledger.periodId} (${options.currency})`;
}
__tests__/fixtures/oversize-member-ts/src/report/store.ts
+18
View File
@@ -0,0 +1,18 @@
import type { ReportRow } from './types';
const saved = new Map<string, ReportRow[]>();
/** Persist a built report for a period. */
export function persistReport(periodId: string, rows: ReportRow[]): void {
saved.set(periodId, rows);
}
/** Read back a persisted report. */
export function loadReport(periodId: string): ReportRow[] {
return saved.get(periodId) ?? [];
}
/** Drop a persisted report. */
export function clearReport(periodId: string): void {
saved.delete(periodId);
}
__tests__/fixtures/oversize-member-ts/src/report/types.ts
+28
View File
@@ -0,0 +1,28 @@
/** One posted ledger entry. */
export interface LedgerEntry {
id: string;
category: string;
amountCents: number;
pending: boolean;
postedAt: string;
}
/** A period's ledger. */
export interface Ledger {
periodId: string;
entries: LedgerEntry[];
}
/** How a report should be built. */
export interface ReportOptions {
currency: string;
includePending: boolean;
includeEmptyCategories: boolean;
}
/** One rendered report line. */
export interface ReportRow {
category: string;
amount: string;
currency: string;
}
__tests__/fixtures/oversize-member-ts/src/report/weekly.ts
+372
View File
@@ -0,0 +1,372 @@
import { formatReportRow } from './format';
import { persistReport } from './store';
import type { Ledger, ReportOptions, ReportRow } from './types';
/** Total the posted entries in one category. */
function sumOf(ledger: Ledger, category: string): number {
return ledger.entries
.filter((entry) => entry.category === category && !entry.pending)
.reduce((sum, entry) => sum + entry.amountCents, 0);
}
/** Total the still-pending entries in one category. */
function pendingOf(ledger: Ledger, category: string): number {
return ledger.entries
.filter((entry) => entry.category === category && entry.pending)
.reduce((sum, entry) => sum + entry.amountCents, 0);
}
/** Build the weekly report for one ledger. */
export function buildWeeklyReport(ledger: Ledger, options: ReportOptions): ReportRow[] {
const rows: ReportRow[] = [];
const totals = new Map<string, number>();
// 1. payroll
{
const gross = sumOf(ledger, 'payroll');
const held = pendingOf(ledger, 'payroll');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('payroll', net, options.currency));
totals.set('payroll', net);
}
}
// 2. benefits
{
const gross = sumOf(ledger, 'benefits');
const held = pendingOf(ledger, 'benefits');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('benefits', net, options.currency));
totals.set('benefits', net);
}
}
// 3. travel
{
const gross = sumOf(ledger, 'travel');
const held = pendingOf(ledger, 'travel');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('travel', net, options.currency));
totals.set('travel', net);
}
}
// 4. equipment
{
const gross = sumOf(ledger, 'equipment');
const held = pendingOf(ledger, 'equipment');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('equipment', net, options.currency));
totals.set('equipment', net);
}
}
// 5. software
{
const gross = sumOf(ledger, 'software');
const held = pendingOf(ledger, 'software');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('software', net, options.currency));
totals.set('software', net);
}
}
// 6. contractors
{
const gross = sumOf(ledger, 'contractors');
const held = pendingOf(ledger, 'contractors');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('contractors', net, options.currency));
totals.set('contractors', net);
}
}
// 7. marketing
{
const gross = sumOf(ledger, 'marketing');
const held = pendingOf(ledger, 'marketing');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('marketing', net, options.currency));
totals.set('marketing', net);
}
}
// 8. training
{
const gross = sumOf(ledger, 'training');
const held = pendingOf(ledger, 'training');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('training', net, options.currency));
totals.set('training', net);
}
}
// 9. utilities
{
const gross = sumOf(ledger, 'utilities');
const held = pendingOf(ledger, 'utilities');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('utilities', net, options.currency));
totals.set('utilities', net);
}
}
// 10. rent
{
const gross = sumOf(ledger, 'rent');
const held = pendingOf(ledger, 'rent');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('rent', net, options.currency));
totals.set('rent', net);
}
}
// 11. insurance
{
const gross = sumOf(ledger, 'insurance');
const held = pendingOf(ledger, 'insurance');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('insurance', net, options.currency));
totals.set('insurance', net);
}
}
// 12. legal
{
const gross = sumOf(ledger, 'legal');
const held = pendingOf(ledger, 'legal');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('legal', net, options.currency));
totals.set('legal', net);
}
}
// 13. shipping
{
const gross = sumOf(ledger, 'shipping');
const held = pendingOf(ledger, 'shipping');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('shipping', net, options.currency));
totals.set('shipping', net);
}
}
// 14. hosting
{
const gross = sumOf(ledger, 'hosting');
const held = pendingOf(ledger, 'hosting');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('hosting', net, options.currency));
totals.set('hosting', net);
}
}
// 15. support
{
const gross = sumOf(ledger, 'support');
const held = pendingOf(ledger, 'support');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('support', net, options.currency));
totals.set('support', net);
}
}
// 16. recruiting
{
const gross = sumOf(ledger, 'recruiting');
const held = pendingOf(ledger, 'recruiting');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('recruiting', net, options.currency));
totals.set('recruiting', net);
}
}
// 17. licenses
{
const gross = sumOf(ledger, 'licenses');
const held = pendingOf(ledger, 'licenses');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('licenses', net, options.currency));
totals.set('licenses', net);
}
}
// 18. taxes
{
const gross = sumOf(ledger, 'taxes');
const held = pendingOf(ledger, 'taxes');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('taxes', net, options.currency));
totals.set('taxes', net);
}
}
// 19. refunds
{
const gross = sumOf(ledger, 'refunds');
const held = pendingOf(ledger, 'refunds');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('refunds', net, options.currency));
totals.set('refunds', net);
}
}
// 20. discounts
{
const gross = sumOf(ledger, 'discounts');
const held = pendingOf(ledger, 'discounts');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('discounts', net, options.currency));
totals.set('discounts', net);
}
}
// 21. interest
{
const gross = sumOf(ledger, 'interest');
const held = pendingOf(ledger, 'interest');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('interest', net, options.currency));
totals.set('interest', net);
}
}
// 22. depreciation
{
const gross = sumOf(ledger, 'depreciation');
const held = pendingOf(ledger, 'depreciation');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('depreciation', net, options.currency));
totals.set('depreciation', net);
}
}
// 23. maintenance
{
const gross = sumOf(ledger, 'maintenance');
const held = pendingOf(ledger, 'maintenance');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('maintenance', net, options.currency));
totals.set('maintenance', net);
}
}
// 24. subscriptions
{
const gross = sumOf(ledger, 'subscriptions');
const held = pendingOf(ledger, 'subscriptions');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('subscriptions', net, options.currency));
totals.set('subscriptions', net);
}
}
// 25. hardware
{
const gross = sumOf(ledger, 'hardware');
const held = pendingOf(ledger, 'hardware');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('hardware', net, options.currency));
totals.set('hardware', net);
}
}
// 26. catering
{
const gross = sumOf(ledger, 'catering');
const held = pendingOf(ledger, 'catering');
const net = options.includePending
? gross
: gross - held;
if (net !== 0) {
rows.push(formatReportRow('catering', net, options.currency));
totals.set('catering', net);
}
}
const grandTotal = [...totals.values()]
.reduce((sum, value) => sum + value, 0);
rows.push(formatReportRow('total', grandTotal, options.currency));
persistReport(ledger.periodId, rows);
return rows;
}
/** Header line for a rendered weekly report. */
export function buildWeeklyReportHeader(ledger: Ledger, options: ReportOptions): string {
return `weekly report ${ledger.periodId} (${options.currency})`;
}
__tests__/fixtures/starved-cluster-ts/package.json
+6
View File
@@ -0,0 +1,6 @@
{
"name": "starved-cluster-fixture",
"private": true,
"version": "0.0.0",
"type": "module"
}
__tests__/fixtures/starved-cluster-ts/src/app/client.ts
+24
View File
@@ -0,0 +1,24 @@
import { RequestChain, describeChain } from '../pipeline/chain';
import type { PipelineRequest, PipelineResponse } from '../pipeline/types';
import { openSocket } from '../transport/socket';
/**
* The entry point a caller reaches for. Everything the chain does happens
* underneath this call, which is why a flow question names it.
*/
export async function sendRequest(request: PipelineRequest): Promise<PipelineResponse> {
const socket = openSocket(request.host, request.port);
const chain = new RequestChain(request, socket);
trace(describeChain(chain));
return chain.proceed(request);
}
export function trace(line: string): void {
if (process.env.PIPELINE_TRACE) process.stderr.write(`${line}\n`);
}
export async function sendAll(requests: PipelineRequest[]): Promise<PipelineResponse[]> {
const out: PipelineResponse[] = [];
for (const request of requests) out.push(await sendRequest(request));
return out;
}
__tests__/fixtures/starved-cluster-ts/src/app/config.ts
+14
View File
@@ -0,0 +1,14 @@
export interface ClientConfig {
host: string;
port: number;
retries: number;
userAgent: string;
}
export function defaultConfig(): ClientConfig {
return { host: 'localhost', port: 8080, retries: 3, userAgent: 'pipeline/1.0' };
}
export function withHost(config: ClientConfig, host: string): ClientConfig {
return { ...config, host };
}
__tests__/fixtures/starved-cluster-ts/src/index.ts
+4
View File
@@ -0,0 +1,4 @@
export { sendRequest, sendAll } from './app/client';
export { RequestChain, describeChain } from './pipeline/chain';
export { openSocket } from './transport/socket';
export { defaultConfig } from './app/config';
__tests__/fixtures/starved-cluster-ts/src/pipeline/chain.ts
+318
View File
@@ -0,0 +1,318 @@
import type { PipelineRequest, PipelineResponse, Interceptor, Socket } from './types';
import { encodeFrame, decodeFrame } from './framing';
import { defaultInterceptors } from './interceptors';
/**
* A one-line summary of a chain, used only by the tracing hook in the caller.
* It is TRIVIAL — it answers nothing about how a request travels — but it sits
* next to the entry point in the call graph, so its cluster carries the file's
* highest per-symbol importance.
*/
export function describeChain(chain: RequestChain): string {
return `chain(${chain.index}/${chain.size}) -> ${chain.hostLabel}`;
}
// ---------------------------------------------------------------------------
//
// Everything below is the part a "how does a request reach the socket" question
// is actually asking about. It is separated from the helper above by more than
// the cluster gap threshold, so it forms its own cluster — a large one, whose
// symbols are reached transitively rather than named.
//
// ---------------------------------------------------------------------------
export class RequestChain {
readonly index: number;
readonly size: number;
readonly hostLabel: string;
private readonly interceptors: Interceptor[];
private readonly socket: Socket;
private readonly request: PipelineRequest;
private connectTimeoutMs = 10_000;
private readTimeoutMs = 10_000;
private writeTimeoutMs = 10_000;
private calls = 0;
constructor(request: PipelineRequest, socket: Socket, index = 0, interceptors?: Interceptor[]) {
this.request = request;
this.socket = socket;
this.index = index;
this.interceptors = interceptors ?? defaultInterceptors();
this.size = this.interceptors.length;
this.hostLabel = `${request.host}:${request.port}`;
}
/**
* Run the request through the remaining interceptors and, once they are
* exhausted, hand it to the transport. This is the method the flow question
* is about: every hop between the caller and the socket passes through here.
*/
async proceed(request: PipelineRequest): Promise<PipelineResponse> {
if (this.index >= this.size) {
return this.writeAndRead(request);
}
this.calls += 1;
if (this.calls > 1) {
throw new Error(`chain link ${this.index} called ${this.calls} times`);
}
const next = this.advance(request);
const interceptor = this.interceptors[this.index]!;
const response = await interceptor.intercept(next);
if (!response) {
throw new Error(`interceptor ${interceptor.name} returned no response`);
}
if (this.index + 1 < this.size && next.callCount() === 0) {
throw new Error(`interceptor ${interceptor.name} must call proceed()`);
}
return response;
}
/**
* The next link in the chain: the same chain with the cursor moved on and the
* timeouts carried over. Cloning here is what keeps each interceptor from
* mutating the chain the one before it is still holding.
*/
advance(request: PipelineRequest): RequestChain {
const next = new RequestChain(request, this.socket, this.index + 1, this.interceptors);
next.connectTimeoutMs = this.connectTimeoutMs;
next.readTimeoutMs = this.readTimeoutMs;
next.writeTimeoutMs = this.writeTimeoutMs;
return next;
}
callCount(): number {
return this.calls;
}
/**
* The end of the chain: frame the request, put the bytes on the socket, wait
* for the reply and decode it. Past this point there is no more pipeline —
* this is the transport hop the question is looking for.
*/
private async writeAndRead(request: PipelineRequest): Promise<PipelineResponse> {
const frame = encodeFrame(request);
await this.socket.connect(this.connectTimeoutMs);
await this.socket.write(frame, this.writeTimeoutMs);
const raw = await this.socket.read(this.readTimeoutMs);
const decoded = decodeFrame(raw);
return {
status: decoded.status,
headers: decoded.headers,
body: decoded.body,
request,
};
}
withConnectTimeout(ms: number): RequestChain {
const next = this.advance(this.request);
next.connectTimeoutMs = checkDuration('connectTimeout', ms);
return next;
}
withReadTimeout(ms: number): RequestChain {
const next = this.advance(this.request);
next.readTimeoutMs = checkDuration('readTimeout', ms);
return next;
}
withWriteTimeout(ms: number): RequestChain {
const next = this.advance(this.request);
next.writeTimeoutMs = checkDuration('writeTimeout', ms);
return next;
}
connectTimeout(): number {
return this.connectTimeoutMs;
}
readTimeout(): number {
return this.readTimeoutMs;
}
writeTimeout(): number {
return this.writeTimeoutMs;
}
/**
* Retry policy for the transport hop. Sits inside the same cluster as the
* proceed/advance pair, so it is part of what a shrink has to choose between.
*/
async retryWrite(request: PipelineRequest, attempts: number): Promise<PipelineResponse> {
let lastError: unknown;
for (let attempt = 0; attempt < attempts; attempt += 1) {
try {
return await this.writeAndRead(request);
} catch (error) {
lastError = error;
await backoff(attempt);
}
}
throw lastError;
}
/** Whether the chain may still be resumed after a transport failure. */
canRetry(error: unknown): boolean {
if (this.index >= this.size) return false;
if (!(error instanceof Error)) return false;
return error.message.includes('timeout') || error.message.includes('reset');
}
/** The interceptor names, in the order the request will visit them. */
route(): string[] {
return this.interceptors.slice(this.index).map((i) => i.name);
}
/** A copy of the chain rewound to the first interceptor. */
rewind(): RequestChain {
return new RequestChain(this.request, this.socket, 0, this.interceptors);
}
/** Drop one interceptor by name and return the shortened chain. */
without(name: string): RequestChain {
const kept = this.interceptors.filter((i) => i.name !== name);
return new RequestChain(this.request, this.socket, this.index, kept);
}
/** Append an interceptor to the end of the chain. */
with(interceptor: Interceptor): RequestChain {
return new RequestChain(
this.request,
this.socket,
this.index,
[...this.interceptors, interceptor],
);
}
/** Close the transport this chain was built around. */
async close(): Promise<void> {
await this.socket.close();
}
/** Headers the transport hop will actually put on the wire. */
effectiveHeaders(): Record<string, string> {
const headers: Record<string, string> = { ...this.request.headers };
headers['host'] = this.hostLabel;
headers['x-chain-index'] = String(this.index);
headers['x-chain-size'] = String(this.size);
if (this.request.body) headers['content-length'] = String(this.request.body.length);
return headers;
}
/** The request as the next link will see it, with the chain's headers merged. */
prepared(): PipelineRequest {
return { ...this.request, headers: this.effectiveHeaders() };
}
/**
* Send the prepared request through the rest of the chain. The convenience
* wrapper most callers use instead of building the request themselves.
*/
async send(): Promise<PipelineResponse> {
return this.proceed(this.prepared());
}
/** Whether the chain has any interceptor left before the transport hop. */
hasNext(): boolean {
return this.index < this.size;
}
/** The interceptor the next `proceed` will run, if there is one. */
peek(): Interceptor | undefined {
return this.interceptors[this.index];
}
/** Total configured wait for one attempt, across all three timeouts. */
totalTimeout(): number {
return this.connectTimeoutMs + this.readTimeoutMs + this.writeTimeoutMs;
}
/** Apply one timeout budget to all three phases at once. */
withTimeout(ms: number): RequestChain {
const next = this.advance(this.request);
const checked = checkDuration('timeout', ms);
next.connectTimeoutMs = checked;
next.readTimeoutMs = checked;
next.writeTimeoutMs = checked;
return next;
}
/**
* Run the chain and translate a transport failure into a response, so a
* caller that only cares about the status code never sees an exception.
*/
async sendOrStatus(status: number): Promise<PipelineResponse> {
try {
return await this.send();
} catch {
return {
status,
headers: this.effectiveHeaders(),
body: new Uint8Array(),
request: this.request,
};
}
}
/** A short description of where in the chain this link sits. */
position(): string {
return `${this.index + 1} of ${this.size + 1}`;
}
/** The chain rebuilt around a different transport. */
onSocket(socket: Socket): RequestChain {
return new RequestChain(this.request, socket, this.index, this.interceptors);
}
/**
* Replay the request through the chain from the start, reusing the transport.
* Used when an interceptor decides the response it got is not usable and the
* whole pipeline has to run again against the same connection.
*/
async replay(): Promise<PipelineResponse> {
const fresh = this.rewind();
try {
return await fresh.send();
} finally {
if (!fresh.hasNext()) await fresh.close();
}
}
/**
* Validate the chain before it runs: every interceptor named once, timeouts
* inside their bounds, and a transport still open at the end of it.
*/
validate(): string[] {
const problems: string[] = [];
const seen = new Set<string>();
for (const interceptor of this.interceptors) {
if (seen.has(interceptor.name)) problems.push(`duplicate interceptor ${interceptor.name}`);
seen.add(interceptor.name);
}
if (this.connectTimeoutMs <= 0) problems.push('connect timeout must be positive');
if (this.readTimeoutMs <= 0) problems.push('read timeout must be positive');
if (this.writeTimeoutMs <= 0) problems.push('write timeout must be positive');
if (this.index > this.size) problems.push('chain cursor is past the end');
return problems;
}
/**
* The transport hop on its own, with the chain's timeouts but none of its
* interceptors — the escape hatch a caller uses to bypass the pipeline.
*/
async direct(request: PipelineRequest): Promise<PipelineResponse> {
const problems = this.validate();
if (problems.length > 0) throw new Error(problems.join('; '));
return this.writeAndRead(request);
}
}
function checkDuration(name: string, ms: number): number {
if (!Number.isFinite(ms) || ms < 0) throw new Error(`${name} must be a positive duration`);
if (ms > 24 * 60 * 60 * 1000) throw new Error(`${name} is longer than a day`);
return Math.round(ms);
}
async function backoff(attempt: number): Promise<void> {
const ms = Math.min(1000, 25 * 2 ** attempt);
await new Promise((resolve) => setTimeout(resolve, ms));
}
__tests__/fixtures/starved-cluster-ts/src/pipeline/framing.ts
+26
View File
@@ -0,0 +1,26 @@
import type { PipelineRequest } from './types';
export function encodeFrame(request: PipelineRequest): Uint8Array {
const head = `${request.method} ${request.path}\n`;
const headers = Object.entries(request.headers).map(([k, v]) => `${k}: ${v}`).join('\n');
const text = `${head}${headers}\n\n`;
const body = request.body ?? new Uint8Array();
const out = new Uint8Array(text.length + body.length);
out.set(new TextEncoder().encode(text), 0);
out.set(body, text.length);
return out;
}
export function decodeFrame(raw: Uint8Array): { status: number; headers: Record<string, string>; body: Uint8Array } {
const text = new TextDecoder().decode(raw);
const split = text.indexOf('\n\n');
const head = split < 0 ? text : text.slice(0, split);
const lines = head.split('\n');
const status = Number.parseInt(lines[0]?.split(' ')[1] ?? '0', 10);
const headers: Record<string, string> = {};
for (const line of lines.slice(1)) {
const at = line.indexOf(': ');
if (at > 0) headers[line.slice(0, at)] = line.slice(at + 2);
}
return { status, headers, body: raw.slice(split < 0 ? raw.length : split + 2) };
}
__tests__/fixtures/starved-cluster-ts/src/pipeline/interceptors.ts
+21
View File
@@ -0,0 +1,21 @@
import type { Interceptor } from './types';
export function defaultInterceptors(): Interceptor[] {
return [retryInterceptor(), headerInterceptor(), logInterceptor()];
}
export function retryInterceptor(): Interceptor {
return { name: 'retry', intercept: (chain) => chain.proceed(currentRequest()) };
}
export function headerInterceptor(): Interceptor {
return { name: 'headers', intercept: (chain) => chain.proceed(currentRequest()) };
}
export function logInterceptor(): Interceptor {
return { name: 'log', intercept: (chain) => chain.proceed(currentRequest()) };
}
function currentRequest() {
return { host: 'localhost', port: 80, method: 'GET', path: '/', headers: {} };
}
__tests__/fixtures/starved-cluster-ts/src/pipeline/types.ts
+27
View File
@@ -0,0 +1,27 @@
export interface PipelineRequest {
host: string;
port: number;
method: string;
path: string;
headers: Record<string, string>;
body?: Uint8Array;
}
export interface PipelineResponse {
status: number;
headers: Record<string, string>;
body: Uint8Array;
request: PipelineRequest;
}
export interface Interceptor {
name: string;
intercept(chain: { proceed(request: PipelineRequest): Promise<PipelineResponse> }): Promise<PipelineResponse>;
}
export interface Socket {
connect(timeoutMs: number): Promise<void>;
write(frame: Uint8Array, timeoutMs: number): Promise<void>;
read(timeoutMs: number): Promise<Uint8Array>;
close(): Promise<void>;
}
__tests__/fixtures/starved-cluster-ts/src/transport/socket.ts
+30
View File
@@ -0,0 +1,30 @@
import type { Socket } from '../pipeline/types';
/** Open a transport socket for a host/port pair. */
export function openSocket(host: string, port: number): Socket {
let open = false;
const inbox: Uint8Array[] = [];
return {
async connect(timeoutMs: number) {
if (open) return;
await settle(timeoutMs);
open = true;
},
async write(frame: Uint8Array, timeoutMs: number) {
if (!open) throw new Error(`socket to ${host}:${port} is not connected`);
await settle(timeoutMs);
inbox.push(frame);
},
async read(timeoutMs: number) {
await settle(timeoutMs);
return inbox.shift() ?? new Uint8Array();
},
async close() {
open = false;
},
};
}
async function settle(timeoutMs: number): Promise<void> {
if (timeoutMs <= 0) throw new Error('timed out');
}
__tests__/fixtures/tail-render-ts/README.md
+39
View File
@@ -0,0 +1,39 @@
# tail-render-ts — CG-38
An agent-named symbol sitting in the TAIL of a large file must render.
This mirrors the geometry of the reported file (a 1,414-line Svelte chat store)
closely enough that the same two defects reproduce, and it is that geometry — not
any individual line — that the fixture exists to hold:
| | line | why it matters |
|---|---|---|
| `QueuedMessage` (interface) | 70 | the DECOY. Same stem as the query token, near the top, cheap to render — it is what the broken build returned *instead of* the functions. |
| `createSessionStore` (function) | 104–1417 | the ENVELOPE. Spans ~92% of the file, and `function` is deliberately **not** in `ENVELOPE_KINDS` (CG-27), so every symbol inside merges into ONE cluster that must then be shrunk and trimmed. |
| `handleStreamMessage` | ~554 | a 290-line god-method in the middle, so the head of the file has plenty to spend the budget on. |
| `queueMessage` | 1088 | TARGET. Past line 1,000. |
| `removeQueuedMessage` | 1096 | TARGET. |
| `flushQueuedMessages` | 1102 | TARGET. Past line 1,000. |
Two more pieces are load-bearing:
- **`queueMessage` never calls `flushQueuedMessages`** (both push to / drain the same
array instead). That absence is what produced no call chain, no synthesized hop and
no dispatch boundary — and so made `buildFlowFromNamedSymbols` throw the
named-symbol identity away along with the narrative it had nothing to print.
- **`types/worker-configuration.d.ts`** — 2,500 lines of generated Wrangler ambient
types, carrying the `Generated by wrangler. DO NOT EDIT.` banner so the ranker flags
and penalises it. It is what makes the fixture able to test the issue's
index-dependence lead: a penalty on this file moves `maxGraph`, which moves the 6%
relevance gate, which moves every other file's allowance — and must still not cost
the top-ranked file the definitions the agent named.
`src/lib/session-store.ts` is machine-generated to hit those line numbers with real,
extractable TypeScript. If you need to change it, change the geometry (the target
line numbers, the closure span, the decoy's position) rather than editing individual
lines — the fixture-shape assertions in
`__tests__/explore-named-symbol-render.test.ts` will tell you if it has rotted.
Gate: `__tests__/explore-named-symbol-render.test.ts`.
Probe: `node scripts/agent-eval/probe-named-symbol.mjs`.
Numbers: `docs/benchmarks/explore-tail-render-cg38.md`.
__tests__/fixtures/tail-render-ts/package.json
+6
View File
@@ -0,0 +1,6 @@
{
"name": "tail-render-fixture",
"private": true,
"version": "0.0.0",
"type": "module"
}
__tests__/fixtures/tail-render-ts/src/components/ChatComposer.ts
+27
View File
@@ -0,0 +1,27 @@
import { createSessionStore } from '../lib/session-store';
/** The composer owns the textarea and decides send-vs-queue. */
export function createComposer(endpoint: string) {
const store = createSessionStore({
getProjectId: () => 'demo',
getEndpoint: () => endpoint,
onError: () => {},
});
let draft = '';
function setDraft(next: string) {
draft = next;
}
function submit(streaming: boolean) {
if (streaming) store.queueMessage(draft);
else store.sendMessage(draft, [], []);
draft = '';
}
function onTurnEnd() {
store.flushQueuedMessages();
}
return { setDraft, submit, onTurnEnd, store };
}
__tests__/fixtures/tail-render-ts/src/lib/message-builder.ts
+32
View File
@@ -0,0 +1,32 @@
import type { AttachedFile, SelectedElementRef } from './session-store';
export interface BuiltMessage {
id: string;
text: string;
attachments: number;
}
/** Render the selected canvas elements as a fenced block above the prose. */
export function renderElementBlock(elements: SelectedElementRef[]): string {
if (elements.length === 0) return '';
const lines = elements.map((e) => `- ${e.kind}: ${e.label} (${e.id})`);
return ['```elements', ...lines, '```'].join('\n');
}
export function formatStylesBlock(files: AttachedFile[]): string {
return files.map((f) => `${f.path} (${f.mime}, ${f.bytes}b)`).join('\n');
}
export function buildMessage(
content: string,
files: AttachedFile[],
elements: SelectedElementRef[],
): BuiltMessage {
const block = renderElementBlock(elements);
const styles = formatStylesBlock(files);
return {
id: `m-${content.length}-${files.length}`,
text: [block, styles, content].filter(Boolean).join('\n\n'),
attachments: files.length,
};
}
__tests__/fixtures/tail-render-ts/src/lib/session-store.ts
+1417
View File
File diff suppressed because it is too large Load Diff
__tests__/fixtures/tail-render-ts/src/lib/socket.ts
+27
View File
@@ -0,0 +1,27 @@
export interface Socket {
emit(event: string, payload: unknown): void;
on(event: string, handler: (chunk: unknown) => void): void;
close(): void;
}
/** One socket per chat session, so two tabs never receive each other's chunks. */
export function createDedicatedSocket(endpoint: string): Socket {
const handlers = new Map<string, Array<(chunk: unknown) => void>>();
return {
emit(event, payload) {
void endpoint;
void event;
void payload;
},
on(event, handler) {
handlers.set(event, [...(handlers.get(event) ?? []), handler]);
},
close() {
handlers.clear();
},
};
}
export function describeSocket(socket: Socket | null): string {
return socket ? 'connected' : 'detached';
}
__tests__/fixtures/tail-render-ts/types/worker-configuration.d.ts
Vendored
+2527
View File
File diff suppressed because it is too large Load Diff
__tests__/generated-detection.test.ts
+16
View File
@@ -118,6 +118,14 @@ describe('hasGeneratedHeader — per-marker coverage (#1500)', () => {
'/* automatically generated by rust-bindgen 0.59.2 */\n\npub const FOO: u32 = 1;\n',
],
['ANTLR — "Generated from … -- DO NOT EDIT"', '// Generated from Expr.g4 by ANTLR 4.9.2 -- DO NOT EDIT\npackage parser;\n'],
[
'Wrangler — "Generated by Wrangler by running `wrangler types`" (CG-25)',
'/* eslint-disable */\n// Generated by Wrangler by running `wrangler types` (hash: adcfde101dd7d9077590b6b39d3eaf8d)\n// Runtime types generated with workerd@1.20260708.1 2026-07-12\ndeclare namespace Cloudflare {\n\tinterface Env {}\n}\n',
],
[
'the same "regenerate by running" shape from an in-house CLI',
'# Generated by ./scripts/schema-gen.py by running `make schema`\n\nfrom typing import Any\n',
],
[
'banner on an unprefixed line INSIDE a block comment',
'/*\n Code generated by ent. DO NOT EDIT.\n*/\npackage ent\n',
@@ -155,6 +163,14 @@ describe('hasGeneratedHeader — per-marker coverage (#1500)', () => {
],
['an email address that happens to contain "@generated"', '// Contact: build@generated.example.com for issues.\npackage main\n'],
['"DO NOT EDIT" with no generation claim', '// DO NOT EDIT THIS FILE BY HAND — run `make fmt` instead.\npackage main\n'],
[
'prose: bare "generated by" naming no tool and no reproduction command (CG-25)',
'// The table below is generated by the build at runtime, so the\n// literal values here are only a fallback.\npackage main\n',
],
[
'prose: "generated by running …" — one "by" clause, not the Wrangler shape (CG-25)',
'// The nightly summary is generated by running the ETL job against\n// yesterday\'s partition.\npackage main\n',
],
['empty file', ''],
];
__tests__/installer-targets.test.ts
+548
View File
@@ -18,6 +18,7 @@ import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { parse as parseJsonc } from 'jsonc-parser';
import { ALL_TARGETS, getTarget, resolveTargetFlag } from '../src/installer/targets/registry';
import { uninstallTargets, refreshTargets } from '../src/installer';
import { upsertTomlTable, removeTomlTable, buildTomlTable } from '../src/installer/targets/toml';
@@ -38,12 +39,14 @@ function setHome(dir: string): { restore: () => void } {
APPDATA: process.env.APPDATA,
XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME,
HERMES_HOME: process.env.HERMES_HOME,
COPILOT_HOME: process.env.COPILOT_HOME,
};
process.env.HOME = dir;
process.env.USERPROFILE = dir;
process.env.APPDATA = path.join(dir, '.config');
process.env.XDG_CONFIG_HOME = path.join(dir, '.config');
delete process.env.HERMES_HOME;
delete process.env.COPILOT_HOME;
return {
restore() {
if (prev.HOME === undefined) delete process.env.HOME; else process.env.HOME = prev.HOME;
@@ -51,6 +54,7 @@ function setHome(dir: string): { restore: () => void } {
if (prev.APPDATA === undefined) delete process.env.APPDATA; else process.env.APPDATA = prev.APPDATA;
if (prev.XDG_CONFIG_HOME === undefined) delete process.env.XDG_CONFIG_HOME; else process.env.XDG_CONFIG_HOME = prev.XDG_CONFIG_HOME;
if (prev.HERMES_HOME === undefined) delete process.env.HERMES_HOME; else process.env.HERMES_HOME = prev.HERMES_HOME;
if (prev.COPILOT_HOME === undefined) delete process.env.COPILOT_HOME; else process.env.COPILOT_HOME = prev.COPILOT_HOME;
},
};
}
@@ -136,6 +140,12 @@ describe('Installer targets — contract', () => {
delete seed.mcpServers;
seed.mcp = { other: { type: 'local', command: ['x'], enabled: true } };
}
// VS Code's mcp.json uses `servers`; the JetBrains Copilot
// plugin's mcp.json is schema-compatible with it.
if (target.id === 'copilot-vscode' || target.id === 'copilot-jetbrains') {
delete seed.mcpServers;
seed.servers = { other: { command: 'x' } };
}
fs.writeFileSync(jsonPath, JSON.stringify(seed, null, 2) + '\n');
target.install(location, { autoAllow: true });
@@ -144,6 +154,9 @@ describe('Installer targets — contract', () => {
if (target.id === 'opencode') {
expect(after.mcp.other).toBeDefined();
expect(after.mcp.codegraph).toBeDefined();
} else if (target.id === 'copilot-vscode' || target.id === 'copilot-jetbrains') {
expect(after.servers.other).toBeDefined();
expect(after.servers.codegraph).toBeDefined();
} else {
expect(after.mcpServers.other).toBeDefined();
expect(after.mcpServers.codegraph).toBeDefined();
@@ -1268,6 +1281,9 @@ describe('Installer targets — registry', () => {
expect(getTarget('gemini')?.id).toBe('gemini');
expect(getTarget('antigravity')?.id).toBe('antigravity');
expect(getTarget('kiro')?.id).toBe('kiro');
expect(getTarget('copilot-vscode')?.id).toBe('copilot-vscode');
expect(getTarget('copilot-cli')?.id).toBe('copilot-cli');
expect(getTarget('copilot-jetbrains')?.id).toBe('copilot-jetbrains');
expect(getTarget('not-a-real-target')).toBeUndefined();
});
@@ -1278,6 +1294,18 @@ describe('Installer targets — registry', () => {
expect(csv.map((t) => t.id)).toEqual(['claude', 'cursor']);
});
it("resolveTargetFlag('all') includes every Copilot target", () => {
const ids = resolveTargetFlag('all', 'global').map((t) => t.id);
expect(ids).toContain('copilot-vscode');
expect(ids).toContain('copilot-cli');
expect(ids).toContain('copilot-jetbrains');
});
it('resolveTargetFlag resolves the Copilot ids from a csv list', () => {
const csv = resolveTargetFlag('copilot-vscode,copilot-cli,copilot-jetbrains', 'global');
expect(csv.map((t) => t.id)).toEqual(['copilot-vscode', 'copilot-cli', 'copilot-jetbrains']);
});
it('resolveTargetFlag throws on unknown id', () => {
expect(() => resolveTargetFlag('claude,bogus', 'global')).toThrow(/Unknown --target/);
});
@@ -1897,3 +1925,523 @@ describe('Installer targets — opencode XDG config path (#535)', () => {
expect(opencode.detect('global').alreadyConfigured).toBe(false);
});
});
// ---------------------------------------------------------------------------
// Copilot family — copilot-vscode / copilot-cli / copilot-jetbrains (CG-5)
//
// The registry-driven contract suite above covers the shared surface
// (install/idempotency/sibling/uninstall/printConfig). These pin the
// target-specific behavior: OS-specific global paths, `--path` injection
// (copilot-vscode mirrors Cursor), global-only skip semantics (cli +
// jetbrains, Codex pattern), COPILOT_HOME resolution, JSONC comment
// preservation, empty-`servers`-wrapper cleanup, and printConfig parity
// with what install writes.
// ---------------------------------------------------------------------------
describe('Installer targets — Copilot family', () => {
let tmpHome: string;
let tmpCwd: string;
let origCwd: string;
let homeRestore: { restore: () => void };
beforeEach(() => {
tmpHome = mkTmpDir('cop-home');
tmpCwd = mkTmpDir('cop-cwd');
origCwd = process.cwd();
process.chdir(tmpCwd);
homeRestore = setHome(tmpHome);
});
afterEach(() => {
homeRestore.restore();
process.chdir(origCwd);
fs.rmSync(tmpHome, { recursive: true, force: true });
fs.rmSync(tmpCwd, { recursive: true, force: true });
});
// printConfig embeds the paste-able snippet after a `# Add to <path>`
// header — extract and parse just the JSON body.
function snippetJson(out: string): any {
const start = out.indexOf('{');
expect(start).toBeGreaterThanOrEqual(0);
return JSON.parse(out.slice(start));
}
// ---- copilot-vscode ----
it('copilot-vscode: local install writes ./.vscode/mcp.json with servers.codegraph and an absolute --path pin', () => {
const t = getTarget('copilot-vscode')!;
const result = t.install('local', { autoAllow: true });
const file = path.join(process.cwd(), '.vscode', 'mcp.json');
expect(result.files[0].path).toBe(file);
expect(result.files[0].action).toBe('created');
const cfg = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(cfg.servers.codegraph.type).toBe('stdio');
expect(cfg.servers.codegraph.command).toBe('codegraph');
// Cursor-mirror: local installs pin the project with an absolute path.
expect(cfg.servers.codegraph.args).toEqual(['serve', '--mcp', '--path', process.cwd()]);
// No mcpServers wrapper — VS Code's mcp.json uses `servers`.
expect(cfg.mcpServers).toBeUndefined();
});
it('copilot-vscode: global install writes a variable-free entry — no --path, no ${workspaceFolder}', () => {
// VS Code refuses to start a user-level server whose entry uses
// ${workspaceFolder} in any window with no folder open, toasting
// "Variable workspaceFolder can not be resolved" (hit live). VS Code
// documents cwd = workspace folder for stdio servers, and the
// codegraph server resolves the project from roots/cwd — so the
// global entry must carry no --path and no variables at all.
const t = getTarget('copilot-vscode')!;
const result = t.install('global', { autoAllow: true });
const cfg = JSON.parse(fs.readFileSync(result.files[0].path, 'utf-8'));
expect(cfg.servers.codegraph.args).toEqual(['serve', '--mcp']);
expect(JSON.stringify(cfg)).not.toContain('${');
});
it.runIf(process.platform === 'darwin')('copilot-vscode: global path is ~/Library/Application Support/Code/User/mcp.json on macOS', () => {
const t = getTarget('copilot-vscode')!;
const expected = path.join(tmpHome, 'Library', 'Application Support', 'Code', 'User', 'mcp.json');
expect(t.describePaths('global')).toEqual([expected]);
const result = t.install('global', { autoAllow: true });
expect(result.files[0].path).toBe(expected);
expect(fs.existsSync(expected)).toBe(true);
});
it.runIf(process.platform === 'linux')('copilot-vscode: global path honors XDG_CONFIG_HOME on Linux', () => {
const t = getTarget('copilot-vscode')!;
// setHome() points XDG_CONFIG_HOME at <home>/.config.
const expected = path.join(tmpHome, '.config', 'Code', 'User', 'mcp.json');
expect(t.describePaths('global')).toEqual([expected]);
const result = t.install('global', { autoAllow: true });
expect(result.files[0].path).toBe(expected);
});
it.runIf(process.platform === 'win32')('copilot-vscode: global path is %APPDATA%\\Code\\User\\mcp.json on Windows', () => {
const t = getTarget('copilot-vscode')!;
// setHome() points APPDATA at <home>/.config.
const expected = path.join(process.env.APPDATA!, 'Code', 'User', 'mcp.json');
expect(t.describePaths('global')).toEqual([expected]);
const result = t.install('global', { autoAllow: true });
expect(result.files[0].path).toBe(expected);
});
it('copilot-vscode: supports both global and local locations', () => {
const t = getTarget('copilot-vscode')!;
expect(t.supportsLocation('global')).toBe(true);
expect(t.supportsLocation('local')).toBe(true);
});
it('copilot-vscode: preserves comments and sibling servers through install + idempotent re-run (JSONC)', () => {
const t = getTarget('copilot-vscode')!;
const dir = path.join(tmpCwd, '.vscode');
fs.mkdirSync(dir, { recursive: true });
const file = path.join(dir, 'mcp.json');
fs.writeFileSync(file, [
'{',
' // my MCP servers',
' "servers": {',
' "other": { "type": "stdio", "command": "other-server" } // keep',
' }',
'}',
'',
].join('\n'));
t.install('local', { autoAllow: true });
const afterInstall = fs.readFileSync(file, 'utf-8');
expect(afterInstall).toContain('// my MCP servers');
expect(afterInstall).toContain('// keep');
expect(afterInstall).toContain('"other-server"');
expect(afterInstall).toContain('"codegraph"');
const second = t.install('local', { autoAllow: true });
expect(second.files[0].action).toBe('unchanged');
expect(fs.readFileSync(file, 'utf-8')).toBe(afterInstall);
});
it('copilot-vscode: uninstall drops an emptied servers wrapper but keeps the file and its siblings (e.g. inputs)', () => {
const t = getTarget('copilot-vscode')!;
const dir = path.join(tmpCwd, '.vscode');
fs.mkdirSync(dir, { recursive: true });
const file = path.join(dir, 'mcp.json');
fs.writeFileSync(file, [
'{',
' // prompt-time inputs',
' "inputs": [{ "id": "api-key", "type": "promptString" }]',
'}',
'',
].join('\n'));
t.install('local', { autoAllow: true });
const result = t.uninstall('local');
expect(result.files[0].action).toBe('removed');
// File survives; our entry and the now-empty `servers` wrapper are gone.
expect(fs.existsSync(file)).toBe(true);
const text = fs.readFileSync(file, 'utf-8');
expect(text).toContain('// prompt-time inputs');
const cfg = parseJsonc(text);
expect(cfg.inputs).toBeDefined();
expect(cfg.servers).toBeUndefined();
expect(text).not.toContain('codegraph');
});
it('copilot-vscode: uninstall keeps a non-empty servers wrapper (sibling server survives)', () => {
const t = getTarget('copilot-vscode')!;
const file = path.join(tmpCwd, '.vscode', 'mcp.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify({
servers: { other: { type: 'stdio', command: 'other-server' } },
}, null, 2) + '\n');
t.install('local', { autoAllow: true });
t.uninstall('local');
const cfg = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(cfg.servers.other).toBeDefined();
expect(cfg.servers.codegraph).toBeUndefined();
});
it('copilot-vscode: uninstall when never installed reports not-found for both locations, no throw', () => {
const t = getTarget('copilot-vscode')!;
for (const loc of ['global', 'local'] as const) {
const result = t.uninstall(loc);
expect(result.files).toHaveLength(1);
expect(result.files[0].action).toBe('not-found');
}
});
it('copilot-vscode: detect() local reports installed only when a .vscode dir exists', () => {
const t = getTarget('copilot-vscode')!;
expect(t.detect('local').installed).toBe(false);
fs.mkdirSync(path.join(tmpCwd, '.vscode'), { recursive: true });
expect(t.detect('local').installed).toBe(true);
expect(t.detect('local').alreadyConfigured).toBe(false);
});
it('copilot-vscode: detect() global falls back to ~/.vscode (extensions dir) as the installed heuristic', () => {
const t = getTarget('copilot-vscode')!;
expect(t.detect('global').installed).toBe(false);
fs.mkdirSync(path.join(tmpHome, '.vscode'), { recursive: true });
expect(t.detect('global').installed).toBe(true);
});
it('copilot-vscode: printConfig matches what install writes, at both locations', () => {
const t = getTarget('copilot-vscode')!;
for (const loc of ['global', 'local'] as const) {
const printed = snippetJson(t.printConfig(loc));
const result = t.install(loc, { autoAllow: true });
const onDisk = JSON.parse(fs.readFileSync(result.files[0].path, 'utf-8'));
expect(printed.servers.codegraph).toEqual(onDisk.servers.codegraph);
}
});
it('copilot-vscode: install note tells the user to restart VS Code', () => {
const t = getTarget('copilot-vscode')!;
const result = t.install('local', { autoAllow: true });
expect(result.notes?.join(' ')).toMatch(/[Rr]estart VS Code/);
});
// ---- copilot-cli ----
it('copilot-cli: global install writes ~/.copilot/mcp-config.json with the documented entry shape (tools: ["*"])', () => {
const t = getTarget('copilot-cli')!;
const result = t.install('global', { autoAllow: true });
const file = path.join(tmpHome, '.copilot', 'mcp-config.json');
expect(result.files[0].path).toBe(file);
expect(result.files[0].action).toBe('created');
const cfg = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(cfg.mcpServers.codegraph).toEqual({
type: 'stdio',
command: 'codegraph',
args: ['serve', '--mcp'],
tools: ['*'],
});
});
it('copilot-cli: is global-only — local install skips with a clear note, uninstall is a no-op', () => {
const t = getTarget('copilot-cli')!;
expect(t.supportsLocation('local')).toBe(false);
expect(t.supportsLocation('global')).toBe(true);
const install = t.install('local', { autoAllow: true });
expect(install.files).toEqual([]);
expect(install.notes?.join(' ')).toMatch(/no project-local config/);
expect(t.uninstall('local').files).toEqual([]);
expect(t.describePaths('local')).toEqual([]);
expect(t.detect('local').installed).toBe(false);
});
it('copilot-cli: honors the COPILOT_HOME override for install, detect, and uninstall', () => {
const t = getTarget('copilot-cli')!;
const custom = path.join(tmpHome, 'copilot-custom');
process.env.COPILOT_HOME = custom;
const result = t.install('global', { autoAllow: true });
const expected = path.join(custom, 'mcp-config.json');
expect(result.files[0].path).toBe(expected);
expect(fs.existsSync(expected)).toBe(true);
expect(t.detect('global').alreadyConfigured).toBe(true);
// The default location was never touched.
expect(fs.existsSync(path.join(tmpHome, '.copilot'))).toBe(false);
t.uninstall('global');
expect(t.detect('global').alreadyConfigured).toBe(false);
});
it('copilot-cli: uninstall removes only codegraph — sibling server and unrelated keys survive', () => {
const t = getTarget('copilot-cli')!;
const file = path.join(tmpHome, '.copilot', 'mcp-config.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify({
mcpServers: { other: { type: 'stdio', command: 'other-server' } },
banner: 'never',
}, null, 2) + '\n');
t.install('global', { autoAllow: true });
t.uninstall('global');
const after = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(after.mcpServers.other).toBeDefined();
expect(after.mcpServers.codegraph).toBeUndefined();
expect(after.banner).toBe('never');
});
it('copilot-cli: uninstall of a from-scratch install deletes the file — no `{}` husk to fool detect()', () => {
const t = getTarget('copilot-cli')!;
t.install('global', { autoAllow: true });
t.uninstall('global');
const file = path.join(tmpHome, '.copilot', 'mcp-config.json');
// A leftover empty mcp-config.json would count as a CLI footprint
// and keep the target showing as detected after uninstall.
expect(fs.existsSync(file)).toBe(false);
});
it('copilot-cli: uninstall keeps the file when unrelated top-level keys remain', () => {
const t = getTarget('copilot-cli')!;
const file = path.join(tmpHome, '.copilot', 'mcp-config.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify({ banner: 'never' }, null, 2) + '\n');
t.install('global', { autoAllow: true });
t.uninstall('global');
const after = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(after.mcpServers).toBeUndefined();
expect(after.banner).toBe('never');
});
it('copilot-cli: uninstall when never installed reports not-found, no throw', () => {
const t = getTarget('copilot-cli')!;
const result = t.uninstall('global');
expect(result.files).toHaveLength(1);
expect(result.files[0].action).toBe('not-found');
// Same when the file exists but holds no codegraph entry.
const file = path.join(tmpHome, '.copilot', 'mcp-config.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify({ mcpServers: { other: { command: 'x' } } }) + '\n');
expect(t.uninstall('global').files[0].action).toBe('not-found');
});
it('copilot-cli: detect() reports installed from CLI artifacts in ~/.copilot', () => {
const t = getTarget('copilot-cli')!;
// The tmp PATH may or may not carry a real `copilot` binary; only
// assert the positive signal we control. The CLI writes config.json
// on first run — that's the footprint.
fs.mkdirSync(path.join(tmpHome, '.copilot'), { recursive: true });
fs.writeFileSync(path.join(tmpHome, '.copilot', 'config.json'), '{}');
expect(t.detect('global').installed).toBe(true);
expect(t.detect('global').alreadyConfigured).toBe(false);
});
it('copilot-cli: detect() is NOT fooled by the VS Code extension\'s ~/.copilot/ide/ locks', () => {
// The VS Code Copilot Chat extension writes MCP socket-handoff lock
// files into ~/.copilot/ide/ on every launch — a machine with only
// the extension has ~/.copilot with a lone `ide` entry and no CLI.
const t = getTarget('copilot-cli')!;
const ideDir = path.join(tmpHome, '.copilot', 'ide');
fs.mkdirSync(ideDir, { recursive: true });
fs.writeFileSync(path.join(ideDir, 'some-uuid.lock'), '{"socketPath":"/tmp/mcp.sock"}');
// Pin PATH to an empty dir so a real `copilot` binary on the host
// can't turn this negative assertion into a false failure.
const prevPath = process.env.PATH;
process.env.PATH = ideDir;
try {
expect(t.detect('global').installed).toBe(false);
// An empty ~/.copilot (no CLI footprint at all) is also not enough.
fs.rmSync(ideDir, { recursive: true });
expect(t.detect('global').installed).toBe(false);
} finally {
process.env.PATH = prevPath;
}
});
it('copilot-cli: printConfig matches what install writes; local variant points at --location=global', () => {
const t = getTarget('copilot-cli')!;
const printed = snippetJson(t.printConfig('global'));
const result = t.install('global', { autoAllow: true });
const onDisk = JSON.parse(fs.readFileSync(result.files[0].path, 'utf-8'));
expect(printed.mcpServers.codegraph).toEqual(onDisk.mcpServers.codegraph);
expect(t.printConfig('local')).toMatch(/--location=global/);
});
// ---- copilot-jetbrains ----
it('copilot-jetbrains: global install writes github-copilot/intellij/mcp.json with the VS Code-compatible servers shape', () => {
const t = getTarget('copilot-jetbrains')!;
const result = t.install('global', { autoAllow: true });
// setHome() sets XDG_CONFIG_HOME, honored on every platform.
const file = path.join(tmpHome, '.config', 'github-copilot', 'intellij', 'mcp.json');
expect(result.files[0].path).toBe(file);
expect(result.files[0].action).toBe('created');
const cfg = JSON.parse(fs.readFileSync(file, 'utf-8'));
// Plain entry — no --path injection for this user-global config.
expect(cfg.servers.codegraph).toEqual({ type: 'stdio', command: 'codegraph', args: ['serve', '--mcp'] });
expect(cfg.mcpServers).toBeUndefined();
});
it.runIf(process.platform !== 'win32')('copilot-jetbrains: falls back to ~/.config/github-copilot when XDG_CONFIG_HOME is unset', () => {
delete process.env.XDG_CONFIG_HOME;
const t = getTarget('copilot-jetbrains')!;
const expected = path.join(tmpHome, '.config', 'github-copilot', 'intellij', 'mcp.json');
expect(t.describePaths('global')).toEqual([expected]);
const result = t.install('global', { autoAllow: true });
expect(result.files[0].path).toBe(expected);
});
it.runIf(process.platform === 'win32')('copilot-jetbrains: falls back to %LOCALAPPDATA%\\github-copilot on Windows when XDG_CONFIG_HOME is unset', () => {
const prevLocal = process.env.LOCALAPPDATA;
delete process.env.XDG_CONFIG_HOME;
process.env.LOCALAPPDATA = path.join(tmpHome, 'AppData', 'Local');
try {
const t = getTarget('copilot-jetbrains')!;
const expected = path.join(tmpHome, 'AppData', 'Local', 'github-copilot', 'intellij', 'mcp.json');
expect(t.describePaths('global')).toEqual([expected]);
const result = t.install('global', { autoAllow: true });
expect(result.files[0].path).toBe(expected);
} finally {
if (prevLocal === undefined) delete process.env.LOCALAPPDATA;
else process.env.LOCALAPPDATA = prevLocal;
}
});
it('copilot-jetbrains: is global-only — local install skips with a clear note, uninstall is a no-op', () => {
const t = getTarget('copilot-jetbrains')!;
expect(t.supportsLocation('local')).toBe(false);
expect(t.supportsLocation('global')).toBe(true);
const install = t.install('local', { autoAllow: true });
expect(install.files).toEqual([]);
expect(install.notes?.join(' ')).toMatch(/no project-local MCP config/);
expect(t.uninstall('local').files).toEqual([]);
expect(t.describePaths('local')).toEqual([]);
expect(t.detect('local').installed).toBe(false);
});
it('copilot-jetbrains: preserves comments and sibling servers through install + idempotent re-run (JSONC)', () => {
const t = getTarget('copilot-jetbrains')!;
const file = path.join(tmpHome, '.config', 'github-copilot', 'intellij', 'mcp.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, [
'{',
' // hand-edited via Settings → Tools → GitHub Copilot',
' "servers": {',
' "other": { "type": "stdio", "command": "other-server" }',
' }',
'}',
'',
].join('\n'));
t.install('global', { autoAllow: true });
const afterInstall = fs.readFileSync(file, 'utf-8');
expect(afterInstall).toContain('// hand-edited via Settings');
expect(afterInstall).toContain('"other-server"');
expect(afterInstall).toContain('"codegraph"');
const second = t.install('global', { autoAllow: true });
expect(second.files[0].action).toBe('unchanged');
expect(fs.readFileSync(file, 'utf-8')).toBe(afterInstall);
});
it('copilot-jetbrains: uninstall removes only codegraph and drops an emptied servers wrapper, keeping the file', () => {
const t = getTarget('copilot-jetbrains')!;
t.install('global', { autoAllow: true });
const file = path.join(tmpHome, '.config', 'github-copilot', 'intellij', 'mcp.json');
const result = t.uninstall('global');
expect(result.files[0].action).toBe('removed');
expect(fs.existsSync(file)).toBe(true);
const cfg = parseJsonc(fs.readFileSync(file, 'utf-8'));
expect(cfg.servers).toBeUndefined();
});
it('copilot-jetbrains: uninstall keeps a sibling server (wrapper not dropped when non-empty)', () => {
const t = getTarget('copilot-jetbrains')!;
const file = path.join(tmpHome, '.config', 'github-copilot', 'intellij', 'mcp.json');
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify({
servers: { other: { type: 'stdio', command: 'other-server' } },
}, null, 2) + '\n');
t.install('global', { autoAllow: true });
t.uninstall('global');
const cfg = JSON.parse(fs.readFileSync(file, 'utf-8'));
expect(cfg.servers.other).toBeDefined();
expect(cfg.servers.codegraph).toBeUndefined();
});
it('copilot-jetbrains: uninstall when never installed reports not-found, no throw', () => {
const t = getTarget('copilot-jetbrains')!;
const result = t.uninstall('global');
expect(result.files).toHaveLength(1);
expect(result.files[0].action).toBe('not-found');
});
it('copilot-jetbrains: detect() reports installed from the intellij config dir', () => {
const t = getTarget('copilot-jetbrains')!;
expect(t.detect('global').installed).toBe(false);
fs.mkdirSync(path.join(tmpHome, '.config', 'github-copilot', 'intellij'), { recursive: true });
expect(t.detect('global').installed).toBe(true);
expect(t.detect('global').alreadyConfigured).toBe(false);
});
it('copilot-jetbrains: printConfig matches what install writes and names the IDE settings path', () => {
const t = getTarget('copilot-jetbrains')!;
const out = t.printConfig('global');
expect(out).toContain('Settings → Tools → GitHub Copilot');
const printed = snippetJson(out);
const result = t.install('global', { autoAllow: true });
const onDisk = JSON.parse(fs.readFileSync(result.files[0].path, 'utf-8'));
expect(printed.servers.codegraph).toEqual(onDisk.servers.codegraph);
expect(t.printConfig('local')).toMatch(/--location=global/);
});
it('copilot-jetbrains: install note tells the user to restart the IDE', () => {
const t = getTarget('copilot-jetbrains')!;
const result = t.install('global', { autoAllow: true });
expect(result.notes?.join(' ')).toMatch(/[Rr]estart your JetBrains IDE/);
});
it('copilot family: all three coexist — uninstalling one leaves the others configured', () => {
const vscode = getTarget('copilot-vscode')!;
const cli = getTarget('copilot-cli')!;
const jetbrains = getTarget('copilot-jetbrains')!;
vscode.install('global', { autoAllow: true });
cli.install('global', { autoAllow: true });
jetbrains.install('global', { autoAllow: true });
cli.uninstall('global');
expect(cli.detect('global').alreadyConfigured).toBe(false);
expect(vscode.detect('global').alreadyConfigured).toBe(true);
expect(jetbrains.detect('global').alreadyConfigured).toBe(true);
});
});
__tests__/sync-rebuild-convergence.test.ts
+440
View File
@@ -0,0 +1,440 @@
/**
* Incremental sync must converge to a full rebuild (CG-33).
*
* A long-lived, auto-synced index silently diverged from a clean rebuild of the
* identical tree: 4.3% of distinct edges wrong, in BOTH directions, on
* codegraph's own repo. Two mechanisms, both exercised here:
*
* 1. Resolution binds a reference to one of the same-named definitions
* PROJECT-WIDE, so adding or removing a definition changes the answer for
* references in files the sync never touches. Those references resolved once
* and their rows were deleted, so nothing revisited them — the index kept an
* answer that was only correct against an older graph.
* 2. When nothing disambiguated the candidates, the winner was whichever row
* the index scan reached first — i.e. the order files were WRITTEN. A full
* index writes in scan order; a sync appends each file as it changes, so the
* same tree resolved differently depending on how the index was built.
*
* The assertions here compare the whole edge SET, never counts: the divergence
* is bidirectional and nets out of a total (raw rows differed by 0.7% while
* 4.3% of edges were wrong), so a count check passes on a broken index.
*
* ---
*
* THIS SUITE MUST FAIL WITH `CODEGRAPH_NO_REBIND=1` (CG-35).
*
* That environment variable is the kill switch on the rebind half of the fix
* (`src/index.ts`, guarding `resurrectStaleResolutionEdges`). The convergence
* cases below are the only coverage that half has, so the check is the suite's
* own regression test:
*
* CODEGRAPH_NO_REBIND=1 npx vitest run __tests__/sync-rebuild-convergence.test.ts
*
* must report failures, and an unset run must be green. If you change a case
* here, re-run both. A version of this suite passed under the kill switch
* because `rebuildEdgeSet` was not rebuilding anything — see the note there.
*/
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import CodeGraph from '../src/index';
import { createDatabase } from '../src/db/sqlite-adapter';
describe('Incremental sync converges to a full rebuild (CG-33)', () => {
let testDir: string;
let cg: CodeGraph;
const write = (rel: string, content: string) => {
const full = path.join(testDir, rel);
fs.mkdirSync(path.dirname(full), { recursive: true });
fs.writeFileSync(full, content);
};
/**
* Every edge as a `source|target|kind` triple, read from the database with a
* second read-only connection. Node ids are `sha256(filePath:kind:name:line)`,
* so for an identical tree they are identical across a sync and a rebuild —
* which is what makes the two sets directly comparable.
*/
const edgeSet = (): Set<string> => {
const { db } = createDatabase(path.join(testDir, '.codegraph', 'codegraph.db'), { readOnly: true });
try {
const rows = db.prepare('SELECT source, target, kind FROM edges').all() as Array<{
source: string;
target: string;
kind: string;
}>;
return new Set(rows.map((r) => `${r.source}|${r.target}|${r.kind}`));
} finally {
db.close();
}
};
/**
* Run `fn` against a second, WRITABLE connection to the same database. Used
* by the two rule tests below to plant edge shapes the extractor cannot
* produce on demand — an edge from an engine older than the refName stamp,
* and a synthesized dispatch edge.
*/
const withDb = <T>(fn: (db: ReturnType<typeof createDatabase>['db']) => T): T => {
const { db } = createDatabase(path.join(testDir, '.codegraph', 'codegraph.db'));
try {
return fn(db);
} finally {
db.close();
}
};
/** Human-readable diff, so a failure names the edges instead of just a count. */
const describeDiff = (synced: Set<string>, rebuilt: Set<string>): string => {
const missing = [...rebuilt].filter((e) => !synced.has(e));
const stale = [...synced].filter((e) => !rebuilt.has(e));
return `missing from synced: ${missing.length}, stale in synced: ${stale.length}`;
};
/**
* Rebuild the index from scratch over the CURRENT tree and return its edge
* set — the ground truth a user gets from `codegraph index`.
*
* It must go through `CodeGraph.recreate`, which is what the CLI's `index`
* command does: it DELETES the database file and builds an empty one. Calling
* `indexAll` on the live handle instead is not a rebuild at all — every file
* hashes identical, so the store writes nothing (`nodesCreated: 0`), no
* reference is re-created, and every existing edge survives untouched. The
* comparison then reads the synced index against ITSELF and can never fail,
* which is exactly how this suite passed with `CODEGRAPH_NO_REBIND=1` (CG-35).
*/
const rebuildEdgeSet = async (): Promise<Set<string>> => {
// Close the live handle first: `recreate` unlinks the database file, and a
// held handle makes that EBUSY on Windows.
cg.destroy();
cg = await CodeGraph.recreate(testDir);
await cg.indexAll();
return edgeSet();
};
beforeEach(() => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg33-'));
});
afterEach(() => {
cg?.destroy();
if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
/**
* The originating shape. `caller.ts` calls `pct` with no import, so it binds
* by name; at index time `zeta.ts` is the only definition. A later sync adds
* `alpha.ts`, which sorts FIRST and is therefore the rebuild's answer — but
* `caller.ts` never changes, so nothing re-resolves it.
*/
it('rebinds references in UNCHANGED files when a sync adds a competing definition', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1);\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
write('src/alpha.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
const result = await cg.sync();
expect(result.filesAdded).toBe(1);
expect(result.definitionDelta).toContain('pct');
const synced = edgeSet();
const rebuilt = await rebuildEdgeSet();
expect(describeDiff(synced, rebuilt)).toBe('missing from synced: 0, stale in synced: 0');
});
/**
* The mirror direction: removing a definition narrows the candidate set too,
* so the delta must include names the sync DROPPED, not just names it added.
*
* This one already converged before the fix — a removal cascades the edge
* away and the #1240 removal path resurrects it, so the reference gets
* re-resolved for free. It is here as a standing guard on the invariant, and
* because the removal half of the delta has no other coverage: an
* implementation that only sampled post-sync names would still pass every
* other test in this file.
*/
it('rebinds references in UNCHANGED files when a sync removes a competing definition', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1);\n}\n`);
write('src/alpha.ts', `export function pct(n: number): number {\n return n;\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
fs.rmSync(path.join(testDir, 'src', 'alpha.ts'));
const result = await cg.sync();
expect(result.filesRemoved).toBe(1);
const synced = edgeSet();
const rebuilt = await rebuildEdgeSet();
expect(describeDiff(synced, rebuilt)).toBe('missing from synced: 0, stale in synced: 0');
});
/**
* The delta must be computed per FILE. Comparing one name set across the whole
* changed batch cancels a name that is added in one changed file while another
* changed file already defined it — which is precisely the shape a commit that
* splits a module out has, and it was the largest residual class in the first
* measurement of this fix.
*/
it('flags a name added in one changed file even when another changed file already defines it', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1);\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n;\n}\nexport function keep(): number {\n return 0;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
// One commit: a NEW file gains `pct`, and the file that already had `pct`
// is edited too (so a batch-wide name set would see `pct` on both sides).
write('src/alpha.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n + 1;\n}\nexport function keep(): number {\n return 0;\n}\n`);
const result = await cg.sync();
expect(result.definitionDelta).toContain('pct');
const synced = edgeSet();
const rebuilt = await rebuildEdgeSet();
expect(describeDiff(synced, rebuilt)).toBe('missing from synced: 0, stale in synced: 0');
});
/**
* The realistic case the issue was filed from: many edits driven through sync
* one after another, the way a watcher or a `git pull` applies them. Drift
* accumulated across syncs, so a single-edit test would not have caught it.
*/
it('stays converged across a sequence of adds, edits, renames and deletes', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1) + fmt(2) + collect(3);\n}\n`);
write('src/util/zeta.ts', `export function pct(n: number): number {\n return n;\n}\n`);
write('src/util/omega.ts', `export function fmt(n: number): number {\n return n;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
// 1. add a competing `pct` that sorts before the existing one
write('src/util/alpha.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
await cg.sync();
// 2. body-only edit — must produce NO definition delta, so the common sync
// pays nothing for this machinery
write('src/util/alpha.ts', `export function pct(n: number): number {\n return n * 3;\n}\n`);
const bodyOnly = await cg.sync();
expect(bodyOnly.filesModified).toBe(1);
expect(bodyOnly.definitionDelta).toBeUndefined();
// 3. a rename: `fmt` moves out of omega.ts into a file that sorts first
write('src/util/omega.ts', `export function other(n: number): number {\n return n;\n}\n`);
write('src/util/beta.ts', `export function fmt(n: number): number {\n return n;\n}\n`);
await cg.sync();
// 4. a symbol appears for a reference that never resolved at all
write('src/util/gamma.ts', `export function collect(n: number): number {\n return n;\n}\n`);
await cg.sync();
// 5. delete the current `pct` winner, so the reference must fall back...
fs.rmSync(path.join(testDir, 'src', 'util', 'alpha.ts'));
await cg.sync();
// 6. ...and then a later sync introduces a new winner ahead of it again.
// Ending here rather than on the delete matters: after the delete the
// binding happens to land back where it started, which a broken index
// also reaches. The final state must be one only re-resolution reaches.
write('src/util/aaa.ts', `export function pct(n: number): number {\n return n * 5;\n}\n`);
await cg.sync();
const synced = edgeSet();
expect(synced.size).toBeGreaterThan(0);
const rebuilt = await rebuildEdgeSet();
expect(describeDiff(synced, rebuilt)).toBe('missing from synced: 0, stale in synced: 0');
});
/**
* The rebind pass DELETES an edge and re-inserts the reference behind it, so
* it may only touch edges it can reconstruct. Two shapes it must leave alone,
* both of which it would otherwise destroy permanently:
*
* - an edge with no `metadata.refName` — written by an engine older than the
* stamp. Rebuilding a reference from the target's plain name would strip the
* receiver context the original text carried (`h.greet` → `greet`);
* - a synthesized dispatch edge (`provenance='heuristic'`), which is not
* resolution output at all: nothing would re-create it, and the synthesizer
* that wired it does not run again on this sync.
*
* Both are planted directly, since extraction cannot be asked to emit them.
* The sync then changes the answer for `pct`, which is exactly the condition
* that makes the pass want to re-open every edge targeting `pct`.
*/
it('never deletes an edge it cannot reconstruct — no refName stamp, or synthesized', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1);\n}\n`);
write('src/other.ts', `export function other(): number {\n return 0;\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
const planted = withDb((db) => {
const pct = db.prepare("SELECT id FROM nodes WHERE name = 'pct'").get() as { id: string };
const other = db.prepare("SELECT id FROM nodes WHERE name = 'other'").get() as { id: string };
// 1. Strip the stamp off the real edge, leaving the rest of its metadata
// intact — the shape an index built before the stamp existed has.
db.prepare(
`UPDATE edges SET metadata = json_remove(metadata, '$.refName')
WHERE target = ? AND kind = 'calls'`
).run(pct.id);
// 2. A synthesized edge that DOES carry a stamp, so only the provenance
// rule can save it.
db.prepare(
`INSERT INTO edges (source, target, kind, metadata, line, col, provenance)
VALUES (?, ?, 'calls', ?, 1, 0, 'heuristic')`
).run(other.id, pct.id, JSON.stringify({ refName: 'pct', synthesizedBy: 'cg35-test' }));
return {
unstamped: `${(db.prepare("SELECT source FROM edges WHERE target = ? AND provenance IS NULL AND kind = 'calls'").get(pct.id) as { source: string }).source}|${pct.id}|calls`,
synthesized: `${other.id}|${pct.id}|calls`,
};
});
const before = edgeSet();
expect(before.has(planted.unstamped)).toBe(true);
expect(before.has(planted.synthesized)).toBe(true);
write('src/alpha.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
const result = await cg.sync();
expect(result.definitionDelta).toContain('pct');
// Both survive: the pass considered them (their target is `pct`) and
// declined. Drift is the acceptable outcome here; an edge that no pass can
// ever restore is not.
const after = edgeSet();
expect(after.has(planted.unstamped)).toBe(true);
expect(after.has(planted.synthesized)).toBe(true);
});
/**
* The per-name ceiling in `getResolutionEdgesByTargetName` (500 by default).
* Above it a name is generic — `push`, `get`, `join` — one new definition
* won't flip most of its references, and rebinding an arbitrary subset would
* manufacture wrong edges while costing the most work. It must DECLINE the
* name outright, and declining must be lossless.
*
* The rare name in the same sync is the control: it proves the pass ran and
* that the ceiling is what spared the generic one, not a dead rebind pass.
*/
it('declines a name over the per-name ceiling instead of rebinding an arbitrary subset', async () => {
// Must exceed the 500 default in getResolutionEdgesByTargetName.
const OVER_CEILING = 501;
const callers = Array.from(
{ length: OVER_CEILING },
(_, i) => `export function hot${i}(): number {\n return push(${i});\n}\n`
).join('');
write('src/hot.ts', callers);
write('src/rare.ts', `export function rare(): number {\n return tug(1);\n}\n`);
write(
'src/zzz_defs.ts',
`export function push(n: number): number {\n return n;\n}\nexport function tug(n: number): number {\n return n;\n}\n`
);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
const targetsOf = (name: string): string[] =>
withDb((db) =>
(
db
.prepare(
`SELECT t.file_path AS file FROM edges e
JOIN nodes t ON t.id = e.target
JOIN nodes s ON s.id = e.source
WHERE t.name = ? AND e.kind = 'calls'`
)
.all(name) as Array<{ file: string }>
).map((r) => r.file)
);
expect(targetsOf('push')).toHaveLength(OVER_CEILING);
expect(new Set(targetsOf('push'))).toEqual(new Set(['src/zzz_defs.ts']));
expect(targetsOf('tug')).toEqual(['src/zzz_defs.ts']);
// One sync adds a competing definition of BOTH names, in a file that sorts
// first and is therefore the rebuild's answer for each.
write(
'src/aaa.ts',
`export function push(n: number): number {\n return n * 2;\n}\nexport function tug(n: number): number {\n return n * 2;\n}\n`
);
const result = await cg.sync();
expect(result.definitionDelta).toContain('push');
expect(result.definitionDelta).toContain('tug');
// `push` is untouched — every edge still there, still on the old target.
// This is knowingly divergent from a rebuild; see "Don't chase the
// residual" in docs/benchmarks/index-drift-cg33.md.
const pushTargets = targetsOf('push');
expect(pushTargets).toHaveLength(OVER_CEILING);
expect(new Set(pushTargets)).toEqual(new Set(['src/zzz_defs.ts']));
// `tug` — the control — rebound.
expect(targetsOf('tug')).toEqual(['src/aaa.ts']);
});
/**
* Guards the escape hatch itself: with the rebind pass off, the same sequence
* must still produce a structurally sound index (no lost or orphaned edges) —
* just a drifted one. If this ever fails, the pass is doing something the
* kill switch cannot undo.
*/
it('CODEGRAPH_NO_REBIND=1 disables the pass without corrupting the index', async () => {
write('src/caller.ts', `export function run(): number {\n return pct(1);\n}\n`);
write('src/zeta.ts', `export function pct(n: number): number {\n return n;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
const before = edgeSet();
process.env.CODEGRAPH_NO_REBIND = '1';
try {
write('src/alpha.ts', `export function pct(n: number): number {\n return n * 2;\n}\n`);
await cg.sync();
} finally {
delete process.env.CODEGRAPH_NO_REBIND;
}
const after = edgeSet();
// Every edge that existed before is still there — the pass is the only
// thing that would have re-opened them, and it did not run.
for (const edge of before) expect(after.has(edge)).toBe(true);
});
});
/**
* Resolution's candidate order must be a property of the CODE, not of the order
* rows were written. This is the half of CG-33 that a re-resolution pass alone
* cannot fix: without it, re-resolving a reference against the very same graph
* can still pick a different winner than a rebuild does.
*/
describe('Same-name candidate order is content-derived, not insertion-derived (CG-33)', () => {
let testDir: string;
let cg: CodeGraph;
afterEach(() => {
cg?.destroy();
if (fs.existsSync(testDir)) fs.rmSync(testDir, { recursive: true, force: true });
});
it('getNodesByName orders by (file_path, start_line) even when rows were written in another order', async () => {
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg33-order-'));
fs.mkdirSync(path.join(testDir, 'src'), { recursive: true });
fs.writeFileSync(path.join(testDir, 'src', 'mid.ts'), `export function pad(): void {}\nexport function dup(): number {\n return 2;\n}\n`);
fs.writeFileSync(path.join(testDir, 'src', 'zeta.ts'), `export function dup(): number {\n return 1;\n}\n`);
cg = CodeGraph.initSync(testDir, { config: { include: ['**/*.ts'], exclude: [] } });
await cg.indexAll();
// A sync APPENDS this file's nodes, so `alpha.ts` gets the highest rowids
// despite sorting first — exactly the divergence a full index never has,
// and the reason candidate order cannot come from the physical row order.
fs.writeFileSync(path.join(testDir, 'src', 'alpha.ts'), `export function dup(): number {\n return 3;\n}\n`);
await cg.sync();
const keys = cg.getNodesByName('dup').map((n) => `${n.filePath}:${String(n.startLine).padStart(6, '0')}`);
expect(keys.length).toBeGreaterThanOrEqual(3);
expect(keys).toEqual([...keys].sort());
expect(keys[0]).toContain('src/alpha.ts');
});
});
docs/benchmarks/explore-cluster-starvation-cg36.md
+164
View File
@@ -0,0 +1,164 @@
# Deterministic measurement — cluster starvation inside one file (task CG-36)
**Date:** 2026-08-06 · **Baseline:** `feature/CG-24` @ `76ab1fe` (the CG-24 epic tip) ·
**Harness:** `scripts/agent-eval/probe-file-spend.mjs` and `probe-suite-envelope.mjs` over the
deterministic 6-repo corpus at `/tmp/codegraph-corpus`, clean full-rebuilt indexes (CG-33), plus
two hermetic fixtures through `probe-allocation.mjs`. No agent A/B: the claim is which bytes go
to which file, and the agent runs are far too noisy to see a 2K shift.
**Verdict: the defect is real, the diagnosis in the issue was half wrong, and the fix holds.**
All 8 starvation flags across the suite clear; net **+1,012 source chars**. One repo (okhttp)
trades its weakest file for +7,196 chars in the two that answer the question — stated in full
below rather than smoothed over.
---
## The defect
A file's ranked clusters were all-or-nothing past the first one. The top-ranked cluster was
always taken — shrunk to the highest-importance whole symbol ranges that fit when it overran —
and every cluster below it was rendered **whole** and then either fit the remainder or was
dropped entirely. On a file whose top-ranked cluster is trivial, that discards the answer.
What made it invisible: the response stays FULL. The unspent reservation carries forward exactly
as CG-31 designed, so a lower-scoring file takes the bytes and every envelope-share measure still
reads healthy. Measured on the epic tip:
| repo | file | score | reserved | spent | share |
|---|---|---|---|---|---|
| django | `db/models/sql/query.py` | 83 | 7,947 | 1,923 | **24%** |
| django | `contrib/admin/filters.py` | 18 | 2,271 | 8,057 | **355%** |
| okhttp | `.../RealInterceptorChain.kt` | 86 | 6,058 | 1,474 | **24%** |
| okhttp | `.../CallServerInterceptor.kt` | 20 | 1,974 | 5,832 | **295%** |
A score-83 file spends a quarter of its reservation while a score-18 file takes 3.5× its own.
## What the issue got wrong
The issue named two candidate fix points and suspected the first: cluster ranking breaks ties on
**density** (`score / span`) after `hasSpine` → `maxImportance`, which structurally favours a
small trivial cluster over a large answer-bearing one. Dumping the cluster set says otherwise —
in **both** real cases the loser lost on `maxImportance`, not on density:
```
django/db/models/sql/query.py budget 10,135 spent 1,923
KEPT 1379–1400 span 22 score 14 maxImp 6 (check_related_objects — a glue symbol)
DROPPED 306– 929 span 624 score 290 maxImp 3 (133 members: the whole Query class)
okhttp .../RealInterceptorChain.kt budget 6,058 spent 1,474
KEPT 16– 44 span 29 score 44 maxImp 6 (package decl + import block)
DROPPED 113– 373 span 261 score 171 maxImp 3 (73 members: the chain itself)
```
`maxImportance` first is deliberate and protective — it is what stops Alamofire's `Session.swift`
from losing its budget to the top-of-file property list — so **ranking was not touched**. The
lever is the second fix point: stop dropping the loser whole.
## The change
Two sites, one rule — *hold the remainder while it is still worth a section*, which is CG-26's
between-FILES lesson applied between CLUSTERS.
1. **Selection.** A later cluster is now shrunk into what is left of the file's budget, by the
same whole-member rule the first cluster already used. Below `MIN_CHARS` (700) the remainder
cannot hold a readable block, so it stays a drop rather than a stutter of fragments the next
call's dedup has to shred around. The never-empty windowing floors may overrun that room; the
first cluster is allowed that overshoot, a later one is not.
2. **The ceiling trim.** When the exact section cost overruns `renderCeiling`, the weakest chosen
cluster is re-rendered into the room that remains before being dropped. This one is worth
naming on its own: on excalidraw's `typeChecks.ts` the section-cost estimate missed by
**13 chars** and a 1,512-char cluster — the file's highest-*scoring* one, last in rank order
only because rank breaks ties on density — was thrown away to pay for it. Recovered 1,501 of
excalidraw's 1,449-char loss.
## Suite result
`node scripts/agent-eval/probe-file-spend.mjs`, 6 repos, clean rebuilds. Only files whose spend
moved are listed; score is the candidate's ranking score, reserved its allocation.
| repo | file | score | reserved | before | after |
|---|---|---|---|---|---|
| django | `db/models/sql/query.py` | 83 | 7,947 | 1,923 | **10,082** |
| django | `contrib/admin/filters.py` | 18 | 2,271 | 8,057 | 2,198 |
| django | `utils/autoreload.py` | 12 | 1,747 | 3,145 | 1,709 |
| django | `db/models/fields/related_descriptors.py` | 11 | 1,660 | 2,516 | 1,493 |
| excalidraw | `element/src/typeChecks.ts` | 23 | 2,740 | 3,102 | 2,573 |
| excalidraw | `excalidraw/types.ts` | 14 | 1,942 | 819 | 1,372 |
| okhttp | `.../RealInterceptorChain.kt` | 86 | 6,058 | 1,474 | **6,038** |
| okhttp | `.../Interceptor.kt` | 64 | 4,697 | 2,027 | **4,659** |
| okhttp | `.../RealCall.kt` | 52 | 3,972 | 3,628 | 3,922 |
| okhttp | `.../Call.kt` | 54 | 4,097 | 4,097 | 2,073 |
| okhttp | `.../CallServerInterceptor.kt` | 20 | 1,974 | 5,832 | 1,959 |
| okhttp | `androidMain/.../AndroidDns.kt` | 21 | 1,999 | 1,812 | **0** |
| tokio | `task/local.rs` | 40 | 4,361 | 4,599 | 4,798 |
| tokio | `runtime/task/harness.rs` | 14 | 1,981 | 2,565 | 2,341 |
| gin | `routergroup.go` | 87 | 5,782 | 3,273 | **5,632** |
| gin | `tree.go` | 17 | 1,693 | 892 | 1,969 |
| gin | `ginS/gins.go` | 26 | 2,213 | 4,431 | 2,171 |
| alamofire | `Source/Core/Session.swift` | 34 | 2,792 | 2,797 | 3,396 |
| alamofire | `Source/Core/Request.swift` | 148 | 9,100 | 8,865 | 8,453 |
Bytes move up the score order in every repo. Envelope totals:
| repo | source before | after | Δ | files | ceiling |
|---|---|---|---|---|---|
| django | 20,878 | 20,719 | −159 | 6 → 6 | 24,963 ≤ 25,000 |
| excalidraw | 19,652 | 19,704 | +52 | 8 → 8 | 24,813 ≤ 25,000 |
| okhttp | 18,870 | 18,651 | −219 | 6 → **5** | 24,985 ≤ 25,000 |
| tokio | 21,607 | 21,582 | −25 | 5 → 5 | 24,777 ≤ 25,000 |
| gin | 10,776 | 11,952 | **+1,176** | 4 → 4 | 14,655 ≤ 19,500 |
| alamofire | 11,662 | 11,849 | +187 | 2 → 2 | 12,862 ≤ 19,500 |
| **total** | **103,445** | **104,457** | **+1,012** | | |
Starvation flags: **8 → 0**.
## The one cost, stated plainly
okhttp drops its rank-6 file, `androidMain/.../AndroidDns.kt` (score 21, a platform DNS helper on
a question about the interceptor chain), and 219 source chars, in exchange for +4,564 to
`RealInterceptorChain.kt` and +2,632 to `Interceptor.kt` — the two files that answer the question.
This is not a new defect and it is not the fix over-reaching. okhttp's reservations are
**structurally over-subscribed**: the allocator splits `maxOutputChars` charging a flat
`FILE_OVERHEAD` of 200 per file while a real header runs 300–500, so the sum of promises
(~22,800 source + ~2,100 of real headers) exceeds what the ~24,760-char render ceiling can hold.
`owedPayableBelow` already refuses to hold bytes back for a file it can see will be dropped, and
AndroidDns.kt is the file past that line. On the epic tip it survived only because the files above
it under-spent — by luck, not by design. Closing the over-subscription means charging the
allocator per-file header estimates rather than the flat 200; that is a wider change than this
issue, and CG-26 deliberately kept `FILE_OVERHEAD` as the allocator's own constant.
## What did NOT change
- **Cluster ranking.** `hasSpine` → `maxImportance` → density → score → span, untouched.
- **Alamofire `Session.swift`.** The shape density-first exists for; it *gains* 599 chars.
- **The factory-closure outcome (CG-27).** `probe-factory-closure.mjs`: 7 of 11 inner closure
definitions delivered, identical to the epic tip.
- **The reservation invariant (CG-31/CG-26).** `explore-reservation-invariant.test.ts` green;
every repo stays at or under its hard ceiling.
- **All four allocation fixtures pass** — `payroll-go`, `self-query`, and the two added here.
## What ships so this stays measurable
- `scripts/agent-eval/probe-file-spend.mjs` — the standing per-file reservation-vs-delivered
sweep. It flags a **pair**, never a single file: a large share unspent *while* a materially
lower-scoring file overspends. Either alone is legitimate (a small file has less to say;
carry-forward is the mechanism that hands its slack down), which is why the envelope probe
could never see this. Exit code 1 on any flag, so it gates.
- `__tests__/fixtures/starved-cluster-ts/` — django's and okhttp's shape reduced to a fixture.
Fails on the epic tip (28.8% of reservation, neither `proceed` nor `writeAndRead` delivered),
passes with the fix.
- `__tests__/fixtures/dense-header-ts/` — the `Session.swift` shape, byte-identical on both
builds. The counterweight: it fails if a future change lets density outrank importance again.
- `spendShareAtLeast` in `probe-allocation.mjs`, and `__tests__/explore-cluster-starvation.test.ts`
pinning both fixtures in `npm test`.
## Method note
None of this is visible in the rendered markdown. To see it you must dump the cluster set —
patch `dist/mcp/tools.js` just before `let assembled = assembleSection(chosenIndices);` and log
`fileBudget` / `projectedChars` / each ranked cluster's span, score, `maxImportance`, chosen flag
and members. Reading only the response makes member-selection effects look like budget effects.
Equally, a source-chars diff between builds is not automatically a regression: excalidraw's
−1,449 on the first cut was the elastic epilogue expanding into room a 13-char accounting error
had released, not source lost to allocation.
docs/benchmarks/explore-declaration-only-cg28.md
+149
View File
@@ -0,0 +1,149 @@
# Deterministic measurement — declaration-only files in the explore envelope (task CG-28)
**Date:** 2026-08-06 · **Baseline:** `feature/CG-24` @ `463f6e7` ·
**Harness:** `scripts/agent-eval/probe-decl-only.mjs` against a hermetic fixture
(`__tests__/fixtures/ambient-decls-ts/`, copied to a temp dir and indexed per run, so two runs on
one build give identical numbers), plus `probe-suite-envelope.mjs` and a corpus-wide flag-rate
survey for the regression side. No agent A/B: the claim under test is which FILES get selected and
in what order, and the agent runs are far too noisy to see that.
**Verdict, both halves:**
- **The motivating file is already handled — CG-25 credited.** The Wrangler `worker-configuration.d.ts`
that opened this issue is demoted by the generated penalty alone, worth 15–46 points of envelope
share on the four flow queries measured. No new mechanism needed for it.
- **The narrower gap is real and was fixed.** A declaration file with NO banner carried `pen 1.00`,
took **rank #1 and 51% of delivered source** on a prose flow query, and displaced the flow's own
entry file out of the response entirely. It is now damped — but only when nothing in the index
depends on it, which is the condition that makes the rule safe.
---
## The fixture
`__tests__/fixtures/ambient-decls-ts/` — an upload path (route → stream → metadata → queue) with
four declaration-shaped files competing against it for one envelope. All four declare nothing but
`interface`/`type_alias` and have no bodies; they differ only in the two properties under test.
| file | banner | depended on | lines |
|---|---|---|---|
| `types/worker-configuration.d.ts` | Wrangler | no | 271 |
| `types/platform-shims.d.ts` | none | no | 212 |
| `src/storage/types.ts` | none | **yes** (2 imports, 3 references) | 18 |
| implementation (`routes/`, `storage/`, `lib/`) | — | — | 37–71 each |
The declaration files carry the same generic identifiers the prose queries use — `Body`, `Message`,
`ImageMetadata`, `ReadableStream`, `Upload*` — which is the whole mechanism of the original report.
## Result 1 — what CG-25 is worth (the obsolescence leg)
Same fixture, same queries, one variable: `--variant strip-banner` deletes the two banner COMMENT
lines from `worker-configuration.d.ts` and changes nothing else, so the two declaration files become
indistinguishable to the ranker. Delivered share of the envelope for that file:
| query | with banner | banner stripped |
|---|---|---|
| flow-upload | not a candidate | 15.1% (2,271 chars) |
| flow-pipe | not a candidate | 38.5% (4,398 chars) |
| flow-generic | cliffed to a pointer, 0 chars | 35.1% (3,076 chars) |
| flow-queue | 9.4% via clusters (1,264 chars) | **46.1%, rank #1, whole file** (7,390 chars) |
The generated penalty alone is the difference between "rank #1 and nearly half the answer" and
"named in the not-shown list". **The file this issue was filed about needs nothing further.**
## Result 2 — the gap that survived
`platform-shims.d.ts` — hand-written, no banner — on the `feature/CG-24` tip:
| query | rank | score | pen | delivered |
|---|---|---|---|---|
| flow-upload | **#1** | 53.0 | 1.00 | 6,044 chars (**50.7%**) |
| flow-queue | **#1** | 21.0 | 1.00 | 6,044 chars (44.9%) |
On `flow-upload` the response carried three files and `src/routes/upload.ts` — the handler the
question is *about* — was not one of them. That is the CG-24 epic symptom, reproduced with no
generated banner anywhere in it.
Note also: `.pyi` is **not an indexed extension**, so Python stubs never enter the graph and cannot
take an envelope. That third of the issue's premise does not occur today.
## The mechanism, and why it is drawn this tight
`AMBIENT_DECLARATION_RANK_PENALTY` (0.5) multiplies score and graph mass in `rankPenalty`, for files
`QueryBuilder.getAmbientDeclarationPathsAmong` flags. Four conditions, all required — the first
three were the obvious rule, the fourth is the one that makes it safe:
1. declares ≥1 symbol;
2. **every** declared symbol is type-level (`interface`, `type_alias`, `enum`, `enum_member`,
`namespace`);
3. originates no `calls`/`instantiates` edge;
4. **nothing outside the file points at it.**
Conditions 2 and 4 were both forced by measurement, not taste:
**Why not just "no callables" (condition 2).** Surveyed across the corpus, a rule of "declares no
callable and calls nothing" flags **1.1%–18.0%** of files, and what it catches is real source:
okhttp's `SocketPolicy.kt` (19 declarations, a Kotlin sealed hierarchy), `BrotliInterceptor.kt`,
`tokio/src/runtime/mod.rs`, Alamofire's umbrella `Alamofire.swift`, and all 500+ of django's
`conf/locale/*/formats.py` constant tables. Requiring every symbol to be type-level drops that to
**0%–4%**.
**Why "nothing depends on it" (condition 4).** Without it the rule also flags
`__tests__/fixtures/displacement-ts/src/pipeline/types.ts` — pure interfaces, no bodies, structurally
identical to an ambient shim — and demoting it **broke the CG-31 displacement gate**, which is a
different invariant entirely. That file carries 13 inbound imports and 21 references: the pipeline
stages that answer a query about the pipeline are typed *by* it, so it is part of that answer's
structure. The ambient shims carry **zero** inbound edges — reachable by name, attached to nothing.
That is the real distinction, and the graph already holds it.
**The counter-case guard.** A query that NAMES a declared type is a question about the declaration,
so its file is exempt and ranks at full weight. Only shape-precise tokens count (the same
NL-stopword reasoning as named-seed selection) — "…the file **body**…" must not exempt a `Body`
interface it never meant to name. This needed its own set: `namedSeedIds` is callable-only by
construction, so a type can never become a named seed.
**No double-charging.** Generated and ambient-declaration are combined with `Math.min`, not
multiplied. A generated `.d.ts` has one property that two signals happen to see; charging it twice
(0.3 × 0.5 = 0.15) is how a file gets cliffed out of answers where it is genuinely relevant. The
low-value multiplier is orthogonal and still compounds.
## Result 3 — after the fix
| query | before | after |
|---|---|---|
| flow-upload | rank **#1**, 50.7% | rank **#2**, 38.6% — and `src/routes/upload.ts` now delivered (2,417 chars) |
| flow-queue | rank **#1**, 44.9% | rank **#3**, 41.3% |
| flow-pipe / flow-generic | not a candidate | unchanged |
| type-shim (`UploadStorage StoredUploadObject ImageMetadataShim`) | rank #1, `pen 1.00` | **unchanged** — exempt |
| type-prose (*what does the UploadStorage interface declare…*) | rank #1, `pen 1.00` | **unchanged** — exempt |
The byte share falls less than the rank does, and that is the correct outcome rather than a weak
fix: on this fixture every implementation file already delivers its entire contents, so the
declaration file is filling envelope nobody else needs. What it was actually taking was a **file
slot** — which is why the entry file came back. The issue explicitly forbids suppression, and a
damped file is still a candidate, still named in the response, and one follow-up explore away.
## Regression evidence
- **`probe-suite-envelope.mjs`, 6 repos, new build vs a clean `feature/CG-24` baseline build:
byte-identical.** django 20,878 · excalidraw 19,652 · okhttp 18,870 · tokio 21,607 · gin 10,776 ·
alamofire 11,662 source chars, same file counts, on both builds.
- **VS Code** — the repo the issue names for `.d.ts` surface — across five flow and type queries:
**zero** ambient-declaration files reach the ranked candidate set, so the output cannot differ.
- **Corpus-wide flag rate:** django 0.00% · okhttp 0.00% · gin 0.00% · alamofire 0.00% ·
tokio 0.12% · vscode 0.53% · excalidraw 0.74%. What it catches is `global.d.ts`, `vite-env.d.ts`,
`css.d.ts`, unreferenced vendored headers and test fixtures — exactly the intended shape.
- `probe-allocation.mjs`: `payroll-go` PASS, `self-query` PASS.
- Full suite: **178 files, 2,978 passed**, 6 skipped.
The change is inert everywhere the shape does not occur, which is most places. That is the point:
the defect is real but rare, and the mechanism costs nothing where it does not apply.
## Reproducing
```bash
npm run build
node scripts/agent-eval/probe-decl-only.mjs # as committed
node scripts/agent-eval/probe-decl-only.mjs --variant strip-banner # what CG-25 is worth
npx vitest run __tests__/explore-declaration-only.test.ts # the standing gate
```
docs/benchmarks/explore-displacement-guard-ab-cg31.md
+140
View File
@@ -0,0 +1,140 @@
# Agent A/B — cluster-path displacement guard (task CG-31)
**Date:** 2026-08-06 · **New:** `bugfix/CG-31` · **Baseline:** `bugfix/CG-30` @ `0d014a6` ·
**Harness:** `scripts/agent-eval/ab-new-vs-baseline.sh`, `--model sonnet --effort high`,
**both arms codegraph-on**, CLI blocked (0 contamination in every run),
`CODEGRAPH_NO_PROMPT_HOOK=1`. Every index measured on was **fully rebuilt**, never
incrementally synced (CG-33).
Baseline is the CG-30 tip, not `main`, so every number here isolates CG-31. CG-30's own A/B
against `main` is `explore-oversize-member-ab-cg30.md`; read them in sequence for the combined
picture the two issues asked for.
CG-31 stops a clustered render from spending a reservation still owed to a file the render loop
has not reached. The whole-file BUY arm has always refused that trade (`owedBelow`); the cluster
path read what was left before the hard ceiling instead of what was still promised.
**Verdict: no regression, and this one is a straight win on both halves.** Four of six suite
repos deliver MORE source and one more file each; the other two are byte-identical. The agent
runs are faster in all three repos measured, with Read at or below baseline.
---
## The two corrections the measurement forced
Worth recording, because the first version of the guard was **wrong in the direction the guard
itself is about**, and only a suite measurement showed it.
**1. Holding back the full owed sum was too much.** The allocator splits the envelope; the render
loop spends against a ceiling that also has to hold the response's own prose, so on a saturated
response the promises are over-subscribed and the tail is going to be dropped whatever happens
above it. Bytes held for a file that is then dropped are bytes nobody receives. Measured: django
−2,319 source, tokio −1,298, both handed to a section the ceiling threw away.
`owedPayableBelow` now holds back only the prefix of what is owed below that the response can
still pay, in rank order.
**2. The final truncation was eating the guard's work.** It cut at the last file-section header,
which drops that whole section *and* the trailing notes. Dropping the notes alone is almost
always enough. The epilogue is a pointer list and two reminders; a section is source the agent
otherwise has to Read. Cutting the epilogue first is what turned the remaining deficits into
gains — and it is the same starvation CG-31 is about, arriving one layer below the guard.
A third, smaller fix: `flow.text` is prepended to `lines` to make the final output but was never
counted in `totalChars`, so the render loop spent against a ceiling it was ~2K under on
symbol-bag queries.
## Deterministic measurement — the primary evidence
Same clean-rebuilt index, same query, both builds. One `codegraph_explore` per repo.
| repo | base source | new source | Δ | base files | new files |
|---|---|---|---|---|---|
| django | 20,033 | **20,791** | +758 | 5 (truncated) | **6** |
| excalidraw | 18,776 | **20,204** | +1,428 | 7 (truncated) | **8** |
| okhttp | 15,628 | **19,034** | +3,406 | 4 (truncated) | **5** |
| tokio | 20,340 | **21,521** | +1,181 | 4 (truncated) | **5** |
| gin | 10,776 | 10,776 | 0 | 4 | 4 |
| alamofire | 11,662 | 11,662 | 0 | 2 | 2 |
Queries: django "How does a QuerySet turn into SQL and fetch rows from the database?";
excalidraw "How does updating an element re-render the canvas on screen?"; gin "How does a
registered route handler get invoked for an incoming HTTP request?"; alamofire "How does a
request get built and sent through the session?"; okhttp "How does a call go through the
interceptor chain to the network?"; tokio "How does a spawned task get scheduled and run by a
worker?".
No repo delivers less. **Four of six stopped truncating**, which is where the extra file comes
from: each of those responses had been throwing a fully-rendered section away.
**gin and alamofire are byte-identical between the builds** — nothing in them is oversize enough
for the guard to engage and neither response was truncated. That is what a control should show,
and it means every gin number in the agent table below is run-to-run variance.
**Fixture** — `__tests__/fixtures/displacement-ts`, four pipeline stages competing for one
envelope, the first a single ~20K function. Padded past 500 indexed files on purpose: the
displacement only exists on the 24K tier, where the reservations plus the preamble genuinely
saturate the render ceiling.
| | baseline | new |
|---|---|---|
| `ingest.ts` | 9,301 chars on a 6,289 spendable, then **dropped whole** by the ceiling — 0 delivered | 4,851, bounded |
| `types.ts` / `sink.ts` | skipped `budget-whole-file` | delivered |
| admitted files delivered | **3 of 6** | **6 of 6** |
| envelope | 14,908 | 22,066 |
Pinned by `__tests__/explore-displacement-guard.test.ts` (11 tests; 3 fail on the baseline).
**Allocation fixtures** — `scripts/agent-eval/allocation-fixtures.json` flips back to
**BOTH PASS**. Its `afterCG30` verdict blamed an over-RESERVED incidental file; the reservation
is identical in both arms — the file was over-SPENDING, which is exactly this defect. Recorded
honestly in `afterCG31`.
## Agent runs
| | django new | django base | okhttp new | okhttp base | gin new | gin base |
|---|---|---|---|---|---|---|
| runs | 3 | 3 | 2 | 2 | 2 | 2 |
| duration (s) | **36** [33–51] | 46 [35–49] | **42** [40–44] | 52 [50–53] | **33** [31–35] | 39 |
| tool calls | 3 [3–4] | 3 [3–4] | **4** [3–4] | 5 [4–5] | **4** [3–4] | 5 [4–5] |
| Read | 0 [0–1] | 0 | **0** | 1 [0–2] | 1 [0–1] | 1 [0–2] |
| Grep/Glob | 0 | 0 | 0 | 0 | 0 | 0 |
| codegraph calls | 2 | 2 [2–3] | 3 [2–3] | 3 [2–3] | 2 | 3 [2–3] |
| occupancy share | **32.1%** [31.1%–36.1%] | 33.8% [28.7%–42.1%] | **40.0%** [36.6%–43.3%] | 40.8% [40.3%–41.4%] | **29.7%** [28.3%–31.1%] | 33.5% [29.8%–37.2%] |
| allocation efficiency | 96.8% | 98.9% | 88.2% | 97.2% | **91.2%** | 85.0% |
Prompts are the deterministic queries above with "Trace the flow end to end." appended.
**Sufficiency, pooled per call.** okhttp: **0** "Read a file we returned" in 5 against the
baseline's 1 in 5 — the arm that returns 3,406 more chars needs fewer follow-up Reads, which is
the mechanism working. django: 1 in 6 against 0 in 7. gin: 1 in 4 against 1 in 5, on a repo where
the builds emit identical bytes. Neither arm produced a single recall miss (a Read of a file we
did NOT return, or a Grep) on any repo.
**Where the new arm looks worse, and why it is not read as a regression:**
- *okhttp allocation efficiency, 88.2% vs 97.2%.* The new arm's envelope is 85,197 chars against
the baseline's 66,014 — it returns substantially more source, and the metric is the share of
returned bytes the answer *cited*. A larger, more complete response with a smaller cited share
and Read driven to 0 is the trade this tool exists to make. The metric's own documentation says
it is relative and must not be read as waste.
- *django, 1 allocation miss in 6 answered calls against 0 in 7.* One run, n=3, and django is the
repo whose duration range overlaps most (33–51 vs 35–49).
## Residual carried forward — for CG-26
Four repos stopped truncating; **okhttp, django, excalidraw and tokio now land at 24,758–24,998
chars against a 25,000 hard ceiling.** That is deliberate (the ceiling exists so the host never
externalizes the result) but it means the render loop's 600-char margin for the epilogue is still
wrong — the epilogue measures 1,064 (gin), 1,788 (django), 2,231 (excalidraw). The response now
survives that by dropping the epilogue rather than a section, which is strictly better, but the
honest fix is for the loop to budget for the epilogue in the first place.
A margin sweep was run and deliberately **not** shipped: at 1,200 django stops truncating on its
own but tokio loses 286 chars; at 2,400 django loses 1,895. Tuning one constant against the suite
is the trap CG-30's own record warns about. Sizing the margin from the epilogue the response is
actually going to emit is the real fix and belongs with the end-to-end reservation invariant.
Second residual: the whole-file BUY arm's fit test (`totalChars + fileContent.length +
FILE_OVERHEAD <= renderCeiling`) has no `owedBelow` term of its own — its displacement guard is
source-space only. It was left alone here to keep this change attributable; the epilogue-first cut
removes the failure mode it would have caused.
docs/benchmarks/explore-factory-closure-cg27.md
+133
View File
@@ -0,0 +1,133 @@
# Deterministic measurement — the factory-closure envelope (task CG-27)
**Date:** 2026-08-06 · **Baseline:** `feature/CG-24` @ `dc4fd75` ·
**Harness:** `scripts/agent-eval/probe-factory-closure.mjs` against a hermetic fixture
(`__tests__/fixtures/factory-closure-ts/`, copied to a temp dir and indexed per run, so two runs
on one build give identical numbers). No agent A/B: the claim under test is which SYMBOLS get
selected inside one file, and the agent runs are far too noisy to see that.
**Verdict: the premise does not survive measurement. CG-27 is closed as obsolete, CG-30 credited.**
The literal change the issue proposes is a large REGRESSION, and a more careful mechanism reaching
the same intent is noise (69 vs 68 inner definitions delivered across nine query shapes).
---
## The claim
`ENVELOPE_KINDS` in `src/mcp/tools.ts` drops a node covering >50% of its file from the cluster
ranges, so the granular symbols inside form their own clusters instead of merging into one blob.
It lists container kinds — `class`, `struct`, `interface`, `enum`, … — and **not `function` or
`method`**. A factory closure (`createFoo()` returning an object of closures) therefore survives
as a file-spanning range. That shape is common, not a one-repo quirk: Svelte 5 `.svelte.ts` rune
stores, React custom-hook modules, IIFE/module-pattern JS, and Zustand's
`create((set, get) => ({ … }))`.
CG-30 already bounds the BYTES such a member may spend, so what remained was a ranking claim:
a file-spanning range merges every inner symbol into one cluster, so selection cannot rank and
pick the relevant closures independently. The issue required that claim be measured before any fix.
## The fixture
`__tests__/fixtures/factory-closure-ts/` — a dashboard app with three stores written as factory
closures, two stateless services and a UI consumer competing for one envelope.
| file | lines | shape |
|---|---|---|
| `src/stores/dashboard-store.ts` | 385 | `createDashboardStore` spans 15–376 (**94%**), 11 closures inside; a tail type alias + helper at file scope |
| `src/stores/alerts-store.ts` | 141 | `createAlertsStore` spans 19–138 (**85%**), 9 closures inside |
| `src/stores/session-store.ts` | 148 | a factory and NOTHING else at file scope — no companion type, no tail helper |
| `src/services/metric-service.ts`, `src/services/filter-parser.ts` | 105, 62 | ordinary top-level functions — the control |
Both factory files are past `WHOLE_FILE_MAX_LINES` where it matters, so they render through the
cluster path and the envelope actually bites.
## Result 1 — the envelope is almost never selected in the first place
`shrinkCluster` orders a cluster's members by **(importance desc, size ASC)** and refuses any
member that overruns the cap once something is kept. A file-spanning member is therefore only ever
selected when it is the FIRST candidate — which requires it to be the *sole* member of the top
importance tier. In eight of the nine query shapes measured, some smaller member shared that tier
(a one-line type alias, a tail helper, another closure), so the factory sorted last and was never
kept. The envelope was inert.
## Result 2 — the proposed change is a large regression
Making the >50% drop kind-independent, measured on the primary query
(*"how does the dashboard store refresh its metrics and apply a filter"*):
| | baseline | drop the range |
|---|---|---|
| `dashboard-store.ts` (rank #1) delivered | 7,539 chars | **397** |
| inner closure definitions delivered | 7 of 11 | **0 of 11** |
| its own reservation left unspent | 0 | ~5,200 of 5,601 |
The mechanism, from the cluster dump: dropping the range **splits** the file into two clusters —
`378-384` (a one-line type alias plus a four-line helper, score 15, span 7) and `4-362` (every
closure, score 116, span 359). Cluster ranking breaks the `maxImportance` tie on **density**, so
the trivial cluster wins, is taken first, and is the only one that may be shrunk. The
answer-bearing cluster then does not fit the remainder and is **dropped whole** — later clusters
are never shrunk, by design.
The enclosing range is what was holding the file together as one cluster, inside which
`shrinkCluster` was already doing exactly the per-symbol ranking the issue asked for.
## Result 3 — the careful version of the same intent is noise
Deferring the envelope MEMBER inside `shrinkCluster` (leaving clustering granularity untouched, so
Result 2's split never happens) reaches the issue's intent by a better mechanism. Nine query
shapes, same fixture, same indexes — inner closure definitions delivered:
| query | target | baseline | deferred |
|---|---|---|---|
| how does the dashboard store refresh its metrics and apply a filter | dashboard | 7/11 | **8/11** |
| createDashboardStore | dashboard | 8/11 | 8/11 |
| how is the dashboard store created and wired up | dashboard | **9/11** | 8/11 |
| createDashboardStore exportCsv summarize | dashboard | 9/11 | 9/11 |
| where is the dashboard store constructed | dashboard | 7/11 | 7/11 |
| how are widgets loaded and the layout reconciled | dashboard | 4/11 | 4/11 |
| createSessionStore (adverse: the factory IS the sole top-tier member) | alerts | 6/9 | **7/9** |
| how are alerts refreshed and acknowledged | alerts | 9/9 | 9/9 |
| createAlertsStore | alerts | 9/9 | 9/9 |
| **total** | | **68** | **69** |
One better, one worse, seven unchanged — on a fixture built specifically to make this pattern
maximally visible. That is not a measurable selection improvement, so nothing shipped.
## Where the envelope DOES get selected, and why CG-30 already covers it
The adverse row above is the one configuration the ordering cannot neutralise: `createAlertsStore`
was the sole importance-10 member, so it was kept first at 3,939 chars against a 2,468 cap and
every closure was skipped. CG-30 then **windowed it on whole lines** rather than emitting it whole
or dropping the file — the response carried lines 16–108, a contiguous, readable head of the
factory carrying 6 of its 9 closure definitions. Bounded, sufficient, never empty. That is the
symptom this issue was filed against, already absorbed.
---
## Byproduct — a real defect this measurement exposed (filed separately)
Result 2's mechanism is not confined to the hypothetical change. Instrumenting the **epic tip**
across the deterministic 6-repo suite for files that drop a cluster while leaving most of their
reservation unspent:
| file | budget | spent | unspent | kept cluster | dropped cluster |
|---|---|---|---|---|---|
| `django/db/models/sql/query.py` | 10,135 | 1,923 | **8,212 (81%)** | 1379–1400, score 14 | 306–929, **score 290** |
| `okhttp .../RealInterceptorChain.kt` | 6,058 | 1,474 | **4,584 (76%)** | 16–44, score 44 | 113–373, score 171 |
| `okhttp .../Interceptor.kt` | 4,697 | 2,027 | 2,670 (57%) | 85–138, score 21 | 154–257, score 10 |
| `gin/routergroup.go` | 5,782 | 3,273 | 2,509 (43%) | 33–91, score 116 | 103–188, score 128 |
A file whose top cluster by density is trivial keeps that one, drops the cluster carrying 20x the
score, and leaves most of its own reservation unspent — because only the first-chosen cluster may
be shrunk. `query.py` is the file CLAUDE.md already names as the `_fetch_all` case.
## Reproducing
```bash
npm run build
node scripts/agent-eval/probe-factory-closure.mjs # primary query
node scripts/agent-eval/probe-factory-closure.mjs \
--target src/stores/alerts-store.ts --factory createAlertsStore \
--query "createSessionStore" # the adverse configuration
npx vitest run __tests__/explore-factory-closure.test.ts # the standing gate
```
docs/benchmarks/explore-noise-epic-cg24.md
+124
View File
@@ -0,0 +1,124 @@
# Epic resolution — explore response noise (CG-24)
Worked 2026-08-05 → 2026-08-06. Started from one bad `codegraph_explore` response
in a real session and ended with four shipped fixes, one open defect, and five
issues closed because measurement contradicted them.
**The headline: the reported symptom was not an explore bug.** It was a degraded
index. The explore defects the investigation found are real and were fixed, but
none of them caused the report.
## The report
A prose flow query returned an unusable response: the symbol the agent had named
never rendered, and a 12k-line generated Cloudflare ambient-types file took 60.7%
of the output envelope.
```
# deliv% bytes reserved score pen flags file
1 1.0% 251 10,970 87.0 1.00 named entry central <the named file>
2 60.7% 15,043 6,484 49.0 1.00 entry central worker-configuration.d.ts
4 — — — 19.9 1.00 dropped: budget <a third file>
```
## Root cause
**Index drift ([CG-33](index-drift-cg33.md)).** The live incrementally-synced index
diverged from a clean rebuild by 4.3% of distinct edges, bidirectionally,
overwhelmingly `calls`. RWR graph mass is relative and normalized, so call edges
missing elsewhere inflate an unaffected file's share — the `.d.ts` carried mass
0.24750 drifted vs 0.13119 rebuilt (~1.9×), score 49.0 vs 27.0.
Two causes, both fixed: incremental sync re-resolved only references *in* changed
files, and `getNodesByName` had no `ORDER BY`, so ties broke by rowid — i.e. by
the order files happened to be **written**. The second is why scope alone could
never converge. Stale edges dropped 671 → 2 across an 80-commit replay.
On a freshly rebuilt index the reported query answers correctly **with no explore
change at all**.
## Shipped
| | what |
|---|---|
| **CG-30** | Bounded how far an oversize cluster member may overshoot; windows on whole lines past 1.5× instead of emitting whole — or, when larger than the response ceiling, dropping the file silently. |
| **CG-31** | Gave the cluster path the `owedBelow` displacement guard the whole-file BUY arm always had, holding back only the prefix of what is owed below that the response can actually pay. |
| **CG-26** | Closed the remaining holes: whole-file arms had no displacement guard at all, section overhead was charged at a flat 200 against a real 300–500, and `owedPayableBelow` held all-or-nothing. |
| **CG-25** | Recognize `Generated by <tool> by running <command>` banners. Precision held by requiring two `by` clauses, so ordinary prose does not match. |
| **CG-28** | Damp declaration-only files that nothing in the index depends on. Does not stack with the generated penalty (`Math.min`), and naming a declaration symbol exempts its file. |
| **CG-33 / CG-35** | Incremental sync converges with a rebuild, plus a regression suite that fails when the fix is disabled. |
| **CG-36** | A later cluster is shrunk into the remainder rather than dropped whole — at selection, and again in the ceiling trim. All 8 starvation flags across the suite clear; +1,012 source chars net. `explore-cluster-starvation-cg36.md`. |
Deterministic across the 6-repo suite: no repo truncates, none loses a file,
okhttp gains one, every repo lands at or under the 25,000 hard ceiling.
## Follow-up — CG-38 (closed)
**Agent-named symbols in the tail of a large file never rendered.** On the
motivating repo, `queueMessage` (line 1087) and `flushQueuedMessages` (1102) in a
1,414-line file were absent from the response on both prose and symbol-bag
queries, even when that file won rank #1 with 67% of the envelope. The response
returned the `QueuedMessage` *interface* at line 70 — a fuzzy near-match on the
query token — instead of the function.
**Pre-existing, not caused by this epic.** A controlled bisect (index held fixed,
engine varied across every merge point) shows the pre-epic engine rendering 12
lines here and CG-36 rendering 463; the symbols render at neither. The epic
strictly improves the case. An earlier claim that the epic regressed it was
wrong — it compared runs across two different indexes.
Two independent causes, both longstanding: `buildFlowFromNamedSymbols` discarded
the named-symbol IDENTITY along with the narrative whenever the named symbols did
not form a call chain, so the importance-9 injection never ran; and the ceiling
trim cut in SOURCE order, so a named def at the end of a large file was always the
first thing dropped. Full account, plus the ranker-penalty lead (real, and
orthogonal — the defs are absent at both `generated` flag states on the old build
and present at both on the new one): `explore-tail-render-cg38.md`.
This epic's probes measure envelope share, starvation, source totals and file
counts. **None measured "did the agent-named symbol render"** — which is why this
survived the whole epic. `scripts/agent-eval/probe-named-symbol.mjs`,
`__tests__/fixtures/tail-render-ts` and
`__tests__/explore-named-symbol-render.test.ts` close that gap: per-symbol and
binary, checking the definition LINE against the response's rendered lines.
CG-36's own measurement is worth carrying forward, because the issue named the wrong
fix point: both real cases (`query.py`, `RealInterceptorChain.kt`) lost on
`maxImportance`, **not** on the density tiebreak the issue suspected. Ranking was
left alone; the never-shrink rule was the lever. Full numbers and the one cost
(okhttp trades its rank-6 file for +7,196 chars in the two that answer the
question) in `explore-cluster-starvation-cg36.md`.
## Closed because measurement contradicted them
Five, which is the story of this epic as much as the fixes are.
| | why |
|---|---|
| **CG-32** | Named file "didn't render first." Drift artifact; on a clean index it renders first and takes 89%. |
| **CG-34** | "Allocator over-reserves for low-scoring files." Filed on a runner's diagnosis without checking the numbers. The file was never over-reserved (4,314 in both arms) — it was over-*spending*, which is CG-31. |
| **CG-27** | Adding `function`/`method` to `ENVELOPE_KINDS` measured as a **large regression** — rank #1 fell from 7,539 delivered chars to 397, 7 of 11 inner closures to 0. The enclosing range was holding the file together as one cluster, inside which `shrinkCluster` already did the per-symbol ranking the issue wanted. A careful version was noise (69 vs 68 across nine queries). |
| **CG-29** | Prose-vs-symbol query gap. Inverted on measurement: prose matches symbol on django and delivers 63% more source on okhttp. The founding observation was drift. |
| **CG-37** | Duplicate of CG-36, filed without seeing it. |
## What this epic is actually a lesson in
**A confident diagnosis is worth less than a cheap measurement.** Every issue
above was filed by someone — human or agent — who had read the code and had a
plausible mechanism. Five were wrong. The ones that survived did so because a
deterministic probe disagreed with them and the probe won.
Two specific traps this cost real time on, both now guarded in tooling:
- **`.codegraph/graph.db` does not exist** — the index is `codegraph.db`, and
`sqlite3` against a mistyped path *creates* an empty database rather than
failing. An empty schema reads exactly like a stale pre-migration index. This
produced a wrong root cause. `diff-index-drift.mjs` refuses a missing path.
- **`ab-new-vs-baseline.sh` checks the engine out at the baseline ref mid-run.**
A commit made while it runs captures baseline sources and silently reverts the
fix under test. This happened during CG-30. Check the `changed:` line before
believing any A/B result.
And one measurement discipline worth keeping: **compare sets, not totals.** The
drift that started all of this shows up as +0.7% on raw edge counts, because it
is bidirectional and nets out. On distinct edge triples it is 4.3%.
docs/benchmarks/explore-oversize-member-ab-cg30.md
+108
View File
@@ -0,0 +1,108 @@
# Agent A/B — bounded oversize cluster member (task CG-30)
**Date:** 2026-08-06 · **New:** `bugfix/CG-30` · **Baseline:** `main` @ `d6d1728` ·
**Harness:** `scripts/agent-eval/ab-new-vs-baseline.sh`, `--model sonnet --effort high`,
**both arms codegraph-on**, CLI blocked (0 contamination in every run),
`CODEGRAPH_NO_PROMPT_HOOK=1`.
CG-30 bounds how far a cluster's top member may overshoot what its file may spend: past 1.5x it
is windowed on whole lines instead of emitted whole. The risk the A/B exists to price is the one
CLAUDE.md names — a section that is no longer sufficient sends the agent to Read, and one or two
of those teach it to stop calling codegraph at all.
**Verdict: no regression, and the deterministic win is unambiguous.** The behavioural bar holds
(Read/Grep ~0, no abandonment, allocation efficiency 100% on the repo where the bound engages),
the cost is a ~10% median duration increase on django inside overlapping ranges, and the one
allocation-miss call the new arm produced is matched by two recall-miss calls in the baseline.
> **Harness note, recorded because it cost a re-run:** `ab-new-vs-baseline.sh` checks the engine
> out at the BASELINE ref while its baseline arm runs and restores it on exit. **Do not commit
> while it is running** — a commit made mid-run captures baseline sources. The first django/gin
> batches were void for exactly this reason (their `changed:` line listed only
> `explore-diagnostics.ts`, i.e. both arms ran identical retrieval code) and were re-run. Check
> that line before believing any A/B in this harness.
---
## Deterministic measurement — where the bound actually engages
Same index, same query, both builds. This is the primary evidence; the agent runs below only
price the risk.
**django** — `codegraph explore "How does a QuerySet turn into SQL and fetch rows from the
database?"`
| | baseline | new |
|---|---|---|
| `django/db/models/query.py` | 7,784 chars on a 3,669 budget — **2.12x** | 5,464 — **1.49x**, windowed |
| `django/contrib/admin/filters.py` | 3,633 (inherited 2,271 spendable) | **8,057** (inherited 9,160) |
| source delivered | 17,929 chars, 5 files | **20,033** chars, 5 files |
The reported CG-30 signature, reproduced on a public repo and then closed: the rank-#1 file took
2.12x its budget, and the files below it inherited the shortfall. Bounding it hands those bytes
straight down the rank order — the response carries the same five files and 2,104 more chars of
actual source.
**gin (control)** — the two builds produce **byte-identical** explore output (13,457 chars) for
the route-dispatch query. Nothing in gin is oversize enough for the bound to engage (max
observed 0.94x of spendable), which is exactly what a control should show — and it means every
gin number in the agent table below is run-to-run variance, not the change.
**Fixture** — `__tests__/fixtures/oversize-member-ts`, three report builders competing for one
envelope, each a single long function:
| File | baseline | new |
|---|---|---|
| `monthly.ts` (24.5K, one ~490-line function) | 12,391 chars on a 3,334 budget — **3.7x** | 4,941 — **1.48x**, windowed |
| `quarterly.ts` (11.4K, one ~200-line function) | **dropped** — `budget-clusters`, no headroom left | 4,004 delivered |
| response | 19,223 chars, 3 files | 15,852 chars, 4 files |
The two rows are the same defect from both sides: a member bigger than the file's share eats the
envelope, and a member bigger than the whole response ceiling makes the file vanish. Pinned by
`__tests__/explore-oversize-member.test.ts` (9 tests; 4 fail on `main`).
## Agent runs
| | django new | django base | gin new | gin base | excalidraw new | excalidraw base |
|---|---|---|---|---|---|---|
| runs | 5 | 5 | 3 | 3 | 2 | 2 |
| duration (s) | 39 [36–71] | 35 [35–60] | 39 [37–51] | 34 [28–46] | 52 [43–60] | 41 [40–42] |
| tool calls | 3 [3–10] | 4 [3–23] | 4 [3–4] | 3 | 4 [3–5] | 4 [3–4] |
| codegraph calls | 2 [2–3] | 2 [0–3] | 2 [2–3] | 2 | 3 [2–4] | 3 [2–3] |
| Read | 0 [0–5] | 0 [0–13] | 0 [0–1] | 0 | 0 | 0 |
| Grep/Glob | 0 | 0 | 0 | 0 | 0 | 0 |
| occupancy share | 33.1% [30.7%–49.5%] | 34.4% [29.3%–47.3%] | 28.9% [26.4%–37.3%] | 30.1% [28.6%–31.9%] | 43.0% | 39.3% |
| allocation efficiency | 100.0% | 98.6% | 88.5% | 98.3% | 85.8% | 90.5% |
django is pooled over two batches (n=2 + n=3). Questions: django "How does a QuerySet turn into
SQL and fetch rows from the database? Trace the flow end to end."; gin "How does a registered
route handler get invoked for an incoming HTTP request?…"; excalidraw "How does updating an
element re-render the canvas on screen?…".
**Sufficiency, pooled per call — the bar that matters.** django: new 1 "Read a file we returned"
in 10 answered calls (the allocation-miss signal a window would trip first) against the
baseline's 1 "Read a file we did not return" + 1 Grep in 10 — a shift in miss type, not an
increase. gin: 1 allocation miss in 7 against 0 in 6, on a repo where the two builds emit
identical bytes, so it is variance by construction. excalidraw: 0 misses in either arm.
**Where the new arm looks worse, and why it is not read as a regression:**
- *django duration, ~10% slower median.* Ranges overlap (36–71 vs 35–60) at n=5, and one
baseline run lost its codegraph attach entirely (0 codegraph calls, 13 Reads, 23 tool calls),
which distorts that arm's spread in both directions.
- *gin allocation efficiency 88.5% vs 98.3%.* The builds are byte-identical on gin. This is the
metric's documented relativity — attribution is by citation and the agent's follow-up queries
differ per run — not an effect of the change.
- *excalidraw occupancy/duration.* Call-count noise: one of the two new-arm runs made a 4th
explore call where the baseline made 2–3, and duration, envelope and occupancy all follow it.
Per-call envelope is flat (20,015 vs 19,446 chars/call), Read/Grep stay 0, tool calls match.
CLAUDE.md's own worked example records 3–10 codegraph calls on this prompt.
## Caveat carried forward
The `self-query` probe fixture in `scripts/agent-eval/allocation-fixtures.json` flips to FAIL
under this change. Allocation is unchanged between arms and `tools.ts` delivers the identical
8,282 chars in both — what changed is that an over-reserved incidental file now *delivers*
instead of being cut by the hard-ceiling truncation, which is what its previous PASS depended
on. Recorded as that fixture's `afterCG30` block. The over-reservation itself is epic CG-24's
subject; it should not be answered by loosening this bound.
docs/benchmarks/explore-reservation-invariant-ab-cg26.md
+156
View File
@@ -0,0 +1,156 @@
# Agent A/B — the end-to-end reservation invariant (task CG-26)
**Date:** 2026-08-06 · **New:** `bugfix/CG-26` @ `7cbde95` · **Baseline:** `bugfix/CG-31` @ `c54e008` ·
**Harness:** `scripts/agent-eval/ab-new-vs-baseline.sh`, `--model sonnet --effort high`,
**both arms codegraph-on**, CLI blocked (0 contamination in every run),
`CODEGRAPH_NO_PROMPT_HOOK=1`. Every index measured on was **fully rebuilt**, never
incrementally synced (CG-33).
Baseline is the CG-31 tip, not `main`, so every number here isolates CG-26. Read the three
in sequence: `explore-oversize-member-ab-cg30.md` → `explore-displacement-guard-ab-cg31.md` →
this one.
**The invariant:** every admitted file receives at least its reservation before any file draws
on carry-forward slack. CG-30 bounded an oversize cluster member; CG-31 gave the cluster path a
displacement guard. This closes the three holes left over — and each one was starving a file that
had been admitted, reserved, and in the worst case *rendered*.
**Verdict: no behavioural regression on three repos, and the response is honest about its own
budget for the first time.** No repo truncates. No repo loses a file; okhttp gains one. Two repos
trade a few hundred source chars on their LAST-ranked file for the pointer list that names what
the response could not cover — bytes the CG-31 tip only had because it over-filled a ceiling it
mis-measured and then discarded the whole epilogue.
---
## The three holes
**1. The whole-file arms had no displacement guard.** The BUY arm's fit test read
`totalChars + fileContent.length + FILE_OVERHEAD <= renderCeiling` — room before the ceiling,
which belongs to every file the loop has not reached — while its own source-space sibling
(`owedBelow`) refused exactly that trade. GRACE was not fit-tested at all. Measured on okhttp:
`CallServerInterceptor.kt` shipped **8,499 chars against a 5,964 funded ceiling**, and the rank-6
file below it delivered nothing. Both arms now test the render they actually produce against
`fundedHeadroom`, and a whole render that does not fit **falls through to clustering** instead of
skipping the file — a clustered section traded for no section is the trade the funding pool exists
to refuse.
**2. Section overhead was charged at a flat 200 chars.** A real header — path plus up to
`maxSymbolsInFileHeader` symbol names — runs 300–500. Everything downstream is expressed in those
units (`headroom`, `fundedHeadroom`, every fit test), so the under-count was not a rounding error:
it funded promises out of bytes that did not exist. okhttp allocated **26,601 chars against a
24,400 ceiling** and the final truncation threw a fully-rendered section away. Sections are
charged their real cost now; `owedPayableBelow` holds back each pending file's reservation *plus a
per-file overhead estimated from that file's own symbols*; and a marginal overrun **trims the
weakest cluster** — or windows the last one into the room that is left — rather than skipping a
file over a ~300-char accounting difference.
**3. `owedPayableBelow` held all-or-nothing.** CG-31 was right that a promise the ceiling cannot
reach is not a claim on this file's bytes — but it dropped the *partial* case. When the last
admitted file's FULL reservation no longer fit, nothing at all was held for it. On the
precise-query fixture the rank-5 file took 4,134 chars against a 2,948 reservation while rank 6 —
admitted, reserved 2,539 — was left **4 chars** and skipped. It now holds the remainder, while
that remainder is still worth a section (`MIN_CHARS`).
## The epilogue, budgeted instead of discarded
CG-31 handed this forward: the loop reserved a flat **600** chars for an epilogue that measures
1,064 (gin), 1,788 (django), 2,231 (excalidraw), and four of six suite repos survived by
discarding the epilogue **whole** — shipping with no pointer list and no reminders at all. A
margin sweep was run and deliberately not shipped, because tuning one constant against the suite
is the trap CG-30's record warns about.
The fix is not a bigger constant. The epilogue is **two things**:
- a **floor** the render loop reserves, sized from the real strings: the one line that says an
uncovered area exists and that another explore — not a Read — reaches it, plus a pointer for
every file whose bytes were deliberately WITHHELD (a cliffed file's bytes were traded away on
the promise that the agent can still name it — CG-12; if the ceiling eats that name the trade
was a silent drop);
- an **elastic tail** — the rest of the pointer list and the reminders — fitted, in priority
order and entry by entry, to the room that is actually left once the loop is done.
So a saturated response now lands with as much of its epilogue as it can pay for, instead of none
of it, and `renderCeiling` is `hardCeiling − floor` rather than `hardCeiling − 600`.
## Deterministic measurement — the primary evidence
Same clean-rebuilt index, same query, both builds. One `codegraph_explore` per repo.
Reproduce with `node scripts/agent-eval/probe-suite-envelope.mjs` (added by this task).
| repo | base source | new source | Δ | base files | new files | ceiling behaviour |
|---|---|---|---|---|---|---|
| django | 20,791 | **20,878** | +87 | 6 | 6 | was discarding its epilogue |
| tokio | 21,521 | **21,607** | +86 | 5 | 5 | was discarding its epilogue |
| okhttp | 19,034 | 18,870 | −164 | 5 | **6** | +1 file delivered; keeps its pointer list |
| excalidraw | 20,204 | 19,652 | −552 | 8 | 8 | keeps its pointer list |
| gin | 10,776 | 10,776 | 0 | 4 | 4 | byte-identical |
| alamofire | 11,662 | 11,662 | 0 | 2 | 2 | byte-identical |
Queries are the CG-30/CG-31 ones, unchanged.
**Read the two negatives honestly.** They are not starvation — they are the reverse. At the CG-31
tip both responses were *over-filled*: the loop under-counted its own section overhead, spent past
the render ceiling, and the hard-ceiling cut then took the epilogue away to pay for it. okhttp
also had a file rendered and dropped. Now the accounting is exact, so the loop stops where it
said it would, and the ~500 chars go to the pointer list naming the files the response could not
cover (2 on excalidraw, both `max-files` skips). No admitted file is starved in either.
**gin and alamofire are byte-identical between the builds** — neither saturates, so neither the
guard nor the epilogue fit engages. That is what a control should show.
**Fixtures.** `__tests__/explore-reservation-invariant.test.ts` (14 tests; **3 fail on the CG-31
tip**) pins the invariant on all three render paths and in both directions — the rank-#1 file when
files below it overspend, and an admitted lower-ranked file when the top one does — plus the two
things the ceiling must no longer do (allocate past itself; drop a rendered section) and the
concentration it must not flatten. `__tests__/explore-displacement-guard.test.ts` (CG-31, 11
tests) still passes unchanged.
**Allocation fixtures** — `scripts/agent-eval/allocation-fixtures.json`: **both PASS**. The
self-query gate changed shape and the reason is recorded in `afterCG26`: the envelope-denominated
`answerShareAtLeast` reads 47.5% here against 51.0% at the CG-31 tip while `tools.ts` delivers
**byte-identical** source in both arms. What moved is the denominator — the response now delivers
a fifth admitted file (`memory-budget.ts`, rank 4, paid its full 3,123-char reservation; the CG-31
tip rendered it and let the ceiling drop the section) and keeps epilogue prose it used to discard.
Both are the improvements this epic exists to make. The gate is now denominated in delivered
SOURCE, where the answer group reads 55.5%, and it passes on both arms.
## Agent runs
| | django new | django base | excalidraw new | excalidraw base | okhttp new | okhttp base |
|---|---|---|---|---|---|---|
| runs | 2 | 2 | 2 | 2 | 2 | 2 |
| duration (s) | **42** | 43 [36–50] | 53 [45–62] | 45 [37–54] | 49 [39–59] | 42 [32–52] |
| tool calls | 4 [3–4] | 4 [3–5] | **3** [2–4] | 5 [4–5] | 4 | 4 [3–5] |
| Read | **0** | 0 | **0** | 0 | **0** | 0 |
| Grep/Glob | 0 | 0 | 0 | 0 | 0 | 0 |
| codegraph calls | 3 [2–3] | 3 [2–3] | **3** [2–3] | 4 [3–4] | 3 | 3 [2–4] |
| occupancy share | **36.6%** | 37.6% | **38.2%** | 47.2% | 44.8% | 40.8% |
| allocation efficiency | **99.2%** | 94.7% | 81.9% | 82.5% | 75.8% | 87.6% |
Prompts are the deterministic queries with "Trace the flow end to end." appended.
**Read is 0 in all 12 runs, in both arms.** Sufficiency, pooled per call: **0 "Read a file we
returned" and 0 recall misses on every repo in both arms** — the responses that deliver a few
hundred fewer chars do not send the agent back to the file.
**Where the new arm looks worse, and why it is not read as a regression:**
- *okhttp allocation efficiency, 75.8% vs 87.6%, and occupancy 44.8% vs 40.8%.* The new arm's
envelope is 99,411 chars against 89,297 — it returns one more file and more source overall, and
the metric is the share of returned bytes the answer *cited*. Same trade the CG-31 record noted
on this repo; the metric's own documentation says it is relative and must not be read as waste.
- *Duration on excalidraw and okhttp.* n=2 with fully overlapping ranges (45–62 vs 37–54;
39–59 vs 32–52), on a machine also running the other arm's build. excalidraw's new arm does the
same work in **3 tool calls against 5** and holds **9 points less context**.
## Residuals
None from this task. The render-loop budget is now exact end to end: `totalChars` counts
`flow.text`, the real per-section cost, and the epilogue floor; `allocatedChars ≤ hardCeiling` on
every suite repo; and the final section-boundary truncation is now unreachable in normal
operation (it stays as the backstop).
One thing deliberately NOT changed: the pointer list still caps at 10 files. Trimming happens
from the bottom of the rank order and the "+N more files" tail is rewritten to confess every entry
dropped, so the count is never silently wrong.
docs/benchmarks/explore-tail-render-cg38.md
+185
View File
@@ -0,0 +1,185 @@
# CG-38 — an agent-named symbol in the tail of a large file never rendered
**Status: fixed.** Two independent causes, both longstanding. Not a CG-24 regression —
the controlled bisect (index held fixed, engine varied across every epic merge point)
found the symptom at every build including pre-epic.
## The report
On a 1,414-line Svelte store, `codegraph_explore` never returned `queueMessage`
(L1087) or `flushQueuedMessages` (L1102) — on a bare symbol bag *or* a prose
question — even though their file won rank #1 with score 127 and 67.3% of the
envelope. What came back instead was the same-stem `QueuedMessage` **interface** at
L70. The agent had to Read the file to find the two functions it had asked for by
name, which is the one outcome explore exists to prevent.
CLAUDE.md's *"guarantee named symbols render"* — the importance-9 named-def
injection — was not holding.
## What it actually was
### 1. The named-symbol IDENTITY was discarded with the narrative
`buildFlowFromNamedSymbols` returns two unrelated things: the Flow prose, and the
SET of node ids the agent named. Downstream, that set is what injects a named def
into its file's cluster ranges and ranks it **importance 9** — the entire mechanism
behind the guarantee.
Its last gate was:
```ts
if (!hasMain && synthLines.length === 0 && !boundaryText && !polyText) return EMPTY;
```
`EMPTY` zeroes `namedNodeIds` too. So whenever the named symbols happened not to
produce anything to *print*, the guarantee silently switched off. Two sibling
closures in one factory are exactly that case: `queueMessage` and
`flushQueuedMessages` never call each other, so there is no chain, no synthesized
hop and no dispatch boundary — and both defs lost importance 9. The file then
rendered from its head, which is how a 6-line interface displaced two functions
1,000 lines below it.
Measured: `flow.namedNodeIds` was **empty** on the reported query, while
`findAllSymbols` resolved both tokens to exactly 1 node each.
The fix separates the two outputs (`identityOnly()`), restricted to **shape-precise
tokens** (camelCase / PascalCase / snake_case / qualified — the same test the gather
path uses). With a narrative present, the prose is itself corroboration and that path
is unchanged; with nothing corroborating it, only an unambiguous symbol reference may
promote, so an English word in a prose question that happens to exact-match a
callable cannot earn importance 9.
### 2. The ceiling trim cut in SOURCE ORDER
Restoring importance 9 was not enough — the symbols still did not render.
`shrinkCluster` *had* kept them: on the reported query it emitted a block spanning
`1022-1121`, which covers both. But the shrink's output measured **26,297 chars
against a 16,532 cap**, so `windowToCeiling` fired, and it fills parts in source
order and drops everything after the first overrun:
```
shrunk: 101-107, 197-226, ..., 648-989, 1022-1121 (26,297)
windowed: 101-107, 197-226, ..., 648-839 (16,532) ← tail gone
```
A trim that cuts in source order will always take the END of a large file first —
which is precisely where an agent-named symbol is most likely to be, and least
likely to be reachable any other way. `windowToCeiling` already had the concept it
needed (`focusLine`, for the spine's next-hop call site, CG-30); it just wasn't told
about named defs. It now takes a `focusLines` list — spine call site plus every
member at importance ≥ 9, capped at 6 — and:
- tries the **full-ceiling** fill FIRST, holding back 40% only when a focus line is
actually left uncovered (so a cluster whose head already reaches its focus keeps
the whole ceiling for source — an improvement on the old unconditional hold-back);
- **splits** the reserve evenly between the uncovered focus lines with carry-forward,
rather than handing it out greedily in source order. Greedy reproduced the bug one
level down: on the prose query, four focus lines resolved and the two earliest took
the entire reserve, dropping `flushQueuedMessages` again.
## The accounting gap — found, measured, deliberately NOT shipped
`shrinkCluster`'s fit test uses the raw source span
(`slice().join('\n').length`) while the render adds `contextPadding` around every
block and a line-number prefix to every line. On the reported file that estimate ran
**~60% under** (16.5K accounted, 26.3K rendered).
An exact projection (prefix-summed line costs, mirroring the merge + padding
`buildSection` performs) was built and measured. **It is worse, and it is not
shipped:**
| | main | exact accounting | exact + spend-the-remainder |
|---|---|---|---|
| django | 20,719 | 20,747 | 20,747 |
| excalidraw | 19,704 | **19,606** | **19,606** |
| okhttp | 18,651 | 18,766 | 18,766 |
| tokio | 21,582 | **21,424** | 21,555 |
| gin | 11,952 | 12,082 | 12,082 |
| alamofire | 11,849 | 11,849 | 11,849 |
| `probe-allocation` | 4 PASS | **payroll-go FAIL** | **payroll-go FAIL** |
The mechanism: exact accounting stops at the last member that fits **whole**, and the
released bytes carry forward to lower-ranked files. On `payroll-go` that moved 1,296
chars out of the rank-#2 answer file `cycle.go` and into the rank-#5
`payslipstore/store.go`, taking `runPayrollCycleAll`'s `s.store.Upsert(ctx, slip)`
call — the "create" half of the query — with it.
So the slack is doing no harm where it is: `bound()` clamps the render to the ceiling
exactly, so the over-keep costs no bytes. What the slack must **not** do is decide
*which* members survive — and that is the ceiling trim's job, which is what this task
fixed. The comment on `shrinkCluster` now says so, so the next reader does not
"fix" it.
## The index-dependence lead — explained, and orthogonal
The issue's sharpest lead was that flagging the ambient `.d.ts` as `generated` seemed
to make an unrelated file's render *worse*. Flipping `files.generated` on that one row
(the CG-25 method — holds the index constant, attributes the delta to the ranker
alone) confirms the mechanism is real:
| | `generated=1` | `generated=0` |
|---|---|---|
| `.d.ts` graphScore | 0.1875 | 0.75 |
| `maxGraph` | 0.3297 | 0.75 |
| gate (6% of max) | 0.0198 | 0.0450 |
| files ranked | 3 | 2 |
| rank-#1 allowance | 9,100 | 8,166 |
`rankPenalty` scales `fileGraphScore`, `fileGraphScore` sets `maxGraph`, and the
relevance gate is 6% of `maxGraph` — so a penalty on one file does move the admitted
set and every other file's allowance. Confirmed.
But it is **not** what hid the symbols. On main they are absent at *both* flag states
(render stops at L316 / L381); with the fix they are present at *both*. The
allocation moves; the guarantee does not depend on it. Pinned by the last case in
`__tests__/explore-named-symbol-render.test.ts`.
## Results
Real repro (`queueMessage` L1087 / `flushQueuedMessages` L1102), all shapes:
| query shape | main | fixed |
|---|---|---|
| symbol bag | absent | **both render** |
| prose, symbols named | absent | **both render** |
| symbols + decoy interface | absent | **both render** |
| prose, no symbols named | absent | **both render** |
Fixture (`__tests__/fixtures/tail-render-ts`, 7 symbol checks over 3 query shapes):
**7/7 fail on main, 7/7 pass** — deterministic over 4 consecutive runs per arm.
Standing bars, all held:
- `probe-allocation.mjs` — payroll-go / starved-cluster / dense-header / self-query all PASS
- `probe-file-spend.mjs` — no starvation flags
- `probe-suite-envelope.mjs` — **byte-identical to main on all six repos** (20,719 /
19,704 / 18,651 / 21,582 / 11,952 / 11,849), same file counts
- full suite green
The suite being byte-identical is the point: the focus windows only change what a
render does once it has *already* overrun its ceiling, which none of the six suite
queries does.
## Instruments
- `scripts/agent-eval/probe-named-symbol.mjs` — the measurement the epic lacked.
Per-SYMBOL and binary: is the symbol's **definition line** among the response's
rendered lines? The name alone proves nothing — it appears in the section header's
symbol list and at call sites whether or not the body was sent, which is exactly how
this hid through a whole epic of aggregate probes.
- `__tests__/fixtures/tail-render-ts` — mirrors the reported file's geometry: decoy
same-stem interface at L70, factory closure at L104 spanning ~92% of the file (so
every symbol merges into ONE cluster), targets at L1088/L1096/L1102, plus a
2,500-line generated `.d.ts` for the ranker to penalise. Generated by script; edit
the geometry, not individual lines.
- `__tests__/explore-named-symbol-render.test.ts` — the standing gate, including the
fixture-shape assertions (if the fixture rots, the gate means nothing).
## Method note
A `git stash -- <path>` "baseline" reverts to **HEAD**, not to `main`. With a WIP
commit on the branch that silently measures your own change against itself — it
produced a clean "passes on main" here that was pure fiction. Use the file swap
(`git show main:<path> > <path>`), as `.kommandr/memory/baseline-builds-use-fresh-file-swap`
already says for builds.
docs/benchmarks/index-drift-cg33.md
+189
View File
@@ -0,0 +1,189 @@
# Index drift: incremental sync vs. full rebuild (CG-33)
Measured 2026-08-06. A live, auto-sync-maintained index **does not converge** to
a clean full rebuild of the identical working tree. On codegraph's own repo,
**4.3% of distinct edges were wrong**, in both directions, overwhelmingly
`calls` edges.
This matters because it is silent: nothing warns, nothing surfaces it, and the
README tells users the index is never stale and there is nothing to re-run.
Retrieval quality decays invisibly, and the user-visible symptom — an agent
falling back to Read — reads as "codegraph isn't very good" rather than "this
index needs rebuilding."
## Result
Subject: codegraph's own `.codegraph/codegraph.db`, long-lived and
incrementally synced, against a full rebuild of the same tree with the same
build. Edges compared as distinct `(source, target, kind)` triples.
| | count |
|---|---|
| distinct edge triples (rebuild) | 28,809 |
| in rebuild but **missing** from live | **751** |
| in live but **absent** from rebuild (stale) | **476** |
| **total divergent** | **1,227 — 4.3%** |
Missing edges by kind: `calls=635`, `contains=38`, `references=34`,
`instantiates=21`, `imports=13`, `extends=10`.
### Raw counts hide it
Raw edge **rows** were 39,845 live vs 40,122 rebuilt — a benign-looking +0.7%.
The divergence is bidirectional, so a net-count check nets it out and reports
almost nothing wrong. **Any drift detector must compare edge sets, not totals.**
### The indexer is deterministic
Control, rebuild vs rebuild on the same tree and build: **0 differing edges**
(28,809 both runs). So the live-vs-rebuild delta is not run-to-run noise.
### It is resolution, not residue
Node sets are identical — `files` 501 = 501, `nodes` 10,110 = 10,110,
heuristic edges 36 = 36 — and every integrity check is 0 on *both* indexes:
no duplicate nodes, no orphan edges, no nodes referencing a missing file row.
Nothing accumulates. Cross-file **resolution** goes stale.
## Mechanism — two causes, both confirmed
`ReferenceResolver` binds a reference to one of the same-named definitions
**project-wide**. Two things follow, and the drift needed both to be fixed.
**1. Scope.** Incremental sync re-resolves only the references *in* the changed
files. Adding or removing a definition of `pct` changes the correct answer for
every `pct(...)` reference in the repo, including references in files the sync
never touches — and those references resolved successfully once, which *deletes*
their `unresolved_refs` row, so nothing existed to revisit them with. (The #1240
retry only revisits refs parked as `status='failed'`.) The index kept an answer
that was correct against an older graph.
**2. Tie-break.** When nothing disambiguated the candidates, `findBestMatch`
kept the first one, and `getNodesByName` had no `ORDER BY` — so the winner was
decided by rowid, i.e. by the order files happened to be **written**. A full
index writes in scan order; a sync appends each file as it changes. The same
tree therefore resolved to different edges depending on how the index was built,
and no amount of re-resolution could converge, because re-resolving against the
identical graph still picked a different candidate.
### The fix
- `getNodesByName` orders by `(file_path, start_line)` — a property of the code,
not of the write order (`src/db/queries.ts`).
- `sync` returns a `definitionDelta`: the names whose set of definitions the sync
changed, computed as the symmetric difference of `file\0name` pairs sampled
before and after the store phase (`ExtractionOrchestrator.sync`).
- For each delta name, `resurrectStaleResolutionEdges` deletes the resolution
edges targeting a symbol of that name whose source is in an *unchanged* file,
and re-inserts each as the reference that created it (the `metadata.refName`
stamp). The existing orphan sweep then resolves them against the post-sync
graph — the same input a rebuild resolves from. Kill switch:
`CODEGRAPH_NO_REBIND=1`.
The delta is compared **per file**, not as one name set over the whole batch: a
commit that adds `collect` to a new file while an unrelated changed file already
defines `collect` cancels out of a batch-wide name set, and that miss was the
largest residual class in the first measurement of this fix.
Conservative by construction, because a wrong deletion is a permanent edge loss
while a missed rebind is only residual drift: an edge with no `refName` stamp
(synthesized, or built by an older engine) is never touched, edges whose source
file the sync already re-extracted are skipped, and a per-name ceiling of 500
edges declines the generic names.
### Result
Replaying real commits of this repo through `sync` one at a time, then diffing
against a clean rebuild of the final tree:
| replay | baseline (`main`) | + ORDER BY only | + rebind pass (shipped) |
|---|---|---|---|
| 16 commits | 48 (24 missing / 24 stale) | 20 | **0 — converged** |
| 80 commits | 1,634 (963 / 671) | 890 | **361 (359 / 2)** |
The direction that actively misleads — **stale** edges the index keeps asserting
— drops from 671 to **2** over 80 commits, a 99.7% reduction.
Index and sync wall-clock are unchanged (392-file repo: index 1.88–2.02s in both
arms, single-file sync 0.183s in both). The `ORDER BY` costs 18% per *uncached*
name lookup in a tight loop (237ms → 280ms over 10,127 lookups), which does not
reach wall-clock because `ReferenceResolver` memoizes the lookup per name. A
composite `(name, file_path, start_line)` index would make the sort free, but it
would widen every node index entry with a full path string on the write-heavy
indexing path — not worth 43ms.
### The residual, and why it is not chased
At 80 commits, 357 of the 361 remaining edges are a single pre-existing class:
references to very generic names (`push` 260, `join` 97) that failed at index
time and stay parked because `getRetryableFailedReferences` declines any name
with more than 500 failed refs (1,412 for `push`, 2,346 for `join`). That
ceiling is #1240/#999 policy, it is present on `main`, and what it declines to
create is cross-language garbage: a TypeScript test file "calling" an R method
named `push`, or a Rust method named `join`. **The full rebuild is the wrong one
here** — converging would mean teaching sync to manufacture thousands of wrong
edges. Left as is, deliberately.
### `codegraph status` — decided: no drift metric
The issue asked whether `status` should surface divergence. Decision: **no**.
A drift number cannot be computed without the full rebuild it would be
recommending, so anything cheap enough to run on `status` would be an estimate —
and an honest estimate is not available. Shipping a proxy would violate the
product rule that a screen must not overclaim, and post-fix it would fire on the
generic-name residual above, training users to ignore it. (`status` already
refuses to warn on parked failed refs for the same reason: every repo with
external-library imports has them, so the warning would be permanent noise.)
The check that *is* exact stays available and is documented below.
## Why it degrades retrieval
Graph mass (RWR) is **relative and normalized**, so call edges missing elsewhere
inflate an unaffected file's share of the mass. Explore ranks files by that mass
(`allocateExploreBudget` weights on it), so drift silently promotes files that
should rank low.
Observed on a private application repo under heavy development: a generated
ambient-types file carried graph mass **0.24750** on the drifted index vs
**0.13119** on a clean rebuild (~1.9×), and score **49.0** vs **27.0**. On the
drifted index it took **60.7%** of an explore envelope and starved the file the
agent had actually named by symbol, which rendered **251 chars of a 10,970
reservation**. After a full re-index — no code change — the same query answers
correctly. That incident is what prompted this measurement; see CG-24.
Severity scales with churn and index age. codegraph's own repo shows 4.3%;
a repo under heavier active development plausibly drifts further.
## Reproducing
`scripts/agent-eval/diff-index-drift.mjs` is read-only and diffs two indexes.
Snapshot the live index **before** rebuilding — the original artifact for this
investigation was destroyed by re-indexing over it:
```bash
cp .codegraph/codegraph.db /tmp/live.db # snapshot FIRST
node dist/bin/codegraph.js index . # full rebuild
node scripts/agent-eval/diff-index-drift.mjs /tmp/live.db .codegraph/codegraph.db
```
Exit code is 0 when converged, 1 when drifted. To re-confirm determinism, diff
two consecutive rebuilds — that must report 0.
To reproduce the *regression* rather than measure a live index, replay real
commits through `sync`: clone the repo, check out `HEAD~N`, index, then
`git checkout <sha> && codegraph sync` for each commit in order, snapshot the
database, and diff it against a rebuild of the final tree. That is what produced
the table above, and the unit-scale version of it is
`__tests__/sync-rebuild-convergence.test.ts`.
## Note on probing an index
The index file is `.codegraph/codegraph.db`. There is no `graph.db`. `sqlite3`
against a mistyped path **creates an empty database** rather than failing, and
every subsequent query then answers from an empty schema — which reads exactly
like a stale pre-migration index. That produced a wrong root cause during this
investigation. `diff-index-drift.mjs` checks `existsSync` before opening for
exactly this reason.
scripts/agent-eval/allocation-fixtures.json
+192 -7
View File
@@ -4,7 +4,14 @@
"explore budget allocation. Run them with `node scripts/agent-eval/probe-allocation.mjs`",
"against a built dist/.",
"",
"STATUS: BOTH FIXTURES PASS. CG-10 (relevance scoring) closed the RANKING half —",
"STATUS: BOTH FIXTURES PASS again as of CG-31. self-query's delivered-share gates",
"failed on the CG-30-only build (`afterCG30`) and CG-31 restored them (`afterCG31`):",
"the incidental file was not over-RESERVED at all — it was over-SPENDING, drawing on",
"the reservations of files the render loop had not reached yet. Bounding that put the",
"response back inside the envelope, so nothing truncates and every admitted file",
"delivers. Read the two blocks together; the CG-30 verdict's diagnosis was wrong.",
"",
"CG-10 (relevance scoring) closed the RANKING half —",
"nothing incidental reaches the envelope any more — and CG-12 (score-proportional",
"allocation with a relative cliff) closed the BYTE SPLIT: each file's share is reserved",
"before anything renders, and a file under 15% of the top weight gets no source at all,",
@@ -15,7 +22,14 @@
"actually about) and `incidental` (what wins the envelope today on name collisions).",
"Assertions are on the DELIVERED envelope unless suffixed `Allocated`; delivered is",
"what the agent got, allocated is what the render loop chose before the hard ceiling.",
"Shares are fractions of the whole response, meta-text included, so they never sum to 1."
"Shares are fractions of the whole response, meta-text included, so they never sum to 1.",
"",
"CG-36 adds two more fixtures and a per-file `spendShareAtLeast` gate. The share gates",
"above ask which files WON the envelope; that one asks whether a file that won its share",
"then spent it. `starved-cluster` and `dense-header` are the two halves of the same",
"tradeoff and must be read together — one fails if a trivial cluster starves the",
"answer-bearing one, the other fails if the fix for that buries a query's own methods",
"under a dense declaration block."
],
"fixtures": [
{
@@ -44,7 +58,9 @@
"internal/domain/**",
"cmd/**"
],
"incidental": ["internal/gen/**"]
"incidental": [
"internal/gen/**"
]
},
"assert": {
"answerShareAtLeast": 0.55,
@@ -96,6 +112,124 @@
"verdict": "ALL GATES PASS. Answer group 78.7% (from 25.6% at baseline), generated layer 0.0% (from 57.4%). All four hand-written files deliver source, including payslip_builder.go — `func (s *Service) BuildPayslip`, the 'calculate' half of the question, finally reaches the agent. The generated files are still NAMED with their symbols and line numbers under 'Not shown above', so withholding their bytes costs ~100 chars each instead of ~4,500 and stays one follow-up explore away."
}
},
{
"id": "starved-cluster",
"title": "CG-36 — a trivial top-ranked cluster starving the answer-bearing one",
"kind": "fixture",
"path": "__tests__/fixtures/starved-cluster-ts",
"query": "how does a request travel from sendRequest to the socket",
"rationale": [
"django's `db/models/sql/query.py` and okhttp's `RealInterceptorChain.kt`, reduced",
"to a fixture. `chain.ts` holds a one-line `describeChain` helper at the top —",
"trivial, but a direct callee of the query's entry point, so its cluster carries",
"the file's highest per-symbol importance — and, past the cluster gap, the",
"`RequestChain` class that actually answers the question. The helper's cluster wins",
"the one guaranteed-and-shrinkable slot; the class then does not fit the remainder.",
"",
"Before CG-36 the class was dropped WHOLE and the file delivered 1,985 of a 6,904",
"reservation. That is not merely unspent budget: the slack carries forward to",
"lower-ranked files, so the response stays full and every envelope-share gate",
"passes while the answer is missing. Hence `spendShareAtLeast`."
],
"groups": {
"answer": [
"src/pipeline/chain.ts",
"src/transport/**",
"src/app/client.ts"
],
"incidental": [
"src/app/config.ts",
"src/pipeline/framing.ts"
]
},
"assert": {
"topFileGroup": "answer",
"spendShareAtLeast": {
"src/pipeline/chain.ts": 0.6
},
"mustDeliverBytes": [
"src/pipeline/chain.ts"
],
"$mustContainComment": "The two ends of the in-file flow: the chain hop and the transport hop it terminates in. Both live in the cluster that used to be dropped whole.",
"mustContain": [
"async proceed(request: PipelineRequest)",
"private async writeAndRead(request: PipelineRequest)"
]
},
"baseline": {
"measuredOn": "2026-08-06",
"note": "The CG-24 epic tip (76ab1fe), before CG-36. 3,725 chars of source delivered in total.",
"delivered": {
"src/pipeline/chain.ts": 1985,
"src/app/client.ts": 997,
"src/pipeline/types.ts": 743
},
"verdict": "FAILS spendShareAtLeast and both needles. chain.ts spends 1,985 of its 6,904 reservation (28.8%) — it keeps the `describeChain` cluster and drops the `RequestChain` cluster whole, so neither `proceed` nor `writeAndRead` reaches the agent. Nothing else in the response is wrong: the file still ranks #1 by score and is still reserved the largest slice."
},
"afterCG36": {
"measuredOn": "2026-08-06",
"note": "10,802 chars of source delivered in total, nothing truncated.",
"delivered": {
"src/pipeline/chain.ts": 9062,
"src/app/client.ts": 997,
"src/pipeline/types.ts": 743
},
"verdict": "ALL GATES PASS. The `RequestChain` cluster is now SHRUNK into the remainder by the same whole-member rule the first cluster already used, instead of being dropped whole, so chain.ts delivers 9,062 chars including `proceed`, `advance` and `writeAndRead` — the whole in-file flow the question asks for."
}
},
{
"id": "dense-header",
"title": "CG-36 — the Session.swift shape density-first ranking exists for",
"kind": "fixture",
"path": "__tests__/fixtures/dense-header-ts",
"query": "how does perform create a URLRequest and start the task",
"rationale": [
"The counterweight to `starved-cluster`, and the reason CG-36 did NOT touch cluster",
"ranking. `session.ts` opens with a 60-line property list and a run of trivial",
"accessors — many adjacent, individually worthless declarations, i.e. the densest",
"block in the file — while `perform`, `didCreateURLRequest` and `task`, which the",
"query names, sit ~200 lines below it.",
"",
"Ranked on density alone the header block takes the file's whole budget and the",
"methods are buried; that is Alamofire's Session.swift, the case the",
"importance-then-density order was built for. Any future change to selection or",
"shrinking has to keep this passing as well as `starved-cluster` — they pull in",
"opposite directions, which is exactly why both are here."
],
"groups": {
"answer": [
"src/net/**",
"src/core/request-builder.ts",
"src/core/task-factory.ts"
],
"incidental": [
"src/core/queue.ts"
]
},
"assert": {
"topFileGroup": "answer",
"answerShareOfSourceAtLeast": 0.8,
"spendShareAtLeast": {
"src/net/session.ts": 0.6
},
"$mustContainComment": "All three named symbols are deep in the file, past the dense header block. If density ever outranks importance again, these are the first thing to go.",
"mustContain": [
"async perform(url: string, method: string",
"didCreateURLRequest(request: URLRequest)",
"task(request: URLRequest, identifier: number)"
]
},
"afterCG36": {
"measuredOn": "2026-08-06",
"note": "11,695 chars of source delivered, BYTE-IDENTICAL to the CG-24 epic tip (76ab1fe) — this fixture pins behaviour CG-36 deliberately left alone.",
"delivered": {
"src/net/session.ts": 9007,
"src/core/types.ts": 1957,
"src/core/task-factory.ts": 731
},
"verdict": "ALL GATES PASS, on the epic tip and on CG-36 alike. session.ts spends 9,007 of its 9,009 reservation and the response carries all three named methods from the bottom of the file. The dense header block is not what won the budget."
}
},
{
"id": "self-query",
"title": "This repo — incidental `explore`/`BUDGET` matches in the agent-eval scripts",
@@ -115,14 +249,31 @@
"The assertions are therefore relative — answer-vs-incidental, not fixed percentages."
],
"groups": {
"answer": ["src/mcp/**"],
"incidental": ["scripts/**"]
"answer": [
"src/mcp/**"
],
"incidental": [
"scripts/**"
]
},
"assert": {
"answerShareAtLeast": 0.5,
"$answerShareComment": [
"Denominated in DELIVERED SOURCE, not in the whole envelope (CG-26). The",
"envelope-denominated form of this gate moved for reasons that have nothing",
"to do with allocation: it fell when the epilogue stopped being discarded,",
"and it fell again when a fifth ADMITTED file finally got paid its",
"reservation instead of being dropped by the ceiling. Both are the",
"improvements this epic exists to make, and a gate that reads them as",
"regressions is measuring the denominator. The fixture's own rationale",
"already says the assertions are relative, answer-vs-incidental, not fixed",
"percentages. 0.5 is unchanged; only what it is a share OF."
],
"answerShareOfSourceAtLeast": 0.5,
"incidentalShareAtMost": 0.25,
"topFileGroup": "answer",
"mustDeliverBytes": ["src/mcp/tools.ts"]
"mustDeliverBytes": [
"src/mcp/tools.ts"
]
},
"baseline": {
"measuredOn": "2026-08-03",
@@ -153,6 +304,40 @@
"src/resolution/lru-cache.ts": 0.111
},
"verdict": "ALL GATES PASS. tools.ts takes 60.6% of the envelope, up from 18.5% at baseline and 32.9% after CG-10 — past the epic's >50% acceptance bar. The reversal is the whole point: memory-budget.ts no longer wins by being small enough to ship whole (it now clusters within its 3.1K reservation), and tools.ts is no longer clipped at maxCharsPerFile (11K reservation, ~3x the old flat cap). Exception to 'no previously-unclipped file becomes clipped': memory-budget.ts was unclipped-whole at 5,672 and is now clipped to its proportional share. That is the epic's own diagnosis of the bug, not a regression — it scored 18 against tools.ts's 58 and was taking the larger slice."
},
"afterCG30": {
"measuredOn": "2026-08-06",
"note": "23,688 delivered of 26,430 allocated, truncated at the 25,000 ceiling. Baseline (main) on the SAME index: 14,851 delivered of 25,221 allocated. tools.ts delivers 8,282 chars in BOTH arms — identical bytes; only the denominator moved.",
"delivered": {
"scripts/agent-eval/parse-run.mjs": 0.361,
"src/mcp/tools.ts": 0.35,
"src/mcp/explore-session-state.ts": 0.147,
"src/resolution/memory-budget.ts": 0.0
},
"verdict": "THREE GATES FAIL — and the cause is not the CG-30 bound. Allocation is unchanged between arms (parse-run.mjs 32.3% here vs 33.9% on main); what changed is that it now DELIVERS. On main its whole 8,548-char section was cut by the hard-ceiling truncation, so the incidental group scored 0.0% by luck, not by design, and the fixture passed on that. Bounding the oversize-member overshoot freed enough headroom that the response no longer truncates the same section away. Every file obeys the new bound on this repo (max ratio 1.40x of spendable, against the 1.5x ceiling). What the failure exposes is real and pre-existing: parse-run.mjs scores 18 against tools.ts's 58 yet is reserved a comparable slice — a low-scoring file taking a top-file share, which is epic CG-24's subject. Fix it there; do not tune the CG-30 bound to restore a pass that depended on truncation. SUPERSEDED by afterCG31 — the diagnosis above is wrong on one load-bearing point, see there."
},
"afterCG31": {
"measuredOn": "2026-08-06",
"note": "23,083 delivered of 23,080 allocated — inside the envelope, nothing truncated. Both arms measured on the SAME clean FULL REBUILD of this repo's index (CG-33: an incrementally-synced index diverges and shifts ranking). CG-30-only arm on that index: 23,692 delivered of 26,410 allocated, TRUNCATED.",
"delivered": {
"src/mcp/tools.ts": 0.359,
"scripts/agent-eval/parse-run.mjs": 0.187,
"src/mcp/explore-session-state.ts": 0.151,
"src/resolution/lru-cache.ts": 0.087
},
"verdict": "ALL FOUR GATES PASS. The afterCG30 verdict called parse-run.mjs over-RESERVED; it was not — its reservation is 4,314 in both arms. It was over-SPENDING: 8,548 chars, drawing on reservations belonging to files the render loop had not reached yet, which is the CG-31 defect. With the displacement guard it renders 4,314, tools.ts's identical 8,282 chars go from 35.0% to 35.9% of a response that no longer overruns, and lru-cache.ts (dropped as memory-budget.ts was on the CG-30 arm) delivers. Note what did NOT change: allocation. This fixture moved because the render loop stopped spending other files' bytes, not because anything was re-ranked."
},
"afterCG26": {
"measuredOn": "2026-08-06",
"note": "24,952 delivered of 24,949 allocated, nothing truncated — against the CG-31 tip's 23,083 on the SAME clean full rebuild of this repo's index. tools.ts delivers 8,282 chars in BOTH arms: identical bytes, unchanged reservation, unchanged rank. Total delivered SOURCE 21,228 against 18,105.",
"delivered": {
"src/mcp/tools.ts": 0.334,
"scripts/agent-eval/parse-run.mjs": 0.174,
"src/mcp/explore-session-state.ts": 0.141,
"src/resolution/memory-budget.ts": 0.126,
"src/resolution/lru-cache.ts": 0.081
},
"verdict": "ALL GATES PASS. The one that changed shape is answerShareAtLeast → answerShareOfSourceAtLeast: on the envelope denominator the answer group reads 47.5% here against 51.0% at the CG-31 tip, and neither number is about allocation. tools.ts's bytes are byte-identical between the arms; what moved is that the response now delivers a FIFTH admitted file (memory-budget.ts, rank 4, paid its full 3,123-char reservation — the CG-31 tip rendered it and then let the hard ceiling drop the whole section) and keeps epilogue prose it used to discard. Answer/incidental separation is unchanged and strong: tools.ts 33.4% against parse-run.mjs's 17.4%, incidental 17.4% (down from 18.7%), top delivered file still tools.ts. Measured in delivered source the answer group is 55.5%."
}
}
]
scripts/agent-eval/diff-index-drift.mjs
+102
View File
@@ -0,0 +1,102 @@
#!/usr/bin/env node
/**
* Diff two CodeGraph indexes of the SAME tree — typically a live,
* incrementally-synced `.codegraph/codegraph.db` against a clean full rebuild
* of the identical working tree (CG-33).
*
* Non-destructive: it only reads. Rebuilding is the caller's job, so the live
* index is never clobbered by the tool measuring it — the mistake that cost the
* original CG-33 artifact.
*
* # snapshot the live index BEFORE touching it
* cp .codegraph/codegraph.db /tmp/live.db
* node dist/bin/codegraph.js index .
* node scripts/agent-eval/diff-index-drift.mjs /tmp/live.db .codegraph/codegraph.db
*
* Edges are compared as distinct `(source, target, kind)` triples. Raw row
* counts are NOT a drift signal: a bidirectional divergence nets out. On the
* codegraph repo the raw counts differed by +0.7% while 4.3% of distinct edges
* were actually wrong.
*/
import { DatabaseSync } from 'node:sqlite';
import { existsSync } from 'node:fs';
const [livePath, rebuiltPath] = process.argv.slice(2);
if (!livePath || !rebuiltPath) {
console.error('usage: diff-index-drift.mjs <live.db> <rebuilt.db>');
process.exit(2);
}
for (const p of [livePath, rebuiltPath]) {
if (!existsSync(p)) {
// node:sqlite CREATES a missing file rather than failing, which silently
// yields an empty schema and a confident, wrong conclusion. Refuse first.
console.error(`not found: ${p}`);
process.exit(2);
}
}
const open = (p) => new DatabaseSync(p, { readOnly: true });
const live = open(livePath);
const rebuilt = open(rebuiltPath);
const scalar = (db, q) => db.prepare(q).get().n;
const edgeKey = (r) => `${r.source}\u0000${r.target}\u0000${r.kind}`;
const liveEdges = live.prepare('select source, target, kind from edges').all();
const rebuiltEdges = rebuilt.prepare('select source, target, kind from edges').all();
const liveSet = new Set(liveEdges.map(edgeKey));
const rebuiltSet = new Set(rebuiltEdges.map(edgeKey));
const missing = rebuiltEdges.filter((r) => !liveSet.has(edgeKey(r))); // should exist, doesn't
const stale = liveEdges.filter((r) => !rebuiltSet.has(edgeKey(r))); // exists, shouldn't
const divergent = missing.length + stale.length;
const byKind = (rows) => {
const m = new Map();
for (const r of rows) m.set(r.kind, (m.get(r.kind) ?? 0) + 1);
return [...m].sort((a, b) => b[1] - a[1]).map(([k, v]) => `${k}=${v}`).join(', ') || '(none)';
};
const pct = (n, d) => (d ? ((n / d) * 100).toFixed(1) : '0.0');
console.log(`live ${livePath}`);
console.log(`rebuilt ${rebuiltPath}`);
console.log('');
console.log('counts live rebuilt');
for (const [label, q] of [
['files', 'select count(*) n from files'],
['nodes', 'select count(*) n from nodes'],
['edges (rows)', 'select count(*) n from edges'],
// Grouped rather than `count(distinct a || b || c)`: bare concatenation has no
// separator, so `(ab, c)` and `(a, bc)` would collapse into one.
['edges (distinct)', 'select count(*) n from (select distinct source, target, kind from edges)'],
['heuristic edges', "select count(*) n from edges where provenance='heuristic'"],
]) {
console.log(` ${label.padEnd(20)} ${String(scalar(live, q)).padEnd(9)} ${scalar(rebuilt, q)}`);
}
console.log('');
console.log('edge divergence (distinct triples)');
console.log(` missing from live: ${missing.length} — ${byKind(missing)}`);
console.log(` stale in live: ${stale.length} — ${byKind(stale)}`);
console.log(` TOTAL divergent: ${divergent} (${pct(divergent, rebuiltSet.size)}% of ${rebuiltSet.size})`);
// Integrity checks — these separate "resolution went stale" (edges wrong, nodes
// identical) from "residue accumulated" (duplicate/orphan rows). CG-33 is the
// former: on the codegraph repo every check below was 0 on BOTH indexes.
console.log('');
console.log('integrity live rebuilt');
for (const [label, q] of [
['duplicate nodes', 'select count(*) n from (select file_path,name,kind,start_line from nodes group by 1,2,3,4 having count(*)>1)'],
['orphan edges', 'select count(*) n from edges e where not exists(select 1 from nodes where id=e.source) or not exists(select 1 from nodes where id=e.target)'],
['nodes w/ missing file row', 'select count(*) n from nodes nd where not exists(select 1 from files f where f.path=nd.file_path)'],
]) {
console.log(` ${label.padEnd(30)} ${String(scalar(live, q)).padEnd(6)} ${scalar(rebuilt, q)}`);
}
console.log('');
console.log(divergent === 0
? 'CONVERGED — the synced index matches a full rebuild.'
: `DRIFTED — ${divergent} edges differ. Rebuild-vs-rebuild is 0 (the indexer is deterministic), so this is sync divergence, not noise.`);
process.exitCode = divergent === 0 ? 0 : 1;
scripts/agent-eval/probe-allocation.mjs
+39
View File
@@ -150,6 +150,28 @@ function evaluate(fixture, report, text) {
`answer ${pct(share('answer'))} delivered (${pct(allocated.get('answer') ?? 0)} allocated)`,
);
}
// Same question against the SOURCE the response delivered rather than the
// whole envelope (CG-26). The envelope-denominated gate above moves whenever
// the response's prose does — the epilogue surviving instead of being
// discarded costs it a point, and every additional admitted file that gets
// paid dilutes it further — so it cannot tell "the answer was starved" from
// "everything else was also delivered". Allocation is about source bytes;
// measure it in source bytes.
if (want.answerShareOfSourceAtLeast !== undefined) {
const sourceBy = new Map();
let totalSource = 0;
for (const f of report.files) {
const g = groupOf(f.path, groups);
sourceBy.set(g, (sourceBy.get(g) ?? 0) + f.finalChars);
totalSource += f.finalChars;
}
const answerSource = totalSource > 0 ? (sourceBy.get('answer') ?? 0) / totalSource : 0;
add(
`answer group takes >= ${pct(want.answerShareOfSourceAtLeast)} of DELIVERED SOURCE`,
answerSource >= want.answerShareOfSourceAtLeast,
`answer ${num(sourceBy.get('answer') ?? 0)} of ${num(totalSource)} source chars (${pct(answerSource)})`,
);
}
if (want.incidentalShareAtMost !== undefined) {
add(
`incidental group takes <= ${pct(want.incidentalShareAtMost)} of the envelope`,
@@ -177,6 +199,23 @@ function evaluate(fixture, report, text) {
: 'not among the ranked candidates',
);
}
// Reservation-vs-delivered, per file (CG-36). The share gates above ask which
// files won the envelope; this asks whether a file that WON its share then
// actually spent it. A file can rank #1, be reserved the largest slice, and
// still deliver a quarter of it because the cluster carrying the answer was
// dropped whole instead of shrunk — and the share gates read that as a pass,
// since the unspent bytes carry forward and the envelope stays full.
for (const [path, floor] of Object.entries(want.spendShareAtLeast ?? {})) {
const rec = report.files.find((f) => f.path === path);
const spent = rec && rec.allowance ? rec.finalChars / rec.allowance : 0;
add(
`${path} spends >= ${pct(floor)} of its reservation`,
!!rec && rec.allowance > 0 && spent >= floor,
rec
? `${num(rec.finalChars)} delivered of a ${num(rec.allowance ?? 0)} reservation (${pct(spent)})`
: 'not among the ranked candidates',
);
}
for (const needle of want.mustContain ?? []) {
add(`response contains "${needle}"`, text.includes(needle), text.includes(needle) ? 'present' : 'absent');
}
scripts/agent-eval/probe-decl-only.mjs
+198
View File
@@ -0,0 +1,198 @@
#!/usr/bin/env node
/**
* CG-28 measurement probe — what a DECLARATION-ONLY file takes from an explore
* envelope, with and without a generated banner.
*
* The issue was filed because a Wrangler `worker-configuration.d.ts` scored 49
* at `pen 1.00` and took 60.7% of an envelope on generic identifier overlap
* (`ReadableStream`, `Body`, `ImageMetadata`, `Message`, …) with a prose query.
* CG-25 has since taught `GENERATED_CONTENT_PATTERNS` the Wrangler banner, so
* the first thing to measure is whether that alone settles it — it does, and
* this probe quantifies it. What CG-25 does NOT cover is a declaration-only file
* that carries no banner at all: a hand-maintained ambient `.d.ts`, vendored
* typings, module augmentation. This probe puts both shapes in ONE fixture
* against ONE envelope so the banner is the only difference between them.
* (`.pyi` is not an indexed extension, so Python stubs never enter the graph.)
*
* Findings and the full regression evidence:
* `docs/benchmarks/explore-declaration-only-cg28.md`.
*
* Fixture: `__tests__/fixtures/ambient-decls-ts/` — an upload path (route →
* stream → metadata → queue) competing with:
* types/worker-configuration.d.ts declaration-only, Wrangler banner (CG-25)
* types/platform-shims.d.ts declaration-only, hand-written, NO banner
* src/storage/types.ts declaration-only but IMPORTED — the control
* that must never be damped
*
* Variants (`--variant`):
* both as committed — the controlled comparison
* strip-banner the banner is deleted from worker-configuration.d.ts, so the
* two declaration files differ in NOTHING the ranker can see;
* the delta against `both` is exactly what CG-25 buys
*
* Usage (needs a current `npm run build`):
* node scripts/agent-eval/probe-decl-only.mjs
* node scripts/agent-eval/probe-decl-only.mjs --variant strip-banner
* node scripts/agent-eval/probe-decl-only.mjs --json
* node scripts/agent-eval/probe-decl-only.mjs --query "..."
*/
import { cpSync, mkdtempSync, readFileSync, writeFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolve(HERE, '../..');
const FIXTURE = join(REPO_ROOT, '__tests__/fixtures/ambient-decls-ts');
const GENERATED_DECL = 'types/worker-configuration.d.ts';
const HANDWRITTEN_DECL = 'types/platform-shims.d.ts';
/**
* The query shapes. The flow ones are prose and name no symbol — the shape that
* let the original file in. The last one is the counter-case the issue requires:
* a question genuinely ABOUT a declared type must still reach the declaration.
*/
const QUERIES = [
{ id: 'flow-upload', kind: 'flow', text: 'how does an upload request stream the file body to storage and record image metadata' },
{ id: 'flow-pipe', kind: 'flow', text: 'where does the upload body get piped into the bucket and the metadata written' },
{ id: 'flow-generic', kind: 'flow', text: 'how are streams and messages and image metadata handled for uploads' },
{ id: 'flow-queue', kind: 'flow', text: 'what happens after an object is stored and the follow-up message is queued' },
{ id: 'type-shim', kind: 'type', text: 'UploadStorage StoredUploadObject ImageMetadataShim' },
{ id: 'type-prose', kind: 'type', text: 'what does the UploadStorage interface declare for putting an object' },
];
const argv = process.argv.slice(2);
const asJson = argv.includes('--json');
const at = (flag) => { const i = argv.indexOf(flag); return i >= 0 ? argv[i + 1] : undefined; };
const VARIANT = at('--variant') ?? 'both';
const ONE_QUERY = at('--query');
const ONE_ID = at('--only');
const say = (s = '') => { if (!asJson) console.log(s); };
const num = (n) => Math.round(n).toLocaleString('en-US');
const pct = (f) => `${(f * 100).toFixed(1)}%`;
if (!existsSync(join(REPO_ROOT, 'dist/index.js'))) {
console.error('dist/ not built — run `npm run build` first.');
process.exit(2);
}
const load = (rel) => import(pathToFileURL(resolve(REPO_ROOT, rel)).href);
const idxMod = await load('dist/index.js');
const toolsMod = await load('dist/mcp/tools.js');
const CodeGraph = idxMod.default?.default ?? idxMod.default ?? idxMod.CodeGraph;
const ToolHandler = toolsMod.ToolHandler ?? toolsMod.default?.ToolHandler;
/** Copy the fixture, apply the variant, index it. Hermetic per run. */
function materialize(variant) {
const dir = mkdtempSync(join(tmpdir(), 'cg-decl-'));
cpSync(FIXTURE, dir, { recursive: true });
rmSync(join(dir, '.codegraph'), { recursive: true, force: true });
if (variant === 'strip-banner') {
const p = join(dir, GENERATED_DECL);
// Drop only the banner comment lines; every declaration stays.
const kept = readFileSync(p, 'utf8').split('\n').filter((l) => !/^\/\/ .*(Generated by Wrangler|Runtime types generated)/.test(l));
writeFileSync(p, kept.join('\n'));
} else if (variant !== 'both') {
rmSync(dir, { recursive: true, force: true });
throw new Error(`unknown --variant ${variant} (both | strip-banner)`);
}
return dir;
}
const queries = ONE_QUERY
? [{ id: 'custom', kind: 'flow', text: ONE_QUERY }]
: QUERIES.filter((q) => !ONE_ID || q.id === ONE_ID);
const dir = materialize(VARIANT);
let rows;
try {
let cg = CodeGraph.initSync(dir);
await cg.indexAll();
cg.close?.();
const sidecar = join(dir, 'diag.jsonl');
rows = [];
for (const q of queries) {
rmSync(sidecar, { force: true });
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
cg = CodeGraph.openSync(dir);
const res = await new ToolHandler(cg).execute('codegraph_explore', { query: q.text });
const text = res.content?.[0]?.text ?? '';
cg.close?.();
delete process.env.CODEGRAPH_EXPLORE_DEBUG;
const report = JSON.parse(readFileSync(sidecar, 'utf8').trim().split('\n').pop());
const pick = (path) => {
const f = report.files.find((x) => x.path === path);
if (!f) return null;
return {
path, rank: f.rank, score: f.score, graph: f.graphScore, hits: f.termHits,
penalty: f.penalty, generated: f.generated, render: f.render,
named: f.named, entry: f.entry, central: f.central,
allocatedShare: f.allocatedShare, share: f.share,
emitted: f.emittedChars, final: f.finalChars, skipped: f.skipped,
};
};
const declPaths = new Set([GENERATED_DECL, HANDWRITTEN_DECL]);
const totalSource = report.files.reduce((a, f) => a + f.finalChars, 0);
const declSource = report.files
.filter((f) => declPaths.has(f.path))
.reduce((a, f) => a + f.finalChars, 0);
// "Named in the response but carrying no source" is the correct outcome for
// a cliffed declaration file — the agent can still fetch it in one call.
const namedInResponse = (p) => text.includes(p);
rows.push({
query: q.id, kind: q.kind, text: q.text,
envelope: report.envelope,
generatedDecl: pick(GENERATED_DECL),
handwrittenDecl: pick(HANDWRITTEN_DECL),
declSourceShare: totalSource > 0 ? declSource / totalSource : 0,
implSourceShare: totalSource > 0 ? (totalSource - declSource) / totalSource : 0,
topFile: report.files.filter((f) => f.finalChars > 0).sort((a, b) => b.finalChars - a.finalChars)[0]?.path ?? null,
generatedNamed: namedInResponse(GENERATED_DECL),
handwrittenNamed: namedInResponse(HANDWRITTEN_DECL),
files: report.files
.filter((f) => f.emittedChars > 0 || f.finalChars > 0)
.map((f) => ({ rank: f.rank, path: f.path, score: f.score, graph: f.graphScore, hits: f.termHits, penalty: f.penalty, generated: f.generated, declOnly: f.ambientDeclaration, named: f.named, entry: f.entry, central: f.central, render: f.render, final: f.finalChars, share: f.share })),
});
}
} finally {
rmSync(dir, { recursive: true, force: true });
}
if (asJson) {
console.log(JSON.stringify({ variant: VARIANT, rows }, null, 2));
} else {
say(`variant ${VARIANT}`);
say('');
for (const r of rows) {
say(`── ${r.query} [${r.kind}] "${r.text}"`);
say(` envelope ${num(r.envelope.chars)} chars · decl-only files hold ${pct(r.declSourceShare)} of delivered source`);
say(' # deliv% bytes score graph hits pen gen flags render file');
for (const f of r.files) {
const flags = [f.named && "named", f.entry && "entry", f.central && "central", f.declOnly && "decl-only"].filter(Boolean).join(" ") || "-";
say(
' ' + String(f.rank).padStart(2) + ' ' +
pct(f.share).padStart(6) + ' ' +
num(f.final).padStart(7) + ' ' +
Number(f.score).toFixed(1).padStart(5) + ' ' +
f.graph.toFixed(5).padStart(7) + ' ' +
String(f.hits).padStart(4) + ' ' +
f.penalty.toFixed(2).padStart(4) + ' ' +
(f.generated ? ' ✓ ' : ' ') + ' ' +
flags.padEnd(18) + ' ' +
(f.render ?? '-').padEnd(9) + ' ' +
f.path,
);
}
for (const [label, d, named] of [
['generated ', r.generatedDecl, r.generatedNamed],
['handwritten', r.handwrittenDecl, r.handwrittenNamed],
]) {
say(` ${label} ${d ? `rank #${d.rank}, score ${Number(d.score).toFixed(1)}, pen ${d.penalty.toFixed(2)}, ${num(d.final)} chars (${pct(d.share)})${d.final === 0 ? ` — ${d.skipped ?? d.render ?? 'not rendered'}` : ''}` : 'not a candidate'}${named ? ' · named in response' : ''}`);
}
say('');
}
}
scripts/agent-eval/probe-factory-closure.mjs
+147
View File
@@ -0,0 +1,147 @@
#!/usr/bin/env node
/**
* CG-27 measurement probe — what a factory-closure file actually delivers.
*
* `probe-allocation.mjs` measures how the envelope is split BETWEEN files. This
* one measures what comes back from WITHIN one file whose top-level symbol spans
* almost all of it: a `createFoo()` factory returning an object of closures
* (Svelte 5 rune stores, React hook modules, Zustand `create((set,get)=>({…}))`,
* IIFE module-pattern JS). The claim under test is a ranking one, not a byte one
* — CG-30 already bounds the bytes — so the number that matters is WHICH inner
* symbols reach the agent, not how many chars did.
*
* Prints, for the factory file: every line range the response delivered, and for
* each inner function whether its DEFINITION LINE is inside one of them.
*
* Usage (needs a current `npm run build`):
* node scripts/agent-eval/probe-factory-closure.mjs
* node scripts/agent-eval/probe-factory-closure.mjs --json
* node scripts/agent-eval/probe-factory-closure.mjs --query "..."
*/
import { cpSync, mkdtempSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join, resolve } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO_ROOT = resolve(HERE, '../..');
const FIXTURE = join(REPO_ROOT, '__tests__/fixtures/factory-closure-ts');
const targetAt = process.argv.indexOf('--target');
const TARGET = targetAt >= 0 ? process.argv[targetAt + 1] : 'src/stores/dashboard-store.ts';
const factoryAt = process.argv.indexOf('--factory');
const FACTORY = factoryAt >= 0 ? process.argv[factoryAt + 1] : 'createDashboardStore';
const argv = process.argv.slice(2);
const asJson = argv.includes('--json');
const queryAt = argv.indexOf('--query');
const QUERY = queryAt >= 0
? argv[queryAt + 1]
: 'how does the dashboard store refresh its metrics and apply a filter';
const say = (s = '') => { if (!asJson) console.log(s); };
const num = (n) => Math.round(n).toLocaleString('en-US');
const load = (rel) => import(pathToFileURL(resolve(REPO_ROOT, rel)).href);
if (!existsSync(join(REPO_ROOT, 'dist/index.js'))) {
console.error('dist/ not built — run `npm run build` first.');
process.exit(2);
}
const idxMod = await load('dist/index.js');
const toolsMod = await load('dist/mcp/tools.js');
const CodeGraph = idxMod.default?.default ?? idxMod.default ?? idxMod.CodeGraph;
const ToolHandler = toolsMod.ToolHandler ?? toolsMod.default?.ToolHandler;
const dir = mkdtempSync(join(tmpdir(), 'cg-factory-'));
cpSync(FIXTURE, dir, { recursive: true });
rmSync(join(dir, '.codegraph'), { recursive: true, force: true });
let out;
try {
let cg = CodeGraph.initSync(dir);
await cg.indexAll();
// Inner function definitions, straight from the index — the symbols the file's
// enclosing factory range would otherwise swallow.
const nodes = cg.getNodesInFile(TARGET);
const factory = nodes.find((n) => n.name === FACTORY);
const inner = nodes
.filter((n) => (n.kind === 'function' || n.kind === 'method')
&& n.name !== FACTORY
&& factory && n.startLine > factory.startLine && n.endLine <= factory.endLine)
.sort((a, b) => a.startLine - b.startLine);
cg.close?.();
const sidecar = join(dir, 'diag.jsonl');
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
cg = CodeGraph.openSync(dir);
const res = await new ToolHandler(cg).execute('codegraph_explore', { query: QUERY });
const text = res.content?.[0]?.text ?? '';
cg.close?.();
delete process.env.CODEGRAPH_EXPLORE_DEBUG;
const report = JSON.parse(readFileSync(sidecar, 'utf8').trim().split('\n').pop());
// Which source lines of the target file the response actually carries. The
// response numbers every delivered line `<n>\t<text>`; match them back against
// the file so a line number that merely appears in prose can't count.
const source = readFileSync(join(dir, TARGET), 'utf8').split('\n');
const delivered = new Set();
for (const line of text.split('\n')) {
const m = /^(\d+)\t(.*)$/.exec(line);
if (!m) continue;
const n = Number(m[1]);
if (n >= 1 && n <= source.length && source[n - 1] === m[2]) delivered.add(n);
}
// Collapse to ranges for display.
const ranges = [];
for (const n of [...delivered].sort((a, b) => a - b)) {
const last = ranges[ranges.length - 1];
if (last && n === last.end + 1) last.end = n;
else ranges.push({ start: n, end: n });
}
const covered = (n) => delivered.has(n.startLine);
const rec = report.files.find((f) => f.path === TARGET) ?? null;
out = {
query: QUERY,
target: TARGET,
fileLines: source.length,
factory: factory ? { name: factory.name, start: factory.startLine, end: factory.endLine } : null,
file: rec && {
rank: rec.rank, render: rec.render, clipped: rec.clipped,
emittedChars: rec.emittedChars, finalChars: rec.finalChars,
allowance: rec.allowance, spendable: rec.spendable, skipped: rec.skipped,
},
deliveredRanges: ranges,
deliveredLines: delivered.size,
inner: inner.map((n) => ({ name: n.name, start: n.startLine, end: n.endLine, delivered: covered(n) })),
innerDelivered: inner.filter(covered).length,
innerTotal: inner.length,
envelope: report.envelope,
allFiles: report.files
.filter((f) => f.emittedChars > 0 || f.finalChars > 0)
.map((f) => ({ rank: f.rank, path: f.path, render: f.render, emitted: f.emittedChars, final: f.finalChars })),
};
} finally {
rmSync(dir, { recursive: true, force: true });
}
if (asJson) {
console.log(JSON.stringify(out, null, 2));
} else {
say(`query "${out.query}"`);
say(`target ${out.target} — ${out.fileLines} lines, factory ${out.factory?.name} spans ${out.factory?.start}–${out.factory?.end}`);
say('');
say(' # render emitted final file');
for (const f of out.allFiles) {
say(` ${String(f.rank).padStart(2)} ${(f.render ?? '-').padEnd(10)} ${num(f.emitted).padStart(7)} ${num(f.final).padStart(7)} ${f.path}`);
}
say('');
say(`delivered lines of ${out.target}: ${out.deliveredLines}`);
say(` ranges: ${out.deliveredRanges.map((r) => `${r.start}-${r.end}`).join(', ') || '(none)'}`);
say('');
say(`inner symbols whose definition reached the agent: ${out.innerDelivered}/${out.innerTotal}`);
for (const n of out.inner) {
say(` ${n.delivered ? '✓' : '·'} ${n.name} (${n.start}–${n.end})`);
}
}
scripts/agent-eval/probe-file-spend.mjs
+195
View File
@@ -0,0 +1,195 @@
#!/usr/bin/env node
/**
* Per-file reservation-vs-delivered sweep for `codegraph_explore` (CG-36).
*
* `probe-suite-envelope.mjs` answers "how much source did the response deliver";
* this answers the question one level down — "did the bytes go to the files that
* earned them". The CG-36 defect was invisible to the envelope probe because the
* envelope stayed full: a rank-#3 file spent 24% of its reservation, the slack
* carried forward exactly as designed, and a far weaker file spent 3.5x its own.
* The response looked healthy; the ANSWER-bearing file had been starved.
*
* So the flag here is a PAIR, not a per-file threshold: a file that leaves a
* large share of its reservation unspent WHILE a materially lower-scoring file
* spends well over its own. Either alone is legitimate — a small file simply has
* less to say, and carry-forward is the mechanism that hands its slack down.
*
* Numbers come from the CG-4 diagnostic (`CODEGRAPH_EXPLORE_DEBUG`), so this
* measures the shipping allocator rather than re-deriving shares from markdown.
*
* Usage (needs a current `npm run build`, and full-REBUILT indexes — CG-33):
* node scripts/agent-eval/probe-file-spend.mjs
* node scripts/agent-eval/probe-file-spend.mjs --json > /tmp/new.json
* node scripts/agent-eval/probe-file-spend.mjs --baseline /tmp/base.json
* node scripts/agent-eval/probe-file-spend.mjs django --all # every file, not just flags
* CORPUS=/tmp/codegraph-corpus node scripts/agent-eval/probe-file-spend.mjs
*
* Exit code is 1 when any repo carries a starvation flag, so this can gate.
*/
import { mkdtempSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const CORPUS = process.env.CORPUS ?? '/tmp/codegraph-corpus';
/** Same six repos and queries the CG-30/CG-31/CG-26 envelope tables use. */
const SUITE = [
{ id: 'django', q: 'How does a QuerySet turn into SQL and fetch rows from the database?' },
{ id: 'excalidraw', q: 'How does updating an element re-render the canvas on screen?' },
{ id: 'okhttp', q: 'How does a call go through the interceptor chain to the network?' },
{ id: 'tokio', q: 'How does a spawned task get scheduled and run by a worker?' },
{ id: 'gin', q: 'How does a registered route handler get invoked for an incoming HTTP request?' },
{ id: 'alamofire', q: 'How does a request get built and sent through the session?' },
];
/**
* Starvation thresholds. A flag needs BOTH sides — the starved file and the
* overspending one it lost the bytes to.
*
* `MIN_RESERVED` keeps the noise out: under it, "80% unspent" is a few hundred
* chars and means nothing. `SCORE_RATIO` is what makes the pair meaningful —
* a higher-scoring file underspending while a *comparable* one overspends is
* ordinary; the defect is a materially weaker file taking the bytes.
*/
const STARVED_SHARE = 0.5; // spent < half its reservation
const OVERSPEND_RATIO = 1.5; // spent > 1.5x its own reservation
const SCORE_RATIO = 2; // ...while scoring less than half the starved file
const MIN_RESERVED = 2000; // ignore files whose reservation is too small to matter
const argv = process.argv.slice(2);
const asJson = argv.includes('--json');
const showAll = argv.includes('--all');
const baselineAt = argv.includes('--baseline') ? argv[argv.indexOf('--baseline') + 1] : null;
const only = argv.filter((a) => !a.startsWith('--') && a !== baselineAt);
const say = (s = '') => { if (!asJson) console.log(s); };
const num = (n) => Math.round(n).toLocaleString('en-US');
const pct = (f) => `${(f * 100).toFixed(1)}%`;
const load = (rel) => import(pathToFileURL(resolve(rel)).href);
const idx = await load('dist/index.js');
const toolsMod = await load('dist/mcp/tools.js');
const CodeGraph = idx.default?.default ?? idx.default ?? idx.CodeGraph;
const ToolHandler = toolsMod.ToolHandler ?? toolsMod.default?.ToolHandler;
if (typeof CodeGraph?.openSync !== 'function' || typeof ToolHandler !== 'function') {
console.error('could not resolve CodeGraph/ToolHandler from dist/ — run `npm run build`');
process.exit(2);
}
/**
* Pair up the starved with the overspenders they lost bytes to. Only files the
* render loop actually reached (a reservation and a render mode) take part —
* a cliffed or max-files file never had bytes to spend.
*/
function findStarvation(files) {
const spenders = files.filter(
(f) => f.allowance !== null && f.allowance > 0 && f.render && f.render !== 'backref',
);
const flags = [];
for (const s of spenders) {
if (s.allowance < MIN_RESERVED) continue;
if (s.finalChars >= s.allowance * STARVED_SHARE) continue;
for (const o of spenders) {
if (o.path === s.path) continue;
if (o.finalChars <= o.allowance * OVERSPEND_RATIO) continue;
if (o.score * SCORE_RATIO > s.score) continue;
flags.push({
starved: s.path,
starvedScore: s.score,
starvedReserved: s.allowance,
starvedSpent: s.finalChars,
overspent: o.path,
overspentScore: o.score,
overspentReserved: o.allowance,
overspentSpent: o.finalChars,
});
}
}
return flags;
}
const tmp = mkdtempSync(join(tmpdir(), 'cg-spend-'));
const results = [];
try {
for (const { id, q } of SUITE) {
if (only.length > 0 && !only.includes(id)) continue;
const repo = join(CORPUS, id);
if (!existsSync(join(repo, '.codegraph', 'codegraph.db'))) {
say(`${id}: no index at ${repo} — skipped`);
continue;
}
const sidecar = join(tmp, `${id}.jsonl`);
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
const cg = CodeGraph.openSync(repo);
const h = new ToolHandler(cg);
await h.execute('codegraph_explore', { query: q });
try { cg.close?.(); } catch { /* best effort */ }
const report = JSON.parse(readFileSync(sidecar, 'utf8').trim().split('\n').pop());
const files = report.files.map((f) => ({
path: f.path,
rank: f.rank,
score: f.score,
allowance: f.allowance,
spendable: f.spendable,
finalChars: f.finalChars,
render: f.render,
skipped: f.skipped,
spent: f.allowance ? f.finalChars / f.allowance : null,
}));
results.push({
repo: id,
sourceChars: report.envelope.sourceChars,
files,
flags: findStarvation(files),
});
}
} finally {
rmSync(tmp, { recursive: true, force: true });
}
if (asJson) {
console.log(JSON.stringify(results, null, 2));
} else {
const base = baselineAt ? JSON.parse(readFileSync(baselineAt, 'utf8')) : null;
const byRepo = new Map((base ?? []).map((r) => [r.repo, r]));
for (const r of results) {
const b = byRepo.get(r.repo);
say(`\n${r.repo} — ${num(r.sourceChars)} source chars`
+ (b ? ` (baseline ${num(b.sourceChars)})` : ''));
say(' # score reserved spent spent% render file');
say('-'.repeat(96));
const flagged = new Set(r.flags.flatMap((f) => [f.starved, f.overspent]));
for (const f of r.files) {
if (f.allowance === null || f.allowance === 0) continue;
if (!showAll && !flagged.has(f.path) && f.spent > STARVED_SHARE && f.spent < OVERSPEND_RATIO) continue;
const mark = flagged.has(f.path) ? '*' : ' ';
say(
`${String(f.rank).padStart(2)}${mark} ${String(Math.round(f.score)).padStart(6)} `
+ `${num(f.allowance).padStart(9)} ${num(f.finalChars).padStart(7)} `
+ `${pct(f.spent).padStart(7)} ${(f.render ?? f.skipped ?? '—').padEnd(13)} ${f.path}`,
);
}
for (const f of r.flags) {
say(` FLAG: ${f.starved} (score ${Math.round(f.starvedScore)}) spent `
+ `${num(f.starvedSpent)}/${num(f.starvedReserved)} while ${f.overspent} `
+ `(score ${Math.round(f.overspentScore)}) spent ${num(f.overspentSpent)}/${num(f.overspentReserved)}`);
}
}
const total = results.reduce((n, r) => n + r.flags.length, 0);
say('');
say(total === 0
? 'No file leaves a large share of its reservation unspent while a weaker file overspends.'
: `STARVATION: ${total} flag(s) across `
+ `${results.filter((r) => r.flags.length > 0).map((r) => r.repo).join(', ')}.`);
if (base) {
const worse = results.filter((r) => {
const b = byRepo.get(r.repo);
return b && (r.flags.length > b.flags.length || r.sourceChars < b.sourceChars);
});
say(worse.length === 0
? 'No repo flags more or delivers less than the baseline.'
: `REGRESSION vs baseline: ${worse.map((r) => r.repo).join(', ')}.`);
}
if (total > 0) process.exitCode = 1;
}
scripts/agent-eval/probe-named-symbol.mjs
Executable
+163
View File
@@ -0,0 +1,163 @@
#!/usr/bin/env node
/**
* "Did the symbol the agent NAMED actually render?" (CG-38).
*
* This is the measurement the whole CG-24 epic was missing. Every other probe
* here scores the response in AGGREGATE — `probe-suite-envelope.mjs` measures how
* much source came back, `probe-file-spend.mjs` measures whether the bytes went
* to the files that earned them, `probe-allocation.mjs` measures group shares.
* All three are green on a response that returns 25K of source from the right
* file and still omits the one function the agent asked for by name. That is
* exactly what CG-38 was: `queueMessage` at L1087 of a 1,414-line file, whose
* file won rank #1 with 67% of the envelope, never rendered — the agent got a
* same-stem `QueuedMessage` INTERFACE at L70 instead and had to Read the file.
*
* So the assertion here is per-SYMBOL and binary: for each named symbol, does its
* definition line appear in the rendered source? Nothing else can substitute —
* not the file being present, not its share, not its byte count.
*
* Usage (needs a current `npm run build`):
* node scripts/agent-eval/probe-named-symbol.mjs
* node scripts/agent-eval/probe-named-symbol.mjs --verbose
* # any indexed repo, ad hoc:
* node scripts/agent-eval/probe-named-symbol.mjs <repo> "<query>" sym1 sym2
*
* Exit code is 1 when any expected symbol is missing, so this can gate.
*/
import { cpSync, mkdtempSync, rmSync, readFileSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve, dirname } from 'node:path';
import { fileURLToPath, pathToFileURL } from 'node:url';
const HERE = dirname(fileURLToPath(import.meta.url));
const REPO = resolve(HERE, '..', '..');
const load = async (rel) => import(pathToFileURL(resolve(REPO, rel)).href);
const idxMod = await load('dist/index.js');
const toolsMod = await load('dist/mcp/tools.js');
const CodeGraph = idxMod.default?.default ?? idxMod.default ?? idxMod.CodeGraph;
const { ToolHandler } = toolsMod;
/**
* The fixture cases. `symbols` are what the agent names; each must come back with
* its DEFINITION rendered. The queries deliberately cover both shapes the bug was
* reported on — a bare symbol bag and a prose question — because the failure had
* a different cause on each and a fix for one does not imply the other.
*/
const FIXTURE = '__tests__/fixtures/tail-render-ts';
const CASES = [
{
id: 'tail-symbol-bag',
why: 'two sibling closures past L1000, named directly; neither calls the other',
query: 'queueMessage flushQueuedMessages',
symbols: ['queueMessage', 'flushQueuedMessages'],
},
{
id: 'tail-prose',
why: 'same two symbols named inside a prose question',
query: 'how does queueMessage hand its entries to flushQueuedMessages',
symbols: ['queueMessage', 'flushQueuedMessages'],
},
{
id: 'tail-with-decoy',
why: 'the same-stem QueuedMessage interface at L70 must not stand in for the functions',
query: 'explain queueMessage, removeQueuedMessage and flushQueuedMessages',
symbols: ['queueMessage', 'removeQueuedMessage', 'flushQueuedMessages'],
},
];
/** Every `<n>\t<text>` line number present in the response's source blocks. */
function renderedLines(response) {
const out = new Set();
for (const m of response.matchAll(/^(\d+)\t/gm)) out.add(Number(m[1]));
return out;
}
/**
* A symbol counts as rendered only when its DECLARATION line is among the lines
* the response actually sent — not when its name merely appears somewhere (it
* shows up in the section header symbol list and in call sites regardless, which
* is precisely how this defect hid for a whole epic).
*/
function check(cg, response, names) {
const lines = renderedLines(response);
return names.map((name) => {
const node = (cg.getNodesByName?.(name) ?? []).find((n) => n.startLine > 0);
return {
name,
file: node?.filePath ?? '(not indexed)',
line: node?.startLine ?? 0,
rendered: !!node && lines.has(node.startLine),
};
});
}
async function runCase(root, { query, symbols }) {
const cg = CodeGraph.openSync(root);
try {
const res = await new ToolHandler(cg).execute('codegraph_explore', { query });
const response = res.content?.[0]?.text ?? '';
return { response, results: check(cg, response, symbols) };
} finally {
try { cg.close?.(); } catch { /* already closed */ }
}
}
const argv = process.argv.slice(2);
const verbose = argv.includes('--verbose');
const positional = argv.filter((a) => !a.startsWith('--'));
let failures = 0;
let checked = 0;
if (positional.length >= 3) {
// Ad-hoc mode: <repo> "<query>" sym...
const [repo, query, ...symbols] = positional;
const { response, results } = await runCase(resolve(repo), { query, symbols });
console.log(`\n${repo}\n query "${query}" · ${response.length} chars\n`);
for (const r of results) {
checked += 1;
if (!r.rendered) failures += 1;
console.log(` ${r.rendered ? 'PASS' : 'FAIL'} ${r.name} ${r.file}:${r.line}`);
}
} else {
const src = join(REPO, FIXTURE);
if (!existsSync(src)) {
console.error(`fixture missing: ${src}`);
process.exit(2);
}
const dir = mkdtempSync(join(tmpdir(), 'cg-named-'));
try {
cpSync(src, dir, { recursive: true });
rmSync(join(dir, '.codegraph'), { recursive: true, force: true });
const cg = CodeGraph.initSync(dir);
await cg.indexAll();
cg.close?.();
console.log(`\ntail-render-ts · agent-named symbols must render\n`);
for (const c of CASES) {
const { response, results } = await runCase(dir, c);
console.log(`── ${c.id} — ${c.why}`);
console.log(` query "${c.query}"`);
console.log(` response ${response.length.toLocaleString()} chars`);
for (const r of results) {
checked += 1;
if (!r.rendered) failures += 1;
console.log(` ${r.rendered ? 'PASS' : 'FAIL'} ${r.name} defined at ${r.file}:${r.line}`
+ (r.rendered ? '' : ' — DEFINITION NOT IN RESPONSE'));
}
if (verbose && failures) {
const spans = [...renderedLines(response)].sort((a, b) => a - b);
console.log(` rendered lines: ${spans[0]}..${spans[spans.length - 1]} (${spans.length} lines)`);
}
console.log();
}
} finally {
rmSync(dir, { recursive: true, force: true });
}
}
console.log(failures === 0
? `Every agent-named symbol rendered (${checked} checked).`
: `${failures} of ${checked} agent-named symbols did NOT render.`);
process.exit(failures === 0 ? 0 : 1);
scripts/agent-eval/probe-suite-envelope.mjs
+124
View File
@@ -0,0 +1,124 @@
#!/usr/bin/env node
/**
* Deterministic 6-repo envelope sweep for `codegraph_explore` (CG-26).
*
* The allocation issues (CG-30 / CG-31 / CG-26) are all decided by how the
* render loop divides a fixed byte ceiling, and the agent A/B is far too noisy
* to see a 2K byte shift. This runs the SAME six queries the CG-30 and CG-31
* benchmark tables use, against the same clean-rebuilt corpus indexes, and
* prints the numbers those tables are made of: source chars delivered, files in
* the final output, whether the hard ceiling cut anything, and whether the
* epilogue survived.
*
* Numbers come from the CG-4 diagnostic (`CODEGRAPH_EXPLORE_DEBUG`), so this
* measures the shipping allocator rather than re-deriving shares from markdown.
*
* Usage (needs a current `npm run build`, and full-REBUILT indexes — CG-33):
* node scripts/agent-eval/probe-suite-envelope.mjs
* node scripts/agent-eval/probe-suite-envelope.mjs --json > /tmp/new.json
* node scripts/agent-eval/probe-suite-envelope.mjs --baseline /tmp/base.json
* CORPUS=/tmp/codegraph-corpus node scripts/agent-eval/probe-suite-envelope.mjs
*/
import { mkdtempSync, readFileSync, rmSync, existsSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join, resolve } from 'node:path';
import { pathToFileURL } from 'node:url';
const CORPUS = process.env.CORPUS ?? '/tmp/codegraph-corpus';
/** The six suite repos + the exact queries the CG-30/CG-31 tables were measured on. */
const SUITE = [
{ id: 'django', q: 'How does a QuerySet turn into SQL and fetch rows from the database?' },
{ id: 'excalidraw', q: 'How does updating an element re-render the canvas on screen?' },
{ id: 'okhttp', q: 'How does a call go through the interceptor chain to the network?' },
{ id: 'tokio', q: 'How does a spawned task get scheduled and run by a worker?' },
{ id: 'gin', q: 'How does a registered route handler get invoked for an incoming HTTP request?' },
{ id: 'alamofire', q: 'How does a request get built and sent through the session?' },
];
const argv = process.argv.slice(2);
const asJson = argv.includes('--json');
const baselineAt = argv.includes('--baseline') ? argv[argv.indexOf('--baseline') + 1] : null;
const only = argv.filter((a) => !a.startsWith('--') && a !== baselineAt);
const say = (s = '') => { if (!asJson) console.log(s); };
const num = (n) => Math.round(n).toLocaleString('en-US');
const load = (rel) => import(pathToFileURL(resolve(rel)).href);
const idx = await load('dist/index.js');
const toolsMod = await load('dist/mcp/tools.js');
const CodeGraph = idx.default?.default ?? idx.default ?? idx.CodeGraph;
const ToolHandler = toolsMod.ToolHandler ?? toolsMod.default?.ToolHandler;
if (typeof CodeGraph?.openSync !== 'function' || typeof ToolHandler !== 'function') {
console.error('could not resolve CodeGraph/ToolHandler from dist/ — run `npm run build`');
process.exit(2);
}
const tmp = mkdtempSync(join(tmpdir(), 'cg-suite-'));
const results = [];
try {
for (const { id, q } of SUITE) {
if (only.length > 0 && !only.includes(id)) continue;
const repo = join(CORPUS, id);
if (!existsSync(join(repo, '.codegraph', 'codegraph.db'))) {
say(`${id}: no index at ${repo} — skipped`);
continue;
}
const sidecar = join(tmp, `${id}.jsonl`);
process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
const cg = CodeGraph.openSync(repo);
const h = new ToolHandler(cg);
const res = await h.execute('codegraph_explore', { query: q });
const text = res.content?.[0]?.text ?? '';
try { cg.close?.(); } catch { /* best effort */ }
const report = JSON.parse(readFileSync(sidecar, 'utf8').trim().split('\n').pop());
results.push({
repo: id,
sourceChars: report.envelope.sourceChars,
envelopeChars: report.envelope.chars,
allocatedChars: report.envelope.allocatedChars,
hardCeiling: report.budget.hardCeiling,
truncated: report.envelope.truncated,
files: report.selection.filesInFinalOutput,
// Did the response keep its trailing pointer list / notes, or did the
// hard ceiling spend them? This is CG-26's residual 1.
epilogueCut: text.includes('omitted for size'),
sectionCut: text.includes('output truncated to budget'),
notShown: text.includes('Not shown above'),
budgetNote: text.includes('**Explore budget:'),
});
}
} finally {
rmSync(tmp, { recursive: true, force: true });
}
if (asJson) {
console.log(JSON.stringify(results, null, 2));
} else {
const base = baselineAt ? JSON.parse(readFileSync(baselineAt, 'utf8')) : null;
const byRepo = new Map((base ?? []).map((r) => [r.repo, r]));
say('repo source Δ env files cut epilogue');
say('-'.repeat(74));
for (const r of results) {
const b = byRepo.get(r.repo);
const delta = b ? (r.sourceChars - b.sourceChars) : null;
const dStr = delta === null ? '' : (delta > 0 ? `+${num(delta)}` : num(delta));
const cut = r.sectionCut ? 'section' : r.epilogueCut ? 'epilogue' : '—';
const epi = [r.notShown ? 'not-shown' : null, r.budgetNote ? 'budget-note' : null]
.filter(Boolean).join('+') || 'none';
say(
`${r.repo.padEnd(12)} ${num(r.sourceChars).padStart(7)} ${dStr.padStart(8)} `
+ `${num(r.envelopeChars).padStart(7)} ${String(r.files).padStart(5)} ${cut.padEnd(12)} ${epi}`,
);
}
if (base) {
const lost = results.filter((r) => {
const b = byRepo.get(r.repo);
return b && (r.sourceChars < b.sourceChars || r.files < b.files);
});
say('');
say(lost.length === 0
? 'No repo delivers less source or fewer files than the baseline.'
: `REGRESSION: ${lost.map((r) => r.repo).join(', ')} deliver less than baseline.`);
}
}
src/bin/codegraph.ts
+2 -2
View File
@@ -2246,7 +2246,7 @@ program
*/
program
.command('install')
.description('Install codegraph MCP server into one or more agents (Claude Code, Cursor, Codex CLI, opencode, Hermes Agent)')
.description('Install codegraph MCP server into one or more agents (Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, Kiro, GitHub Copilot)')
.option('-t, --target <ids>', 'Target agent(s): comma-separated ids, or "auto"|"all"|"none". Default: prompt')
.option('-l, --location <where>', 'Install location: "global" or "local". Default: prompt')
.option('-y, --yes', 'Non-interactive: defaults to --location=global --target=auto, auto-allow on')
@@ -2346,7 +2346,7 @@ program
*/
program
.command('uninstall')
.description('Remove codegraph from your agents (Claude Code, Cursor, Codex CLI, opencode, Hermes Agent)')
.description('Remove codegraph from your agents (Claude Code, Cursor, Codex CLI, opencode, Hermes Agent, Gemini CLI, Antigravity IDE, Kiro, GitHub Copilot)')
.option('-t, --target <ids>', 'Target agent(s): comma-separated ids, or "all". Default: all')
.option('-l, --location <where>', 'Uninstall location: "global" or "local". Default: prompt')
.option('-y, --yes', 'Non-interactive: defaults to --location=global --target=all')
src/db/queries.ts
+234 -2
View File
@@ -1113,11 +1113,28 @@ export class QueryBuilder {
}
/**
* Get nodes by exact name match (uses idx_nodes_name index)
* Get nodes by exact name match (uses idx_nodes_name index).
*
* This is resolution's candidate list, and the ORDER BY is load-bearing for
* index correctness, not cosmetic (CG-33). When a reference names a symbol
* that several files define and nothing disambiguates them, resolution binds
* to the first candidate — so without an ORDER BY the winner was decided by
* rowid, i.e. by the order files happened to be WRITTEN. A full index writes
* them in scan order; an incremental sync appends each file as it changes, so
* the same tree resolved to different edges depending on how the index was
* built, and a long-lived synced index drifted away from a rebuild of itself
* (measured at 4.3% of distinct edges, mostly `calls`).
*
* `(file_path, start_line)` is a property of the CODE, so both paths now pick
* the same candidate. The sort is paid once per distinct name per resolution
* run — ReferenceResolver memoizes this in its nameCache — and the population
* is capped by AMBIGUOUS_NAME_CEILING (#999).
*/
getNodesByName(name: string): Node[] {
if (!this.stmts.getNodesByName) {
this.stmts.getNodesByName = this.db.prepare('SELECT * FROM nodes WHERE name = ?');
this.stmts.getNodesByName = this.db.prepare(
'SELECT * FROM nodes WHERE name = ? ORDER BY file_path, start_line'
);
}
const rows = this.stmts.getNodesByName.all(name) as NodeRow[];
return rows.map(rowToNode);
@@ -1944,6 +1961,101 @@ export class QueryBuilder {
return (filePath: string) => flagged.has(filePath) || isGeneratedFile(filePath);
}
/**
* Which of `filePaths` are AMBIENT DECLARATION files — they declare nothing
* but types, and nothing in the index depends on them (CG-28). A hand-written
* ambient `.d.ts` of global shims, a vendored typings file, module
* augmentation: reachable only by name, structurally attached to nothing.
*
* Structural, not extension-based, so a hand-written `types.ts` and a `.d.ts`
* are judged by the same rule and a `.d.ts` that does declare a class or a
* const is (correctly) not caught. Four conditions, all required:
*
* 1. it declares at least one symbol — an empty or unparsed file is not a
* declaration file, it is a file we know nothing about;
* 2. EVERY declared symbol is a type-level kind (interface / type alias /
* enum / namespace). The narrowness is deliberate and measured: a rule
* of "no callables" alone flags 1–18% of a repo, including Kotlin sealed
* classes, Rust `mod.rs` re-exports and django's locale constant tables —
* real source that must not be demoted. This rule flags 0–4%;
* 3. no symbol in it originates a `calls`/`instantiates` edge — the direct
* evidence that nothing here has a body;
* 4. NOTHING ELSE IN THE INDEX points at it. This is the condition that
* separates an ambient shim from a working type module, and it is why
* the flag is narrow enough to be safe: `displacement-ts`'s pipeline
* `types.ts` passes 1–3 identically but carries 13 inbound imports and
* 21 references, so the files that answer a query about the pipeline are
* typed BY it — it is part of that answer's structure. An ambient
* `declare global` shim has zero. Deliberately index-wide rather than
* restricted to the candidate list: the file that imports it is usually
* not itself a candidate.
*
* Bounded-lookup like {@link getGeneratedPathsAmong}: callers hold a ranked
* candidate list, so this is a partial-index probe over a handful of paths.
*/
getAmbientDeclarationPathsAmong(filePaths: Iterable<string>): Set<string> {
const unique = [...new Set(filePaths)];
const found = new Set<string>();
if (unique.length === 0) return found;
for (let i = 0; i < unique.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = unique.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
// `file`/`import`/`export`/`parameter` are structural bookkeeping, not
// things the file declares, so they neither qualify nor disqualify.
const rows = this.db
.prepare(`
SELECT file_path,
SUM(CASE WHEN kind NOT IN ('file','import','export','parameter')
THEN 1 ELSE 0 END) AS declared,
SUM(CASE WHEN kind IN ('interface','type_alias','enum','enum_member','namespace')
THEN 1 ELSE 0 END) AS typeDeclared
FROM nodes
WHERE file_path IN (${placeholders})
GROUP BY file_path
`)
.all(...chunk) as Array<{ file_path: string; declared: number; typeDeclared: number }>;
let candidates = rows
.filter((r) => r.declared > 0 && r.declared === r.typeDeclared)
.map((r) => r.file_path);
if (candidates.length === 0) continue;
const disqualify = (sql: string): void => {
if (candidates.length === 0) return;
const hit = new Set(
(this.db
.prepare(sql.replace('$IN$', candidates.map(() => '?').join(',')))
.all(...candidates) as Array<{ file_path: string }>).map((r) => r.file_path),
);
candidates = candidates.filter((p) => !hit.has(p));
};
// (3) originates behaviour
disqualify(`
SELECT DISTINCT n.file_path AS file_path
FROM edges e JOIN nodes n ON n.id = e.source
WHERE e.kind IN ('calls','instantiates') AND n.file_path IN ($IN$)
`);
// (4) something outside the file depends on it
disqualify(`
SELECT DISTINCT t.file_path AS file_path
FROM edges e JOIN nodes t ON t.id = e.target JOIN nodes s ON s.id = e.source
WHERE t.file_path IN ($IN$) AND s.file_path <> t.file_path
`);
for (const path of candidates) found.add(path);
}
return found;
}
/**
* A reusable `(path) => boolean` ambient-declaration test over a bounded
* candidate list — the shape a ranking comparator wants: one query up front,
* O(1) per comparison.
*/
ambientDeclarationPredicateFor(filePaths: Iterable<string>): (filePath: string) => boolean {
const flagged = this.getAmbientDeclarationPathsAmong(filePaths);
return (filePath: string) => flagged.has(filePath);
}
/** How many indexed files carry the generated flag. Surfaced by `status`. */
countGeneratedFiles(): number {
const row = this.db
@@ -2445,6 +2557,99 @@ export class QueryBuilder {
}));
}
/**
* Resolution edges whose TARGET symbol is named one of `names` — the edges a
* sync must re-resolve after `names` gained or lost a definition (CG-33).
*
* Resolution binds a reference to a node whose name matches the reference's
* tail, and it picks among ALL same-named definitions project-wide. So adding
* or removing one definition of `pct` changes the answer for every `pct(...)`
* reference in the repo — including references in files this sync never
* touches, whose edges nothing else revisits. Those edges' current target is,
* by that same rule, a node named `pct`, which is why the target's name is a
* sufficient (and index-backed, via idx_nodes_name) way to find them without
* a schema change or a scan of edge metadata.
*
* Returns the source file/language alongside each edge so the caller can
* resurrect it as its original reference. Excludes `provenance='heuristic'`
* (synthesized dispatch edges are not resolution output and carry no refName
* stamp to resurrect from — deleting one would be a permanent loss).
*
* Names matching more than `perNameCeiling` edges are skipped entirely, same
* rationale and same default as {@link getRetryableFailedReferences}: at that
* population the name is generic (`get`, `clear`, …), one definition changing
* won't flip most of them, and rebinding an arbitrary subset is both wasted
* work and incoherent coverage.
*/
getResolutionEdgesByTargetName(
names: string[],
perNameCeiling: number = 500
): Array<Edge & { edgeId: number; sourceFilePath: string; sourceLanguage: Language }> {
if (names.length === 0) return [];
// Pass 1: per-name edge counts, chunked under the SQLite parameter limit.
const keep: string[] = [];
for (let i = 0; i < names.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = names.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const counts = this.db
.prepare(
`SELECT tgt.name AS name, COUNT(*) AS count
FROM edges e
JOIN nodes tgt ON tgt.id = e.target
WHERE tgt.name IN (${placeholders})
AND (e.provenance IS NULL OR e.provenance != 'heuristic')
GROUP BY tgt.name`
)
.all(...chunk) as Array<{ name: string; count: number }>;
for (const row of counts) {
if (row.count <= perNameCeiling) keep.push(row.name);
}
}
if (keep.length === 0) return [];
// Pass 2: load the surviving edges with the source file context a
// resurrection needs.
const out: Array<Edge & { edgeId: number; sourceFilePath: string; sourceLanguage: Language }> = [];
for (let i = 0; i < keep.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = keep.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const rows = this.db
.prepare(
`SELECT e.*, src.file_path AS source_file_path, src.language AS source_language
FROM edges e
JOIN nodes tgt ON tgt.id = e.target
JOIN nodes src ON src.id = e.source
WHERE tgt.name IN (${placeholders})
AND (e.provenance IS NULL OR e.provenance != 'heuristic')`
)
.all(...chunk) as Array<EdgeRow & { source_file_path: string; source_language: Language }>;
for (const row of rows) {
out.push({
...rowToEdge(row),
edgeId: row.id,
sourceFilePath: row.source_file_path,
sourceLanguage: row.source_language,
});
}
}
return out;
}
/** Delete edges by primary key — the rebind pass's half of a re-resolution. */
deleteEdgesByIds(edgeIds: number[]): number {
if (edgeIds.length === 0) return 0;
let changed = 0;
this.db.transaction(() => {
for (let i = 0; i < edgeIds.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = edgeIds.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
changed += this.db.prepare(`DELETE FROM edges WHERE id IN (${placeholders})`).run(...chunk).changes;
}
})();
return changed;
}
/**
* Distinct node names present in the given files — the symbol names a sync
* pass uses to look up retryable failed refs after those files changed.
@@ -2463,6 +2668,33 @@ export class QueryBuilder {
return [...names];
}
/**
* Distinct `file\0name` pairs defined by the given files — the shape sync's
* definition delta needs (CG-33).
*
* Deliberately NOT `getNodeNamesByFiles`: a bare name set is taken over the
* WHOLE changed batch, so a name that moves between two files in one commit
* (or exists in one changed file and is newly added to another) appears on
* both sides and cancels out of the symmetric difference — even though a
* definition genuinely appeared or vanished and every reference to that name
* repo-wide may now bind elsewhere. Keying by file makes each definition its
* own fact, so the move is seen as one removal plus one addition.
*/
getNodeNamePairsByFiles(filePaths: string[]): Set<string> {
const pairs = new Set<string>();
if (filePaths.length === 0) return pairs;
for (let i = 0; i < filePaths.length; i += SQLITE_PARAM_CHUNK_SIZE) {
const chunk = filePaths.slice(i, i + SQLITE_PARAM_CHUNK_SIZE);
const placeholders = chunk.map(() => '?').join(',');
const rows = this.db
.prepare(`SELECT DISTINCT file_path, name FROM nodes WHERE file_path IN (${placeholders})`)
.all(...chunk) as Array<{ file_path: string; name: string }>;
// NUL-joined: a path or a symbol name can contain a space, never a NUL.
for (const row of rows) pairs.add(`${row.file_path}\0${row.name}`);
}
return pairs;
}
// ===========================================================================
// Statistics
// ===========================================================================
src/extraction/generated-detection.ts
+8
View File
@@ -181,6 +181,14 @@ const GENERATED_CONTENT_PATTERNS: ReadonlyArray<RegExp> = [
// "by" is required — bare "automatically generated" appears in hand-written
// prose ("the table below is automatically generated at runtime").
/\b(?:automatically generated|auto[- ]?generated|autogenerated) by\b/i,
// The "run this command to regenerate" shape: Cloudflare Wrangler
// ("Generated by Wrangler by running `wrangler types` (hash: …)"), and the
// same phrasing used by other CLI-driven emitters. Bare "generated by" is
// deliberately NOT enough — it is ordinary prose — so the reproduction
// instruction is the discriminator: the banner must name a tool AND then
// say `by running`, i.e. TWO separate "by" clauses. That rules out
// "the report is generated by running the nightly job", which has only one.
/\bgenerated by\s+\S.{0,80}?\bby running\b/i,
// Self-declaring in-house banners that name no tool.
/\bthis (?:file|class|code|module) (?:is|was) (?:auto[- ]?)?generated\b/i,
// The reverse ordering: "DO NOT EDIT — this is a generated file".
src/extraction/index.ts
+107
View File
@@ -116,6 +116,20 @@ export interface SyncResult {
nodesUpdated: number;
durationMs: number;
changedFilePaths?: string[];
/**
* Symbol names whose set of definitions this sync CHANGED — names the synced
* files gained or lost, as the symmetric difference of their `file\0name`
* definition pairs before and after the store phase (per file, so a name
* moving between two changed files does not cancel itself out).
* Resolution picks among all same-named definitions project-wide,
* so these are exactly the names whose already-resolved edges — in files this
* sync never touched — may now bind elsewhere and must be re-resolved for the
* index to stay convergent with a full rebuild (CG-33).
*
* A body-only edit leaves this empty, which is the common case and costs
* nothing downstream.
*/
definitionDelta?: string[];
}
/**
@@ -2491,6 +2505,64 @@ export class ExtractionOrchestrator {
}
}
/**
* Re-open, for re-resolution, every resolution edge whose answer this sync
* may have changed — the fix for index drift (CG-33).
*
* Incremental sync re-resolves only the references IN the changed files, but
* resolution's answer is a function of the WHOLE graph: a reference binds to
* one of the same-named definitions project-wide, so adding or removing a
* definition of `pct` can change which `pct` every other file's `pct(...)`
* should bind to. Those other files are never revisited, and their references
* resolved successfully once and were deleted from `unresolved_refs`, so
* nothing existed to revisit them with — the index kept an answer that was
* correct against an older graph. Measured on codegraph's own long-lived
* index: 4.3% of distinct edges differed from a clean rebuild, in BOTH
* directions, overwhelmingly `calls`. See docs/benchmarks/index-drift-cg33.md.
*
* This deletes each affected edge and re-inserts it as the reference that
* created it (the refName/refKind stamp), status='pending', for the sync's
* resolution sweep to bind against the post-sync graph — the same input a
* full rebuild resolves from, which is what makes the two converge.
*
* Deliberately conservative in three ways, because a wrong deletion is a
* permanent edge loss while a missed rebind is only residual drift:
* - an edge with no refName stamp (synthesized, or built by an engine older
* than the stamp) is left ALONE rather than reconstructed from the target's
* plain name, same rule as `resurrectRefFromDroppedEdge`;
* - edges whose source is in a file this sync already re-extracted are
* skipped — their references were re-resolved from scratch moments ago;
* - very common names are skipped by the per-name ceiling in
* `getResolutionEdgesByTargetName`.
*
* Returns the number of references resurrected.
*/
resurrectStaleResolutionEdges(definitionDelta: string[], changedFilePaths: string[]): number {
if (definitionDelta.length === 0) return 0;
const alreadyFresh = new Set(changedFilePaths);
const candidates = this.queries.getResolutionEdgesByTargetName(definitionDelta);
const edgeIds: number[] = [];
const refs: UnresolvedReference[] = [];
for (const e of candidates) {
if (alreadyFresh.has(e.sourceFilePath)) continue;
const ref = resurrectRefFromDroppedEdge(e);
if (!ref) continue; // no stamp — never delete what we cannot restore
edgeIds.push(e.edgeId);
refs.push(ref);
}
if (refs.length === 0) return 0;
// Delete first. The sweep re-inserts whichever edge resolution now picks,
// and `insertEdges` is INSERT OR IGNORE against idx_edges_identity — so a
// rebind to the same target is a clean no-op, but leaving the old row in
// place for a rebind ELSEWHERE would keep both, turning drift into
// duplication.
this.queries.deleteEdgesByIds(edgeIds);
this.queries.insertUnresolvedRefsBatch(refs);
return refs.length;
}
/**
* Sync the index with the current file state.
*
@@ -2520,6 +2592,10 @@ export class ExtractionOrchestrator {
let filesRemoved = 0;
let nodesUpdated = 0;
const changedFilePaths: string[] = [];
// `file\0name` definition pairs for the files this sync touches, sampled
// BEFORE their nodes are replaced/deleted. Compared against the post-store
// pairs below to derive `definitionDelta` (CG-33).
const pairsBefore = new Set<string>();
onProgress?.({
phase: 'scanning',
@@ -2585,6 +2661,9 @@ export class ExtractionOrchestrator {
// failed until the symbol reappears somewhere. (A deleted file whose
// CALLERS are also being deleted is fine: their nodes cascade later
// in this loop and take the resurrected rows with them.)
// Every name this file defined is about to stop existing here, which
// narrows the candidate set for that name repo-wide (CG-33).
for (const pair of this.queries.getNodeNamePairsByFiles([tracked.path])) pairsBefore.add(pair);
const incoming = this.queries.getCrossFileIncomingEdgesWithTarget(tracked.path);
if (incoming.length > 0) {
const resurrected = incoming
@@ -2651,6 +2730,14 @@ export class ExtractionOrchestrator {
}
}
// Sampled here — after the add/modify classification, before any file is
// re-extracted — because `storeExtractionResult` deletes a file's nodes
// before inserting the new ones, so this is the last point the pre-edit
// definition set is readable (CG-33).
if (filesToIndex.length > 0) {
for (const pair of this.queries.getNodeNamePairsByFiles(filesToIndex)) pairsBefore.add(pair);
}
// Load only grammars needed for changed files
if (filesToIndex.length > 0) {
const overrides = loadExtensionOverrides(this.rootDir);
@@ -2677,6 +2764,25 @@ export class ExtractionOrchestrator {
nodesUpdated += result.nodes.length;
}
// Names whose definition set this sync changed: a `file\0name` pair present
// before but not after (removed/renamed away) or after but not before
// (added). A pair on both sides is untouched as far as resolution's
// candidate set is concerned — only its node id moved, which
// reattachCrossFileEdges already follows — so an edit that only changes
// bodies yields an empty delta and no downstream rebind work (CG-33).
//
// Compared per FILE, not as one name set over the whole batch: a commit
// that adds `collect` to a new file while an unrelated changed file already
// defined `collect` must still flag the name, and a bare name set cancels
// exactly that case out. That miss left the largest residual class in the
// first measurement of this fix.
const pairsAfter = this.queries.getNodeNamePairsByFiles(filesToIndex);
const deltaNames = new Set<string>();
const nameOf = (pair: string) => pair.slice(pair.indexOf('\0') + 1);
for (const pair of pairsBefore) if (!pairsAfter.has(pair)) deltaNames.add(nameOf(pair));
for (const pair of pairsAfter) if (!pairsBefore.has(pair)) deltaNames.add(nameOf(pair));
const definitionDelta = [...deltaNames];
return {
filesChecked,
filesAdded,
@@ -2685,6 +2791,7 @@ export class ExtractionOrchestrator {
nodesUpdated,
durationMs: Date.now() - startTime,
changedFilePaths: changedFilePaths.length > 0 ? changedFilePaths : undefined,
definitionDelta: definitionDelta.length > 0 ? definitionDelta : undefined,
};
}
src/index.ts
+39
View File
@@ -883,6 +883,32 @@ export class CodeGraph {
}
}
// Re-open resolution edges this sync may have invalidated ELSEWHERE in
// the repo (CG-33). Everything above re-resolves references in the
// changed files; this covers the opposite direction — references in
// files the sync never touched whose answer depended on a definition
// that just appeared or disappeared. Without it a synced index never
// converges to a full rebuild: measured at 4.3% of distinct edges wrong
// on codegraph's own index, in both directions, mostly `calls`. The
// resurrected refs are pending rows, so the orphan sweep immediately
// below is what resolves them — batched, yielding, multi-pass, exactly
// as a full index resolves.
//
// `definitionDelta` is empty for a body-only edit, so the overwhelmingly
// common sync pays one branch. CODEGRAPH_NO_REBIND=1 disables it.
if (result.definitionDelta && process.env.CODEGRAPH_NO_REBIND !== '1') {
const tRebind = Date.now();
const rebound = this.orchestrator.resurrectStaleResolutionEdges(
result.definitionDelta,
result.changedFilePaths ?? []
);
if (process.env.CODEGRAPH_SYNTH_TIMINGS) {
console.error(
`[phase-timing] sync-rebind: ${Date.now() - tRebind}ms (${result.definitionDelta.length} changed names, ${rebound} edges re-opened)`
);
}
}
// Orphan sweep (#1187). A resolution pass that dies mid-run — the #850
// daemon liveness watchdog's SIGKILL (#1122), Ctrl-C, a crash — leaves
// the refs it never reached in unresolved_refs, and the git-scoped fast
@@ -1550,6 +1576,19 @@ export class CodeGraph {
return this.queries.generatedPredicateFor(filePaths);
}
/**
* A `(path) => boolean` ambient-declaration test over a BOUNDED candidate
* list: true for a file that declares nothing but types, originates no call
* edge, and that nothing in the index depends on — an ambient `.d.ts` of
* global shims, vendored typings, module augmentation (CG-28). Structural
* rather than extension-based, and deliberately narrow: see
* `QueryBuilder.getAmbientDeclarationPathsAmong` for why each condition is
* there, in particular why a `types.ts` the codebase imports is NOT flagged.
*/
ambientDeclarationFilePredicate(filePaths: Iterable<string>): (filePath: string) => boolean {
return this.queries.ambientDeclarationPredicateFor(filePaths);
}
/** How many indexed files are flagged tool-generated. Reported by `status`. */
getGeneratedFileCount(): number {
return this.queries.countGeneratedFiles();
src/installer/index.ts
+4 -3
View File
@@ -3,7 +3,8 @@
*
* Multi-target: writes MCP server config + instructions for the
* agents the user picks (Claude Code, Cursor, Codex CLI, opencode,
* Hermes Agent, Gemini CLI, Antigravity IDE).
* Hermes Agent, Gemini CLI, Antigravity IDE, Kiro, and GitHub
* Copilot in VS Code / the Copilot CLI / JetBrains IDEs).
* Defaults to the Claude-only behavior for backwards compatibility
* when no targets are explicitly chosen and nothing else is detected.
*
@@ -467,8 +468,8 @@ export async function runUninstaller(opts: RunUninstallerOptions): Promise<void>
const sel = await clack.select({
message: 'Remove CodeGraph from all your projects, or just this one?',
options: [
{ value: 'global' as const, label: 'All projects (global)', hint: '~/.claude, ~/.cursor, ~/.codex, ~/.config/opencode, ~/.hermes, ~/.gemini, ~/.kiro' },
{ value: 'local' as const, label: 'Just this project (local)', hint: './.claude, ./.cursor, ./opencode.jsonc, ./.gemini, ./.kiro' },
{ value: 'global' as const, label: 'All projects (global)', hint: '~/.claude, ~/.cursor, ~/.codex, ~/.config/opencode, ~/.hermes, ~/.gemini, ~/.kiro, ~/.copilot, ~/.config/github-copilot' },
{ value: 'local' as const, label: 'Just this project (local)', hint: './.claude, ./.cursor, ./.vscode, ./opencode.jsonc, ./.gemini, ./.kiro' },
],
initialValue: 'global' as const,
});
src/installer/targets/copilot-cli.ts
+192
View File
@@ -0,0 +1,192 @@
/**
* GitHub Copilot CLI target.
*
* - MCP server entry to `~/.copilot/mcp-config.json` under the
* `mcpServers` key (same wrapper as Claude/Cursor). Entry shape per
* the GitHub docs: `{ "type": "stdio", "command", "args", "tools" }`
* — `type` accepts `"local"` or `"stdio"`; we write `"stdio"` (the
* standard MCP name, recommended by the docs for cross-client
* compatibility). `"tools": ["*"]` mirrors the docs' example and is
* the documented default.
* - The config dir is `~/.copilot` unless the user moved it via
* `COPILOT_HOME` (documented override) — we honor it so install and
* detect follow the CLI's own resolution.
*
* Copilot CLI as of 2026-07 has no project-local MCP config — per-repo
* config (`.github/mcp.json`) is an open feature request
* (github/copilot-cli#2528). `supportsLocation('local')` returns false;
* the orchestrator skips this target for local installs with a clear
* message (same pattern as Codex).
*
* The file is machine-written by the CLI's own `/mcp add` flow, so it's
* plain JSON — no JSONC handling needed; surgical edits go through the
* shared read/mutate/write helpers (Cursor pattern), preserving sibling
* servers.
*
* No instructions file (MCP `initialize` instructions are the single
* source of truth, #529) and no permissions concept — `autoAllow` is
* silently ignored.
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import {
AgentTarget,
DetectionResult,
InstallOptions,
Location,
WriteResult,
} from './types';
import {
getMcpServerConfig,
jsonDeepEqual,
readJsonFile,
writeJsonFile,
} from './shared';
function configDir(): string {
const override = process.env.COPILOT_HOME;
if (override && override.trim().length > 0) return override;
return path.join(os.homedir(), '.copilot');
}
function mcpConfigPath(): string {
return path.join(configDir(), 'mcp-config.json');
}
/**
* `~/.copilot` existing is NOT proof the CLI is installed: the VS Code
* Copilot Chat extension drops MCP socket-handoff lock files into
* `~/.copilot/ide/` on launch, so a machine with only the VS Code
* extension still has the dir (with a lone `ide` entry). Count the dir
* as a CLI footprint only when it holds anything besides `ide` — the
* CLI writes `config.json` (and later `mcp-config.json`, history state)
* on first run.
*/
function cliConfigDirPresent(): boolean {
let entries: string[];
try {
entries = fs.readdirSync(configDir());
} catch {
return false;
}
return entries.some((e) => e !== 'ide');
}
/**
* Best-effort check that the `copilot` binary is reachable on PATH.
* A plain fs scan (no shell-out) — cheap enough to run inside
* `detectAll()` for the multiselect prompt.
*/
function copilotOnPath(): boolean {
const pathVar = process.env.PATH || '';
const exts = process.platform === 'win32'
? ['.exe', '.cmd', '.bat', '.ps1']
: [''];
for (const dir of pathVar.split(path.delimiter)) {
if (!dir) continue;
for (const ext of exts) {
try {
if (fs.existsSync(path.join(dir, 'copilot' + ext))) return true;
} catch { /* ignore unreadable PATH entries */ }
}
}
return false;
}
function buildCopilotMcpConfig(): { type: string; command: string; args: string[]; tools: string[] } {
const base = getMcpServerConfig();
return { ...base, tools: ['*'] };
}
class CopilotCliTarget implements AgentTarget {
readonly id = 'copilot-cli' as const;
readonly displayName = 'GitHub Copilot CLI';
readonly docsUrl = 'https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/add-mcp-servers';
supportsLocation(loc: Location): boolean {
return loc === 'global';
}
detect(loc: Location): DetectionResult {
if (loc !== 'global') {
return { installed: false, alreadyConfigured: false };
}
const file = mcpConfigPath();
const config = readJsonFile(file);
const alreadyConfigured = !!config.mcpServers?.codegraph;
const installed = cliConfigDirPresent() || copilotOnPath();
return { installed, alreadyConfigured, configPath: file };
}
install(loc: Location, _opts: InstallOptions): WriteResult {
if (loc !== 'global') {
return {
files: [],
notes: ['Copilot CLI has no project-local config — re-run with --location=global to install.'],
};
}
return {
files: [writeMcpEntry()],
notes: ['Restart any running Copilot CLI session to pick up the MCP server.'],
};
}
uninstall(loc: Location): WriteResult {
if (loc !== 'global') return { files: [] };
const file = mcpConfigPath();
if (!fs.existsSync(file)) {
return { files: [{ path: file, action: 'not-found' }] };
}
const config = readJsonFile(file);
if (!config.mcpServers?.codegraph) {
return { files: [{ path: file, action: 'not-found' }] };
}
delete config.mcpServers.codegraph;
if (Object.keys(config.mcpServers).length === 0) {
delete config.mcpServers;
}
if (Object.keys(config).length === 0) {
// Nothing left but the `{}` we'd write back — delete the file so
// uninstall fully reverses a from-scratch install. A leftover
// empty file would keep detect() reporting the CLI as installed.
fs.unlinkSync(file);
} else {
writeJsonFile(file, config);
}
return { files: [{ path: file, action: 'removed' }] };
}
printConfig(loc: Location): string {
if (loc !== 'global') {
return '# Copilot CLI has no project-local config — use --location=global.\n';
}
const snippet = JSON.stringify({ mcpServers: { codegraph: buildCopilotMcpConfig() } }, null, 2);
return `# Add to ${mcpConfigPath()}\n\n${snippet}\n`;
}
describePaths(loc: Location): string[] {
if (loc !== 'global') return [];
return [mcpConfigPath()];
}
}
function writeMcpEntry(): WriteResult['files'][number] {
const file = mcpConfigPath();
const existing = readJsonFile(file);
const before = existing.mcpServers?.codegraph;
const after = buildCopilotMcpConfig();
if (jsonDeepEqual(before, after)) {
return { path: file, action: 'unchanged' };
}
const existed = fs.existsSync(file);
if (!existing.mcpServers) existing.mcpServers = {};
existing.mcpServers.codegraph = after;
writeJsonFile(file, existing);
return { path: file, action: existed ? 'updated' : 'created' };
}
export const copilotCliTarget: AgentTarget = new CopilotCliTarget();
src/installer/targets/copilot-jetbrains.ts
+230
View File
@@ -0,0 +1,230 @@
/**
* JetBrains IDEs (GitHub Copilot plugin) target.
*
* - MCP server entry to the plugin's user-level `mcp.json`, which
* lives under the shared `github-copilot` config dir (the same dir
* the Copilot ecosystem uses for `hosts.json`):
*
* macOS/Linux: $XDG_CONFIG_HOME|~/.config/github-copilot/intellij/mcp.json
* Windows: %LOCALAPPDATA%\github-copilot\intellij\mcp.json
*
* `$XDG_CONFIG_HOME` is honored on every platform when set —
* matching the plugin family's own resolution (copilot.vim /
* copilot-language-server check it before the OS default).
* - Shape is VS Code-compatible: `{ "servers": { "<name>": { "type":
* "stdio", "command", "args" } } }` — the plugin documents mcp.json
* parity with `.vscode/mcp.json`.
* - **Global-only.** The plugin reads exactly one user-level file; a
* project-level mcp.json is an open feature request
* (microsoft/copilot-intellij-feedback#701, still open 2026-07).
* `supportsLocation('local')` returns false so the orchestrator
* skips local installs with a clear message (Codex pattern).
* - No `--path` injection: the config is user-global and the plugin
* documents no `${workspaceFolder}`-style variable expansion for
* this file, so we ship the plain entry and let the MCP server
* resolve the project from the client's roots/cwd as with other
* global installs.
* - No instructions file (MCP `initialize` instructions are the
* single source of truth, #529) and no permissions concept —
* `autoAllow` is silently ignored.
*
* The IDE opens this file in a JSON editor for hand-editing (Settings →
* Tools → GitHub Copilot → MCP → Configure), so reads + writes go
* through `jsonc-parser` — surgical edits that preserve sibling
* servers, user comments, and formatting (same approach as the
* copilot-vscode target).
*
* The plugin only re-reads mcp.json on IDE restart
* (microsoft/copilot-intellij-feedback#1139) — hence the restart note.
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { parse as parseJsonc, modify, applyEdits } from 'jsonc-parser';
import {
AgentTarget,
DetectionResult,
InstallOptions,
Location,
WriteResult,
} from './types';
import {
atomicWriteFileSync,
getMcpServerConfig,
jsonDeepEqual,
} from './shared';
/**
* The `github-copilot` config root, resolved the way the Copilot
* plugin family resolves it: `$XDG_CONFIG_HOME` first on every
* platform, then `%LOCALAPPDATA%` on Windows, then `~/.config`.
*/
function copilotConfigRoot(): string {
const xdg = process.env.XDG_CONFIG_HOME;
if (xdg && xdg.trim().length > 0) {
return path.join(xdg, 'github-copilot');
}
if (process.platform === 'win32') {
const localAppData = process.env.LOCALAPPDATA && process.env.LOCALAPPDATA.trim().length > 0
? process.env.LOCALAPPDATA
: path.join(os.homedir(), 'AppData', 'Local');
return path.join(localAppData, 'github-copilot');
}
return path.join(os.homedir(), '.config', 'github-copilot');
}
function intellijDir(): string {
return path.join(copilotConfigRoot(), 'intellij');
}
function mcpJsonPath(): string {
return path.join(intellijDir(), 'mcp.json');
}
/**
* Best-effort "a JetBrains IDE exists here" heuristic for the
* multiselect default — the per-OS dir every JetBrains IDE creates on
* first launch. False positives (IDE without the Copilot plugin) are
* acceptable per the `DetectionResult` contract.
*/
function jetbrainsConfigDirExists(): boolean {
const home = os.homedir();
if (process.platform === 'darwin') {
return fs.existsSync(path.join(home, 'Library', 'Application Support', 'JetBrains'));
}
if (process.platform === 'win32') {
const appData = process.env.APPDATA && process.env.APPDATA.trim().length > 0
? process.env.APPDATA
: path.join(home, 'AppData', 'Roaming');
return fs.existsSync(path.join(appData, 'JetBrains'));
}
const xdg = process.env.XDG_CONFIG_HOME && process.env.XDG_CONFIG_HOME.trim().length > 0
? process.env.XDG_CONFIG_HOME
: path.join(home, '.config');
return fs.existsSync(path.join(xdg, 'JetBrains'));
}
function readConfigText(file: string): string {
if (!fs.existsSync(file)) return '';
return fs.readFileSync(file, 'utf-8');
}
function parseConfig(text: string): Record<string, any> {
if (!text.trim()) return {};
const errors: any[] = [];
const result = parseJsonc(text, errors, { allowTrailingComma: true });
if (result == null || typeof result !== 'object' || Array.isArray(result)) {
return {};
}
return result as Record<string, any>;
}
const FORMATTING = { tabSize: 2, insertSpaces: true, eol: '\n' };
class CopilotJetbrainsTarget implements AgentTarget {
readonly id = 'copilot-jetbrains' as const;
readonly displayName = 'JetBrains IDEs (Copilot plugin)';
readonly docsUrl = 'https://docs.github.com/en/copilot/how-tos/provide-context/use-mcp/extend-copilot-chat-with-mcp';
supportsLocation(loc: Location): boolean {
return loc === 'global';
}
detect(loc: Location): DetectionResult {
if (loc !== 'global') {
return { installed: false, alreadyConfigured: false };
}
const file = mcpJsonPath();
const config = parseConfig(readConfigText(file));
const alreadyConfigured = !!config.servers?.codegraph;
// The `intellij/` subdir is created by the Copilot plugin itself;
// fall back to "some JetBrains IDE is installed" for first-time
// plugin users.
const installed = fs.existsSync(intellijDir()) || jetbrainsConfigDirExists();
return { installed, alreadyConfigured, configPath: file };
}
install(loc: Location, _opts: InstallOptions): WriteResult {
if (loc !== 'global') {
return {
files: [],
notes: ['The JetBrains Copilot plugin has no project-local MCP config — re-run with --location=global to install.'],
};
}
return {
files: [writeMcpEntry()],
notes: ['Restart your JetBrains IDE — the Copilot plugin only reads mcp.json on startup.'],
};
}
uninstall(loc: Location): WriteResult {
if (loc !== 'global') return { files: [] };
return { files: [removeMcpEntry()] };
}
printConfig(loc: Location): string {
if (loc !== 'global') {
return '# The JetBrains Copilot plugin has no project-local MCP config — use --location=global.\n';
}
const snippet = JSON.stringify({ servers: { codegraph: getMcpServerConfig() } }, null, 2);
return `# Add to ${mcpJsonPath()}\n# (Settings → Tools → GitHub Copilot → Model Context Protocol → Configure)\n\n${snippet}\n`;
}
describePaths(loc: Location): string[] {
if (loc !== 'global') return [];
return [mcpJsonPath()];
}
}
function writeMcpEntry(): WriteResult['files'][number] {
const file = mcpJsonPath();
const existed = fs.existsSync(file);
let text = readConfigText(file);
if (!text.trim()) text = '{}\n';
const config = parseConfig(text);
const before = config.servers?.codegraph;
const after = getMcpServerConfig();
if (jsonDeepEqual(before, after)) {
return { path: file, action: 'unchanged' };
}
// Surgical edit — preserves comments, formatting, and sibling
// servers ("servers" is created when missing).
const edits = modify(text, ['servers', 'codegraph'], after, {
formattingOptions: FORMATTING,
});
const updated = applyEdits(text, edits);
atomicWriteFileSync(file, updated);
return { path: file, action: existed ? 'updated' : 'created' };
}
function removeMcpEntry(): WriteResult['files'][number] {
const file = mcpJsonPath();
if (!fs.existsSync(file)) return { path: file, action: 'not-found' };
const text = readConfigText(file);
const config = parseConfig(text);
if (!config.servers?.codegraph) return { path: file, action: 'not-found' };
let edits = modify(text, ['servers', 'codegraph'], undefined, {
formattingOptions: FORMATTING,
});
let updated = applyEdits(text, edits);
// Drop an emptied `servers` wrapper; the file itself is left in
// place — the plugin owns it and siblings may remain.
const afterParsed = parseConfig(updated);
if (afterParsed.servers && typeof afterParsed.servers === 'object' &&
Object.keys(afterParsed.servers).length === 0) {
edits = modify(updated, ['servers'], undefined, { formattingOptions: FORMATTING });
updated = applyEdits(updated, edits);
}
atomicWriteFileSync(file, updated);
return { path: file, action: 'removed' };
}
export const copilotJetbrainsTarget: AgentTarget = new CopilotJetbrainsTarget();
src/installer/targets/copilot-vscode.ts
+212
View File
@@ -0,0 +1,212 @@
/**
* VS Code (GitHub Copilot Chat) target.
*
* - MCP server entry to `.vscode/mcp.json` (local, workspace-scoped)
* or the user-level `mcp.json` in the VS Code User dir (global):
*
* macOS: ~/Library/Application Support/Code/User/mcp.json
* Windows: %APPDATA%\Code\User\mcp.json
* Linux: $XDG_CONFIG_HOME|~/.config/Code/User/mcp.json
*
* VS Code moved MCP config out of settings.json into this dedicated
* `mcp.json` (v1.102, "MCP: Open User Configuration"). Shape is
* `{ "servers": { "<name>": { "type": "stdio", "command", "args" } } }`
* — note `servers`, not the `mcpServers` wrapper Claude/Cursor use.
* - No instructions file: Copilot Chat consumes the MCP `initialize`
* instructions, the single source of truth (#529).
* - No permissions concept — `autoAllow` is silently ignored.
*
* ## Why `--path` only for local installs (NOT the Cursor pattern)
*
* Unlike Cursor, VS Code DOCUMENTS the launch cwd for stdio MCP
* servers: "Working directory for the server command. Defaults to the
* workspace folder when run in a workspace" (mcp-configuration
* reference). The codegraph server resolves its project via the MCP
* roots/list dance with a cwd fallback, so cwd alone is sufficient:
*
* - `local` install: absolute `--path` (known at install time) —
* deterministic, and free of variables.
* - `global` install: NO `--path`. Do not be tempted to pin it with
* `${workspaceFolder}`: VS Code refuses to start a user-level
* server whose entry uses that variable whenever a window has no
* folder open (loose files, welcome tab), surfacing an error toast
* "Variable workspaceFolder can not be resolved" in every such
* window — exactly the error-noise that teaches users to disable
* the server. With no `--path`, a folderless window still starts
* the server fine and it serves the "no project" guidance.
*
* ## JSONC
*
* VS Code parses its config files as JSONC (comments + trailing commas
* allowed), so reads + writes go through `jsonc-parser` — surgical
* edits that preserve sibling servers, user comments, and formatting
* across install / re-install / uninstall (same approach as opencode).
*/
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
import { parse as parseJsonc, modify, applyEdits } from 'jsonc-parser';
import {
AgentTarget,
DetectionResult,
InstallOptions,
Location,
WriteResult,
} from './types';
import {
atomicWriteFileSync,
getMcpServerConfig,
jsonDeepEqual,
} from './shared';
function vscodeUserDir(): string {
const home = os.homedir();
if (process.platform === 'win32') {
const appData = process.env.APPDATA && process.env.APPDATA.trim().length > 0
? process.env.APPDATA
: path.join(home, 'AppData', 'Roaming');
return path.join(appData, 'Code', 'User');
}
if (process.platform === 'darwin') {
return path.join(home, 'Library', 'Application Support', 'Code', 'User');
}
const xdg = process.env.XDG_CONFIG_HOME && process.env.XDG_CONFIG_HOME.trim().length > 0
? process.env.XDG_CONFIG_HOME
: path.join(home, '.config');
return path.join(xdg, 'Code', 'User');
}
function mcpJsonPath(loc: Location): string {
return loc === 'global'
? path.join(vscodeUserDir(), 'mcp.json')
: path.join(process.cwd(), '.vscode', 'mcp.json');
}
/**
* Build the codegraph server entry for VS Code at the given location.
* Local installs pin `--path`; global installs rely on VS Code's
* documented workspace-folder cwd — see file header for why the global
* entry must stay variable-free.
*/
function buildVscodeServerEntry(loc: Location): { type: string; command: string; args: string[] } {
const base = getMcpServerConfig();
if (loc === 'local') {
return { ...base, args: [...base.args, '--path', process.cwd()] };
}
return { ...base, args: [...base.args] };
}
function readConfigText(file: string): string {
if (!fs.existsSync(file)) return '';
return fs.readFileSync(file, 'utf-8');
}
function parseConfig(text: string): Record<string, any> {
if (!text.trim()) return {};
const errors: any[] = [];
const result = parseJsonc(text, errors, { allowTrailingComma: true });
if (result == null || typeof result !== 'object' || Array.isArray(result)) {
return {};
}
return result as Record<string, any>;
}
const FORMATTING = { tabSize: 2, insertSpaces: true, eol: '\n' };
class CopilotVscodeTarget implements AgentTarget {
readonly id = 'copilot-vscode' as const;
readonly displayName = 'VS Code (Copilot Chat)';
readonly docsUrl = 'https://code.visualstudio.com/docs/copilot/customization/mcp-servers';
supportsLocation(_loc: Location): boolean {
return true;
}
detect(loc: Location): DetectionResult {
const file = mcpJsonPath(loc);
const config = parseConfig(readConfigText(file));
const alreadyConfigured = !!config.servers?.codegraph;
// "Installed" heuristic: the VS Code User dir (created on first
// launch) or ~/.vscode (extensions dir) for global; an existing
// .vscode/ dir in the project for local.
const installed = loc === 'global'
? fs.existsSync(vscodeUserDir()) || fs.existsSync(path.join(os.homedir(), '.vscode'))
: fs.existsSync(path.join(process.cwd(), '.vscode'));
return { installed, alreadyConfigured, configPath: file };
}
install(loc: Location, _opts: InstallOptions): WriteResult {
return {
files: [writeMcpEntry(loc)],
notes: ['Restart VS Code for MCP changes to take effect.'],
};
}
uninstall(loc: Location): WriteResult {
return { files: [removeMcpEntry(loc)] };
}
printConfig(loc: Location): string {
const target = mcpJsonPath(loc);
const snippet = JSON.stringify({ servers: { codegraph: buildVscodeServerEntry(loc) } }, null, 2);
return `# Add to ${target}\n\n${snippet}\n`;
}
describePaths(loc: Location): string[] {
return [mcpJsonPath(loc)];
}
}
function writeMcpEntry(loc: Location): WriteResult['files'][number] {
const file = mcpJsonPath(loc);
const existed = fs.existsSync(file);
let text = readConfigText(file);
if (!text.trim()) text = '{}\n';
const config = parseConfig(text);
const before = config.servers?.codegraph;
const after = buildVscodeServerEntry(loc);
if (jsonDeepEqual(before, after)) {
return { path: file, action: 'unchanged' };
}
// Surgical edit — preserves comments, formatting, and sibling
// servers ("servers" is created when missing).
const edits = modify(text, ['servers', 'codegraph'], after, {
formattingOptions: FORMATTING,
});
const updated = applyEdits(text, edits);
atomicWriteFileSync(file, updated);
return { path: file, action: existed ? 'updated' : 'created' };
}
function removeMcpEntry(loc: Location): WriteResult['files'][number] {
const file = mcpJsonPath(loc);
if (!fs.existsSync(file)) return { path: file, action: 'not-found' };
const text = readConfigText(file);
const config = parseConfig(text);
if (!config.servers?.codegraph) return { path: file, action: 'not-found' };
let edits = modify(text, ['servers', 'codegraph'], undefined, {
formattingOptions: FORMATTING,
});
let updated = applyEdits(text, edits);
// Drop an emptied `servers` wrapper; the file itself is left in
// place — VS Code recreates/reads it and siblings like `inputs`
// may remain.
const afterParsed = parseConfig(updated);
if (afterParsed.servers && typeof afterParsed.servers === 'object' &&
Object.keys(afterParsed.servers).length === 0) {
edits = modify(updated, ['servers'], undefined, { formattingOptions: FORMATTING });
updated = applyEdits(updated, edits);
}
atomicWriteFileSync(file, updated);
return { path: file, action: 'removed' };
}
export const copilotVscodeTarget: AgentTarget = new CopilotVscodeTarget();
src/installer/targets/registry.ts
+6
View File
@@ -16,6 +16,9 @@ import { hermesTarget } from './hermes';
import { geminiTarget } from './gemini';
import { antigravityTarget } from './antigravity';
import { kiroTarget } from './kiro';
import { copilotVscodeTarget } from './copilot-vscode';
import { copilotCliTarget } from './copilot-cli';
import { copilotJetbrainsTarget } from './copilot-jetbrains';
export const ALL_TARGETS: readonly AgentTarget[] = Object.freeze([
claudeTarget,
@@ -26,6 +29,9 @@ export const ALL_TARGETS: readonly AgentTarget[] = Object.freeze([
geminiTarget,
antigravityTarget,
kiroTarget,
copilotVscodeTarget,
copilotCliTarget,
copilotJetbrainsTarget,
]);
export function getTarget(id: string): AgentTarget | undefined {
src/installer/targets/types.ts
+1 -1
View File
@@ -19,7 +19,7 @@ export type Location = 'global' | 'local';
* lookup. New targets add a value here when they're added to the
* registry. Keep these short and lowercase.
*/
export type TargetId = 'claude' | 'cursor' | 'codex' | 'opencode' | 'hermes' | 'gemini' | 'antigravity' | 'kiro';
export type TargetId = 'claude' | 'cursor' | 'codex' | 'opencode' | 'hermes' | 'gemini' | 'antigravity' | 'kiro' | 'copilot-vscode' | 'copilot-cli' | 'copilot-jetbrains';
/**
* Result of `target.detect(location)`.

Some files were not shown because too many files have changed in this diff Show More