Merge pull request #1527 from colbymchenry/bugfix/CG-38
CG-38: guarantee an agent-named symbol renders, wherever it sits
This commit is contained in:
Colby Mchenry
@@ -36,6 +36,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
- Every file `codegraph_explore` decides to include now actually arrives. A file shown in full could still spend room set aside for files below it — the fix above covered files shown as excerpts but not files shown whole — and the answer's own size bookkeeping under-counted each file's heading, so the answer ran past its limit and a fully prepared file was discarded at the end. A file that no longer fits whole is now shown as excerpts instead of vanishing, and one that overshoots by a little is trimmed to fit rather than dropped.
|
||||
- The list of files an answer could not cover — the "explore these names for their source" pointers — is no longer thrown away when the answer is full. It is now budgeted for and trimmed to fit, so a full answer still tells you what it left out and which names to ask for next, instead of ending with no pointers at all.
|
||||
- When a `codegraph_explore` answer shows a file as excerpts, a large excerpt that no longer fit was dropped entirely instead of being shortened. If the file's first excerpt happened to be a trivial one — an import block, a one-line helper next to the code you asked about — the excerpt carrying the actual answer was the one thrown away, and the file came back with a quarter of the room it had been given. On real projects that meant the top-ranked file delivered a fraction of its share while a far less relevant file took the rest. Excerpts are now shortened to fit, whole method by whole method, and only dropped when what is left is too small to hold anything readable.
|
||||
- When you name a symbol in a `codegraph_explore` query, its definition now actually comes back. Two cases previously lost it. If the symbols you named don't call one another — sibling functions inside the same factory or module are the everyday example — CodeGraph stopped treating them as symbols you had asked for, and answered with whatever sat at the top of their file instead; on one 1,400-line file that meant a same-stem `QueuedMessage` interface on line 70 came back while the `queueMessage` function on line 1087 did not. And when an answer had to be trimmed to fit, it was trimmed from the bottom of the file down, so a symbol near the end of a long file was always the first thing cut. Trimming now protects the definitions you named wherever they sit in the file.
|
||||
- The blast-radius section of `codegraph_explore` flagged "no covering tests found" whenever no test called a symbol directly — falsely branding helpers that tests exercise through their callers as untested (about 40% of flagged symbols in a measured sample). The check now follows caller chains up to 3 hops and reports indirect coverage as "tested via callers"; when nothing is found it states exactly what was checked instead of an unconditional warning. Thanks @inth3shadows for measuring the false-positive rate. (#1475)
|
||||
|
||||
## [1.5.0] - 2026-07-21
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* Standing gate for THE GUARANTEE (task CG-38): if the agent names a symbol and
|
||||
* that symbol's file is admitted to the response, the symbol's DEFINITION renders.
|
||||
*
|
||||
* This is the measurement the CG-24 epic never had. Its probes all score the
|
||||
* response in aggregate — envelope share, per-file spend, source totals, file
|
||||
* counts — and every one of them is green on a response that returns 25K of
|
||||
* source from the right file and still omits the function the agent asked for by
|
||||
* name. That is what CG-38 was: on a 1,414-line Svelte store, `queueMessage`
|
||||
* (L1087) and `flushQueuedMessages` (L1102) never rendered even though their file
|
||||
* won rank #1 with 67% of the envelope; the agent got the same-stem
|
||||
* `QueuedMessage` INTERFACE at L70 and had to Read the file to find the
|
||||
* functions. Longstanding, not an epic regression — the controlled bisect (index
|
||||
* held fixed, engine varied across every epic merge point) found it at every
|
||||
* build including pre-epic.
|
||||
*
|
||||
* Two independent causes, and the fixture below fails on either:
|
||||
*
|
||||
* 1. `buildFlowFromNamedSymbols` returned EMPTY — throwing away the NAMED-SYMBOL
|
||||
* IDENTITY along with the narrative — whenever the named symbols happened not
|
||||
* to form a call chain. Two sibling closures in one factory produce no chain,
|
||||
* no synthesized hop and no dispatch boundary, so both defs lost the
|
||||
* importance-9 rank that the named-def injection exists to give them.
|
||||
* 2. The ceiling trim cut in SOURCE ORDER, so whatever survived the shrink at
|
||||
* the END of a large file was always the first thing dropped.
|
||||
*
|
||||
* The fixture mirrors the reported file's geometry deliberately: a decoy
|
||||
* same-stem interface at L70, a factory closure at L104 spanning ~92% of the file
|
||||
* (so every symbol merges into ONE cluster), the target functions past L1000, and
|
||||
* a 2,500-line generated `.d.ts` for the ranker to penalise.
|
||||
*/
|
||||
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
|
||||
import * as fs from 'fs';
|
||||
import * as path from 'path';
|
||||
import * as os from 'os';
|
||||
import CodeGraph from '../src/index';
|
||||
import { ToolHandler } from '../src/mcp/tools';
|
||||
|
||||
const FIXTURE = 'tail-render-ts';
|
||||
const TARGET = 'src/lib/session-store.ts';
|
||||
|
||||
let dir: string;
|
||||
let cg: CodeGraph;
|
||||
|
||||
/** Every `<n>\t<text>` line number the response actually sent. */
|
||||
function renderedLines(response: string): Set<number> {
|
||||
const out = new Set<number>();
|
||||
for (const m of response.matchAll(/^(\d+)\t/gm)) out.add(Number(m[1]));
|
||||
return out;
|
||||
}
|
||||
|
||||
async function explore(query: string): Promise<string> {
|
||||
const res = await new ToolHandler(cg).execute('codegraph_explore', { query });
|
||||
return res.content?.[0]?.text ?? '';
|
||||
}
|
||||
|
||||
function defLineOf(name: string): number {
|
||||
const node = cg.getNodesByName(name).find((n) => n.filePath === TARGET && n.startLine > 0);
|
||||
expect(node, `${name} is not indexed in ${TARGET}`).toBeDefined();
|
||||
return node!.startLine;
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg38-'));
|
||||
fs.cpSync(path.join(__dirname, 'fixtures', FIXTURE), dir, { recursive: true });
|
||||
fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
|
||||
cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
}, 180_000);
|
||||
|
||||
afterAll(() => {
|
||||
cg?.destroy();
|
||||
if (dir && fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('CG-38 fixture shape — if this rots, the gate below means nothing', () => {
|
||||
it('puts the target functions past L1000 of a ~1,400-line file', () => {
|
||||
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
|
||||
expect(lines.length).toBeGreaterThan(1300);
|
||||
expect(defLineOf('queueMessage')).toBeGreaterThan(1000);
|
||||
expect(defLineOf('flushQueuedMessages')).toBeGreaterThan(1000);
|
||||
});
|
||||
|
||||
it('wraps them in a closure spanning most of the file, so they all cluster as one', () => {
|
||||
const lines = fs.readFileSync(path.join(dir, TARGET), 'utf-8').split('\n');
|
||||
const factory = cg.getNodesByName('createSessionStore')
|
||||
.find((n) => n.filePath === TARGET)!;
|
||||
expect(factory).toBeDefined();
|
||||
expect(factory.endLine - factory.startLine + 1).toBeGreaterThan(lines.length * 0.5);
|
||||
});
|
||||
|
||||
it('carries the same-stem decoy near the top', () => {
|
||||
const decoy = cg.getNodesByName('QueuedMessage').find((n) => n.filePath === TARGET)!;
|
||||
expect(decoy).toBeDefined();
|
||||
expect(decoy.kind).toBe('interface');
|
||||
expect(decoy.startLine).toBeLessThan(100);
|
||||
});
|
||||
|
||||
it('carries a generated declaration file for the ranker to penalise', () => {
|
||||
const dts = path.join(dir, 'types/worker-configuration.d.ts');
|
||||
expect(fs.existsSync(dts)).toBe(true);
|
||||
expect(fs.readFileSync(dts, 'utf-8').split('\n').length).toBeGreaterThan(2000);
|
||||
});
|
||||
|
||||
it('neither target calls the other — that absence is what produced no flow', () => {
|
||||
const queue = cg.getNodesByName('queueMessage').find((n) => n.filePath === TARGET)!;
|
||||
const flush = cg.getNodesByName('flushQueuedMessages').find((n) => n.filePath === TARGET)!;
|
||||
const between = [...cg.getCallees(queue.id), ...cg.getCallees(flush.id)]
|
||||
.filter(({ node }) => node.id === queue.id || node.id === flush.id);
|
||||
expect(between).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CG-38 — an agent-named symbol renders its definition', () => {
|
||||
/**
|
||||
* Both reported query shapes. They fail for different reasons — the symbol bag
|
||||
* never built a flow at all, the prose question built one and then lost the
|
||||
* tail to the ceiling trim — so a fix for one does not imply the other.
|
||||
*/
|
||||
const CASES: Array<{ shape: string; query: string; symbols: string[] }> = [
|
||||
{
|
||||
shape: 'symbol bag',
|
||||
query: 'queueMessage flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
shape: 'prose question',
|
||||
query: 'how does queueMessage hand its entries to flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
shape: 'three siblings, with the decoy interface competing',
|
||||
query: 'explain queueMessage, removeQueuedMessage and flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'removeQueuedMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
];
|
||||
|
||||
for (const { shape, query, symbols } of CASES) {
|
||||
it(`renders every named definition — ${shape}`, async () => {
|
||||
const response = await explore(query);
|
||||
const lines = renderedLines(response);
|
||||
for (const name of symbols) {
|
||||
const line = defLineOf(name);
|
||||
// The NAME alone proves nothing: it appears in the section header's
|
||||
// symbol list and at call sites whether or not the body was sent. Only
|
||||
// the definition LINE being among the rendered lines counts.
|
||||
expect(lines.has(line), `${name} (${TARGET}:${line}) did not render for "${query}"`)
|
||||
.toBe(true);
|
||||
}
|
||||
}, 120_000);
|
||||
}
|
||||
|
||||
it('never steers the agent to Read', async () => {
|
||||
const response = await explore('queueMessage flushQueuedMessages');
|
||||
expect(response).not.toMatch(/\buse Read\b|\bRead the file\b/i);
|
||||
}, 120_000);
|
||||
});
|
||||
|
||||
describe('CG-38 — a penalty on one file cannot shrink an unrelated file\'s render', () => {
|
||||
/**
|
||||
* The issue's sharpest lead: on an index where the generated `.d.ts` was NOT
|
||||
* flagged, the target file rendered ~581 lines including both symbols; on an
|
||||
* index where it WAS flagged, the same engine rendered 12. `rankPenalty` scales
|
||||
* `fileGraphScore`, which moves the relevance gate (6% of max) and so reshuffles
|
||||
* the admitted set — a demotion of one file must not cost an unrelated
|
||||
* top-ranked file its source.
|
||||
*
|
||||
* Flipping `files.generated` on that one row holds the INDEX constant and
|
||||
* attributes any delta to the ranker alone (the CG-25 method).
|
||||
*/
|
||||
const DTS = 'types/worker-configuration.d.ts';
|
||||
const QUERY = 'queueMessage flushQueuedMessages';
|
||||
|
||||
it('renders the same named definitions with the .d.ts flagged and unflagged', async () => {
|
||||
const setGenerated = (value: number) => {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const db = (cg as any).db?.getDatabase?.() ?? (cg as any).db?.db;
|
||||
db.prepare('UPDATE files SET generated = ? WHERE path = ?').run(value, DTS);
|
||||
};
|
||||
const linesFor = async () => renderedLines(await explore(QUERY));
|
||||
|
||||
const flagged = await linesFor();
|
||||
setGenerated(0);
|
||||
try {
|
||||
const unflagged = await linesFor();
|
||||
for (const name of ['queueMessage', 'flushQueuedMessages']) {
|
||||
const line = defLineOf(name);
|
||||
expect(flagged.has(line), `${name} missing with the .d.ts FLAGGED`).toBe(true);
|
||||
expect(unflagged.has(line), `${name} missing with the .d.ts UNFLAGGED`).toBe(true);
|
||||
}
|
||||
// The guarantee is about the named defs, not byte equality — the penalty is
|
||||
// supposed to move bytes around. What it must never do is cost the
|
||||
// top-ranked file the source the agent asked for.
|
||||
expect(unflagged.size).toBeGreaterThan(0);
|
||||
} finally {
|
||||
setGenerated(1);
|
||||
}
|
||||
}, 180_000);
|
||||
});
|
||||
@@ -0,0 +1,39 @@
|
||||
# tail-render-ts — CG-38
|
||||
|
||||
An agent-named symbol sitting in the TAIL of a large file must render.
|
||||
|
||||
This mirrors the geometry of the reported file (a 1,414-line Svelte chat store)
|
||||
closely enough that the same two defects reproduce, and it is that geometry — not
|
||||
any individual line — that the fixture exists to hold:
|
||||
|
||||
| | line | why it matters |
|
||||
|---|---|---|
|
||||
| `QueuedMessage` (interface) | 70 | the DECOY. Same stem as the query token, near the top, cheap to render — it is what the broken build returned *instead of* the functions. |
|
||||
| `createSessionStore` (function) | 104–1417 | the ENVELOPE. Spans ~92% of the file, and `function` is deliberately **not** in `ENVELOPE_KINDS` (CG-27), so every symbol inside merges into ONE cluster that must then be shrunk and trimmed. |
|
||||
| `handleStreamMessage` | ~554 | a 290-line god-method in the middle, so the head of the file has plenty to spend the budget on. |
|
||||
| `queueMessage` | 1088 | TARGET. Past line 1,000. |
|
||||
| `removeQueuedMessage` | 1096 | TARGET. |
|
||||
| `flushQueuedMessages` | 1102 | TARGET. Past line 1,000. |
|
||||
|
||||
Two more pieces are load-bearing:
|
||||
|
||||
- **`queueMessage` never calls `flushQueuedMessages`** (both push to / drain the same
|
||||
array instead). That absence is what produced no call chain, no synthesized hop and
|
||||
no dispatch boundary — and so made `buildFlowFromNamedSymbols` throw the
|
||||
named-symbol identity away along with the narrative it had nothing to print.
|
||||
- **`types/worker-configuration.d.ts`** — 2,500 lines of generated Wrangler ambient
|
||||
types, carrying the `Generated by wrangler. DO NOT EDIT.` banner so the ranker flags
|
||||
and penalises it. It is what makes the fixture able to test the issue's
|
||||
index-dependence lead: a penalty on this file moves `maxGraph`, which moves the 6%
|
||||
relevance gate, which moves every other file's allowance — and must still not cost
|
||||
the top-ranked file the definitions the agent named.
|
||||
|
||||
`src/lib/session-store.ts` is machine-generated to hit those line numbers with real,
|
||||
extractable TypeScript. If you need to change it, change the geometry (the target
|
||||
line numbers, the closure span, the decoy's position) rather than editing individual
|
||||
lines — the fixture-shape assertions in
|
||||
`__tests__/explore-named-symbol-render.test.ts` will tell you if it has rotted.
|
||||
|
||||
Gate: `__tests__/explore-named-symbol-render.test.ts`.
|
||||
Probe: `node scripts/agent-eval/probe-named-symbol.mjs`.
|
||||
Numbers: `docs/benchmarks/explore-tail-render-cg38.md`.
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"name": "tail-render-fixture",
|
||||
"private": true,
|
||||
"version": "0.0.0",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import { createSessionStore } from '../lib/session-store';
|
||||
|
||||
/** The composer owns the textarea and decides send-vs-queue. */
|
||||
export function createComposer(endpoint: string) {
|
||||
const store = createSessionStore({
|
||||
getProjectId: () => 'demo',
|
||||
getEndpoint: () => endpoint,
|
||||
onError: () => {},
|
||||
});
|
||||
let draft = '';
|
||||
|
||||
function setDraft(next: string) {
|
||||
draft = next;
|
||||
}
|
||||
|
||||
function submit(streaming: boolean) {
|
||||
if (streaming) store.queueMessage(draft);
|
||||
else store.sendMessage(draft, [], []);
|
||||
draft = '';
|
||||
}
|
||||
|
||||
function onTurnEnd() {
|
||||
store.flushQueuedMessages();
|
||||
}
|
||||
|
||||
return { setDraft, submit, onTurnEnd, store };
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import type { AttachedFile, SelectedElementRef } from './session-store';
|
||||
|
||||
export interface BuiltMessage {
|
||||
id: string;
|
||||
text: string;
|
||||
attachments: number;
|
||||
}
|
||||
|
||||
/** Render the selected canvas elements as a fenced block above the prose. */
|
||||
export function renderElementBlock(elements: SelectedElementRef[]): string {
|
||||
if (elements.length === 0) return '';
|
||||
const lines = elements.map((e) => `- ${e.kind}: ${e.label} (${e.id})`);
|
||||
return ['```elements', ...lines, '```'].join('\n');
|
||||
}
|
||||
|
||||
export function formatStylesBlock(files: AttachedFile[]): string {
|
||||
return files.map((f) => `${f.path} (${f.mime}, ${f.bytes}b)`).join('\n');
|
||||
}
|
||||
|
||||
export function buildMessage(
|
||||
content: string,
|
||||
files: AttachedFile[],
|
||||
elements: SelectedElementRef[],
|
||||
): BuiltMessage {
|
||||
const block = renderElementBlock(elements);
|
||||
const styles = formatStylesBlock(files);
|
||||
return {
|
||||
id: `m-${content.length}-${files.length}`,
|
||||
text: [block, styles, content].filter(Boolean).join('\n\n'),
|
||||
attachments: files.length,
|
||||
};
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,27 @@
|
||||
export interface Socket {
|
||||
emit(event: string, payload: unknown): void;
|
||||
on(event: string, handler: (chunk: unknown) => void): void;
|
||||
close(): void;
|
||||
}
|
||||
|
||||
/** One socket per chat session, so two tabs never receive each other's chunks. */
|
||||
export function createDedicatedSocket(endpoint: string): Socket {
|
||||
const handlers = new Map<string, Array<(chunk: unknown) => void>>();
|
||||
return {
|
||||
emit(event, payload) {
|
||||
void endpoint;
|
||||
void event;
|
||||
void payload;
|
||||
},
|
||||
on(event, handler) {
|
||||
handlers.set(event, [...(handlers.get(event) ?? []), handler]);
|
||||
},
|
||||
close() {
|
||||
handlers.clear();
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function describeSocket(socket: Socket | null): string {
|
||||
return socket ? 'connected' : 'detached';
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -52,13 +52,13 @@ change at all**.
|
||||
Deterministic across the 6-repo suite: no repo truncates, none loses a file,
|
||||
okhttp gains one, every repo lands at or under the 25,000 hard ceiling.
|
||||
|
||||
## Open
|
||||
## Follow-up — CG-38 (closed)
|
||||
|
||||
**CG-38** — agent-named symbols in the tail of a large file never render. On the
|
||||
**Agent-named symbols in the tail of a large file never rendered.** On the
|
||||
motivating repo, `queueMessage` (line 1087) and `flushQueuedMessages` (1102) in a
|
||||
1,414-line file are absent from the response on both prose and symbol-bag
|
||||
queries, even when that file wins rank #1 with 67% of the envelope. The response
|
||||
returns the `QueuedMessage` *interface* at line 70 — a fuzzy near-match on the
|
||||
1,414-line file were absent from the response on both prose and symbol-bag
|
||||
queries, even when that file won rank #1 with 67% of the envelope. The response
|
||||
returned the `QueuedMessage` *interface* at line 70 — a fuzzy near-match on the
|
||||
query token — instead of the function.
|
||||
|
||||
**Pre-existing, not caused by this epic.** A controlled bisect (index held fixed,
|
||||
@@ -67,15 +67,20 @@ lines here and CG-36 rendering 463; the symbols render at neither. The epic
|
||||
strictly improves the case. An earlier claim that the epic regressed it was
|
||||
wrong — it compared runs across two different indexes.
|
||||
|
||||
Sharpest lead: an earlier index of the same repo with the `.d.ts` **not** flagged
|
||||
generated rendered 581 lines including both symbols on the pre-epic engine, where
|
||||
the current flagged index renders 12. A penalty on one file should not shrink an
|
||||
unrelated top-ranked file's render; `rankPenalty` scales `fileGraphScore`, which
|
||||
moves the relevance gate and reshuffles the admitted set.
|
||||
Two independent causes, both longstanding: `buildFlowFromNamedSymbols` discarded
|
||||
the named-symbol IDENTITY along with the narrative whenever the named symbols did
|
||||
not form a call chain, so the importance-9 injection never ran; and the ceiling
|
||||
trim cut in SOURCE order, so a named def at the end of a large file was always the
|
||||
first thing dropped. Full account, plus the ranker-penalty lead (real, and
|
||||
orthogonal — the defs are absent at both `generated` flag states on the old build
|
||||
and present at both on the new one): `explore-tail-render-cg38.md`.
|
||||
|
||||
This epic's probes measure envelope share, starvation, source totals and file
|
||||
counts. **None measures "did the agent-named symbol render"** — which is why this
|
||||
survived the whole epic. CG-38 requires the fixture that closes that gap.
|
||||
counts. **None measured "did the agent-named symbol render"** — which is why this
|
||||
survived the whole epic. `scripts/agent-eval/probe-named-symbol.mjs`,
|
||||
`__tests__/fixtures/tail-render-ts` and
|
||||
`__tests__/explore-named-symbol-render.test.ts` close that gap: per-symbol and
|
||||
binary, checking the definition LINE against the response's rendered lines.
|
||||
|
||||
CG-36's own measurement is worth carrying forward, because the issue named the wrong
|
||||
fix point: both real cases (`query.py`, `RealInterceptorChain.kt`) lost on
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
# CG-38 — an agent-named symbol in the tail of a large file never rendered
|
||||
|
||||
**Status: fixed.** Two independent causes, both longstanding. Not a CG-24 regression —
|
||||
the controlled bisect (index held fixed, engine varied across every epic merge point)
|
||||
found the symptom at every build including pre-epic.
|
||||
|
||||
## The report
|
||||
|
||||
On a 1,414-line Svelte store, `codegraph_explore` never returned `queueMessage`
|
||||
(L1087) or `flushQueuedMessages` (L1102) — on a bare symbol bag *or* a prose
|
||||
question — even though their file won rank #1 with score 127 and 67.3% of the
|
||||
envelope. What came back instead was the same-stem `QueuedMessage` **interface** at
|
||||
L70. The agent had to Read the file to find the two functions it had asked for by
|
||||
name, which is the one outcome explore exists to prevent.
|
||||
|
||||
CLAUDE.md's *"guarantee named symbols render"* — the importance-9 named-def
|
||||
injection — was not holding.
|
||||
|
||||
## What it actually was
|
||||
|
||||
### 1. The named-symbol IDENTITY was discarded with the narrative
|
||||
|
||||
`buildFlowFromNamedSymbols` returns two unrelated things: the Flow prose, and the
|
||||
SET of node ids the agent named. Downstream, that set is what injects a named def
|
||||
into its file's cluster ranges and ranks it **importance 9** — the entire mechanism
|
||||
behind the guarantee.
|
||||
|
||||
Its last gate was:
|
||||
|
||||
```ts
|
||||
if (!hasMain && synthLines.length === 0 && !boundaryText && !polyText) return EMPTY;
|
||||
```
|
||||
|
||||
`EMPTY` zeroes `namedNodeIds` too. So whenever the named symbols happened not to
|
||||
produce anything to *print*, the guarantee silently switched off. Two sibling
|
||||
closures in one factory are exactly that case: `queueMessage` and
|
||||
`flushQueuedMessages` never call each other, so there is no chain, no synthesized
|
||||
hop and no dispatch boundary — and both defs lost importance 9. The file then
|
||||
rendered from its head, which is how a 6-line interface displaced two functions
|
||||
1,000 lines below it.
|
||||
|
||||
Measured: `flow.namedNodeIds` was **empty** on the reported query, while
|
||||
`findAllSymbols` resolved both tokens to exactly 1 node each.
|
||||
|
||||
The fix separates the two outputs (`identityOnly()`), restricted to **shape-precise
|
||||
tokens** (camelCase / PascalCase / snake_case / qualified — the same test the gather
|
||||
path uses). With a narrative present, the prose is itself corroboration and that path
|
||||
is unchanged; with nothing corroborating it, only an unambiguous symbol reference may
|
||||
promote, so an English word in a prose question that happens to exact-match a
|
||||
callable cannot earn importance 9.
|
||||
|
||||
### 2. The ceiling trim cut in SOURCE ORDER
|
||||
|
||||
Restoring importance 9 was not enough — the symbols still did not render.
|
||||
|
||||
`shrinkCluster` *had* kept them: on the reported query it emitted a block spanning
|
||||
`1022-1121`, which covers both. But the shrink's output measured **26,297 chars
|
||||
against a 16,532 cap**, so `windowToCeiling` fired, and it fills parts in source
|
||||
order and drops everything after the first overrun:
|
||||
|
||||
```
|
||||
shrunk: 101-107, 197-226, ..., 648-989, 1022-1121 (26,297)
|
||||
windowed: 101-107, 197-226, ..., 648-839 (16,532) ← tail gone
|
||||
```
|
||||
|
||||
A trim that cuts in source order will always take the END of a large file first —
|
||||
which is precisely where an agent-named symbol is most likely to be, and least
|
||||
likely to be reachable any other way. `windowToCeiling` already had the concept it
|
||||
needed (`focusLine`, for the spine's next-hop call site, CG-30); it just wasn't told
|
||||
about named defs. It now takes a `focusLines` list — spine call site plus every
|
||||
member at importance ≥ 9, capped at 6 — and:
|
||||
|
||||
- tries the **full-ceiling** fill FIRST, holding back 40% only when a focus line is
|
||||
actually left uncovered (so a cluster whose head already reaches its focus keeps
|
||||
the whole ceiling for source — an improvement on the old unconditional hold-back);
|
||||
- **splits** the reserve evenly between the uncovered focus lines with carry-forward,
|
||||
rather than handing it out greedily in source order. Greedy reproduced the bug one
|
||||
level down: on the prose query, four focus lines resolved and the two earliest took
|
||||
the entire reserve, dropping `flushQueuedMessages` again.
|
||||
|
||||
## The accounting gap — found, measured, deliberately NOT shipped
|
||||
|
||||
`shrinkCluster`'s fit test uses the raw source span
|
||||
(`slice().join('\n').length`) while the render adds `contextPadding` around every
|
||||
block and a line-number prefix to every line. On the reported file that estimate ran
|
||||
**~60% under** (16.5K accounted, 26.3K rendered).
|
||||
|
||||
An exact projection (prefix-summed line costs, mirroring the merge + padding
|
||||
`buildSection` performs) was built and measured. **It is worse, and it is not
|
||||
shipped:**
|
||||
|
||||
| | main | exact accounting | exact + spend-the-remainder |
|
||||
|---|---|---|---|
|
||||
| django | 20,719 | 20,747 | 20,747 |
|
||||
| excalidraw | 19,704 | **19,606** | **19,606** |
|
||||
| okhttp | 18,651 | 18,766 | 18,766 |
|
||||
| tokio | 21,582 | **21,424** | 21,555 |
|
||||
| gin | 11,952 | 12,082 | 12,082 |
|
||||
| alamofire | 11,849 | 11,849 | 11,849 |
|
||||
| `probe-allocation` | 4 PASS | **payroll-go FAIL** | **payroll-go FAIL** |
|
||||
|
||||
The mechanism: exact accounting stops at the last member that fits **whole**, and the
|
||||
released bytes carry forward to lower-ranked files. On `payroll-go` that moved 1,296
|
||||
chars out of the rank-#2 answer file `cycle.go` and into the rank-#5
|
||||
`payslipstore/store.go`, taking `runPayrollCycleAll`'s `s.store.Upsert(ctx, slip)`
|
||||
call — the "create" half of the query — with it.
|
||||
|
||||
So the slack is doing no harm where it is: `bound()` clamps the render to the ceiling
|
||||
exactly, so the over-keep costs no bytes. What the slack must **not** do is decide
|
||||
*which* members survive — and that is the ceiling trim's job, which is what this task
|
||||
fixed. The comment on `shrinkCluster` now says so, so the next reader does not
|
||||
"fix" it.
|
||||
|
||||
## The index-dependence lead — explained, and orthogonal
|
||||
|
||||
The issue's sharpest lead was that flagging the ambient `.d.ts` as `generated` seemed
|
||||
to make an unrelated file's render *worse*. Flipping `files.generated` on that one row
|
||||
(the CG-25 method — holds the index constant, attributes the delta to the ranker
|
||||
alone) confirms the mechanism is real:
|
||||
|
||||
| | `generated=1` | `generated=0` |
|
||||
|---|---|---|
|
||||
| `.d.ts` graphScore | 0.1875 | 0.75 |
|
||||
| `maxGraph` | 0.3297 | 0.75 |
|
||||
| gate (6% of max) | 0.0198 | 0.0450 |
|
||||
| files ranked | 3 | 2 |
|
||||
| rank-#1 allowance | 9,100 | 8,166 |
|
||||
|
||||
`rankPenalty` scales `fileGraphScore`, `fileGraphScore` sets `maxGraph`, and the
|
||||
relevance gate is 6% of `maxGraph` — so a penalty on one file does move the admitted
|
||||
set and every other file's allowance. Confirmed.
|
||||
|
||||
But it is **not** what hid the symbols. On main they are absent at *both* flag states
|
||||
(render stops at L316 / L381); with the fix they are present at *both*. The
|
||||
allocation moves; the guarantee does not depend on it. Pinned by the last case in
|
||||
`__tests__/explore-named-symbol-render.test.ts`.
|
||||
|
||||
## Results
|
||||
|
||||
Real repro (`queueMessage` L1087 / `flushQueuedMessages` L1102), all shapes:
|
||||
|
||||
| query shape | main | fixed |
|
||||
|---|---|---|
|
||||
| symbol bag | absent | **both render** |
|
||||
| prose, symbols named | absent | **both render** |
|
||||
| symbols + decoy interface | absent | **both render** |
|
||||
| prose, no symbols named | absent | **both render** |
|
||||
|
||||
Fixture (`__tests__/fixtures/tail-render-ts`, 7 symbol checks over 3 query shapes):
|
||||
**7/7 fail on main, 7/7 pass** — deterministic over 4 consecutive runs per arm.
|
||||
|
||||
Standing bars, all held:
|
||||
|
||||
- `probe-allocation.mjs` — payroll-go / starved-cluster / dense-header / self-query all PASS
|
||||
- `probe-file-spend.mjs` — no starvation flags
|
||||
- `probe-suite-envelope.mjs` — **byte-identical to main on all six repos** (20,719 /
|
||||
19,704 / 18,651 / 21,582 / 11,952 / 11,849), same file counts
|
||||
- full suite green
|
||||
|
||||
The suite being byte-identical is the point: the focus windows only change what a
|
||||
render does once it has *already* overrun its ceiling, which none of the six suite
|
||||
queries does.
|
||||
|
||||
## Instruments
|
||||
|
||||
- `scripts/agent-eval/probe-named-symbol.mjs` — the measurement the epic lacked.
|
||||
Per-SYMBOL and binary: is the symbol's **definition line** among the response's
|
||||
rendered lines? The name alone proves nothing — it appears in the section header's
|
||||
symbol list and at call sites whether or not the body was sent, which is exactly how
|
||||
this hid through a whole epic of aggregate probes.
|
||||
- `__tests__/fixtures/tail-render-ts` — mirrors the reported file's geometry: decoy
|
||||
same-stem interface at L70, factory closure at L104 spanning ~92% of the file (so
|
||||
every symbol merges into ONE cluster), targets at L1088/L1096/L1102, plus a
|
||||
2,500-line generated `.d.ts` for the ranker to penalise. Generated by script; edit
|
||||
the geometry, not individual lines.
|
||||
- `__tests__/explore-named-symbol-render.test.ts` — the standing gate, including the
|
||||
fixture-shape assertions (if the fixture rots, the gate means nothing).
|
||||
|
||||
## Method note
|
||||
|
||||
A `git stash -- <path>` "baseline" reverts to **HEAD**, not to `main`. With a WIP
|
||||
commit on the branch that silently measures your own change against itself — it
|
||||
produced a clean "passes on main" here that was pure fiction. Use the file swap
|
||||
(`git show main:<path> > <path>`), as `.kommandr/memory/baseline-builds-use-fresh-file-swap`
|
||||
already says for builds.
|
||||
Executable
+163
@@ -0,0 +1,163 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* "Did the symbol the agent NAMED actually render?" (CG-38).
|
||||
*
|
||||
* This is the measurement the whole CG-24 epic was missing. Every other probe
|
||||
* here scores the response in AGGREGATE — `probe-suite-envelope.mjs` measures how
|
||||
* much source came back, `probe-file-spend.mjs` measures whether the bytes went
|
||||
* to the files that earned them, `probe-allocation.mjs` measures group shares.
|
||||
* All three are green on a response that returns 25K of source from the right
|
||||
* file and still omits the one function the agent asked for by name. That is
|
||||
* exactly what CG-38 was: `queueMessage` at L1087 of a 1,414-line file, whose
|
||||
* file won rank #1 with 67% of the envelope, never rendered — the agent got a
|
||||
* same-stem `QueuedMessage` INTERFACE at L70 instead and had to Read the file.
|
||||
*
|
||||
* So the assertion here is per-SYMBOL and binary: for each named symbol, does its
|
||||
* definition line appear in the rendered source? Nothing else can substitute —
|
||||
* not the file being present, not its share, not its byte count.
|
||||
*
|
||||
* Usage (needs a current `npm run build`):
|
||||
* node scripts/agent-eval/probe-named-symbol.mjs
|
||||
* node scripts/agent-eval/probe-named-symbol.mjs --verbose
|
||||
* # any indexed repo, ad hoc:
|
||||
* node scripts/agent-eval/probe-named-symbol.mjs <repo> "<query>" sym1 sym2
|
||||
*
|
||||
* Exit code is 1 when any expected symbol is missing, so this can gate.
|
||||
*/
|
||||
import { cpSync, mkdtempSync, rmSync, readFileSync, existsSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join, resolve, dirname } from 'node:path';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const REPO = resolve(HERE, '..', '..');
|
||||
|
||||
const load = async (rel) => import(pathToFileURL(resolve(REPO, rel)).href);
|
||||
const idxMod = await load('dist/index.js');
|
||||
const toolsMod = await load('dist/mcp/tools.js');
|
||||
const CodeGraph = idxMod.default?.default ?? idxMod.default ?? idxMod.CodeGraph;
|
||||
const { ToolHandler } = toolsMod;
|
||||
|
||||
/**
|
||||
* The fixture cases. `symbols` are what the agent names; each must come back with
|
||||
* its DEFINITION rendered. The queries deliberately cover both shapes the bug was
|
||||
* reported on — a bare symbol bag and a prose question — because the failure had
|
||||
* a different cause on each and a fix for one does not imply the other.
|
||||
*/
|
||||
const FIXTURE = '__tests__/fixtures/tail-render-ts';
|
||||
const CASES = [
|
||||
{
|
||||
id: 'tail-symbol-bag',
|
||||
why: 'two sibling closures past L1000, named directly; neither calls the other',
|
||||
query: 'queueMessage flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
id: 'tail-prose',
|
||||
why: 'same two symbols named inside a prose question',
|
||||
query: 'how does queueMessage hand its entries to flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
{
|
||||
id: 'tail-with-decoy',
|
||||
why: 'the same-stem QueuedMessage interface at L70 must not stand in for the functions',
|
||||
query: 'explain queueMessage, removeQueuedMessage and flushQueuedMessages',
|
||||
symbols: ['queueMessage', 'removeQueuedMessage', 'flushQueuedMessages'],
|
||||
},
|
||||
];
|
||||
|
||||
/** Every `<n>\t<text>` line number present in the response's source blocks. */
|
||||
function renderedLines(response) {
|
||||
const out = new Set();
|
||||
for (const m of response.matchAll(/^(\d+)\t/gm)) out.add(Number(m[1]));
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* A symbol counts as rendered only when its DECLARATION line is among the lines
|
||||
* the response actually sent — not when its name merely appears somewhere (it
|
||||
* shows up in the section header symbol list and in call sites regardless, which
|
||||
* is precisely how this defect hid for a whole epic).
|
||||
*/
|
||||
function check(cg, response, names) {
|
||||
const lines = renderedLines(response);
|
||||
return names.map((name) => {
|
||||
const node = (cg.getNodesByName?.(name) ?? []).find((n) => n.startLine > 0);
|
||||
return {
|
||||
name,
|
||||
file: node?.filePath ?? '(not indexed)',
|
||||
line: node?.startLine ?? 0,
|
||||
rendered: !!node && lines.has(node.startLine),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async function runCase(root, { query, symbols }) {
|
||||
const cg = CodeGraph.openSync(root);
|
||||
try {
|
||||
const res = await new ToolHandler(cg).execute('codegraph_explore', { query });
|
||||
const response = res.content?.[0]?.text ?? '';
|
||||
return { response, results: check(cg, response, symbols) };
|
||||
} finally {
|
||||
try { cg.close?.(); } catch { /* already closed */ }
|
||||
}
|
||||
}
|
||||
|
||||
const argv = process.argv.slice(2);
|
||||
const verbose = argv.includes('--verbose');
|
||||
const positional = argv.filter((a) => !a.startsWith('--'));
|
||||
|
||||
let failures = 0;
|
||||
let checked = 0;
|
||||
|
||||
if (positional.length >= 3) {
|
||||
// Ad-hoc mode: <repo> "<query>" sym...
|
||||
const [repo, query, ...symbols] = positional;
|
||||
const { response, results } = await runCase(resolve(repo), { query, symbols });
|
||||
console.log(`\n${repo}\n query "${query}" · ${response.length} chars\n`);
|
||||
for (const r of results) {
|
||||
checked += 1;
|
||||
if (!r.rendered) failures += 1;
|
||||
console.log(` ${r.rendered ? 'PASS' : 'FAIL'} ${r.name} ${r.file}:${r.line}`);
|
||||
}
|
||||
} else {
|
||||
const src = join(REPO, FIXTURE);
|
||||
if (!existsSync(src)) {
|
||||
console.error(`fixture missing: ${src}`);
|
||||
process.exit(2);
|
||||
}
|
||||
const dir = mkdtempSync(join(tmpdir(), 'cg-named-'));
|
||||
try {
|
||||
cpSync(src, dir, { recursive: true });
|
||||
rmSync(join(dir, '.codegraph'), { recursive: true, force: true });
|
||||
const cg = CodeGraph.initSync(dir);
|
||||
await cg.indexAll();
|
||||
cg.close?.();
|
||||
|
||||
console.log(`\ntail-render-ts · agent-named symbols must render\n`);
|
||||
for (const c of CASES) {
|
||||
const { response, results } = await runCase(dir, c);
|
||||
console.log(`── ${c.id} — ${c.why}`);
|
||||
console.log(` query "${c.query}"`);
|
||||
console.log(` response ${response.length.toLocaleString()} chars`);
|
||||
for (const r of results) {
|
||||
checked += 1;
|
||||
if (!r.rendered) failures += 1;
|
||||
console.log(` ${r.rendered ? 'PASS' : 'FAIL'} ${r.name} defined at ${r.file}:${r.line}`
|
||||
+ (r.rendered ? '' : ' — DEFINITION NOT IN RESPONSE'));
|
||||
}
|
||||
if (verbose && failures) {
|
||||
const spans = [...renderedLines(response)].sort((a, b) => a - b);
|
||||
console.log(` rendered lines: ${spans[0]}..${spans[spans.length - 1]} (${spans.length} lines)`);
|
||||
}
|
||||
console.log();
|
||||
}
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
console.log(failures === 0
|
||||
? `Every agent-named symbol rendered (${checked} checked).`
|
||||
: `${failures} of ${checked} agent-named symbols did NOT render.`);
|
||||
process.exit(failures === 0 ? 0 : 1);
|
||||
+138
-31
@@ -2542,6 +2542,14 @@ export class ToolHandler {
|
||||
// fed only to the dynamic-dispatch-links scan below.
|
||||
const dynNamed = new Map<string, Node>();
|
||||
const DYN_KINDS = new Set(['constant', 'variable', 'field', 'property']);
|
||||
// Nodes resolved from a SHAPE-PRECISE token (camelCase / PascalCase /
|
||||
// snake_case / qualified) — the same test the gather path uses. It is the
|
||||
// difference between "the agent named this symbol" and "an ordinary English
|
||||
// word in a prose question collided with a callable", and it is what makes
|
||||
// the narrative-less return below safe (see `identityOnly`).
|
||||
const isPreciseToken = (x: string) =>
|
||||
/[._$]|::|\//.test(x) || /[a-z][A-Z]/.test(x) || /^[A-Z]/.test(x);
|
||||
const preciseNamedIds = new Set<string>();
|
||||
const hasHeuristicEdge = (id: string): boolean =>
|
||||
[...cg.getCallers(id), ...cg.getCallees(id)].some(({ edge }) => edge.provenance === 'heuristic');
|
||||
for (const t of tokens) {
|
||||
@@ -2560,9 +2568,11 @@ export class ToolHandler {
|
||||
});
|
||||
const kept = pick.slice(0, 6);
|
||||
tokenNodes.set(t, kept.map((n) => n.id));
|
||||
const precise = isPreciseToken(t);
|
||||
for (const n of kept) {
|
||||
named.set(n.id, n);
|
||||
if (specific) uniqueNamedNodeIds.add(n.id);
|
||||
if (precise) preciseNamedIds.add(n.id);
|
||||
}
|
||||
// Same token, non-callable synth endpoints (capped, precision-gated on an
|
||||
// actual heuristic edge so plain config constants never qualify).
|
||||
@@ -2575,6 +2585,7 @@ export class ToolHandler {
|
||||
if (CALLABLE.has(n.kind) || !DYN_KINDS.has(n.kind) || dynNamed.has(n.id)) continue;
|
||||
if (hasHeuristicEdge(n.id)) {
|
||||
dynNamed.set(n.id, n);
|
||||
if (precise) preciseNamedIds.add(n.id);
|
||||
tokenDyn++;
|
||||
}
|
||||
if (dynNamed.size >= 12 || tokenDyn >= 4) break;
|
||||
@@ -2606,6 +2617,35 @@ export class ToolHandler {
|
||||
}
|
||||
return synthLines;
|
||||
};
|
||||
/**
|
||||
* No narrative to print — but the agent still NAMED symbols, and their
|
||||
* identity is a separate output from the prose (CG-38).
|
||||
*
|
||||
* `namedNodeIds` is not decoration: downstream it injects the named def into
|
||||
* the file's cluster ranges and ranks it importance 9, which is the whole
|
||||
* mechanism behind "a symbol the agent named renders" (the assembler's
|
||||
* named-def injection). Returning EMPTY here threw that away whenever the
|
||||
* named symbols happened not to form a call chain — two sibling closures in
|
||||
* one factory (`queueMessage` / `flushQueuedMessages`, neither calling the
|
||||
* other) produce no chain, no synth hop and no dispatch boundary, so BOTH
|
||||
* defs lost importance 9 and the file rendered from its head instead: the
|
||||
* agent got the `QueuedMessage` interface at L70 and had to Read the file
|
||||
* for the functions at L1087/L1102 it had asked for by name.
|
||||
*
|
||||
* Restricted to SHAPE-PRECISE tokens. With a narrative present the prose is
|
||||
* itself corroboration that the resolution was right, so that path keeps
|
||||
* every named id as before; with nothing corroborating it, only an
|
||||
* unambiguous symbol reference may promote — an English word in a prose
|
||||
* question that happens to exact-match a callable must not earn importance 9.
|
||||
* Same distinction, same test, as the gather path's `isPreciseToken`.
|
||||
*/
|
||||
const identityOnly = () => (preciseNamedIds.size === 0 ? EMPTY : {
|
||||
text: '',
|
||||
pathNodeIds: new Set<string>(),
|
||||
namedNodeIds: new Set<string>(preciseNamedIds),
|
||||
uniqueNamedNodeIds: new Set<string>([...uniqueNamedNodeIds].filter((id) => preciseNamedIds.has(id))),
|
||||
spineCallSites: new Map<string, number>(),
|
||||
});
|
||||
if (named.size < 2) {
|
||||
// <2 CALLABLES resolved. Two recoveries before giving up: (1) synthesized
|
||||
// edges among named CONSTANT/VARIABLE endpoints — RTK thunk→thunk is
|
||||
@@ -2614,7 +2654,7 @@ export class ToolHandler {
|
||||
// dynamic-dispatch site that EXPLAINS a half-connected flow.
|
||||
const synthLines = collectSynthLinks(null);
|
||||
const boundaries = named.size === 0 ? '' : (this.buildDynamicBoundaries(cg, [...named.values()], named) || '');
|
||||
if (synthLines.length === 0 && !boundaries) return EMPTY;
|
||||
if (synthLines.length === 0 && !boundaries) return identityOnly();
|
||||
const out: string[] = [];
|
||||
if (synthLines.length) out.push(
|
||||
'**Dynamic-dispatch links among your symbols**',
|
||||
@@ -2729,7 +2769,7 @@ export class ToolHandler {
|
||||
hasMain ? (e: Edge) => pathIds.has(e.source) && pathIds.has(e.target) : null
|
||||
);
|
||||
|
||||
if (!hasMain && synthLines.length === 0 && !boundaryText && !polyText) return EMPTY;
|
||||
if (!hasMain && synthLines.length === 0 && !boundaryText && !polyText) return identityOnly();
|
||||
const out: string[] = [];
|
||||
if (hasMain) {
|
||||
out.push('**Flow (call path among the symbols you queried)**', '');
|
||||
@@ -4996,6 +5036,19 @@ export class ToolHandler {
|
||||
* keeps every rule that matters: only whole symbol ranges are emitted, so a
|
||||
* body is never cut, and the members are chosen by the same importance the
|
||||
* cluster ranking uses. Returns null when nothing needed shrinking.
|
||||
*
|
||||
* `sizeOf` measures the RAW source span, while the render adds
|
||||
* `contextPadding` around every block and a line-number prefix to every
|
||||
* line — so this over-keeps (measured ~60% under on a 1,414-line file:
|
||||
* 16.5K accounted, 26.3K rendered). That is deliberate, not an oversight:
|
||||
* `bound()` clamps the result to the ceiling exactly, so the slack costs no
|
||||
* bytes, and making the estimate exact instead measured WORSE — it stops at
|
||||
* the last member that fits whole, and the released bytes carry forward to
|
||||
* lower-ranked files (payroll-go's `runPayrollCycleAll` body lost its
|
||||
* `s.store.Upsert` call to a rank-5 file). What the slack must NOT do is
|
||||
* decide WHICH members survive: that is the ceiling trim's job, and CG-38 is
|
||||
* why that trim now protects the named spans instead of cutting in source
|
||||
* order. See `docs/benchmarks/explore-tail-render-cg38.md`.
|
||||
*/
|
||||
const shrinkCluster = (c: ExploreCluster, cap: number): SectionPart[] | null => {
|
||||
if (c.members.length < 2) return null;
|
||||
@@ -5088,45 +5141,81 @@ export class ToolHandler {
|
||||
* it or re-send it. Below that floor the part is simply dropped — unless
|
||||
* nothing has been emitted at all, where the floor wins over the ceiling
|
||||
* because an empty section is the one outcome worse than an oversize one.
|
||||
*
|
||||
* `focusLines` are the lines this trim must not lose: the spine's next-hop
|
||||
* call site (CG-30) and every definition the agent NAMED inside the cluster
|
||||
* (CG-38). The head fill is source-ordered, so a named def in the TAIL of a
|
||||
* large file is otherwise always the first thing an over-ceiling render
|
||||
* drops — the one span the agent asked for by name, cut in favour of
|
||||
* head-of-file filler it did not ask for. The full-ceiling fill is tried
|
||||
* FIRST and the 60% hold-back applies only when a focus line is actually
|
||||
* left uncovered, so a cluster whose head already reaches its focus keeps
|
||||
* the whole ceiling for source.
|
||||
*/
|
||||
const windowToCeiling = (
|
||||
parts: ReadonlyArray<SectionPart>,
|
||||
ceiling: number,
|
||||
focusLine?: number,
|
||||
focusLines: ReadonlyArray<number> = [],
|
||||
): SectionPart[] => {
|
||||
const emit: ExploreLineRange[] = [];
|
||||
const inParts = (line: number) =>
|
||||
parts.some((p) => line >= p.range.start && line <= p.range.end);
|
||||
const needFocus = typeof focusLine === 'number' && focusLine > 0 && inParts(focusLine);
|
||||
// Hold room back for the call site so the head window can't eat all of it.
|
||||
const headRoom = needFocus ? Math.floor(ceiling * 0.6) : ceiling;
|
||||
let used = 0;
|
||||
for (const p of parts) {
|
||||
const join = emit.length > 0 ? GAP_MARKER.length : 0;
|
||||
if (used + join + p.text.length <= headRoom) {
|
||||
emit.push(p.range);
|
||||
used += join + p.text.length;
|
||||
continue;
|
||||
const focus = [...new Set(focusLines)]
|
||||
.filter((l) => typeof l === 'number' && l > 0 && inParts(l))
|
||||
.sort((a, b) => a - b);
|
||||
/** Source-ordered fill of whole parts, the overrunning one cut to a head window. */
|
||||
const fill = (room: number): { emit: ExploreLineRange[]; used: number } => {
|
||||
const emit: ExploreLineRange[] = [];
|
||||
let used = 0;
|
||||
for (const p of parts) {
|
||||
const join = emit.length > 0 ? GAP_MARKER.length : 0;
|
||||
if (used + join + p.text.length <= room) {
|
||||
emit.push(p.range);
|
||||
used += join + p.text.length;
|
||||
continue;
|
||||
}
|
||||
const first = emit.length === 0;
|
||||
const win = headWindowOf(
|
||||
p.range, Math.max(0, room - used - join), first ? MIN_WINDOW_LINES : 0);
|
||||
if (win && (first || win.end - win.start + 1 >= MIN_WINDOW_LINES)) {
|
||||
emit.push(win);
|
||||
used += join + renderSpan(win).length;
|
||||
}
|
||||
break;
|
||||
}
|
||||
const first = emit.length === 0;
|
||||
const win = headWindowOf(
|
||||
p.range, Math.max(0, headRoom - used - join), first ? MIN_WINDOW_LINES : 0);
|
||||
if (win && (first || win.end - win.start + 1 >= MIN_WINDOW_LINES)) {
|
||||
emit.push(win);
|
||||
used += join + renderSpan(win).length;
|
||||
}
|
||||
break;
|
||||
return { emit, used };
|
||||
};
|
||||
let { emit, used } = fill(ceiling);
|
||||
const reached = () => (emit.length ? emit[emit.length - 1]!.end : 0);
|
||||
if (focus.some((l) => l > reached())) {
|
||||
// Hold room back for the focus windows so the head can't eat all of it.
|
||||
({ emit, used } = fill(Math.floor(ceiling * 0.6)));
|
||||
}
|
||||
const last = emit[emit.length - 1];
|
||||
if (needFocus && (!last || focusLine! > last.end)) {
|
||||
const host = parts.find((p) => focusLine! >= p.range.start && focusLine! <= p.range.end)!;
|
||||
const lo = Math.max(host.range.start, focusLine! - SPINE_WINDOW, last ? last.end + 1 : 0);
|
||||
const hi = Math.min(host.range.end, focusLine! + SPINE_WINDOW);
|
||||
const win = centeredWindowOf(
|
||||
focusLine!, lo, hi, Math.max(0, ceiling - used - GAP_MARKER.length));
|
||||
// What is left is SPLIT between the uncovered focus lines rather than
|
||||
// handed to them in order. Greedy-in-source-order reproduces the very bug
|
||||
// this guards: on a prose query resolving four focus lines, the two
|
||||
// earliest took the whole reserve and `flushQueuedMessages` at L1102 —
|
||||
// named in the question — was dropped again. A skipped or undersized
|
||||
// window returns its share to the pool for the ones after it.
|
||||
let covered = reached();
|
||||
let room = Math.max(0, ceiling - used);
|
||||
const pending = focus.filter((l) => l > covered);
|
||||
for (let i = 0; i < pending.length; i++) {
|
||||
const line = pending[i]!;
|
||||
if (line <= covered) continue; // an earlier window already reached it
|
||||
const share = Math.floor(room / (pending.length - i)) - GAP_MARKER.length;
|
||||
if (share <= 0) continue;
|
||||
const host = parts.find((p) => line >= p.range.start && line <= p.range.end)!;
|
||||
const lo = Math.max(host.range.start, line - SPINE_WINDOW, covered + 1);
|
||||
const hi = Math.min(host.range.end, line + SPINE_WINDOW);
|
||||
const win = centeredWindowOf(line, lo, hi, share);
|
||||
// Same sliver floor as the head window — a two-line peek at the call
|
||||
// site teaches the next call's dedup to shred the block around it.
|
||||
if (win && win.end - win.start + 1 >= MIN_WINDOW_LINES) emit.push(win);
|
||||
if (!win || win.end - win.start + 1 < MIN_WINDOW_LINES) continue;
|
||||
emit.push(win);
|
||||
const cost = GAP_MARKER.length + renderSpan(win).length;
|
||||
used += cost;
|
||||
room -= cost;
|
||||
covered = win.end;
|
||||
}
|
||||
// Never empty: a section with no source sends the agent to Read.
|
||||
if (emit.length === 0 && parts.length > 0) {
|
||||
@@ -5138,6 +5227,24 @@ export class ToolHandler {
|
||||
.map((r) => ({ range: r, text: renderSpan(r) }));
|
||||
};
|
||||
|
||||
/**
|
||||
* The lines a ceiling trim of this cluster must not lose: the spine's
|
||||
* next-hop call site, and the definition line of every member the agent
|
||||
* NAMED or that is a query entry point (importance >= 9). Capped, because
|
||||
* each one costs a window and too many turn a section into confetti; the
|
||||
* most important come first, source order within a tier so the windows read
|
||||
* top-down.
|
||||
*/
|
||||
const MAX_FOCUS_LINES = 6;
|
||||
const focusLinesOf = (c: ExploreCluster): number[] => {
|
||||
const named = c.members
|
||||
.filter((m) => m.importance >= 9)
|
||||
.sort((a, b) => b.importance - a.importance || a.start - b.start)
|
||||
.slice(0, MAX_FOCUS_LINES)
|
||||
.map((m) => m.start);
|
||||
return c.spineCallLine ? [c.spineCallLine, ...named] : named;
|
||||
};
|
||||
|
||||
/**
|
||||
* One cluster's final parts: built, shrunk if it overruns `cap`, then
|
||||
* passed through the session history (CG-18).
|
||||
@@ -5165,7 +5272,7 @@ export class ToolHandler {
|
||||
if (!Number.isFinite(ceiling) || sectionText(r.parts).length <= ceiling) return r;
|
||||
// Windows are subsets of spans dedupeSpans already cleared, so the record
|
||||
// still only ever claims source that was actually sent.
|
||||
const parts = windowToCeiling(r.parts, ceiling, c.spineCallLine);
|
||||
const parts = windowToCeiling(r.parts, ceiling, focusLinesOf(c));
|
||||
return { parts, covered: r.covered, shrunk: true };
|
||||
};
|
||||
if (sectionText(base.parts).length <= cap) {
|
||||
|
||||
Reference in New Issue
Block a user