colbymchenry--codegraph/docs/benchmarks/explore-displacement-guard-ab-cg31.md

Agent A/B — cluster-path displacement guard (task CG-31)

Date: 2026-08-06 · New: bugfix/CG-31 · Baseline: bugfix/CG-30 @ 0d014a6 · Harness: scripts/agent-eval/ab-new-vs-baseline.sh, --model sonnet --effort high, both arms codegraph-on, CLI blocked (0 contamination in every run), CODEGRAPH_NO_PROMPT_HOOK=1. Every index measured on was fully rebuilt, never incrementally synced (CG-33).

Baseline is the CG-30 tip, not main, so every number here isolates CG-31. CG-30's own A/B against main is explore-oversize-member-ab-cg30.md; read them in sequence for the combined picture the two issues asked for.

CG-31 stops a clustered render from spending a reservation still owed to a file the render loop has not reached. The whole-file BUY arm has always refused that trade (owedBelow); the cluster path read what was left before the hard ceiling instead of what was still promised.

Verdict: no regression, and this one is a straight win on both halves. Four of six suite repos deliver MORE source and one more file each; the other two are byte-identical. The agent runs are faster in all three repos measured, with Read at or below baseline.

---

The two corrections the measurement forced

Worth recording, because the first version of the guard was wrong in the direction the guard itself is about, and only a suite measurement showed it.

1. Holding back the full owed sum was too much. The allocator splits the envelope; the render loop spends against a ceiling that also has to hold the response's own prose, so on a saturated response the promises are over-subscribed and the tail is going to be dropped whatever happens above it. Bytes held for a file that is then dropped are bytes nobody receives. Measured: django −2,319 source, tokio −1,298, both handed to a section the ceiling threw away. owedPayableBelow now holds back only the prefix of what is owed below that the response can still pay, in rank order.

2. The final truncation was eating the guard's work. It cut at the last file-section header, which drops that whole section and the trailing notes. Dropping the notes alone is almost always enough. The epilogue is a pointer list and two reminders; a section is source the agent otherwise has to Read. Cutting the epilogue first is what turned the remaining deficits into gains — and it is the same starvation CG-31 is about, arriving one layer below the guard.

A third, smaller fix: flow.text is prepended to lines to make the final output but was never counted in totalChars, so the render loop spent against a ceiling it was ~2K under on symbol-bag queries.

Deterministic measurement — the primary evidence

Same clean-rebuilt index, same query, both builds. One codegraph_explore per repo.

repo	base source	new source	Δ	base files	new files
django	20,033	20,791	+758	5 (truncated)	6
excalidraw	18,776	20,204	+1,428	7 (truncated)	8
okhttp	15,628	19,034	+3,406	4 (truncated)	5
tokio	20,340	21,521	+1,181	4 (truncated)	5
gin	10,776	10,776	0	4	4
alamofire	11,662	11,662	0	2	2

Queries: django "How does a QuerySet turn into SQL and fetch rows from the database?"; excalidraw "How does updating an element re-render the canvas on screen?"; gin "How does a registered route handler get invoked for an incoming HTTP request?"; alamofire "How does a request get built and sent through the session?"; okhttp "How does a call go through the interceptor chain to the network?"; tokio "How does a spawned task get scheduled and run by a worker?".

No repo delivers less. Four of six stopped truncating, which is where the extra file comes from: each of those responses had been throwing a fully-rendered section away.

gin and alamofire are byte-identical between the builds — nothing in them is oversize enough for the guard to engage and neither response was truncated. That is what a control should show, and it means every gin number in the agent table below is run-to-run variance.

Fixture — __tests__/fixtures/displacement-ts, four pipeline stages competing for one envelope, the first a single ~20K function. Padded past 500 indexed files on purpose: the displacement only exists on the 24K tier, where the reservations plus the preamble genuinely saturate the render ceiling.

	baseline	new
ingest.ts	9,301 chars on a 6,289 spendable, then dropped whole by the ceiling — 0 delivered	4,851, bounded
types.ts / sink.ts	skipped budget-whole-file	delivered
admitted files delivered	3 of 6	6 of 6
envelope	14,908	22,066

Pinned by __tests__/explore-displacement-guard.test.ts (11 tests; 3 fail on the baseline).

Allocation fixtures — scripts/agent-eval/allocation-fixtures.json flips back to BOTH PASS. Its afterCG30 verdict blamed an over-RESERVED incidental file; the reservation is identical in both arms — the file was over-SPENDING, which is exactly this defect. Recorded honestly in afterCG31.

Agent runs

	django new	django base	okhttp new	okhttp base	gin new	gin base
runs	3	3	2	2	2	2
duration (s)	36 [33–51]	46 [35–49]	42 [40–44]	52 [50–53]	33 [31–35]	39
tool calls	3 [3–4]	3 [3–4]	4 [3–4]	5 [4–5]	4 [3–4]	5 [4–5]
Read	0 [0–1]	0	0	1 [0–2]	1 [0–1]	1 [0–2]
Grep/Glob	0	0	0	0	0	0
codegraph calls	2	2 [2–3]	3 [2–3]	3 [2–3]	2	3 [2–3]
occupancy share	32.1% [31.1%–36.1%]	33.8% [28.7%–42.1%]	40.0% [36.6%–43.3%]	40.8% [40.3%–41.4%]	29.7% [28.3%–31.1%]	33.5% [29.8%–37.2%]
allocation efficiency	96.8%	98.9%	88.2%	97.2%	91.2%	85.0%

Prompts are the deterministic queries above with "Trace the flow end to end." appended.

Sufficiency, pooled per call. okhttp: 0 "Read a file we returned" in 5 against the baseline's 1 in 5 — the arm that returns 3,406 more chars needs fewer follow-up Reads, which is the mechanism working. django: 1 in 6 against 0 in 7. gin: 1 in 4 against 1 in 5, on a repo where the builds emit identical bytes. Neither arm produced a single recall miss (a Read of a file we did NOT return, or a Grep) on any repo.

Where the new arm looks worse, and why it is not read as a regression:

• okhttp allocation efficiency, 88.2% vs 97.2%. The new arm's envelope is 85,197 chars against the baseline's 66,014 — it returns substantially more source, and the metric is the share of returned bytes the answer cited. A larger, more complete response with a smaller cited share and Read driven to 0 is the trade this tool exists to make. The metric's own documentation says it is relative and must not be read as waste.
• django, 1 allocation miss in 6 answered calls against 0 in 7. One run, n=3, and django is the repo whose duration range overlaps most (33–51 vs 35–49).

Residual carried forward — for CG-26

Four repos stopped truncating; okhttp, django, excalidraw and tokio now land at 24,758–24,998 chars against a 25,000 hard ceiling. That is deliberate (the ceiling exists so the host never externalizes the result) but it means the render loop's 600-char margin for the epilogue is still wrong — the epilogue measures 1,064 (gin), 1,788 (django), 2,231 (excalidraw). The response now survives that by dropping the epilogue rather than a section, which is strictly better, but the honest fix is for the loop to budget for the epilogue in the first place.

A margin sweep was run and deliberately not shipped: at 1,200 django stops truncating on its own but tokio loses 286 chars; at 2,400 django loses 1,895. Tuning one constant against the suite is the trap CG-30's own record warns about. Sizing the margin from the epilogue the response is actually going to emit is the real fix and belongs with the end-to-end reservation invariant.

Second residual: the whole-file BUY arm's fit test (totalChars + fileContent.length + FILE_OVERHEAD <= renderCeiling) has no owedBelow term of its own — its displacement guard is source-space only. It was left alone here to keep this change attributable; the epilogue-first cut removes the failure mode it would have caused.