Files
colbymchenry--codegraph/scripts
Colby McHenry db3b8d2a1e test(agent-eval): report what share of explore's bytes the answer used (CG-9)
The envelope view needed a human to say which files answer the question
(`--answer <glob>`). This reads it off the agent's own final answer and
reports one number per run and per call: bytes returned for files the
answer cited, over all bytes returned. That is the #1500 defect as a
number instead of a hunch.

Attribution has two channels, ranked so the weaker one stays separable:
the answer naming the file (reported alone as the conservative floor),
and the answer citing, in a code span, a symbol only that file DEFINES.
Three guards keep the error from leaning optimistic — the direction a
tuning metric must not lean:

  * Only symbols the file defines. Section headers render `name(kind)`
    for call sites too, and crediting those marked excalidraw's
    dragElements.ts used because the answer named `mutateElement`.
  * A definition beats an import alias of the same name (`variable`),
    or `lib/application.js` gets credit for `require('./utils')`.
  * A name on 3+ returned files identifies none of them.

Bare basenames count as citations (agents write `utils.js:225` in prose)
but only for extensions the envelope shipped, so `res.send` and
`mime.contentType` — the same token shape — do not read as files.

Both the envelope view and this share one parse of the rendered markdown
(parseExploreCall), still not the CG-4 sidecar: the sidecar exists only
on a post-CG-4 build and so cannot measure a baseline arm.
2026-08-05 00:42:59 -05:00
..