Files
Colby McHenry 4f8782cbe5 test(agent-eval): add output-style A/B harness, cost/token analyzer, and DISALLOW/REP_START controls
Three additions to tighten the eval loop:

- offload-eval-styles.sh: new 4-arm eval (raw/refs/map/src) isolating the Worker's
  output shape's effect on main-session tokens, latency, and accuracy. Delegation
  blocked by default (DISALLOW=Agent) so variance from Haiku subagent spawning doesn't
  contaminate the measurement.
- offload-eval-cost.mjs: cost/token analyzer that reads Claude Code's own per-model
  accounting (modelUsage.costUSD) rather than re-deriving from raw token counts,
  giving a correct main(Sonnet)/sub(Haiku) split with proper per-tier pricing.
- offload-eval-3arm.sh: adds DISALLOW env to block sub-agent delegation across all
  arms, and REP_START to append reps to an existing run without clobbering earlier
  jsonls (e.g. REP_START=4 REPS=3 → reps 4,5,6).

Also adds CODEGRAPH_OFFLOAD_STYLE forwarding to the managed gateway so the styles
eval can drive output shape end-to-end; the field is stripped before the upstream
model call and never sent to BYO endpoints.
2026-06-19 16:43:10 -05:00
..