feat(beacon): allowlist the routing summary key (#2818)
One line in the receiver's allowlist. No client change; the proxy's own
payload is untouched.
## Why
A routing extension sees things the proxy alone cannot, and they are all
measurements rather than opinions:
- **Empirical `min_cacheable` per provider.** Fireworks, Together and
DeepInfra publish no minimum and litellm carries no value for them, so a
router has to guess. But the number is directly observable — send prefix
length L, see whether the repeat reports cached tokens. Across enough
installs the step function falls out.
- **TTL survival.** Currently modelled as a constant.
- **Conversation length distribution.** The horizon is the only free
parameter in a cache-aware cost model, and it decides the answer: at a
900-token prefix, 1 remaining turn and 20 remaining turns route to
different models.
- **Predicted vs actual cache hits.** Every response carries
`cache_read_input_tokens`. Comparing it to what was predicted is the
only way to find out when the cost model is lying.
## What lands here
`'routing'` added to `ALLOWED_KEYS`, and the comment above the list
corrected — it claimed the set mirrors `_Session.payload()`, which is no
longer the whole story now that an extension can emit its own event
carrying one of these keys.
The ordering constraint is the reason this is its own PR: **allowlisting
is a write-side gate**, so anything sent before the key exists is
dropped and unrecoverable. This has to be deployed before any client
starts emitting it, not alongside.
## Shape of the block
Same rule as every other key — counters and model ids, no free text:
```json
"routing": {
"harness": "claude-code",
"decisions": 47, "would_change": 12, "enforced": 9, "holdout": 3,
"at_free_boundary": 4, "cross_protocol": 0,
"picked": {"claude-haiku-4-5": 12, "claude-opus-5": 35},
"requested": {"claude-opus-5": 47},
"mean_prefix_tokens": 7514,
"measured_cost": 0.0236, "modelled_cost": 0.0376,
"cache_read_tokens": 3200, "cache_write_tokens": 0,
"predicted_hits": 4, "actual_hits": 4
}
```
`measured_cost` comes from the provider's own usage; `modelled_cost`
from the router's cost function. They stay separate because the
difference is the only thing that means anything.
The extension's `reason` string is deliberately absent. It is
code-generated, so it carries no user content, but it is unbounded — it
stays out rather than being reasoned about.
`holdout` is the count of turns deliberately left unrouted as a control.
Without it the rest is observational: once a router is acting on every
request, the corpus is entirely that router's own policy.
`sample-event.json` is unchanged on purpose — it mirrors
`_Session.payload()`, which does not produce this key, and adding it
there would suggest the proxy emits it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+13
-3
@@ -28,9 +28,12 @@
|
||||
* deanonymise install_id, so it is never read.
|
||||
*/
|
||||
|
||||
// Mirrors the payload built by _Session.payload(). A key absent here is
|
||||
// dropped, not stored. Adding a metric means adding it here first — that
|
||||
// friction is the point.
|
||||
// Mostly mirrors the payload built by _Session.payload(); an extension may
|
||||
// also emit its own event carrying one of these top-level keys. A key absent
|
||||
// here is dropped, not stored. Adding a metric means adding it here first —
|
||||
// that friction is the point, and it is also the only privacy control that
|
||||
// works retroactively, so it must land BEFORE any client starts sending the
|
||||
// key or that traffic is silently discarded and unrecoverable.
|
||||
const ALLOWED_KEYS = [
|
||||
'schema_version',
|
||||
'session',
|
||||
@@ -43,6 +46,13 @@ const ALLOWED_KEYS = [
|
||||
'models',
|
||||
'failures',
|
||||
'failure_statuses',
|
||||
// Model-routing summary. Emitted by a routing extension rather than by the
|
||||
// proxy itself -- see proxy/route_advice.py for the decision seam. Same rule
|
||||
// as everything above: counters and model ids, no free text. Allowlisted
|
||||
// here so the corpus can answer what the proxy alone cannot -- a provider's
|
||||
// real minimum cacheable prefix, how long a cache actually survives, and how
|
||||
// far predicted cache hits are from the ones that happened.
|
||||
'routing',
|
||||
];
|
||||
|
||||
// Resource attributes we keep. Same rule: allowlist, not denylist.
|
||||
|
||||
Reference in New Issue
Block a user