-
feat(fleet_monitoring): migrate model pricing to litellm's price table (#4047)
frostbyte_neo 发布于
2026-07-16 20:56:28 +00:00 - feat(fleet_monitoring): migrate model pricing to litellm's price table
Replace the ~80-entry hand-vendored MODEL_PRICES dict with lookups
against litellm's bundled community-maintained price table (already a
project dependency), so new/updated model pricing no longer needs a
manual code change per model. Reads litellm's local snapshot directly
rather than the shared litellm.model_cost global, which is populated
from a live network fetch on whichever import touches it first and can
silently diverge from the deterministic offline table this always-on
dashboard sampler needs.A small local override table covers the two confirmed gaps in
litellm's snapshot (the GPT-5.6 family, and the retired
claude-3-5-sonnet-20241022 direct-API id), plus a narrow fallback for
Groq/MiniMax, which OpenSRE's OpenAI-compat client reports as bare
model names that litellm only keys under a provider prefix. Verified
against every provider in config/llm_auth/provider_catalog.py; NVIDIA
NIM has zero coverage in litellm today and self-hosted (Ollama) models
have no per-token price to look up — both correctly resolve to
unpriced rather than a guessed rate.Public API of pricing.py is unchanged, so sampler.py needs no changes.
Fixes #4035.
- fix: address greptile review feedback (greploop iteration 1)
- normalize_model_name: exclude @-suffixed candidates from the
"canonical" preference so a future litellm @default-style key
can't leak a routing artifact into the displayed model name. - Read litellm's local price snapshot via its packaged JSON file
directly instead of the private GetModelCostMap internal, and
degrade to an empty table (unpriced) rather than raise if the file
is ever missing — this module is imported unconditionally by the
always-on dashboard sampler. - Reject bool explicitly when validating litellm rate fields,
matching the existing safe_int guard in meters/__init__.py
(isinstance(True, int) is True).
- fix: address greptile review feedback (greploop iteration 2)
- Narrow _litellm_local_cost_map's return type to dict[str, Any]
(its values are dicts, not opaque objects) so subscripting entries
in tests type-checks correctly. - Add a startup-time smoke test asserting the local snapshot loads a
substantial table, so a broken/missing packaged data file fails
loudly in CI instead of silently degrading every model to unpriced.
下载附件