-
[BE] fix: apply above_200k tier pricing in cost calculation (#7023)
frostbyte_neo 发布于
2026-06-12 07:57:09 +00:00 Closes #6982. Several models in model_prices_and_context_window.json (e.g. gemini-2.5-pro, vertex_ai/claude-sonnet-4-5) carry *_above_200k_tokens rate variants that LiteLLM consumes as whole-prompt tiers — once prompt_tokens > threshold, every token is billed at the tier rate. Opik wasn't reading these fields, so long-context requests on those models were billed at the base rate and total_estimated_cost was under-reported.
Plumbs the four above_200k rates through ModelCostData (with explicit @JsonProperty since Jackson's snake-case strategy doesn't underscore-separate digits) into ModelPrice, exposes effective*Price(int totalPromptTokens) helpers, and wires the Google + shared Anthropic/Bedrock calculators to use them. Boundary integration tests on gemini-2.5-pro at 200_000 and 300_000 prompt tokens. 128k/256k/272k thresholds follow the same shape and can be added as follow-ups.
Co-authored-by: Andres Cruz andresc@comet.com
下载附件