-
[OPIK-7180] [SDK] fix: capture proxy cost and allow provider override in LangChain integration (#7450)
frostbyte_neo 发布于
2026-07-14 14:07:33 +00:00 - [OPIK-7180] [SDK] fix: capture proxy cost and allow provider override in LangChain integration
When LangChain's ChatOpenAI targets an OpenAI-compatible proxy (e.g. a
LiteLLM gateway), the provider was auto-detected as the proxy hostname (so
the backend could not price the call) and the proxy-reported cost was never
captured, resulting in $0.00 span cost.- Add a
providerparameter to OpikTracer, resolved per LLM run. Accepts a
string/LLMProvider for the common single-provider case, or a callable for
chains/graphs that mix providers (returning None falls back to auto-detect). - Add an extensible response-cost extractor registry; ship a LiteLLM extractor
that readsx-litellm-response-cost(nested under response_metadata.headers,
verified against a real LiteLLM proxy) and records it as the span total_cost,
taking priority over Opik's own estimate. - Document the proxy setup, including the required ChatOpenAI
include_response_headers=True.
Implements OPIK-7180.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- test(langchain): drive proxy cost tests through mocked HTTP transport
Replace the fake-chat-model end-to-end tests with ones that route the real
langchain_openai + openai stack through an httpx.MockTransport mimicking a
LiteLLM proxy, so the x-litellm-response-cost header is parsed into
response_metadata exactly as against a live proxy. Cheap and deterministic:
no server, no network, no spend.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- fix(python): exclude litellm 1.92.* to avoid proxy import crash
litellm 1.92.0 made core completion() eagerly import litellm.proxy modules
that require fastapi/orjson (proxy-only extras), so any completion crashes
without litellm[proxy]. Reverted upstream in 1.93. Excluding 1.92.* resolves
to 1.91.3 now and picks up 1.93 once stable.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- test(openai): accept new cache_write_tokens usage field
OpenAI's Responses API now returns cache_write_tokens under
input_tokens_details, which flows through to logged usage. Add it to the
expected usage format so the live integration tests match the current shape.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- test(openai): subset-match Responses usage to tolerate provider extras
cache_write_tokens is an undeclared passthrough field (openai SDK only
declares cached_tokens; it leaks via extra="allow"). Asserting it exactly
is brittle across API changes, so drop it from the shared expected dict
and switch the responses/agents call sites to ANY_DICT.containing(...),
matching the langchain suite.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- style(openai): ruff-format wrap ANY_DICT.containing usage assertions
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- chore(sdk): split openai usage assert + litellm 1.92 fixes to #7465
These CI-hygiene changes are unrelated to the langchain proxy-cost
feature and now live in the independent PR #7465. Reverting them here
so this PR is scoped to OPIK-7180 only.Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- fix(langchain): friendlier provider resolver context and safe callback
Address PR #7450 review:
- Pass ProviderResolverContext (.model parsed name, .run raw dict) to the
provider resolver instead of the raw LangChain run dict, so the common case
isctx.modelrather than a deep run-dict dig. - Isolate the resolver call in try/except: a throwing user callback now warns
and falls back to auto-detection instead of aborting end-span processing. - Split the 89-char x-litellm-model-id header literal to satisfy the 88-char
limit; rename proxy tests to the WHAT__CASE__EXPECTED convention. - Update LangChain docs to the friendlier resolver and note that ctx.model is
normalized (not proxy-dependent).
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Co-authored-by: Claude Opus 4.8 noreply@anthropic.com
下载附件