-
[issue-7526][BE] fix: support Anthropic adaptive-thinking models as online-rule judges (#7531)
frostbyte_neo 发布于
2026-07-22 13:23:08 +00:00 - [issue-7526] [BE] fix: support Anthropic adaptive-thinking models as online-rule judges
Online LLM-as-judge rules failed with Anthropic adaptive-thinking models
(claude-sonnet-5, claude-opus-4-7/4-8): the judge-path builder forwarded
temperature unconditionally (400 "temperature is deprecated for this model")
and never set max_tokens or read the rule's custom_parameters, so thinking
consumed the whole budget and the judge returned an empty response
(finishReason=LENGTH).In AnthropicClientGenerator.newChatLanguageModel (the judge path):
- Gate temperature server-side via AnthropicModelName.supportsSamplingParams,
and also skip it whenever thinking is enabled per-rule via custom_parameters
(Anthropic rejects sampling params while thinking is on) — covers API-created
rules that bypass the FE sanitizer. - Forward custom_parameters (thinking type/budget, max_tokens) onto the native
langchain4j builder. - Always send max_tokens, defaulting to the Playground default (4096) and
stacking it above any thinking budget so max_tokens > budget_tokens holds. - Ignore non-positive max_tokens/budget_tokens from the bypass path.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- fix(online-rules): address Anthropic judge review feedback
Address baz-reviewer comments on the Anthropic adaptive-thinking judge fix:
- Fail closed for unknown model names in AnthropicModelName.supportsSamplingParams
(default false) so a new adaptive-thinking model can't reintroduce the temperature
400 before it is added to the enum; back the lookup with a cached static map instead
of streaming values() on every call. - Treat thinking as enabled unless the type is explicitly "disabled" (covers "adaptive"
and any future type), and decode the thinking block once via parseThinking(), reused
by both the temperature gate and the builder wiring. - Clamp max_tokens above the thinking budget even when an explicit max_tokens is set, so
max_tokens <= budget_tokens can no longer reach Anthropic and 400. - Rename the headroom test to reflect what it asserts.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- fix(online-rules): harden Anthropic judge thinking/max_tokens parsing
Second round of baz-reviewer feedback on the adaptive-thinking judge fix:
- Default unknown model names back to true in supportsSamplingParams so registry-only
Anthropic models keep temperature support; adaptive models remain explicitly opted out. - Treat thinking as enabled only for an explicit non-blank type other than "disabled", so
an empty/blank thinking block no longer suppresses temperature or shapes max_tokens. - Forward thinking.budget_tokens only when thinking is enabled, so a budget-only (no type)
block can't reach Anthropic as a partial config and 400. - Widen to long before adding headroom in resolveMaxTokens so an extreme budget can't
overflow max_tokens to a negative int.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
- test(online-rules): assert default max_tokens in Anthropic disabled-thinking case
Harden dropsBudgetWhenThinkingDisabled so a regression that changes max_tokens
(not just the thinking budget) is caught.Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Co-authored-by: Claude Opus 4.8 (1M context) noreply@anthropic.com
下载附件