发布

RSS 订阅源

2.0.16-5147 07638aa2cc

比较
[OPIK-4551] [BE] [FE] [DOCS] feat: support Azure OpenAI REST API via Custom LLM provider (#6474)

frostbyte_neo 发布于 2026-04-27 10:18:16 +00:00
- [OPIK-4551] [BE] test: add WireMock baseline for Custom LLM URL + error behavior
Adds 7 integration tests documenting current Custom LLM provider behavior
as a regression baseline for OPIK-4551:
- legacyCustomProviderBehavesAsTodayWhenNoNewKeysSet — backward-compat
  contract lock: no URL/header mutation when no new config keys are set.
- openAiStyleErrorBodyIsPropagated / azureStyleErrorBodyIsPropagated —
  current error-body propagation; Azure-shape falls through to 500 today.
- baseUrlPathSuffixIsPreserved / queryParamInBaseUrlIsMangled — URL
  construction; proves a dedicated url_query_params config is needed.
- structuredContentIsPreservedForVisionModels /
  structuredContentIsFlattenedForNonVisionModels — AC #5 holds for
  gpt-4o family; documents pre-existing flattenContent bug for non-vision
  models (separate ticket).
No production code changed.
- [OPIK-4551] [BE] feat: support custom query params for Custom LLM providers
Introduces a thin HTTP-client decorator over LangChain4j's JdkHttpClient that
mutates outgoing Custom LLM requests based on optional keys in the existing
provider configuration map. The first configurable key is:
- configuration["url_query_params"]: JSON-encoded Map<String,String> whose
  entries are URL-encoded and appended to every outbound request URL. This
  unblocks Azure OpenAI APIM gateways that require the mandatory
  api-version query parameter, which cannot be smuggled via base_url
  because LangChain4j's string concatenation of /chat/completions would
  corrupt the query string.
The decorator is a pure no-op when no configuration keys it understands are
set, preserving the existing Custom LLM contract byte-for-byte for Ollama,
vLLM, and every other OpenAI-compatible provider in use today. Guarded by
the legacyCustomProviderBehavesAsTodayWhenNoNewKeysSet test introduced in
the baseline commit.

Adds urlQueryParamsAreAppendedToUpstreamRequest integration test that
verifies both api-version and a second arbitrary query parameter reach the
upstream cleanly via WireMock verify assertions.
- [OPIK-4551] [BE] feat: support custom auth header and optional Bearer suppression
Adds two new optional configuration keys for Custom LLM providers:
- configuration["auth_header_name"]: when set, appends a custom auth header
  {name}: {apiKey} alongside LangChain4j's default Authorization: Bearer.
  Used by Azure APIM gateways that expect api-key (or a tenant-specific
  header name) rather than Bearer tokens.
- configuration["suppress_default_auth"]: when "true", removes the default
  Authorization: Bearer header for gateways whose policy rejects it.
  Defaults to false to preserve today's behavior for every existing
  Custom LLM provider.
Plumbs the decrypted apiKey through InterceptingHttpClientBuilder so the
decorator can use it as the custom header value. The apiKey is already
decrypted by LlmProviderFactoryImpl.buildConfig at call time (line 74);
the decorator receives the same plaintext LangChain4j would have used for
Bearer — no new secret-handling code path.

Adds two integration tests:
- authHeaderNameAddsCustomHeaderAlongsideBearer verifies both headers
  reach the upstream when suppress_default_auth is off.
- suppressDefaultAuthDropsBearer verifies only the custom header is sent
  when suppression is on.
The backward-compat tripwire (legacyCustomProviderBehavesAsTodayWhenNoNewKeysSet)
continues to pass, confirming the decorator is a no-op for providers
without any of the new configuration keys.
- [OPIK-4551] [BE] feat: substitute {model} placeholder in Custom LLM base URL
Lets a single Custom LLM provider entry serve many deployments on gateways
that bake the deployment name into the path. When the configured base_url
contains the literal '{model}' placeholder, InterceptingHttpClient parses
the outgoing request body, reads the model field (already stripped of the
'custom-llm/<provider_name>/' prefix by CustomLlmProvider.cleanModelName
and OpikOpenAiChatModel respectively), and substitutes it into the URL.

Use case: Azure APIM gateways expect paths like
/openai/deployments/{deployment}/chat/completions
Operators can configure a single provider entry with
base_url = https://gw/openai/deployments/{model}
and list 20+ deployments in the Models field. End users pick any
deployment from the single dropdown; the decorator routes each request to
the matching upstream path at request time.

If the URL contains no placeholder, this code path is entirely skipped —
Ollama, vLLM, bare OpenAI-compat setups remain unaffected (validated by
the legacyCustomProviderBehavesAsTodayWhenNoNewKeysSet tripwire).

Adds modelPlaceholderIsSubstitutedInUrl integration test that reuses one
provider entry to fire two requests against two deployments and verifies
each lands on the correct upstream path, with no literal '{model}' reaching
the gateway.
- [OPIK-4551] [BE] feat: surface Azure-shape error responses cleanly
Extends CustomLlmErrorMessage to understand OpenAI's and Azure's nested
object error shape (for example {"error": {"code": "InvalidAPIVersion",
"message": "..."}}) alongside the historical OpenAI-compat text shape
({"error": "message"}). Before this change, object-shaped error bodies
could not be parsed into CustomLlmErrorMessage's String error field, so
ChatCompletionService fell through to the generic 500 path with the raw
upstream JSON embedded inside "Unexpected error calling LLM provider: ...".
Now the nested message surfaces as the user-visible text.

The record field is promoted to JsonNode so Jackson can accept either
shape. toErrorMessage mirrors OpenAiErrorMessage.getCode's status-code
mapping for OpenAI-compat codes (invalid_api_key to 401, rate_limit_exceeded
to 429, etc.) so Custom LLM providers fronting a real OpenAI-compat
backend behave identically to the native OpenAI provider. InvalidAPIVersion
is listed explicitly since it is the one Azure-specific code we have seen
in the ticket. Unrecognized codes fall back to 400 rather than 500 because
a provider that returned an error body at all is almost always signalling
a client-side fault rather than a server-side blow-up. The flip from
OpenAiErrorMessage's 500 default is deliberate.

Flips azureStyleErrorBodyIsPropagated from the previous "documents-the-bug"
shape (asserting 500 with raw body embedded) to the fixed expectation
(400 with the clean message text). openAiStyleErrorBodyIsPropagated and
legacyCustomProviderBehavesAsTodayWhenNoNewKeysSet remain green, proving
the string-error path and no-config-key path are unchanged.
- [OPIK-4551] [FE] feat: expose query params, auth header name, and suppress Bearer in Custom Provider form
Extends the v2 Custom Provider configuration dialog with three optional
fields that round-trip to the new backend configuration keys introduced in
this ticket:
- Query parameters: reuses the CustomHeadersField key/value pattern
  (generalized here with name/label/placeholder props). Serialized into
  configuration.url_query_params as a JSON Map<String,String> on save and
  parsed back on load. The empty case is collapsed to undefined so the
  configuration key is omitted for providers that don't need it.
- Auth header name: single text input. Sent alongside the default
  Authorization: Bearer unless suppression is on.
- Suppress default Authorization header: Switch toggle, default off so
  existing custom providers are unchanged.
Adds a helper hint under the URL field explaining the {model} placeholder
for multi-deployment gateways (Azure APIM-style) and extends
ProviderKeyConfiguration in types/providers.ts with the three new optional
fields.

Extracts the form <-> API serialization helpers into customProviderConfig.ts
and covers them with 15 Vitest unit tests (15/15 green) in
customProviderConfig.test.ts, including round-trip, blank-key handling,
and the empty-array-while-editing behavior convertHeadersForAPI already
has. v1 form is untouched per repo convention.
- [OPIK-4551] [DOCS] docs: document query params, auth header, and {model} placeholder
Extends the AI Providers admin documentation with the new Custom Provider
configuration options introduced in OPIK-4551:
- Query parameters section explaining how to add key/value pairs that
  are appended to every outbound URL (covers the api-version case).
- Auth header name and suppress-default-Authorization section. Makes
  clear that suppression is opt-in and default-off matches today's
  behavior for every existing provider.
- URL templates with {model} for multi-deployment gateways (Azure APIM
  style), plus a worked example.
- End-to-end Azure OpenAI via APIM subsection that ties all the new
  options together with realistic enterprise gateway URL shapes.
Also notes that the plaintext API key remains encrypted at rest using
the same mechanism as every other provider, so users understand the new
fields don't change the secret-handling contract.
- [OPIK-4551] [BE] [FE] refactor: address PR review feedback on OPIK-4551 decorator
Addresses five reviewer findings on #6474:

Backend (InterceptingHttpClient):
- Normalize null configuration to Map.of() in the constructor so
  applyAuthHeaders / applyQueryParams can safely call configuration.get
  when {model}-only providers reach mutate() without any new config keys.
  Fixes the potential NullPointerException flagged on the review.
- Narrow the broad catch (RuntimeException) around JSON parsing to
  catch (UncheckedIOException) — the concrete type JsonUtils throws — so
  only config-parse / body-parse failures are swallowed while other
  unchecked bugs propagate.
- Log the offending url_query_params raw value when the map fails to
  parse, to aid debugging misconfigured providers.
- Use Boolean.TRUE.toString() instead of a bare "true" string literal.
Backend (CustomLlmClientGenerator):
- Extract a shared newInterceptingHttpClientBuilder(config) helper so the
  HTTP/1.1-pinned JdkHttpClientBuilder + decorator wiring is defined once and
  reused by both newCustomLlmClient and newCustomProviderChatLanguageModel.
- Only call OpenAiClient.builder().httpClientBuilder(...) when the provider
  actually needs request mutation — i.e. when it declares one of the
  OPIK-4551 configuration keys (url_query_params, auth_header_name,
  suppress_default_auth) or its base_url contains a {model} placeholder.
  Legacy providers (Ollama, vLLM, bare OpenAI-compat) skip the decorator
  entirely and keep LangChain4j's default HTTP client path, so their request
  behaviour is byte-identical to pre-OPIK-4551.
Backend (new test):
- Adds InterceptingHttpClientTest with three unit tests locking in the
  behaviours that are awkward to reach through the WireMock integration
  suite: null-configuration with a {model} placeholder (regression guard
  for the NPE fix above), empty-configuration-and-no-placeholder pure
  pass-through, and malformed JSON body with a placeholder forwarding the
  URL unchanged.
Frontend (CustomHeadersField):
- Replace the manual field.onChange([...entries]) array handling with
  React Hook Form's useFieldArray, per the project forms guideline. React
  keys now come from RHF's auto-managed field.id; text inputs are wired
  via inner FormField with indexed paths so RHF tracks value/onChange
  automatically. Array data shape ({key, value, id}) unchanged, so
  serialization helpers / Vitest round-trip tests keep passing unchanged.
- [OPIK-4551] [FE] polish: generalize Azure-specific help text in Custom Provider form
Addresses review feedback on UI copy:
- Drop Azure-specific phrasing from the "Auth header name" helper (the
  custom provider isn't Azure-only). The what-it-does sentence is kept,
  the "Used by Azure APIM gateways that expect…" tail is removed.
- Similarly generalize the URL {model}-placeholder helper (drop
  "(e.g. Azure APIM)") and the "Query parameters" helper ("Azure OpenAI
  gateways" → "Some gateways").
- Drop "Default off, matching today's behavior for all existing custom
  providers." from the "Suppress default Authorization header" helper —
  reads as noise to someone configuring their first custom provider.
- Fix the pre-existing "Models list" example that read as slash-separated
  (meta-llama/Meta-Llama-3.1-70B,mistralai/Mistral-7B) even though the
  text said "comma separated". Switched to gpt-4o, gpt-4o-mini, llama-3.1-70b so the commas are the obvious separator.
- [OPIK-4551] [DOCS] polish: generalize Azure-specific framing in Custom Provider docs
Mirrors the UI-copy cleanup in the previous commit. The generic "Query
Parameters", "Auth Header Name", and "URL Templates with {model}"
sub-sections now describe the feature in terms of any gateway — Azure is
no longer singled out in the intros.

The "Azure OpenAI via API Management gateway" worked example later in the
page is kept as-is; its whole purpose is to show how to apply these
generic building blocks to a specific Azure APIM deployment. Pre-existing
"Azure OpenAI" section (direct-endpoint setup) is also untouched.

Also drops "Leave off for every existing provider — the default preserves
today's behavior." from the suppress-default-auth bullet since, like its
UI counterpart, the "today's behavior" wording is noise for a first-time
custom provider reader.

The {model} placeholder example URL was switched from
my-gateway.azure-api.net/openai/deployments/{model} to
my-gateway.example.com/deployments/{model} so the abstract pattern
doesn't look like it requires an Azure-shaped host.
- [OPIK-4551] [BE] fix: always pass the JdkHttpClientBuilder to LangChain4j, only wrap with decorator conditionally
The previous revision of this commit dropped the .httpClientBuilder(...)
call entirely for legacy providers, which also dropped the HTTP/1.1
pinning the original CustomLlmClientGenerator code put in place for
vLLM / FastAPI servers (a pre-OPIK-4551 behaviour that must be
preserved).

The correct split is:
- .httpClientBuilder(...) is always called, passing a JdkHttpClientBuilder
  that still forces HTTP/1.1 — same object LC4j received before OPIK-4551.
- Only the InterceptingHttpClientBuilder wrapper is conditional. It is
  applied via requiresInterceptingBuilder(config) when the provider
  actually uses an OPIK-4551 feature (url_query_params, auth_header_name,
  suppress_default_auth, or a {model} placeholder in the base URL).
Legacy providers (Ollama, vLLM, bare OpenAI-compat) therefore see a plain
JdkHttpClientBuilder — byte-identical to the pre-OPIK-4551 setup — with
no decorator in the way. OPIK-4551-enabled providers see the same
JdkHttpClientBuilder wrapped by our decorator so URL / header mutation
can take effect.

All 14 unit and integration tests still green.
- [OPIK-4551] [FE] refactor: extract shared KeyValueFieldArray for headers + query-param editors
Addresses PR review: CustomHeadersField and WebhookHeaders.tsx both
implemented their own key/value field editor with useFieldArray, which
would have meant every future header/query-param tweak (styling,
validation, accessibility, etc.) had to land in two places.

Introduces a generic shared component at
src/shared/KeyValueFieldArray/KeyValueFieldArray.tsx that:
- Is parameterised over the form type T extends FieldValues and takes
  name: ArrayPath<T>, so any form with a {key, value}-shaped array
  field can plug in without schema coupling.
- Accepts a newItem factory so callers control the exact shape of
  freshly-appended rows. Custom LLM providers need {key, value, id}
  to satisfy their Zod contract; Webhook Alert headers only need
  {key, value}. Both consumers pass the right shape in one place.
- Exposes configurable label / description / placeholders / add-button
  label, plus an optional showColumnHeaders toggle that matches the
  existing WebhookHeaders layout.
Both consumers are migrated:
- CustomHeadersField.tsx becomes a thin wrapper that preserves its
  current prop surface (used twice from CustomProviderDetails — once
  for custom headers, once for query parameters) so CustomProviderDetails
  is unchanged.
- v2/pages/AlertsPage/AddEditAlertPage/WebhookHeaders.tsx now renders
  the shared component with showColumnHeaders and its existing label /
  description strings. Visually the only drift is the remove icon (Trash2
  instead of X) and slightly less vertical padding — net cleaner.
v1 WebhookHeaders.tsx is intentionally untouched per the v1/v2
separation rule.

Frontend gates still green: tsc --noEmit, eslint --max-warnings=0,
15/15 Vitest unit tests in customProviderConfig.test.ts.
- docs(ai-providers): apply reviewer wording polish on Query Parameters and Auth header sections
- Reword the Query Parameters intro with a concrete Azure api-version example
- Tweak bullet "Add multiple parameters as needed" -> "Add as many parameters as needed"
- Drop redundant "the default" qualifier before Authorization: Bearer header
- chore(custom-llm): drop customer names from comments
Scrub customer references from Javadoc on CustomLlmErrorMessage.getStatusCode
and from a test comment in ChatCompletionsResourceTest. Customer names
should not ship in the public repo; OPIK-4551 remains the traceable reference.
- refactor(custom-llm): use URIBuilder for safer query-param append
Replace hand-rolled "?"/"&" logic with Apache HttpComponents URIBuilder
so malformed, empty, or fragment-bearing URLs are handled safely. If the
URL fails to parse, log a warning and forward unchanged instead of
producing a corrupt output.

Adds tests covering the fragment case (query goes before "#") and the
malformed-URL fallback.
- fix(custom-llm): guard against unauthenticated upstream + nested error paths in KeyValueFieldArray
- InterceptingHttpClient: ignore suppress_default_auth when no
  auth_header_name is configured, keep the default Authorization
  header and log a warning. Previously this combination dropped the
  header without replacing it, silently sending an unauthenticated
  request upstream.
- KeyValueFieldArray: split dotted name (ArrayPath) into path
  segments before lodash get, so nested RHF error trees like
  errors.provider.headers[0].key are read correctly when the
  component is used with a dotted path.
- Add unit test covering the suppress-auth fallback.
- refactor(custom-llm): address PR review — centralize status mapping, javadoc, redact URL log, drop prints
- Extract OpenAiCompatStatusCodes as a shared utility so OpenAiErrorMessage
  and CustomLlmErrorMessage no longer duplicate the OpenAI-compat code -> HTTP
  status switch. Azure-specific codes (InvalidAPIVersion) layer on top in
  CustomLlmErrorMessage.
- Convert /// comment blocks to proper /** */ javadoc on public /
  package-private types, constructors, and helper methods that this PR
  introduced, so the javadoc tool picks them up on Java 21.
- Stop logging the user-controlled URL in the URIBuilder failure warn path;
  log only the exception. Base URLs can contain secrets in their query
  string or userinfo segment.
- Remove diagnostic System.out.printf calls from ChatCompletionsResourceTest.
- refactor(custom-llm): warn on slashed model names, simplify error switch, drop stale comment
- InterceptingHttpClient.applyModelPlaceholder now logs a warning when the
  substituted model name contains '/'. The substitution stays raw (some
  gateways expect literal slashed segments) so the request still goes
  through, but operators get visibility when a HuggingFace-style name
  silently turns into extra path segments.
- CustomLlmErrorMessage.getStatusCode collapses the InvalidAPIVersion
  branch into the default since both already returned 400. The intent is
  preserved as a comment.
- ChatCompletionsResourceTest comment drops the stale :10 line ref —
  class name alone is enough.
- FE form help text and ai_providers.mdx now call out the alphanumeric /
  hyphen guideline for {model} substitution.
- refactor(custom-llm): use @UtilityClass; apply reviewer wording polish on ai_providers docs
- OpenAiCompatStatusCodes is now annotated with Lombok's @UtilityClass; the
  explicit private constructor and static modifier on fromCode are removed
  since the annotation contributes both.
- ai_providers.mdx wording polish across the Query Parameters, Auth Header,
  URL Templates and Azure APIM subsections (Juan's review): "key-value",
  "automatically URL-encoded", "differently named", "Enable this", "using
  {model} as a placeholder. This avoids creating...", split slashed-name
  warning into two sentences, "instead of Authorization: Bearer", and
  "full gateway path, including the deployment placeholder".
下载附件
- 源代码 (ZIP)
- 源代码 (TAR.GZ)

0 发布 8,340 Git 标签

[OPIK-4551] [BE] [FE] [DOCS] feat: support Azure OpenAI REST API via Custom LLM provider (#6474)