-
[OPIK-7641] [BE][FE] fix: resolve Vertex AI multi-region endpoints and unhide flash-lite models (#7682)
frostbyte_neo 发布于
2026-08-03 06:48:54 +00:00 - [OPIK-7641] [BE][FE] fix: resolve Vertex AI multi-region endpoints and unhide flash-lite models
The Vertex AI SDK derives its host from the configured location as
%s-aiplatform.googleapis.com, which only holds for single-region locations. A provider
configured withglobaltherefore targetedglobal-aiplatform.googleapis.com, which does
not resolve, and every completion failed with a 404. Set the API endpoint explicitly for the
multi-region locations while keeping the location itself, so the request path still carries
locations/global. Single-region locations keep the SDK default untouched.gemini-3.1-flash-litewas missing from the model list because the model-sync script only
read thevertex_ai-chat-modelsandvertex_aiLiteLLM provider tags, while the flash-lite
variants are taggedvertex_ai-language-models. Fixing the extractor rather than hand-adding
the enum rows also unhidesgemini-3.1-flash-lite-previewandgemini-3.5-flash-lite, and
stops the next flash-lite release from regressing the same way. Generated definitions are the
script's own output, limited to the Vertex AI provider.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- fix(vertexai): canonicalise the configured location once
The location lands in the
locations/%sresource path as well as the host, so normalising
it only for the endpoint lookup left a configured" GLOBAL "reaching the correct global
host while requesting a malformed path. Canonicalise once at the configuration boundary and
derive both from that value.The existing casing test only exercised the lookup helper in isolation, which is why it
passed while the client was misconfigured; it now asserts the generated client's location
too, and fails without the fix.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- fix(vertexai): treat a blank location as unset
The location is not validated for blankness at the API boundary, and the SDK rejects an empty
one outright, so canonicalising a whitespace-only value into "" made client creation throw
where it previously succeeded. The failure also surfaced as a 500 blaming credentials, since
the builder call sits inside the credentials try block. Skip blank values so they fall back to
the SDK default, exactly as an absent location key does.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- refactor(vertexai): make multi-region endpoints configurable
Move the multi-region endpoint hosts out of the generator and into
vertexAIClientconfig, with
the current values as defaults, so an endpoint change does not need a code release. Both config.yml
and config-test.yml declare them, and a test parses the shipped YAML so a misnamed key fails the
build instead of silently falling back to the defaults.Also address review feedback:
strip()overtrim()for modern whitespace, and both helpers are
private again with the tests reaching them through the public generate() entry point rather than
unit-testing internals.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- test(vertexai): stub the Vertex endpoint with WireMock
Replace the reflection-based inspection of the generated client with WireMock stubs, per review
feedback. The multi-region cases now issue a real completion against the stub and verify the
request path, which is stronger than the previous assertions:locations/{location}is checked as
the SDK actually built it rather than inferred from the resolved host.The service-account fixture points
token_uriat WireMock too, so the OAuth exchange is stubbed
and nothing authenticates against Google. Transport becomes configurable (defaulting to GRPC, as
before) because WireMock speaks HTTP rather than gRPC.Locations that keep the SDK-derived host still assert on the resolved host instead of calling it:
those hosts are real Google endpoints, and issuing the request cost ~70s of DNS timeouts per run.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- fix(vertexai): overlay configured endpoints on the defaults
Returning a configured endpoint map verbatim meant overriding one location dropped the others, so
they fell back to the SDK-derived host that this fix exists to avoid. Overlay the configured entries
on the defaults instead, and canonicalise their keys: the lookup uses a canonicalised location, so a
configuredGlobal:would never have matched and would have silently taken the derived host too.Canonicalisation now lives on the config record, keeping the map keys and the lookup on one
definition. Also extends the fallback test to the empty-map branch.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- refactor(vertexai): resolve the endpoint map at construction
Adds
@Builder(toBuilder = true)per the backend record convention and switches the tests off
positional constructors, which matters now the record has three components.Moves the merge into a compact constructor so it runs once instead of on every lookup, and rejects
blank locations and endpoints with a message naming the offending entry: a YAML entry like
global:with no value deserialises to null and previously surfaced as a bare NullPointerException
fromstrip().Co-Authored-By: Claude Opus 5 noreply@anthropic.com
- refactor(vertexai): make the config file the only source of the endpoints
The multi-region endpoint hosts had a copy in Java that the configured map
was overlaid on. Two values for one setting is confusing to debug in
production: an operator reading config.yml could not tell whether those
hosts were the ones actually applied. The Java copy is gone and the
configuration is now mandatory, so what the file says is always what the
client uses.Also moves the LLM provider client records to top-level files in the llm
package, since they are public configuration types, and replaces the
hand-rolled checks in the Vertex record with the Dropwizard/Jakarta
annotations the rest of the configuration uses:- @NotNull on vertexAIClient and on transport, @NotBlank on scope
- @NotEmpty on multiRegionApiEndpoints, with @NotBlank on the values
- @Pattern on the keys, because the lookup canonicalises the location:
a configured 'Global:' would never match and would silently fall back to
the SDK-derived host, so it now fails at startup instead
That also removes the last public helper on the record, so canonicalisation
lives in the generator that needs it.No validation is added to the OpenAI and Anthropic records on purpose: both
are optional in the shipped configuration (openAiClient.url defaults to an
empty string) and every caller reads them through
Optional.ofNullable(...).filter(isNotBlank), so a @NonNull or @NotBlank
there would turn a supported configuration into a startup failure.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01W4ttXw2winYUAneiAeWoCo- test(vertexai): drop the config test and load config-test.yml via TestConfigUtils
VertexAIClientConfigTest tested configuration values, which belongs to
infrastructure testing rather than to this service, and it parsed the YAML
itself instead of using the existing helper.The generator test now builds its config from TestConfigUtils.loadConfigTest,
overriding only what WireMock needs (the endpoints and the REST transport).
That covers what the deleted class was worth keeping for: loadConfigTest
validates, so a config-test.yml missing or misspelling the endpoint block
fails here, and the functional assertions still prove the client calls the
configured endpoint for every multi-region location.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01W4ttXw2winYUAneiAeWoCo- refactor(config): validate the client configuration string fields
Adds @NotBlank to the Anthropic url and version, and a host constraint on
the Vertex endpoint values.@URL does not fit the endpoint values: the SDK takes a bare host and appends
':443' itself on the gRPC transport, so @URL rejects all three hosts we ship
(verified against hibernate-validator 8.0.3). The @Pattern is the equivalent
constraint for a host and catches the realistic mistake of pasting a full
'https://' URL, which would otherwise only surface as a failed completion.OpenAiClientConfig.url stays unconstrained, with the reason in the javadoc:
the shipped config.yml has 'url: ${LLM_PROVIDER_OPENAI_URL:-}', which
deserialises to null and means "use the provider default", so any constraint
there fails startup on a default installation. Verified by loading config.yml
through the same substituting provider and validator the application uses.Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01W4ttXw2winYUAneiAeWoCo
Co-authored-by: Claude Opus 5 noreply@anthropic.com
下载附件