feat: split LLM Gateway into two provider catalogs (#4011)

* feat: split LLM Gateway into two provider catalogs

Renames the existing llmgateway provider to "DevPass (LLM Gateway)" (id
and models unchanged: the aggregated, auto-routed root-model catalog) and
adds llmgateway-providers ("LLM Gateway"): one entry per upstream
provider mapping, addressed as provider/model-id, synced from
/v1/models?mapped=true. The catalog starts empty and is populated by the
scheduled sync automation; the sync refuses to run against a deployment
without the mapped view so it fails loudly instead of syncing wrong ids.

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* fix: apply deployment data on mapped factored entries

Addresses the PR review: brand-new factored mapped entries now carry the
mapping's own capability flags (attachment/tool_call/reasoning and
structured_output) as overrides, translate the deployment's declared
reasoning_efforts into reasoning_options instead of stamping [], prefer
the gateway's served max_output over inherited/authored output limits,
and only fall back to context when the base metadata declares no output.
Adds unit tests for mapped factoring, capability overrides, max_output
preference, and the unprefixed-id refusal guard.

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* chore: seed the llmgateway-providers catalog

The dev branch now rejects providers with zero models, so the empty
.gitkeep-anchored catalog no longer validates. Seed it with a small
representative set generated by the sync (factored, full, duplicate
deployments of one model, capability deltas); the scheduled sync fills
in the rest once the gateway's mapped view is live.

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* fix: honor base and sibling reasoning data on mapped sync

Round 2 of review feedback:
- Factored resyncs no longer stamp context as limit.output when the
  gateway omits max_output and the base declares an output to inherit;
  the served max_output still wins whenever reported (creates and
  resyncs), and reasoning_options now refresh from deployment efforts.
- A deployment whose only accepted effort is "none" is a plain on/off
  switch, so it translates to a toggle (matches the lab's control).
- When a deployment declares no efforts, mapped entries reuse the
  aggregated llmgateway catalog's curated reasoning_options for the
  same root model instead of ending up with []; a curated [] counts as
  unknown so a bad first stamp is not sticky. The runner also stops
  stamping [] onto factored reasoners whose base metadata already
  declares reasoning_options (it would shadow the base's controls).
- perplexity added to the canonical prefixes so Sonar models factor
  against their lab metadata; the sonar-pro seed is now override-only.

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* fix: harden mapped sync guards and seed curation

Round 3 of review feedback:
- Both LLM Gateway syncs now reject an empty (or fully filtered)
  response instead of authoritatively deleting the catalog through the
  delete-missing pass; the every() prefix guard alone passed on [].
- A vision-less deployment also overrides modalities on factored
  creates, so attachment=false can no longer coexist with inherited
  image input (sonar-pro seed regenerated accordingly).
- Mapped entries copy the interleaved reasoning side-channel from the
  aggregated llmgateway catalog when the deployment reasons (same wire
  surface); glm-5.1 and kimi-k2.6 seeds now carry it.
- Toggle seeds carry the required leading wire-path comment.
- gpt-5.5 seeds author the 272k context pricing tier so resync
  preserves it, matching the first-party and aggregated entries.

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* fix: never author zero limits, enforce vision on modalities

Round 4 of review feedback:
- A missing/zero context_length is no longer written as limit.context=0:
  factored entries leave context unset and inherit the base, and
  unfactored creates without a positive served context are skipped
  (reported via sourceID) instead of publishing unusable limits. Applies
  to both the aggregated and mapped builders.
- vision=false now forces non-image input modalities from the mapping
  itself instead of trusting the model-level architecture, on both the
  factored and unfactored create paths (and the existing-full fallback).

Claude-Session: https://claude.ai/code/session_017pReWhniXJcDL9aiQHqoFQ

* fix: scalable logo, require one mapping per entry

Review round 5: drop the fixed width/height from the new provider logo
(AGENTS.md blocker), and fail the mapped sync loudly when a kept model
does not carry exactly one providers[] mapping instead of letting the
builder silently fall back to noisy supported_parameters defaults.

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF

* fix: inherit lab descriptions, author toggle headers

Review round 6: mapped factored resyncs no longer stamp a synthesized
describeModel blurb as a sticky description override (unset keeps
inheriting the lab text, matching merge-gateway/cortecs), and mapped
sync writes now author the required leading wire-path comment on files
that carry a toggle reasoning control via a new optional header on the
translateModel result (an existing on-disk header always wins).

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF

* fix: keep mapping flags authoritative on resyncs

Review round 7: mapped existing-entry resyncs (factored and full) now
apply the deployment mapping's reasoning/vision/tools/structured-output
flags with the same authority as creates, so the written booleans and
the reasoning_options derived from them always move together and drift
self-heals hourly; prior curation only fills in where the mapping is
silent. Also documents in the together-ai/kimi-k2.6 seed header why
that pin is intentionally weaker than Together's first-party row (the
gateway serves it with tools/JSON off and a 32k output cap per its own
e2e'd catalog mapping).

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF

* fix: realign vision modalities in both directions

Review round 8: mapped resyncs no longer keep a stale text-only
modalities override once the deployment's vision returns — a declared
vision=true clears the override on factored entries (base image/pdf
inputs inherit again) and recomputes from the served architecture on
full entries, mirroring how vision=false already strips them; only a
silent mapping leaves curated modalities untouched.

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF

* fix: local perplexity resolution, no zero limits

Review round 9: drop the perplexity entry from the shared
CANONICAL_PROVIDER_PREFIXES (it would silently start factoring other
hosts' standalone perplexity files) — the llmgateway sync now resolves
lab IDs through resolveModelMetadataBaseModel, whose exact models/ path
match covers perplexity without touching other providers. Full-row
resyncs in both builders no longer fall back to the zero/absent
reported context: authored limits only ever carry known-positive
values, an authored 0 on disk counts as unusable, and a full row with
no usable context anywhere fails loudly (skipping would hand the file
to the delete-missing pass).

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF

* fix: merge deployment efforts with curated controls

Review round 10: deployment reasoning_efforts now own only the
effort/toggle surface — curated non-effort controls such as
budget_tokens (the same host's $.reasoning.max_tokens path, mirroring
DigitalOcean's sync) survive from the existing file or the aggregated
sibling instead of being wiped on every resync. Mapped creates also
seed cost.tiers from the aggregated sibling's curated tiers, since the
gateway API exposes none and the bulk sync would otherwise author
tiered models at flat long-context rates; authored tiers still win on
resync.

Claude-Session: https://claude.ai/code/session_0131ZfUnTfJrCzygw3wE4bNF
This commit is contained in:
Ismail Ghallou
2026-08-20 20:39:11 +02:00
committed by GitHub
parent 38d785c8e3
commit 82345e0bf3
14 changed files with 1066 additions and 51 deletions
+1
View File
@@ -28,6 +28,7 @@
"huggingface:sync": "bun ./packages/core/script/sync-models.ts huggingface",
"kilo:sync": "bun ./packages/core/script/sync-models.ts kilo",
"llmgateway:sync": "bun ./packages/core/script/sync-models.ts llmgateway",
"llmgateway-providers:sync": "bun ./packages/core/script/sync-models.ts llmgateway-providers",
"requesty:sync": "bun ./packages/core/script/sync-models.ts requesty",
"merge-gateway:sync": "bun ./packages/core/script/sync-models.ts merge-gateway",
"nano-gpt:sync": "bun ./packages/core/script/sync-models.ts nano-gpt",
+25 -4
View File
@@ -21,7 +21,7 @@ import { hyper } from "./providers/hyper.js";
import { huggingface } from "./providers/huggingface.js";
import { inceptron } from "./providers/inceptron.js";
import { kilo } from "./providers/kilo.js";
import { llmgateway } from "./providers/llmgateway.js";
import { llmgateway, llmgatewayProviders } from "./providers/llmgateway.js";
import { mergeGateway } from "./providers/merge-gateway.js";
import { nanoGpt } from "./providers/nano-gpt.js";
import { openai } from "./providers/openai.js";
@@ -98,7 +98,17 @@ export interface SyncProvider<SourceModel> {
existing(id: string): ExistingModel | undefined;
authored(id: string): ExistingModel | undefined;
},
): { id: string; model: SyncedModel; metadata?: { id: string; model: SyncedMetadata } } | undefined;
): {
id: string;
model: SyncedModel;
metadata?: { id: string; model: SyncedMetadata };
/**
* Leading comment block for the written file when it has none of its own
* (e.g. the wire-path header every toggle reasoning control requires). A
* header already present on the existing file always wins.
*/
header?: string;
} | undefined;
}
export interface SyncResult {
@@ -131,6 +141,7 @@ export const providers: {
inceptron: SyncProvider<any>;
kilo: SyncProvider<any>;
llmgateway: SyncProvider<any>;
"llmgateway-providers": SyncProvider<any>;
"merge-gateway": SyncProvider<any>;
"nano-gpt": SyncProvider<any>;
ofox: SyncProvider<any>;
@@ -162,6 +173,7 @@ export const providers: {
inceptron,
kilo,
llmgateway,
"llmgateway-providers": llmgatewayProviders,
"merge-gateway": mergeGateway,
"nano-gpt": nanoGpt,
ofox,
@@ -186,6 +198,7 @@ export const groups = {
"inceptron",
"kilo",
"llmgateway",
"llmgateway-providers",
"merge-gateway",
"nano-gpt",
"ofox",
@@ -270,13 +283,16 @@ export async function syncProvider<SourceModel>(
: preserveBaseModel(translated.model, existing.get(relativePath)?.authored);
const translatedBase = "base_model" in translatedModel ? translatedModel.base_model : undefined;
let resolvedReasoning: boolean | undefined;
let baseReasoningOptions: unknown;
if (translatedBase !== undefined) {
if (translated.metadata?.id === translatedBase) {
resolvedReasoning = translated.metadata.model.reasoning;
baseReasoningOptions = translated.metadata.model.reasoning_options;
} else {
modelMetadata ??= await readModelMetadata(provider.modelsDir);
const canonicalReasoning = modelMetadata[translatedBase]?.reasoning;
resolvedReasoning = typeof canonicalReasoning === "boolean" ? canonicalReasoning : undefined;
baseReasoningOptions = modelMetadata[translatedBase]?.reasoning_options;
}
} else {
resolvedReasoning = existing.get(relativePath)?.toml.reasoning;
@@ -285,6 +301,7 @@ export async function syncProvider<SourceModel>(
translatedModel,
existing.get(relativePath)?.authored,
resolvedReasoning,
baseReasoningOptions,
);
const withDescription = provider.preserveDescriptions === false
? withReasoningOptions
@@ -300,7 +317,7 @@ export async function syncProvider<SourceModel>(
desired.set(relativePath, {
model: parsed.data,
content: (existing.get(relativePath)?.header ?? "") + formatToml(parsed.data),
content: ((existing.get(relativePath)?.header || translated.header) ?? "") + formatToml(parsed.data),
});
}
@@ -476,6 +493,7 @@ export function preserveReasoningOptions(
model: SyncedModel,
existing: ExistingModel | undefined,
resolvedReasoning: boolean | undefined = existing?.reasoning,
baseReasoningOptions: unknown = undefined,
): SyncedModel {
if ((model.reasoning ?? resolvedReasoning) === false) {
const { reasoning_options: _reasoningOptions, ...withoutReasoningOptions } = model;
@@ -483,7 +501,10 @@ export function preserveReasoningOptions(
}
if (model.reasoning_options !== undefined) return model;
if (existing?.reasoning_options === undefined) {
return (model.reasoning ?? resolvedReasoning) === true
// When the base model already declares reasoning_options, leave the field
// unset so the factored file inherits them — stamping [] here would
// shadow the base's real controls with "no controls".
return (model.reasoning ?? resolvedReasoning) === true && baseReasoningOptions === undefined
? { ...model, reasoning_options: [] }
: model;
}
+486 -45
View File
@@ -1,16 +1,21 @@
import { z } from "zod";
import { existsSync, readFileSync } from "node:fs";
import path from "node:path";
import { describeModel } from "../../describe.js";
import { inferKimiFamily, ModelFamilyValues } from "../../family.js";
import { ReasoningOption } from "../../schema.js";
import type { ExistingModel, SyncProvider, SyncedFullModel, SyncedModel } from "../index.js";
import { factorBaseModel, resolveCanonicalBaseModel } from "./openrouter.js";
import { factorBaseModel, resolveModelMetadataBaseModel } from "./openrouter.js";
const API_ENDPOINT = "https://api.llmgateway.io/v1/models";
// LLM Gateway names the originating lab in `family`; most already match the
// canonical prefixes understood by resolveCanonicalBaseModel. Alias the few that
// spell the lab differently. (Mirrors huggingface's CANONICAL_ORG_PREFIXES.)
// canonical prefixes understood by resolveModelMetadataBaseModel, and labs
// outside that shared table (e.g. perplexity) resolve through its exact
// `models/` path match without widening the OpenRouter prefix map for every
// other provider. Alias the few that spell the lab differently. (Mirrors
// huggingface's CANONICAL_ORG_PREFIXES.)
const CANONICAL_FAMILY_ALIASES: Record<string, string> = {
grok: "xai",
mistral: "mistralai",
@@ -30,10 +35,6 @@ const Pricing = z.object({
input_cache_write: z.string().optional(),
});
const LLMGatewayProvider = z.object({
reasoning_efforts: z.array(z.string()).optional(),
}).passthrough();
const ReasoningEffortOrder = new Map([
"none",
"minimal",
@@ -54,9 +55,22 @@ export const LLMGatewayModel = z.object({
input_modalities: z.array(z.string()),
output_modalities: z.array(z.string()),
}),
providers: z.array(
z.object({
providerId: z.string().optional(),
vision: z.boolean().optional(),
tools: z.boolean().optional(),
reasoning: z.boolean().optional(),
reasoning_efforts: z.array(
z.enum(["none", "minimal", "low", "medium", "high", "xhigh", "max", "default"]),
).optional(),
}).passthrough(),
).optional(),
pricing: Pricing,
providers: z.array(LLMGatewayProvider),
context_length: z.number(),
// Absent for pseudo-models (custom/auto) and some non-text mappings; text
// models always report it.
context_length: z.number().optional(),
max_output: z.number().optional(),
supported_parameters: z.array(z.string()),
structured_outputs: z.boolean().optional(),
}).passthrough();
@@ -67,31 +81,117 @@ export const LLMGatewayResponse = z.object({
export type LLMGatewayModel = z.infer<typeof LLMGatewayModel>;
async function fetchLLMGatewayModels(url: string) {
const headers = process.env.LLMGATEWAY_API_KEY
? { Authorization: `Bearer ${process.env.LLMGATEWAY_API_KEY}` }
: undefined;
const response = await fetch(url, { headers });
if (!response.ok) {
throw new Error(`LLM Gateway request failed: ${response.status} ${response.statusText}`);
}
return response.json();
}
function textOnly(model: LLMGatewayModel) {
const output = model.architecture.output_modalities;
return output.length === 1 && output[0] === "text";
}
// The DevPass (LLM Gateway) provider: the gateway's aggregated catalog of root
// model IDs, auto-routed across upstream providers.
export const llmgateway = {
id: "llmgateway",
name: "LLM Gateway",
name: "DevPass (LLM Gateway)",
modelsDir: "providers/llmgateway/models",
async fetchModels() {
const headers = process.env.LLMGATEWAY_API_KEY
? { Authorization: `Bearer ${process.env.LLMGATEWAY_API_KEY}` }
: undefined;
const response = await fetch(API_ENDPOINT, { headers });
if (!response.ok) {
throw new Error(`LLM Gateway request failed: ${response.status} ${response.statusText}`);
}
return response.json();
return fetchLLMGatewayModels(API_ENDPOINT);
},
parseModels(raw) {
return LLMGatewayResponse.parse(raw).data.filter((model) => {
const output = model.architecture.output_modalities;
return output.length === 1 && output[0] === "text";
});
const data = LLMGatewayResponse.parse(raw).data.filter(textOnly);
// An empty catalog is an upstream fault; syncing it would delete every
// model file, so fail loudly instead.
if (data.length === 0) {
throw new Error("LLM Gateway returned no text models");
}
return data;
},
translateModel(model, context) {
return {
id: model.id,
model: buildLLMGatewayModel(model, context.existing(model.id)),
};
const translated = buildLLMGatewayModel(model, context.existing(model.id));
if (translated === undefined) {
return undefined;
}
return { id: model.id, model: translated };
},
sourceID(model) {
return model.id;
},
} satisfies SyncProvider<LLMGatewayModel>;
// Every toggle reasoning control requires a leading wire-path comment, and the
// sync runner only carries over headers that already exist on disk. Files this
// sync writes with a toggle get the gateway-wide default; a hand-written
// header on the existing file always wins.
const TOGGLE_HEADER = `# Toggle: $.reasoning_effort = "none" disables thinking; any other accepted
# value (or omitting the field) leaves it on. The gateway maps it to the
# deployment's thinking switch.
# https://docs.llmgateway.io/features/reasoning
`;
function toggleHeader(model: SyncedModel) {
return model.reasoning_options?.some((option) => option.type === "toggle")
? TOGGLE_HEADER
: undefined;
}
// The LLM Gateway provider: one entry per upstream provider mapping, addressed
// the way the gateway accepts provider-pinned requests (`provider/model-id`).
export const llmgatewayProviders = {
id: "llmgateway-providers",
name: "LLM Gateway",
modelsDir: "providers/llmgateway-providers/models",
async fetchModels() {
return fetchLLMGatewayModels(`${API_ENDPOINT}?mapped=true`);
},
parseModels(raw) {
const data = LLMGatewayResponse.parse(raw).data;
// A deployment without the mapped view ignores the query param and returns
// aggregated root IDs (no provider prefix); syncing those here would wipe
// the provider-pinned catalog, so refuse to proceed. An empty response (or
// one left empty after filtering) would silently do the same via the
// delete-missing pass, so it is equally fatal.
if (data.length === 0 || !data.every((model) => model.id.includes("/"))) {
throw new Error("LLM Gateway mapped view unavailable: response is empty or contains unprefixed model ids");
}
// llmgateway/custom is the BYO-model placeholder and llmgateway/auto the
// auto-router; pinning either to a provider is meaningless in this catalog
// (the aggregated llmgateway provider carries `auto`).
const mapped = data.filter((model) => !model.id.startsWith("llmgateway/") && textOnly(model));
if (mapped.length === 0) {
throw new Error("LLM Gateway mapped view returned no text models");
}
// Every mapped entry is one specific provider deployment whose single
// providers[] mapping drives capabilities and reasoning controls. A kept
// entry with zero or several mappings would make the builder silently fall
// back to noisy supported_parameters / sibling defaults, so fail loudly.
const malformed = mapped.filter((model) => model.providers?.length !== 1);
if (malformed.length > 0) {
throw new Error(
`LLM Gateway mapped view returned entries without exactly one provider mapping: ${
malformed.map((model) => model.id).join(", ")
}`,
);
}
return mapped;
},
translateModel(model, context) {
const translated = buildLLMGatewayMappedModel(model, context.existing(model.id));
if (translated === undefined) {
return undefined;
}
return { id: model.id, model: translated, header: toggleHeader(translated) };
},
sourceID(model) {
return model.id;
},
} satisfies SyncProvider<LLMGatewayModel>;
@@ -125,12 +225,75 @@ function modalities(values: string[], fallback: Modality[]): Modality[] {
return [...new Set(result.length > 0 ? result : fallback)];
}
function resolveLLMGatewayBaseModel(model: LLMGatewayModel) {
const alias = BASE_MODEL_ALIASES[model.id];
// Modalities as served by a specific deployment: a mapping without vision must
// not carry image/pdf input, regardless of what the model-level architecture
// claims — attachment=false with image input is contradictory.
function deploymentModalities(model: LLMGatewayModel, vision: boolean | undefined) {
const base = defaultModalities(model);
if (vision !== false) {
return base;
}
const input = base.input.filter((value) => value !== "image" && value !== "pdf");
return {
input: input.length > 0 ? input : (["text"] satisfies Modality[]),
output: base.output,
};
}
const MODELS_DIR = path.join(import.meta.dirname, "..", "..", "..", "..", "..", "models");
const AGGREGATED_MODELS_DIR = path.join(import.meta.dirname, "..", "..", "..", "..", "..", "providers", "llmgateway", "models");
const canonicalOutputLimitByID = new Map<string, number | undefined>();
interface SiblingCuration {
reasoning_options?: SyncedFullModel["reasoning_options"];
interleaved?: SyncedFullModel["interleaved"];
cost_tiers?: NonNullable<SyncedFullModel["cost"]>["tiers"];
}
const siblingCurationByID = new Map<string, SiblingCuration>();
// The aggregated llmgateway catalog curates reasoning controls, the reasoning
// side-channel, and context pricing tiers for the same gateway surface; mapped
// deployments of the same root model reuse them when the deployment does not
// declare its own.
function siblingCuration(rootID: string): SiblingCuration {
let curation = siblingCurationByID.get(rootID);
if (curation === undefined) {
const filePath = path.join(AGGREGATED_MODELS_DIR, `${rootID}.toml`);
const authored = existsSync(filePath)
? Bun.TOML.parse(readFileSync(filePath, "utf8")) as SiblingCuration & {
cost?: { tiers?: NonNullable<SyncedFullModel["cost"]>["tiers"] };
}
: undefined;
curation = {
reasoning_options: authored?.reasoning_options?.length ? authored.reasoning_options : undefined,
interleaved: authored?.interleaved,
cost_tiers: authored?.cost?.tiers,
};
siblingCurationByID.set(rootID, curation);
}
return curation;
}
// Whether the canonical metadata declares limit.output; factored entries can
// only omit their own output override when the base has one to inherit.
function canonicalOutputLimit(modelID: string) {
if (!canonicalOutputLimitByID.has(modelID)) {
const filePath = path.join(MODELS_DIR, `${modelID}.toml`);
const metadata = existsSync(filePath)
? Bun.TOML.parse(readFileSync(filePath, "utf8")) as { limit?: { output?: number } }
: undefined;
canonicalOutputLimitByID.set(modelID, metadata?.limit?.output);
}
return canonicalOutputLimitByID.get(modelID);
}
function resolveLLMGatewayBaseModel(model: LLMGatewayModel, modelID = model.id) {
const alias = BASE_MODEL_ALIASES[modelID];
if (alias !== undefined) return alias;
if (model.family === undefined) return undefined;
const prefix = CANONICAL_FAMILY_ALIASES[model.family] ?? model.family;
return resolveCanonicalBaseModel(`${prefix}/${model.id}`);
return resolveModelMetadataBaseModel(`${prefix}/${modelID}`);
}
function inferFamily(model: LLMGatewayModel, name: string) {
@@ -152,15 +315,19 @@ function inferFamily(model: LLMGatewayModel, name: string) {
export function buildLLMGatewayModel(
model: LLMGatewayModel,
existing: ExistingModel | undefined,
): SyncedModel {
): SyncedModel | undefined {
const prompt = price(model.pricing.prompt);
const completion = price(model.pricing.completion);
const reasoning = model.supported_parameters.includes("reasoning")
|| model.supported_parameters.includes("include_reasoning");
const reasoningOptions = llmGatewayReasoningOptions(model, existing);
const context = model.context_length > 0
? model.context_length
: existing?.limit?.context ?? model.context_length;
const reported = model.context_length ?? 0;
// A missing/zero context must never be authored as limit.context = 0:
// factored entries leave it unset and inherit the base, and unfactored
// creates are skipped entirely. An authored 0 on the existing file is
// equally unusable and must not be re-stamped.
const servedContext = reported > 0 ? reported : undefined;
const context = servedContext ?? (existing?.limit?.context || undefined);
// The gateway is authoritative for the volatile, gateway-specific data — cost,
// served limits, and explicitly advertised reasoning efforts. Its
@@ -179,15 +346,24 @@ export function buildLLMGatewayModel(
tiers: existing?.cost?.tiers,
}
: existing?.cost;
const limit = {
context,
input: existing?.limit?.input,
output: existing?.limit?.output ?? context,
};
// Authored limits carry only known-positive values — never the zero/absent
// `reported` fallback.
const limit = context !== undefined
? {
context,
input: existing?.limit?.input,
output: (existing?.limit?.output || undefined) ?? context,
}
: undefined;
// Existing factored model: refresh cost + limit, keep every authored override
// as-is (undefined fields keep inheriting the base model).
if (existing?.base_model !== undefined) {
const factoredLimit = {
context,
input: existing.limit?.input,
output: existing.limit?.output ?? context,
};
return factorBaseModel(
existing.base_model,
{
@@ -200,7 +376,7 @@ export function buildLLMGatewayModel(
tool_call: existing.tool_call,
structured_output: existing.structured_output,
open_weights: existing.open_weights,
limit,
limit: factoredLimit,
modalities: existing.modalities,
}),
reasoning: existing.reasoning,
@@ -212,16 +388,22 @@ export function buildLLMGatewayModel(
interleaved: existing.interleaved,
knowledge: existing.knowledge,
modalities: existing.modalities,
limit,
limit: factoredLimit,
cost,
},
limit,
factoredLimit,
existing.base_model_omit,
);
}
// Existing full model: refresh cost + limit, preserve curated metadata.
if (existing !== undefined) {
// With no usable context from the API or the file there is nothing valid
// to author, and skipping would hand the file to the delete-missing pass —
// fail loudly rather than write limit.context = 0.
if (limit === undefined) {
throw new Error(`LLM Gateway entry ${model.id} has no usable context to author`);
}
return {
name: existing.name ?? model.name,
description: existing.description ?? describeModel({
@@ -271,7 +453,12 @@ export function buildLLMGatewayModel(
}
// Brand-new model: best-effort translation from the gateway. Capability and
// modality data are unreliable here and should be hand-reviewed.
// modality data are unreliable here and should be hand-reviewed. Without a
// positive served context there is nothing usable to author, so skip.
if (servedContext === undefined) {
return undefined;
}
const createdLimit = limit ?? { context: servedContext, input: undefined, output: servedContext };
const { input, output } = defaultModalities(model);
return {
name: model.name,
@@ -284,7 +471,7 @@ export function buildLLMGatewayModel(
|| model.supported_parameters.includes("tool_choice"),
structured_output: model.structured_outputs ?? false,
open_weights: false,
limit,
limit: createdLimit,
modalities: { input, output },
}),
family: inferFamily(model, model.name),
@@ -299,7 +486,261 @@ export function buildLLMGatewayModel(
structured_output: model.structured_outputs ?? false,
open_weights: false,
cost,
limit,
limit: createdLimit,
modalities: { input, output },
} satisfies SyncedFullModel;
}
export function buildLLMGatewayMappedModel(
model: LLMGatewayModel,
existing: ExistingModel | undefined,
): SyncedModel | undefined {
// Mapped entries carry exactly one provider mapping; its capability flags
// describe that specific deployment, unlike the aggregated view where
// supported_parameters are too noisy to trust.
const mapping = model.providers?.[0];
const rootID = model.id.split("/").slice(1).join("/");
const prompt = price(model.pricing.prompt);
const completion = price(model.pricing.completion);
// The mapping's flag stays authoritative on resyncs too, so the written
// reasoning boolean and the reasoning_options derived from it always move
// together; prior curation only fills in when the mapping is silent, then
// the noisy supported_parameters signal as a last resort.
const reasoning = mapping?.reasoning
?? existing?.reasoning
?? (model.supported_parameters.includes("reasoning")
|| model.supported_parameters.includes("include_reasoning"));
// The exact reasoning_effort values this deployment accepts. A deployment
// whose only accepted effort is "none" exposes a plain on/off switch (the
// gateway honours it through the thinking toggle), not effort tiers.
const deploymentOptions = mapping?.reasoning_efforts?.length
? mapping.reasoning_efforts.length === 1 && mapping.reasoning_efforts[0] === "none"
? [{ type: "toggle" as const }]
: [{ type: "effort" as const, values: mapping.reasoning_efforts }]
: undefined;
// Deployment-declared efforts own the effort/toggle surface; curation falls
// back from non-empty options on this file to the aggregated llmgateway
// catalog's controls for the same root model on the same gateway surface.
// Curated non-effort controls (e.g. budget_tokens for $.reasoning.max_tokens,
// which this host serves regardless of the effort list) survive alongside
// deployment efforts instead of being wiped by them. A curated [] counts as
// unknown so a bad first stamp is not sticky. Non-reasoning deployments
// carry none; the same applies to the interleaved reasoning side-channel.
const sibling = siblingCuration(rootID);
const curatedOptions = (existing?.reasoning_options?.length ? existing.reasoning_options : undefined)
?? sibling.reasoning_options;
const reasoningOptions = reasoning
? deploymentOptions !== undefined
? [
...(curatedOptions ?? []).filter((option) => option.type !== "effort" && option.type !== "toggle"),
...deploymentOptions,
]
: curatedOptions
: undefined;
const interleaved = reasoning
? existing?.interleaved ?? sibling.interleaved
: undefined;
const reported = model.context_length ?? 0;
// Same zero-context rule as the aggregated builder: never author 0, inherit
// on factored entries, skip unfactored creates. An authored 0 on the
// existing file is equally unusable.
const servedContext = reported > 0 ? reported : undefined;
const context = servedContext ?? (existing?.limit?.context || undefined);
const cost = prompt !== undefined && completion !== undefined
? {
input: prompt,
output: completion,
reasoning: reasoning ? nonZeroPrice(model.pricing.internal_reasoning) ?? existing?.cost?.reasoning : existing?.cost?.reasoning,
cache_read: nonZeroPrice(model.pricing.input_cache_read) ?? existing?.cost?.cache_read,
cache_write: nonZeroPrice(model.pricing.input_cache_write) ?? existing?.cost?.cache_write,
// The gateway API does not expose context pricing tiers, so authored
// tiers stick and new files seed from the aggregated sibling's curated
// tiers rather than silently under-stating long-context pricing.
tiers: existing?.cost?.tiers ?? sibling.cost_tiers,
}
: existing?.cost;
// The gateway's max_output is the deployment's real served limit, so it wins
// over inherited/authored values, unlike the aggregated view.
const servedOutput = (model.max_output || undefined) ?? (existing?.limit?.output || undefined);
// Authored limits carry only known-positive values — never the zero/absent
// `reported` fallback.
const limit = context !== undefined
? {
context,
input: existing?.limit?.input,
output: servedOutput ?? context,
}
: undefined;
// Existing factored model: refresh cost + limit, keep every authored override
// as-is. Unlike the aggregated provider, the name override must be carried
// forward: mapped names disambiguate deployments of the same model (e.g.
// "GPT-5.5 (Azure)" vs "GPT-5.5 (OpenAI)") and must not collapse back to the
// base metadata name.
if (existing?.base_model !== undefined) {
// Mirror the brand-new factored path: without a served or authored output,
// keep inheriting the base's output rather than stamping context over it.
const factoredLimit = {
context,
input: existing.limit?.input,
output: servedOutput ?? (canonicalOutputLimit(existing.base_model) !== undefined ? undefined : context),
};
// Deployment capability flags keep their create-path authority on
// resyncs: a mapping that gains or loses reasoning/vision/tools/structured
// outputs realigns the written flags together with the reasoning_options
// computed from them, instead of freezing stale curation forever.
return factorBaseModel(
existing.base_model,
{
name: existing.name ?? model.name,
attachment: mapping?.vision ?? existing.attachment,
// No describeModel fallback: synthesizing a description here would
// stamp a sticky generic override on every name-pinned factored entry;
// leaving it unset keeps inheriting the lab text from the base.
description: existing.description,
reasoning: mapping?.reasoning ?? existing.reasoning,
reasoning_options: reasoningOptions,
temperature: existing.temperature,
tool_call: mapping?.tools ?? existing.tool_call,
structured_output: model.structured_outputs ?? existing.structured_output,
status: existing.status,
interleaved,
knowledge: existing.knowledge,
// Vision realigns modalities in both directions: false strips
// image/pdf, true clears any stale stripped override so the base's
// richer inputs inherit again; only a silent mapping keeps curation.
modalities: mapping?.vision === undefined
? existing.modalities
: mapping.vision
? undefined
: deploymentModalities(model, false),
limit: factoredLimit,
cost,
},
factoredLimit,
existing.base_model_omit,
);
}
// Existing full model: refresh cost + limit, preserve curated metadata.
// Capability flags follow the same rule as the factored path above: the
// deployment mapping wins, curation fills the gaps.
if (existing !== undefined) {
// With no usable context from the API or the file there is nothing valid
// to author, and skipping would hand the file to the delete-missing pass —
// fail loudly rather than write limit.context = 0.
if (limit === undefined) {
throw new Error(`LLM Gateway mapped entry ${model.id} has no usable context to author`);
}
const resolved = {
attachment: mapping?.vision ?? existing.attachment ?? false,
tool_call: mapping?.tools ?? existing.tool_call ?? false,
structured_output: model.structured_outputs ?? existing.structured_output,
// Same bidirectional vision rule as the factored path; with no base to
// inherit from, a declared vision recomputes from the served
// architecture instead of clearing.
modalities: mapping?.vision === undefined
? existing.modalities ?? deploymentModalities(model, undefined)
: deploymentModalities(model, mapping.vision),
};
return {
name: existing.name ?? model.name,
description: existing.description ?? describeModel({
id: model.id,
name: existing.name ?? model.name,
family: existing.family,
reasoning,
tool_call: resolved.tool_call,
structured_output: resolved.structured_output,
open_weights: existing.open_weights,
limit,
modalities: resolved.modalities,
}),
family: existing.family,
release_date: existing.release_date ?? dateFromTimestamp(model.created),
last_updated: existing.last_updated ?? dateFromTimestamp(model.created),
attachment: resolved.attachment,
reasoning,
reasoning_options: reasoningOptions,
temperature: existing.temperature ?? false,
tool_call: resolved.tool_call,
structured_output: resolved.structured_output,
knowledge: existing.knowledge,
open_weights: existing.open_weights ?? false,
status: existing.status,
interleaved,
cost,
limit,
modalities: resolved.modalities,
} satisfies SyncedFullModel;
}
// Brand-new model with a reviewed metadata entry: factor against the
// canonical base. The mapped ID is `serving-provider/model-id` and the
// serving provider is unrelated to the originating lab, so resolve the base
// from the root model ID + family, and keep the disambiguating name. The
// mapping's own capability flags describe this specific deployment, so they
// go in as overrides (factorBaseModel drops the ones equal to the base).
const canonical = resolveLLMGatewayBaseModel(model, rootID);
if (canonical !== undefined) {
const factoredLimit = {
context,
input: undefined,
// Without a served limit, inherit the base's output; only fall back to
// context when the base declares none (output is required downstream).
output: model.max_output ?? (canonicalOutputLimit(canonical) !== undefined ? undefined : context),
};
return factorBaseModel(canonical, {
name: model.name,
attachment: mapping?.vision,
reasoning: mapping?.reasoning,
reasoning_options: reasoningOptions,
interleaved,
tool_call: mapping?.tools,
structured_output: model.structured_outputs,
// A deployment without vision must not inherit image/pdf inputs from
// the base — attachment=false with image input is contradictory.
modalities: mapping?.vision === false ? deploymentModalities(model, false) : undefined,
limit: factoredLimit,
cost,
}, factoredLimit);
}
// Brand-new model without metadata: best-effort translation. The mapping's
// own capability flags are reliable here; modalities mirror the mapping too.
// Without a positive served context there is nothing usable to author.
if (servedContext === undefined) {
return undefined;
}
const createdLimit = limit ?? { context: servedContext, input: undefined, output: servedOutput ?? servedContext };
const { input, output } = deploymentModalities(model, mapping?.vision);
return {
name: model.name,
description: describeModel({
id: model.id,
name: model.name,
family: inferFamily(model, model.name),
reasoning,
tool_call: mapping?.tools ?? false,
structured_output: model.structured_outputs ?? false,
open_weights: false,
limit: createdLimit,
modalities: { input, output },
}),
family: inferFamily(model, model.name),
release_date: dateFromTimestamp(model.created),
last_updated: dateFromTimestamp(model.created),
attachment: mapping?.vision ?? input.some((value) => value !== "text"),
reasoning,
reasoning_options: reasoningOptions,
interleaved,
temperature: model.supported_parameters.includes("temperature"),
tool_call: mapping?.tools ?? false,
structured_output: model.structured_outputs ?? false,
open_weights: false,
cost,
limit: createdLimit,
modalities: { input, output },
} satisfies SyncedFullModel;
}
@@ -308,7 +749,7 @@ function llmGatewayReasoningOptions(
model: LLMGatewayModel,
existing: ExistingModel | undefined,
): SyncedFullModel["reasoning_options"] {
const advertised = new Set(model.providers.flatMap((provider) => provider.reasoning_efforts ?? []));
const advertised = new Set((model.providers ?? []).flatMap((provider) => provider.reasoning_efforts ?? []));
if (advertised.size === 0) return undefined;
const efforts = [...advertised].sort((a, b) => {
+407 -1
View File
@@ -56,7 +56,13 @@ import {
resolveCanonicalBaseModel,
type OpenRouterModel,
} from "../src/sync/providers/openrouter.js";
import { buildLLMGatewayModel, type LLMGatewayModel } from "../src/sync/providers/llmgateway.js";
import {
buildLLMGatewayMappedModel,
buildLLMGatewayModel,
llmgateway,
llmgatewayProviders,
type LLMGatewayModel,
} from "../src/sync/providers/llmgateway.js";
import {
buildMergeGatewayModel,
fetchMergeGatewayModels,
@@ -2675,6 +2681,11 @@ test("defaults new reasoning models to empty reasoning options", () => {
});
});
test("inherits base reasoning options instead of stamping empty ones", () => {
expect(preserveReasoningOptions({ reasoning: true }, undefined, undefined, [{ type: "toggle" }]))
.toEqual({ reasoning: true });
});
test("normalizes Cortecs file modalities to pdf", () => {
const [model] = cortecs.parseModels({
object: "list",
@@ -3021,6 +3032,52 @@ test("factors aliased LLM Gateway routes against canonical metadata", () => {
});
});
test("factors mapped LLM Gateway entries against the root model metadata", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel(), undefined);
expect(model).toEqual({
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
reasoning_options: [{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] }],
structured_output: true,
cost: {
input: 10,
output: 50,
cache_read: 1,
cache_write: 12.5,
},
});
});
test("applies deployment capability flags on mapped factored entries", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
providers: [{ providerId: "anthropic", vision: false, tools: false, reasoning: false }],
architecture: { input_modalities: ["text"], output_modalities: ["text"] },
max_output: 64_000,
}), undefined);
expect(model).toEqual({
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
attachment: false,
reasoning: false,
tool_call: false,
structured_output: true,
modalities: {
input: ["text"],
},
limit: {
output: 64_000,
},
cost: {
input: 10,
output: 50,
cache_read: 1,
cache_write: 12.5,
},
});
});
test("factors Grok LLM Gateway routes against xAI metadata", () => {
const model = buildLLMGatewayModel(llmGatewayModel({
id: "grok-4-6",
@@ -3044,6 +3101,339 @@ test("factors Grok LLM Gateway routes against xAI metadata", () => {
});
});
test("prefers the gateway max_output over authored output on mapped resyncs", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({ max_output: 32_000 }), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
description: "Claude Fable 5 served by Anthropic",
limit: { output: 64_000 },
});
expect(model).toEqual({
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
description: "Claude Fable 5 served by Anthropic",
reasoning_options: [{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] }],
structured_output: true,
limit: {
output: 32_000,
},
cost: {
input: 10,
output: 50,
cache_read: 1,
cache_write: 12.5,
},
});
});
test("translates a none-only effort list into a reasoning toggle", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
providers: [{ providerId: "anthropic", vision: true, tools: true, reasoning: true, reasoning_efforts: ["none"] }],
}), undefined);
expect(model).toMatchObject({
base_model: "anthropic/claude-fable-5",
reasoning_options: [{ type: "toggle" }],
});
});
test("realigns capability flags from the mapping on mapped factored resyncs", () => {
// The deployment dropped reasoning and gained tools since the file was
// written: the resync must move the booleans and the reasoning controls
// together instead of clearing options under a frozen reasoning = true.
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
providers: [{ providerId: "anthropic", vision: true, tools: true, reasoning: false }],
}), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
reasoning: true,
reasoning_options: [{ type: "toggle" }],
tool_call: false,
});
expect(model).toMatchObject({ reasoning: false });
expect(model!.reasoning_options).toBeUndefined();
// Realigned to the mapping and now equal to the base, the stale
// tool_call = false override is dropped and inherits the base again.
expect(model!.tool_call).toBeUndefined();
});
test("restores image input when vision returns on mapped resyncs", () => {
// The file was written while the deployment had no vision (text-only
// stripped modalities); vision is back, so the stale override must clear.
const factored = buildLLMGatewayMappedModel(llmGatewayMappedModel(), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
attachment: false,
modalities: { input: ["text"] },
});
expect(factored!.modalities).toBeUndefined();
expect(factored!.attachment).toBeUndefined();
const full = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "acme/mystery-model",
name: "Mystery Model (Acme)",
family: undefined,
providers: [{ providerId: "acme", vision: true, tools: true, reasoning: false }],
}), {
name: "Mystery Model (Acme)",
attachment: false,
modalities: { input: ["text"], output: ["text"] },
});
expect(full).toMatchObject({
attachment: true,
modalities: { input: ["text", "image"], output: ["text"] },
});
});
test("never synthesizes a description on mapped factored resyncs", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel(), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
});
// An unset description must keep inheriting the base's lab text instead of
// being stamped with a sticky synthesized override on the first resync.
expect(model).toBeDefined();
expect(model!.description).toBeUndefined();
});
test("authors the toggle wire-path header on mapped sync creates", () => {
const context = { existing: () => undefined, authored: () => undefined };
const toggle = llmgatewayProviders.translateModel(llmGatewayMappedModel({
providers: [{ providerId: "anthropic", vision: true, tools: true, reasoning: true, reasoning_efforts: ["none"] }],
}), context);
expect(toggle?.header).toStartWith("# Toggle: $.reasoning_effort");
const effort = llmgatewayProviders.translateModel(llmGatewayMappedModel(), context);
expect(effort?.model).toMatchObject({
reasoning_options: [{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] }],
});
expect(effort?.header).toBeUndefined();
});
test("keeps inheriting base output on factored resyncs without max_output", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({ max_output: undefined }), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
description: "Claude Fable 5 served by Anthropic",
});
expect(model).toEqual({
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
description: "Claude Fable 5 served by Anthropic",
reasoning_options: [{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] }],
structured_output: true,
cost: {
input: 10,
output: 50,
cache_read: 1,
cache_write: 12.5,
},
});
});
test("skips unfactorable LLM Gateway creates without a served context", () => {
// Unknown family, so no canonical base to inherit a context from.
const mapped = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "acme/mystery-model",
name: "Mystery Model (Acme)",
family: undefined,
context_length: undefined,
}), undefined);
expect(mapped).toBeUndefined();
const aggregated = buildLLMGatewayModel(llmGatewayModel({
id: "mystery-model",
name: "Mystery Model",
family: undefined,
context_length: undefined,
}), undefined);
expect(aggregated).toBeUndefined();
});
test("keeps curated budget controls under deployment efforts", () => {
// Deployment efforts own only the effort/toggle surface: the hand-authored
// budget_tokens control (this host's $.reasoning.max_tokens path) survives
// the resync, while the stale effort list is replaced.
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel(), {
base_model: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
reasoning_options: [
{ type: "effort", values: ["low", "high"] },
{ type: "budget_tokens", min: 1_024, max: 63_999 },
],
});
expect(model!.reasoning_options).toEqual([
{ type: "budget_tokens", min: 1_024, max: 63_999 },
{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] },
]);
// Same merge on creates, with the budget coming from the aggregated
// sibling's curation for the same root model.
const seeded = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "anthropic/claude-sonnet-4-6",
name: "Claude Sonnet 4.6 (Anthropic)",
}), undefined);
expect(seeded!.reasoning_options).toEqual([
{ type: "budget_tokens", min: 1_024, max: 63_999 },
{ type: "effort", values: ["low", "medium", "high", "xhigh", "max"] },
]);
});
test("seeds context pricing tiers from the aggregated sibling on creates", () => {
// The gateway API carries no tier pricing; without the sibling's curated
// tiers the bulk sync would author tiered models at flat long-context rates.
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "openai/gpt-5.5",
name: "GPT-5.5 (OpenAI)",
family: "openai",
}), undefined);
expect(model!.cost?.tiers).toEqual([
{ tier: { type: "context", size: 272_000 }, input: 10, output: 45, cache_read: 1 },
]);
});
test("factors perplexity entries without widening the shared prefix map", () => {
// The perplexity family resolves through resolveModelMetadataBaseModel's
// exact models/ path match; CANONICAL_PROVIDER_PREFIXES stays untouched so
// other hosts' standalone perplexity files keep their current behavior.
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "perplexity/sonar-pro",
name: "Sonar Pro (Perplexity)",
family: "perplexity",
}), undefined);
expect(model).toMatchObject({ base_model: "perplexity/sonar-pro" });
});
test("refuses to author a zero context on full LLM Gateway resyncs", () => {
// Existing full rows (no base to inherit from) with nothing usable from the
// API or the file must fail loudly instead of being rewritten with
// limit.context = 0.
expect(() => buildLLMGatewayMappedModel(llmGatewayMappedModel({
context_length: undefined,
max_output: undefined,
}), {
name: "Claude Fable 5 (Anthropic)",
})).toThrow("no usable context");
// An authored 0 on disk is as unusable as an absent context.
expect(() => buildLLMGatewayMappedModel(llmGatewayMappedModel({
context_length: 0,
max_output: undefined,
}), {
name: "Claude Fable 5 (Anthropic)",
limit: { context: 0 },
})).toThrow("no usable context");
expect(() => buildLLMGatewayModel(llmGatewayModel({
context_length: undefined,
}), {
name: "Claude Fable 5",
})).toThrow("no usable context");
});
test("leaves context unset on mapped factored creates without a served context", () => {
const model = buildLLMGatewayMappedModel(llmGatewayMappedModel({
context_length: undefined,
max_output: undefined,
}), undefined);
// Everything limit-related inherits from the base; no zero is authored.
expect(model).toBeDefined();
expect("limit" in model!).toBe(false);
});
test("strips image input when the deployment has no vision", () => {
// The model-level architecture still claims image input; the deployment
// flag must win on both the factored and the unfactored path.
const factored = buildLLMGatewayMappedModel(llmGatewayMappedModel({
providers: [{ providerId: "anthropic", vision: false, tools: true, reasoning: false }],
}), undefined);
expect(factored).toMatchObject({
base_model: "anthropic/claude-fable-5",
attachment: false,
modalities: { input: ["text"] },
});
const full = buildLLMGatewayMappedModel(llmGatewayMappedModel({
id: "acme/mystery-model",
name: "Mystery Model (Acme)",
family: undefined,
providers: [{ providerId: "acme", vision: false, tools: true, reasoning: false }],
}), undefined);
expect(full).toMatchObject({
attachment: false,
modalities: { input: ["text"], output: ["text"] },
});
});
test("refuses empty responses in both LLM Gateway syncs", () => {
expect(() => llmgateway.parseModels({ data: [] })).toThrow("no text models");
expect(() => llmgatewayProviders.parseModels({ data: [] })).toThrow("mapped view unavailable");
});
test("refuses aggregated responses in the mapped LLM Gateway sync", () => {
expect(() => llmgatewayProviders.parseModels({ data: [llmGatewayModel()] }))
.toThrow("mapped view unavailable");
});
test("filters pseudo and non-text entries from the mapped LLM Gateway sync", () => {
const parsed = llmgatewayProviders.parseModels({
data: [
llmGatewayMappedModel(),
llmGatewayMappedModel({ id: "llmgateway/auto", name: "Auto Route (LLM Gateway)" }),
llmGatewayMappedModel({
id: "openai/sora-2",
name: "Sora 2 (OpenAI)",
architecture: { input_modalities: ["text"], output_modalities: ["video"] },
}),
],
});
expect(parsed.map((model) => model.id)).toEqual(["anthropic/claude-fable-5"]);
});
test("refuses mapped LLM Gateway entries without exactly one provider mapping", () => {
expect(() => llmgatewayProviders.parseModels({
data: [llmGatewayMappedModel({ providers: undefined })],
})).toThrow("without exactly one provider mapping");
expect(() => llmgatewayProviders.parseModels({
data: [llmGatewayMappedModel({ providers: [] })],
})).toThrow("without exactly one provider mapping");
expect(() => llmgatewayProviders.parseModels({
data: [
llmGatewayMappedModel(),
llmGatewayMappedModel({
id: "azure/gpt-5.5",
name: "GPT-5.5 (Azure)",
providers: [{ providerId: "azure" }, { providerId: "openai" }],
}),
],
})).toThrow("azure/gpt-5.5");
// Entries the sync drops anyway (pseudo-models, non-text) may lack a
// mapping without tripping the guard.
const parsed = llmgatewayProviders.parseModels({
data: [
llmGatewayMappedModel(),
llmGatewayMappedModel({
id: "llmgateway/auto",
name: "Auto Route (LLM Gateway)",
providers: undefined,
}),
],
});
expect(parsed.map((model) => model.id)).toEqual(["anthropic/claude-fable-5"]);
});
// Ensures catalog pagination preserves authentication and returns every page.
test("fetches every page of the Merge Gateway catalog", async () => {
const requests: string[] = [];
@@ -3913,6 +4303,22 @@ function llmGatewayModel(overrides: Partial<LLMGatewayModel> = {}): LLMGatewayMo
};
}
function llmGatewayMappedModel(overrides: Partial<LLMGatewayModel> = {}): LLMGatewayModel {
return llmGatewayModel({
id: "anthropic/claude-fable-5",
name: "Claude Fable 5 (Anthropic)",
providers: [{
providerId: "anthropic",
vision: true,
tools: true,
reasoning: true,
reasoning_efforts: ["low", "medium", "high", "xhigh", "max"],
}],
max_output: 128_000,
...overrides,
});
}
function mergeGatewayVendor(
overrides: Partial<MergeGatewayModel["vendors"][string]> = {},
): MergeGatewayModel["vendors"][string] {
+6
View File
@@ -0,0 +1,6 @@
<svg viewBox="0 0 40 40" xmlns="http://www.w3.org/2000/svg">
<g transform="translate(1.21, 0) scale(0.17241)">
<path d="M218 59.4686c0-4.1697-2.351-7.9813-6.071-9.8441L119.973 3.58361s2.926 3.32316 2.926 7.01529V218.833c0 4.081-2.926 7.016-2.926 7.016l15.24-7.468c2.964-2.232 7.187-7.443 7.438-16.006.293-9.976.61-84.847.732-121.0353.487-3.6678 4.096-11.0032 14.63-11.0032 10.535 0 29.262 5.1348 37.309 7.7022 2.439.7336 7.608 4.1812 8.779 12.1036 1.17 7.9223.975 59.0507.731 83.6247 0 2.445.137 7.069 6.653 7.069 6.515 0 6.515-7.069 6.515-7.069V59.4686Z" fill="currentColor"/>
<path d="M149.235 86.323c0-5.5921 5.132-9.7668 10.589-8.6132l31.457 6.6495c4.061.8585 6.967 4.4207 6.967 8.5824v81.9253c0 5.868 5.121 9.169 5.121 9.169l-51.9-12.658c-1.311-.32-2.234-1.498-2.234-2.852V86.323ZM99.7535 1.15076c7.2925-3.60996 15.8305 1.71119 15.8305 9.86634V220.983c0 8.155-8.538 13.476-15.8305 9.866L6.11596 184.496C2.37105 182.642 0 178.818 0 174.63v-17.868l49.7128 19.865c4.0474 1.617 8.4447-1.372 8.4449-5.741 0-2.66-1.6975-5.022-4.2142-5.863L0 146.992v-14.305l40.2756 7.708c3.9656.759 7.6405-2.289 7.6405-6.337 0-3.286-2.4628-6.048-5.7195-6.413L0 122.917V108.48l78.5181-3.014c4.1532-.16 7.4381-3.582 7.4383-7.7498 0-4.6256-4.0122-8.2229-8.5964-7.7073L0 98.7098V82.4399l53.447-17.8738c2.3764-.7948 3.9791-3.0254 3.9792-5.5374 0-4.0961-4.0978-6.9185-7.9106-5.4486L0 72.6695V57.3696c.0000304-4.1878 2.37107-8.0125 6.11596-9.8664L99.7535 1.15076Z" fill="currentColor"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.4 KiB

@@ -0,0 +1,13 @@
base_model = "anthropic/claude-opus-4-8"
name = "Claude Opus 4.8 (Anthropic)"
structured_output = true
[[reasoning_options]]
type = "effort"
values = ["low", "medium", "high", "xhigh", "max"]
[cost]
input = 5
output = 25
cache_read = 0.5
cache_write = 6.25
@@ -0,0 +1,13 @@
base_model = "anthropic/claude-sonnet-5"
name = "Claude Sonnet 5 (Anthropic)"
structured_output = true
[[reasoning_options]]
type = "effort"
values = ["low", "medium", "high", "xhigh", "max"]
[cost]
input = 2
output = 10
cache_read = 0.2
cache_write = 2.5
@@ -0,0 +1,17 @@
base_model = "openai/gpt-5.5"
name = "GPT-5.5 (Azure)"
[[reasoning_options]]
type = "effort"
values = ["none", "low", "medium", "high", "xhigh"]
[cost]
input = 5
output = 30
cache_read = 0.5
[[cost.tiers]]
tier = { type = "context", size = 272_000 }
input = 10
output = 45
cache_read = 1
@@ -0,0 +1,22 @@
# Toggle: $.reasoning_effort = "none" disables thinking; any other accepted
# value (or omitting the field) leaves it on. The gateway maps it to the
# deployment's thinking switch; thinking streams back in reasoning_content.
# https://docs.llmgateway.io/features/reasoning (accessed 2026-08-04)
base_model = "zhipuai/glm-5.1"
name = "GLM-5.1 (EmberCloud)"
structured_output = false
[interleaved]
field = "reasoning_content"
[[reasoning_options]]
type = "toggle"
[cost]
input = 0.931
output = 2.93
cache_read = 0.173
[limit]
context = 203_000
output = 131_000
@@ -0,0 +1,17 @@
base_model = "openai/gpt-5.5"
name = "GPT-5.5 (OpenAI)"
[[reasoning_options]]
type = "effort"
values = ["none", "low", "medium", "high", "xhigh"]
[cost]
input = 5
output = 30
cache_read = 0.5
[[cost.tiers]]
tier = { type = "context", size = 272_000 }
input = 10
output = 45
cache_read = 1
@@ -0,0 +1,11 @@
base_model = "perplexity/sonar-pro"
name = "Sonar Pro (Perplexity)"
attachment = false
structured_output = true
[cost]
input = 3
output = 15
[modalities]
input = ["text"]
@@ -0,0 +1,28 @@
# Toggle: $.reasoning_effort = "none" disables thinking via Together's
# thinking switch. Together accepts any other effort string without validating
# it and no tier changes reasoning length, so on/off is the only real control;
# thinking streams back in reasoning_content.
# Intentionally weaker than Together's first-party row: the gateway's own
# together-ai mapping serves this pin with tool calls and JSON mode off and a
# 32k output cap, from gateway e2e against the deployment (no streaming
# content with tools; theopenco/llmgateway#2094) — this catalog mirrors what
# the gateway actually accepts and bills, and resyncs pick up any later fix.
# https://api.llmgateway.io/v1/models?mapped=true providers[].reasoning_efforts (accessed 2026-08-04)
base_model = "moonshotai/kimi-k2.6"
name = "Kimi K2.6 (Together AI)"
tool_call = false
structured_output = false
[interleaved]
field = "reasoning_content"
[[reasoning_options]]
type = "toggle"
[cost]
input = 1.2
output = 4.5
cache_read = 0.2
[limit]
output = 32_768
@@ -0,0 +1,19 @@
# The provider-pinned LLM Gateway catalog: one entry per upstream provider
# mapping, addressed as `provider/model-id` — the format the gateway accepts to
# pin a request to a specific upstream provider. The aggregated, auto-routed
# catalog of root model IDs lives in the `llmgateway` provider
# ("DevPass (LLM Gateway)"). Both are served by the same endpoint and API key.
#
# POST /v1/chat/completions accepts $.reasoning_effort = none|minimal|low|
# medium|high|xhigh|max. Its raw schema lists $.reasoning.effort = low|medium|
# high; the two effort paths are mutually exclusive. $.reasoning.max_tokens
# overrides either effort path. Anthropic budgets are clamped to 1024..128000.
# POST /v1/messages translates $.thinking to unified reasoning controls; its
# $.output_config.effort controls adaptive depth on Opus 4.7+.
# https://docs.llmgateway.io/features/reasoning (accessed 2026-06-25)
# https://docs.llmgateway.io/v1_messages (accessed 2026-06-25)
name = "LLM Gateway"
env = ["LLMGATEWAY_API_KEY"]
npm = "@ai-sdk/openai-compatible"
doc = "https://llmgateway.io/docs"
api = "https://api.llmgateway.io/v1"
+1 -1
View File
@@ -6,7 +6,7 @@
# $.output_config.effort controls adaptive depth on Opus 4.7+.
# https://docs.llmgateway.io/features/reasoning (accessed 2026-06-25)
# https://docs.llmgateway.io/v1_messages (accessed 2026-06-25)
name = "LLM Gateway"
name = "DevPass (LLM Gateway)"
env = ["LLMGATEWAY_API_KEY"]
npm = "@ai-sdk/openai-compatible"
doc = "https://llmgateway.io/docs"