Eric Allam 1cfc296c6b feat(ai): LLM metrics tracking and AI span inspector (#3213) ...

- Automatic LLM cost enrichment for AI SDK spans (streamText,
generateText, generateObject) or any other spans that use semantic
gen_ai attributes with support for 145+ models
- New AI span inspector sidebar showing model, tokens, cost, messages,
tool calls, and response text
- LLM metrics dual-write to ClickHouse `llm_metrics_v1` table for
analytics
- LLM metrics built-in dashboard (unlinked at the moment)
- Provider cost fallback — uses gateway/OpenRouter reported costs from
`providerMetadata` when registry pricing is unavailable
- Prefix-stripping for gateway/OpenRouter model names (e.g.
`mistral/mistral-large-3` matches `mistral-large-3` pricing)
- Admin dashboard for managing LLM model pricing (list, create, edit,
delete, search, test pattern matching)
- Missing models detection page — queries ClickHouse for unpriced models
with sample spans and Claude Code-ready prompts for adding pricing
- AI span seed script (`pnpm run db:seed:ai-spans`) with 51 spans across
12 provider systems for local dev testing
- UI fixes: `completionTokens`/`promptTokens` aliases,
`ai.response.object` display for generateObject, cache read/write token
breakdown

## Screenshots:

<img width="1030" height="104" alt="CleanShot 2026-03-17 at 16 48 54@2x"
src="https://github.com/user-attachments/assets/bc8fccda-e48b-4d0c-bfb1-e620064e5979"
/>

<img width="1094" height="1512" alt="CleanShot 2026-03-17 at 16 49
23@2x"
src="https://github.com/user-attachments/assets/c2424569-d07e-4d67-a436-e8250043a1ee"
/>

<img width="1074" height="1412" alt="CleanShot 2026-03-17 at 16 49
18@2x"
src="https://github.com/user-attachments/assets/22342ac4-4769-45d1-a328-a24fb9a82a50"
/>

<img width="1012" height="2292" alt="CleanShot 2026-03-17 at 16 39
01@2x"
src="https://github.com/user-attachments/assets/59e327d1-6652-4293-8be0-bb8326e5fbc5"
/>

<img width="3680" height="2392" alt="CleanShot 2026-03-15 at 08 29
38@2x"
src="https://github.com/user-attachments/assets/1f77beb8-de67-495b-b890-bcdb8d7f1fe8"
/>

---------

Co-authored-by: James Ritchie <james@trigger.dev>

2026-03-17 18:26:43 +00:00

..

cache

fix(run-engine): fix queue cache memory leak and replace MemoryStore with LRU cache (#2945)

2026-01-26 22:23:23 +00:00

clickhouse

feat(ai): LLM metrics tracking and AI span inspector (#3213)

2026-03-17 18:26:43 +00:00

database

feat(ai): LLM metrics tracking and AI span inspector (#3213)

2026-03-17 18:26:43 +00:00

emails

feat(webapp): Vercel / Slack integrations improvements (#3108)

2026-02-23 13:48:09 +00:00

llm-pricing

feat(ai): LLM metrics tracking and AI span inspector (#3213)

2026-03-17 18:26:43 +00:00

otlp-importer

feat: OTEL metrics pipeline for task workers (#3061)

2026-02-20 13:16:34 +00:00

redis

…

replication

perf(runs-replication): Improve the CPU efficiency and throughput of the runs replication to clickhouse (#2866)

2026-01-13 12:18:30 +00:00

run-engine

fix(engine): lockless waitpoint insert for batch items to eliminate lock contention (#3232)

2026-03-17 16:42:01 +00:00

run-queue/src/run-queue/tests

…

schedule-engine

Add TaskScheduleInstance projectId (#2897)

2026-01-15 15:06:52 +00:00

sdk-compat-tests

fix(core): vendor superjson to fix ESM/CJS compatibility (#2949)

2026-01-30 09:15:02 +00:00

testcontainers

Upgrade to electricsql 1.2.4 (#2668)

2025-11-13 15:19:59 +00:00

tracing

chore(redis-worker): add otel spans to fair queue processing pipeline (#2815)

2025-12-24 00:37:24 +00:00

tsql

feat(ai): LLM metrics tracking and AI span inspector (#3213)

2026-03-17 18:26:43 +00:00

zod-worker

Update zod package to version 3.25.76 across all modules (#2352)

2025-08-06 14:48:43 +01:00