Files
Katia Bulatova 4569657923 feat(webapp): dashboard agent — chat, reports, investigate (#4418)
## What & why

This is the system behind the Dashboard Agent — an assistant that
answers questions about a project's runs, errors, queues, deploys and
health, and can investigate failures end to end.

The agent runs as a chat.agent task in its own Trigger project. It has
no access to the main database or ClickHouse; all platform data is read
through the public API using a delegated, read-only user token.

Everything here is behind `canAccessDashboardAgent` and inert with the
flag off. The UI that mounts the panel lands in #4529.

## Stack

`#4418` (this, base) ← `#4529` UI ← `#4525` Watch ← `#4516` storybook
gallery. The scenario/contract reference for the whole stack is
`internal-packages/dashboard-agent/GUIDEBOOK.md` (it lands on the Watch
branch): it states, per feature, what makes each thing happen and where
that is decided.

## What's inside

**Agent runtime and tools** — `internal-packages/dashboard-agent`:
prompt, tool set (API reads, TRQL query, docs, navigation,
evidence/investigations, repo source), conversation compaction, a
prompt-prefix token budget pinned by snapshot test, and sampled
LLM-judged turn evals. The package cannot import webapp server code,
which is what makes the "no DB access" claim structural rather than a
convention.

**Contracts** — `internal-packages/dashboard-agent-contracts`:
`trigger://` URIs, intents, and the block envelope every rendered card
travels in.

**Conversation store** — `internal-packages/dashboard-agent-db`: drizzle
over postgres-js in its own `trigger_dashboard_agent` Postgres schema,
plus one additive migration.

**Auth boundary** — the user-actor token gains an optional environment
claim; one guard (`userActorEnvironment.server.ts`) enforces it so
routes don't each re-derive the rule. Token minting, cap ceiling, and
the RBAC fallback path for self-hosted.

**Transport** — webapp resource routes that mint the token and proxy
each turn, and SDK-side mid-turn reconnect.

**Public API the agent reads through** — orgs, projects, environments,
runs, queue metrics, workers, a run's commit metadata, repo snapshot,
reports, and `POST /api/v1/query`.

**Reports** — the health report's layout is declared once and shared by
the card, the markdown surface and the JSON/MCP surface, so the same
report reads the same in the dashboard, the terminal and an editor.

**Block renderers** — the report and investigation cards the flows above
already emit (`app/components/dashboard-agent/`). The panel that hosts
them, and the rest of the chat UI, is #4529.

**Query safety and CSP** — see below.

## Key decisions

- **The agent is a separate Trigger project, not webapp code.** It reads
platform data over the public API with a delegated user-actor token
whose `cap` ceilings it to read scopes. No Prisma, no ClickHouse, no
webapp imports.
- **The PAT-only auth helper now refuses user-actor tokens.** This is an
intentional behavioral change: its callers consume only a bare userId
and do not enforce delegated-token capabilities. Actor-aware routes
continue through the scoped route builders instead.
- **RBAC fallback builds a delegated token's ability from its own cap**,
never the blanket ability a PAT gets (read-only when the token declares
none). Without this, the agent's read-only cap would buy a write JWT on
self-hosted.
- **Org creation checks RBAC only for user-actor tokens, and only after
the env gate**, so an install with `ORG_CREATION_API_ENABLED` off
returns 404 rather than 403, and an ordinary PAT never consults an
ability the route has no org to scope. Both orderings are pinned by
test.
- **The query path is read-only in depth.** TRQL rejects write
statements at the grammar level (they don't parse, rather than being
filtered), ClickHouse runs with `readonly=1`, and the org/project/env
filters are injected server-side from the credential — the request body
cannot widen scope. An unparseable query denies instead of falling
through to the permissive resource.
- **Document-wide img-src CSP.** Remote images are an
outbound-request/exfiltration surface, so the policy permits only
own-origin/data/blob, the required SSO avatar hosts, and the favicon
endpoint. Operators can add exact origins through CSP_IMG_SRC_ALLOWLIST;
wildcard hosts and bare schemes are intentionally not allowed.
- **The chat transport reconnects on a mid-turn EOF**
(`@trigger.dev/sdk`). A body that ends without a turn-complete is
terminal only when the server says `X-Session-Settled: true`; otherwise
the transport resubscribes from `lastEventId` with bounded backoff, and
any record re-earns the budget. Previously a closed long-poll window or
a proxy restart left the reply stuck as if still generating.
- **Conversations live in their own datastore**, schema-scoped and
foreign-key-free (it references `organizationId`/`userId` by id, because
in cloud it is a different database). It is a display read-model for the
History tab and transport resume; `chat.agent`'s object-store snapshot
remains the model's source of truth.
- **Deterministic first.** Reports and health checks contain no LLM —
they are computed from the same data the dashboard shows, and the model
only narrates and links them. That is what makes a number in an answer
auditable.

## Testing

- 63 new test files, run with `pnpm run test --filter webapp` and
per-package vitest. Heaviest coverage on the auth boundary
(`userActorPatOnlyBoundary`, `userActorTokenClaimsAndScopes`,
`contextlessPatRoutes`, `rbacFallbackBranch`), TRQL read-only, the
report layout, and the SDK reconnect.
- The agent package has a separate eval lane (`pnpm run test:evals`,
`vitest.eval.config.ts`) that hits the real model, so it never runs in
`pnpm test`.
- Live-tested against a local stack scenario by scenario; the GUIDEBOOK
lists the condition each behaviour is expected under, which is what
those runs were checked against.

## Changelog

`.server-changes/dashboard-agent.md`, plus changesets for
`@trigger.dev/core` (report schemas), `@trigger.dev/sdk` (chat
reconnect) and the CLI's `mint-token` help text.
2026-08-11 18:56:14 +02:00

224 lines
6.9 KiB
TypeScript

import { type z } from "zod";
import type { PrismaClient } from "@trigger.dev/database";
import { prisma, type PrismaClientOrTransaction } from "~/db.server";
import {
FEATURE_FLAG,
type FeatureFlagCatalogSchema,
type FeatureFlagKey,
FeatureFlagCatalog,
} from "~/v3/featureFlags";
import { stampMintKindFlip } from "~/v3/runOpsMigration/mintFlipGrace";
export type FlagsOptions<T extends FeatureFlagKey> = {
key: T;
defaultValue?: z.infer<(typeof FeatureFlagCatalog)[T]>;
overrides?: Record<string, unknown>;
};
export function makeFlag(_prisma: PrismaClientOrTransaction = prisma) {
function flag<T extends FeatureFlagKey>(
opts: FlagsOptions<T> & { defaultValue: z.infer<(typeof FeatureFlagCatalog)[T]> }
): Promise<z.infer<(typeof FeatureFlagCatalog)[T]>>;
function flag<T extends FeatureFlagKey>(
opts: FlagsOptions<T>
): Promise<z.infer<(typeof FeatureFlagCatalog)[T]> | undefined>;
async function flag<T extends FeatureFlagKey>(
opts: FlagsOptions<T>
): Promise<z.infer<(typeof FeatureFlagCatalog)[T]> | undefined> {
const flagSchema = FeatureFlagCatalog[opts.key];
const override = opts.overrides?.[opts.key];
if (override !== undefined) {
const parsed = flagSchema.safeParse(override);
if (parsed.success) {
return parsed.data;
}
// an override that fails the schema is ignored: the global value still wins
}
const value = await _prisma.featureFlag.findFirst({
where: {
key: opts.key,
},
});
if (value !== null) {
const parsed = flagSchema.safeParse(value.value);
if (parsed.success) {
return parsed.data;
}
}
return opts.defaultValue;
}
return flag;
}
const cachedFlagStore = new Map<string, { value: unknown; expiresAt: number }>();
/**
* flag() behind a short process-level TTL cache, for global flags read on hot
* paths (e.g. the root loader) where a database round-trip per request is too
* expensive. Flips propagate within ttlMs per process. Overrides are rejected
* by the type: a scoped resolution must never be reused across scopes.
*/
export async function cachedFlag<T extends FeatureFlagKey>(
opts: Omit<FlagsOptions<T>, "overrides"> & {
defaultValue: z.infer<(typeof FeatureFlagCatalog)[T]>;
},
ttlMs = 30_000
): Promise<z.infer<(typeof FeatureFlagCatalog)[T]>> {
// defaultValue resolves the flag when the row is absent, so it's part of the key
const cacheKey = `${opts.key}:${JSON.stringify(opts.defaultValue)}`;
const hit = cachedFlagStore.get(cacheKey);
if (hit && hit.expiresAt > Date.now()) {
return hit.value as z.infer<(typeof FeatureFlagCatalog)[T]>;
}
const value = await flag(opts);
cachedFlagStore.set(cacheKey, { value, expiresAt: Date.now() + ttlMs });
return value;
}
export function makeSetFlag(_prisma: PrismaClientOrTransaction = prisma) {
return async function setFlag<T extends FeatureFlagKey>(
opts: FlagsOptions<T> & { value: z.infer<(typeof FeatureFlagCatalog)[T]> }
): Promise<void> {
await _prisma.featureFlag.upsert({
where: {
key: opts.key,
},
create: {
key: opts.key,
value: opts.value,
},
update: {
value: opts.value,
},
});
};
}
export type AllFlagsOptions = {
defaultValues?: Partial<FeatureFlagCatalog>;
overrides?: Record<string, unknown>;
};
export function makeFlags(_prisma: PrismaClientOrTransaction = prisma) {
return async function flags(options?: AllFlagsOptions): Promise<Partial<FeatureFlagCatalog>> {
const rows = await _prisma.featureFlag.findMany();
// Build a map of key -> value from database
const dbValues = new Map<string, unknown>();
for (const row of rows) {
dbValues.set(row.key, row.value);
}
const result: Partial<FeatureFlagCatalog> = {};
// Process each flag in the catalog
for (const key of Object.keys(FeatureFlagCatalog) as FeatureFlagKey[]) {
const schema = FeatureFlagCatalog[key];
// Priority: overrides > database > defaultValues
if (options?.overrides?.[key] !== undefined) {
const parsed = schema.safeParse(options.overrides[key]);
if (parsed.success) {
(result as any)[key] = parsed.data;
continue;
}
}
if (dbValues.has(key)) {
const parsed = schema.safeParse(dbValues.get(key));
if (parsed.success) {
(result as any)[key] = parsed.data;
continue;
}
}
if (options?.defaultValues?.[key] !== undefined) {
const parsed = schema.safeParse(options.defaultValues[key]);
if (parsed.success) {
(result as any)[key] = parsed.data;
}
}
}
return result;
};
}
export const flag = makeFlag();
export const flags = makeFlags();
export const setFlag = makeSetFlag();
// Utility function to set multiple feature flags at once
export function makeSetMultipleFlags(_prisma: PrismaClientOrTransaction = prisma) {
return async function setMultipleFlags(
flags: Partial<z.infer<typeof FeatureFlagCatalogSchema>>
): Promise<{ key: string; value: any }[]> {
const setFlag = makeSetFlag(_prisma);
const updatedFlags: { key: string; value: any }[] = [];
for (const [key, value] of Object.entries(flags)) {
if (value !== undefined) {
await setFlag({
key: key as any,
value: value as any,
});
updatedFlags.push({ key, value });
}
}
return updatedFlags;
};
}
// Read -> stamp -> write the global mint-kind grace metadata in one transaction. The three
// FeatureFlag rows may not exist yet, so a row FOR UPDATE can't lock them; an advisory xact lock
// serializes concurrent global flips so one can't clobber another's grace stamp (mirrors per-org).
export async function applyGlobalMintKindFlip(
client: PrismaClient,
requestedFlags: Partial<z.infer<typeof FeatureFlagCatalogSchema>>,
graceMs: number
): Promise<{ key: string; value: any }[]> {
return client.$transaction(async (tx) => {
await tx.$executeRaw`SELECT pg_advisory_xact_lock(hashtext('runops-global-mint-kind-flip'))`;
const existingRows = await tx.featureFlag.findMany({
where: {
key: {
in: [
FEATURE_FLAG.runOpsMintKind,
FEATURE_FLAG.runOpsMintKindPrev,
FEATURE_FLAG.runOpsMintKindFlippedAt,
],
},
},
select: { key: true, value: true },
});
const existingGlobal: Record<string, unknown> = {};
for (const row of existingRows) {
existingGlobal[row.key] = row.value;
}
// Anchor the cutover to the control-plane DB clock, not this process's wall clock.
const [{ now }] = await tx.$queryRaw<{ now: Date }[]>`SELECT now() AS now`;
const stamped = stampMintKindFlip(
existingGlobal,
{ ...requestedFlags },
now.getTime(),
graceMs
) as Partial<z.infer<typeof FeatureFlagCatalogSchema>>;
return makeSetMultipleFlags(tx)(stamped);
});
}