Files
triggerdotdev--trigger.dev/apps/webapp/test/userActorProjectWideScope.test.ts
Katia Bulatova 4569657923 feat(webapp): dashboard agent — chat, reports, investigate (#4418)
## What & why

This is the system behind the Dashboard Agent — an assistant that
answers questions about a project's runs, errors, queues, deploys and
health, and can investigate failures end to end.

The agent runs as a chat.agent task in its own Trigger project. It has
no access to the main database or ClickHouse; all platform data is read
through the public API using a delegated, read-only user token.

Everything here is behind `canAccessDashboardAgent` and inert with the
flag off. The UI that mounts the panel lands in #4529.

## Stack

`#4418` (this, base) ← `#4529` UI ← `#4525` Watch ← `#4516` storybook
gallery. The scenario/contract reference for the whole stack is
`internal-packages/dashboard-agent/GUIDEBOOK.md` (it lands on the Watch
branch): it states, per feature, what makes each thing happen and where
that is decided.

## What's inside

**Agent runtime and tools** — `internal-packages/dashboard-agent`:
prompt, tool set (API reads, TRQL query, docs, navigation,
evidence/investigations, repo source), conversation compaction, a
prompt-prefix token budget pinned by snapshot test, and sampled
LLM-judged turn evals. The package cannot import webapp server code,
which is what makes the "no DB access" claim structural rather than a
convention.

**Contracts** — `internal-packages/dashboard-agent-contracts`:
`trigger://` URIs, intents, and the block envelope every rendered card
travels in.

**Conversation store** — `internal-packages/dashboard-agent-db`: drizzle
over postgres-js in its own `trigger_dashboard_agent` Postgres schema,
plus one additive migration.

**Auth boundary** — the user-actor token gains an optional environment
claim; one guard (`userActorEnvironment.server.ts`) enforces it so
routes don't each re-derive the rule. Token minting, cap ceiling, and
the RBAC fallback path for self-hosted.

**Transport** — webapp resource routes that mint the token and proxy
each turn, and SDK-side mid-turn reconnect.

**Public API the agent reads through** — orgs, projects, environments,
runs, queue metrics, workers, a run's commit metadata, repo snapshot,
reports, and `POST /api/v1/query`.

**Reports** — the health report's layout is declared once and shared by
the card, the markdown surface and the JSON/MCP surface, so the same
report reads the same in the dashboard, the terminal and an editor.

**Block renderers** — the report and investigation cards the flows above
already emit (`app/components/dashboard-agent/`). The panel that hosts
them, and the rest of the chat UI, is #4529.

**Query safety and CSP** — see below.

## Key decisions

- **The agent is a separate Trigger project, not webapp code.** It reads
platform data over the public API with a delegated user-actor token
whose `cap` ceilings it to read scopes. No Prisma, no ClickHouse, no
webapp imports.
- **The PAT-only auth helper now refuses user-actor tokens.** This is an
intentional behavioral change: its callers consume only a bare userId
and do not enforce delegated-token capabilities. Actor-aware routes
continue through the scoped route builders instead.
- **RBAC fallback builds a delegated token's ability from its own cap**,
never the blanket ability a PAT gets (read-only when the token declares
none). Without this, the agent's read-only cap would buy a write JWT on
self-hosted.
- **Org creation checks RBAC only for user-actor tokens, and only after
the env gate**, so an install with `ORG_CREATION_API_ENABLED` off
returns 404 rather than 403, and an ordinary PAT never consults an
ability the route has no org to scope. Both orderings are pinned by
test.
- **The query path is read-only in depth.** TRQL rejects write
statements at the grammar level (they don't parse, rather than being
filtered), ClickHouse runs with `readonly=1`, and the org/project/env
filters are injected server-side from the credential — the request body
cannot widen scope. An unparseable query denies instead of falling
through to the permissive resource.
- **Document-wide img-src CSP.** Remote images are an
outbound-request/exfiltration surface, so the policy permits only
own-origin/data/blob, the required SSO avatar hosts, and the favicon
endpoint. Operators can add exact origins through CSP_IMG_SRC_ALLOWLIST;
wildcard hosts and bare schemes are intentionally not allowed.
- **The chat transport reconnects on a mid-turn EOF**
(`@trigger.dev/sdk`). A body that ends without a turn-complete is
terminal only when the server says `X-Session-Settled: true`; otherwise
the transport resubscribes from `lastEventId` with bounded backoff, and
any record re-earns the budget. Previously a closed long-poll window or
a proxy restart left the reply stuck as if still generating.
- **Conversations live in their own datastore**, schema-scoped and
foreign-key-free (it references `organizationId`/`userId` by id, because
in cloud it is a different database). It is a display read-model for the
History tab and transport resume; `chat.agent`'s object-store snapshot
remains the model's source of truth.
- **Deterministic first.** Reports and health checks contain no LLM —
they are computed from the same data the dashboard shows, and the model
only narrates and links them. That is what makes a number in an answer
auditable.

## Testing

- 63 new test files, run with `pnpm run test --filter webapp` and
per-package vitest. Heaviest coverage on the auth boundary
(`userActorPatOnlyBoundary`, `userActorTokenClaimsAndScopes`,
`contextlessPatRoutes`, `rbacFallbackBranch`), TRQL read-only, the
report layout, and the SDK reconnect.
- The agent package has a separate eval lane (`pnpm run test:evals`,
`vitest.eval.config.ts`) that hits the real model, so it never runs in
`pnpm test`.
- Live-tested against a local stack scenario by scenario; the GUIDEBOOK
lists the condition each behaviour is expected under, which is what
those runs were checked against.

## Changelog

`.server-changes/dashboard-agent.md`, plus changesets for
`@trigger.dev/core` (report schemas), `@trigger.dev/sdk` (chat
reconnect) and the CLI's `mint-token` help text.
2026-08-11 18:56:14 +02:00

347 lines
11 KiB
TypeScript

/**
* The project-wide PAT routes (`/projects/:ref/environments`, `/projects/:ref/runs`) are the door
* a delegated user-actor token could walk around its environment claim through: they list across a
* project, so org membership alone would answer for every environment. These tests drive both real
* routes against a real database with real signed tokens.
*/
import { postgresTest } from "@internal/testcontainers";
import type { PrismaClient } from "@trigger.dev/database";
import { signUserActorToken } from "@trigger.dev/rbac";
import { expect, vi } from "vitest";
const SESSION_SECRET = "test-session-secret-for-project-wide-scope";
const ctx = vi.hoisted(() => ({
prisma: undefined as unknown as PrismaClient,
patUserId: undefined as string | undefined,
presenterEnvironments: [] as Array<{ id: string; organizationId: string } | undefined>,
}));
vi.mock("~/db.server", () => {
const proxy = new Proxy(
{},
{ get: (_target, prop) => (ctx.prisma as unknown as Record<string, unknown>)[prop as string] }
);
return { prisma: proxy, $replica: proxy, sqlDatabaseSchema: undefined };
});
vi.mock("~/env.server", () => ({
env: { SESSION_SECRET: "test-session-secret-for-project-wide-scope" },
}));
vi.mock("~/services/logger.server", () => ({
logger: { debug: vi.fn(), error: vi.fn(), warn: vi.fn(), info: vi.fn() },
}));
vi.mock("~/services/personalAccessToken.server", () => ({
updateLastAccessedAtIfStale: vi.fn(),
// The plugin already verified the claims; test tokens carry no source PAT, so the
// liveness recheck is a no-op that hands the claims straight back.
resolveAndRecheckUserActorClaims: async (claims: unknown) => claims,
}));
vi.mock("~/services/authTelemetry.server", () => ({
authenticateBearerWithTelemetry: vi.fn(),
}));
vi.mock("~/services/tenantContext.server", () => ({
tenantContext: { enrich: vi.fn() },
tenantContextFromAuthEnvironment: vi.fn(),
}));
vi.mock("~/v3/services/worker/workerGroupTokenService.server", () => ({
WorkerGroupTokenService: class {},
}));
vi.mock("~/v3/services/common.server", () => ({
ServiceValidationError: class extends Error {},
}));
vi.mock("@internal/run-engine", () => ({
EngineServiceValidationError: class extends Error {},
}));
vi.mock("~/services/clickhouse/clickhouseFactoryInstance.server", () => ({
clickhouseFactory: { getClickhouseForOrganization: vi.fn() },
}));
// The run list itself isn't under test — which environment the presenter is handed is.
vi.mock("~/presenters/v3/ApiRunListPresenter.server", async () => {
const actual: any = await vi.importActual("~/presenters/v3/ApiRunListPresenter.server");
return {
ApiRunListSearchParams: actual.ApiRunListSearchParams,
ApiRunListPresenter: class {
async call(
_project: unknown,
_searchParams: unknown,
_apiVersion: unknown,
environment?: any
) {
ctx.presenterEnvironments.push(
environment
? { id: environment.id, organizationId: environment.organizationId }
: undefined
);
return { data: [] };
}
},
};
});
// The RBAC controller is the OSS fallback's behaviour: verify the token, ability from its own cap.
vi.mock("~/services/rbac.server", async () => {
const { buildJwtAbility, verifyUserActorToken } = await import("@trigger.dev/rbac");
const bearerOf = (request: Request) =>
request.headers
.get("Authorization")
?.replace(/^Bearer /, "")
.trim() ?? "";
return {
rbac: {
authenticateUserActor: async (request: Request, context: any) => {
const claims = await verifyUserActorToken(
"test-session-secret-for-project-wide-scope",
bearerOf(request)
);
if (!claims) return { ok: false, status: 401, error: "Invalid user-actor token" };
return {
ok: true,
userId: claims.userId,
claims,
subject: {
type: "userActor",
userId: claims.userId,
organizationId: context.organizationId ?? "",
},
ability: buildJwtAbility(claims.cap ?? ["read:all"]),
};
},
authenticatePat: async (_request: Request, context: any) => ({
ok: true,
tokenId: "tok_test",
userId: ctx.patUserId,
lastAccessedAt: null,
subject: {
type: "personalAccessToken",
tokenId: "tok_test",
organizationId: context.organizationId ?? "",
},
ability: buildJwtAbility(["admin"]),
}),
},
};
});
const { loader: environmentsLoader } =
await import("~/routes/api.v1.projects.$projectRef.environments");
const { loader: runsLoader } = await import("~/routes/api.v1.projects.$projectRef.runs");
function suffix() {
return Math.random().toString(36).slice(2, 10);
}
/** An org with one project and two environments (prod + staging), and a member user. */
async function seedProject(prisma: PrismaClient) {
const slug = `scope_${suffix()}`;
const user = await prisma.user.create({
data: { email: `${slug}@example.com`, authenticationMethod: "MAGIC_LINK" },
});
const organization = await prisma.organization.create({ data: { title: slug, slug } });
await prisma.orgMember.create({
data: { organizationId: organization.id, userId: user.id, role: "ADMIN" },
});
const project = await prisma.project.create({
data: { name: slug, slug, organizationId: organization.id, externalRef: `proj_${slug}` },
});
const environmentFor = (envSlug: "prod" | "stg") =>
prisma.runtimeEnvironment.create({
data: {
slug: envSlug,
type: envSlug === "prod" ? "PRODUCTION" : "STAGING",
projectId: project.id,
organizationId: organization.id,
apiKey: `tr_${envSlug}_${slug}`,
pkApiKey: `pk_${envSlug}_${slug}`,
shortcode: `${envSlug}${suffix()}`,
},
});
return {
user,
organization,
project,
envA: await environmentFor("prod"),
envB: await environmentFor("stg"),
};
}
function agentToken(userId: string, environmentId?: string, client = "dashboard-agent") {
return signUserActorToken(SESSION_SECRET, {
userId,
client,
...(environmentId ? { environmentId } : {}),
cap: ["read:runs", "read:environments"],
});
}
async function call(
loader: typeof environmentsLoader | typeof runsLoader,
opts: { projectRef: string; token: string; search?: string }
) {
const url = `https://api.trigger.dev/api/v1/projects/${opts.projectRef}/x${opts.search ?? ""}`;
try {
const response = await (loader as any)({
request: new Request(url, { headers: { Authorization: `Bearer ${opts.token}` } }),
params: { projectRef: opts.projectRef },
context: {},
});
return { status: response.status, body: await response.json() };
} catch (thrown) {
if (thrown instanceof Response) {
return { status: thrown.status, body: await thrown.json() };
}
throw thrown;
}
}
/** A PAT is prefixed `tr_pat_` so the route builder takes the PAT branch. */
const PAT = "tr_pat_testtoken";
postgresTest(
"a user-actor token scoped to one environment lists only that environment",
async ({ prisma }) => {
ctx.prisma = prisma;
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
const scoped = await call(environmentsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, seeded.envA.id),
});
expect(scoped.status).toBe(200);
expect(scoped.body.map((env: any) => env.id)).toEqual([seeded.envA.id]);
},
60_000
);
postgresTest(
"a user-actor token sees only its own environment's runs",
async ({ prisma }) => {
ctx.prisma = prisma;
ctx.presenterEnvironments = [];
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
const scoped = await call(runsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, seeded.envA.id),
});
expect(scoped.status).toBe(200);
expect(ctx.presenterEnvironments).toEqual([
{ id: seeded.envA.id, organizationId: seeded.organization.id },
]);
},
60_000
);
postgresTest(
"a user-actor token asking for another environment is refused, not overridden",
async ({ prisma }) => {
ctx.prisma = prisma;
ctx.presenterEnvironments = [];
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
const conflicting = await call(runsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, seeded.envA.id),
search: `?filter[env]=${seeded.envB.slug}`,
});
expect(conflicting.status).toBe(403);
expect(conflicting.body.code).toBe("forbidden_environment");
expect(ctx.presenterEnvironments).toEqual([]);
},
60_000
);
postgresTest(
"a claimless dashboard-agent token is refused",
async ({ prisma }) => {
ctx.prisma = prisma;
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
// The agent always mints per-environment, so a claimless one of its own is a bug, not a flow.
const claimless = await call(environmentsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, undefined),
});
expect(claimless.status).toBe(403);
expect(claimless.body.code).toBe("forbidden_environment");
},
60_000
);
postgresTest(
"a claimless user-actor token from another client still gets the project-wide answer",
async ({ prisma }) => {
ctx.prisma = prisma;
ctx.presenterEnvironments = [];
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
// The public PAT exchange mints claimless tokens, so narrowing one would be a breaking
// change: it reads the whole project as it always has.
const environments = await call(environmentsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, undefined, "mcp"),
});
expect(environments.status).toBe(200);
expect(environments.body.map((env: any) => env.id).sort()).toEqual(
[seeded.envA.id, seeded.envB.id].sort()
);
const runs = await call(runsLoader, {
projectRef: seeded.project.externalRef,
token: await agentToken(seeded.user.id, undefined, "mcp"),
search: `?filter[env]=${seeded.envB.slug}`,
});
// No forced environment, and its own filter is honoured rather than refused.
expect(runs.status).toBe(200);
expect(ctx.presenterEnvironments).toEqual([undefined]);
},
60_000
);
postgresTest(
"a personal access token still gets the project-wide answer",
async ({ prisma }) => {
ctx.prisma = prisma;
ctx.presenterEnvironments = [];
const seeded = await seedProject(prisma);
ctx.patUserId = seeded.user.id;
const environments = await call(environmentsLoader, {
projectRef: seeded.project.externalRef,
token: PAT,
});
expect(environments.status).toBe(200);
expect(environments.body.map((env: any) => env.id).sort()).toEqual(
[seeded.envA.id, seeded.envB.id].sort()
);
const runs = await call(runsLoader, {
projectRef: seeded.project.externalRef,
token: PAT,
search: `?filter[env]=${seeded.envB.slug}`,
});
// No forced environment: the request's own filter decides, as before.
expect(runs.status).toBe(200);
expect(ctx.presenterEnvironments).toEqual([undefined]);
},
60_000
);