4569657923
## What & why This is the system behind the Dashboard Agent — an assistant that answers questions about a project's runs, errors, queues, deploys and health, and can investigate failures end to end. The agent runs as a chat.agent task in its own Trigger project. It has no access to the main database or ClickHouse; all platform data is read through the public API using a delegated, read-only user token. Everything here is behind `canAccessDashboardAgent` and inert with the flag off. The UI that mounts the panel lands in #4529. ## Stack `#4418` (this, base) ← `#4529` UI ← `#4525` Watch ← `#4516` storybook gallery. The scenario/contract reference for the whole stack is `internal-packages/dashboard-agent/GUIDEBOOK.md` (it lands on the Watch branch): it states, per feature, what makes each thing happen and where that is decided. ## What's inside **Agent runtime and tools** — `internal-packages/dashboard-agent`: prompt, tool set (API reads, TRQL query, docs, navigation, evidence/investigations, repo source), conversation compaction, a prompt-prefix token budget pinned by snapshot test, and sampled LLM-judged turn evals. The package cannot import webapp server code, which is what makes the "no DB access" claim structural rather than a convention. **Contracts** — `internal-packages/dashboard-agent-contracts`: `trigger://` URIs, intents, and the block envelope every rendered card travels in. **Conversation store** — `internal-packages/dashboard-agent-db`: drizzle over postgres-js in its own `trigger_dashboard_agent` Postgres schema, plus one additive migration. **Auth boundary** — the user-actor token gains an optional environment claim; one guard (`userActorEnvironment.server.ts`) enforces it so routes don't each re-derive the rule. Token minting, cap ceiling, and the RBAC fallback path for self-hosted. **Transport** — webapp resource routes that mint the token and proxy each turn, and SDK-side mid-turn reconnect. **Public API the agent reads through** — orgs, projects, environments, runs, queue metrics, workers, a run's commit metadata, repo snapshot, reports, and `POST /api/v1/query`. **Reports** — the health report's layout is declared once and shared by the card, the markdown surface and the JSON/MCP surface, so the same report reads the same in the dashboard, the terminal and an editor. **Block renderers** — the report and investigation cards the flows above already emit (`app/components/dashboard-agent/`). The panel that hosts them, and the rest of the chat UI, is #4529. **Query safety and CSP** — see below. ## Key decisions - **The agent is a separate Trigger project, not webapp code.** It reads platform data over the public API with a delegated user-actor token whose `cap` ceilings it to read scopes. No Prisma, no ClickHouse, no webapp imports. - **The PAT-only auth helper now refuses user-actor tokens.** This is an intentional behavioral change: its callers consume only a bare userId and do not enforce delegated-token capabilities. Actor-aware routes continue through the scoped route builders instead. - **RBAC fallback builds a delegated token's ability from its own cap**, never the blanket ability a PAT gets (read-only when the token declares none). Without this, the agent's read-only cap would buy a write JWT on self-hosted. - **Org creation checks RBAC only for user-actor tokens, and only after the env gate**, so an install with `ORG_CREATION_API_ENABLED` off returns 404 rather than 403, and an ordinary PAT never consults an ability the route has no org to scope. Both orderings are pinned by test. - **The query path is read-only in depth.** TRQL rejects write statements at the grammar level (they don't parse, rather than being filtered), ClickHouse runs with `readonly=1`, and the org/project/env filters are injected server-side from the credential — the request body cannot widen scope. An unparseable query denies instead of falling through to the permissive resource. - **Document-wide img-src CSP.** Remote images are an outbound-request/exfiltration surface, so the policy permits only own-origin/data/blob, the required SSO avatar hosts, and the favicon endpoint. Operators can add exact origins through CSP_IMG_SRC_ALLOWLIST; wildcard hosts and bare schemes are intentionally not allowed. - **The chat transport reconnects on a mid-turn EOF** (`@trigger.dev/sdk`). A body that ends without a turn-complete is terminal only when the server says `X-Session-Settled: true`; otherwise the transport resubscribes from `lastEventId` with bounded backoff, and any record re-earns the budget. Previously a closed long-poll window or a proxy restart left the reply stuck as if still generating. - **Conversations live in their own datastore**, schema-scoped and foreign-key-free (it references `organizationId`/`userId` by id, because in cloud it is a different database). It is a display read-model for the History tab and transport resume; `chat.agent`'s object-store snapshot remains the model's source of truth. - **Deterministic first.** Reports and health checks contain no LLM — they are computed from the same data the dashboard shows, and the model only narrates and links them. That is what makes a number in an answer auditable. ## Testing - 63 new test files, run with `pnpm run test --filter webapp` and per-package vitest. Heaviest coverage on the auth boundary (`userActorPatOnlyBoundary`, `userActorTokenClaimsAndScopes`, `contextlessPatRoutes`, `rbacFallbackBranch`), TRQL read-only, the report layout, and the SDK reconnect. - The agent package has a separate eval lane (`pnpm run test:evals`, `vitest.eval.config.ts`) that hits the real model, so it never runs in `pnpm test`. - Live-tested against a local stack scenario by scenario; the GUIDEBOOK lists the condition each behaviour is expected under, which is what those runs were checked against. ## Changelog `.server-changes/dashboard-agent.md`, plus changesets for `@trigger.dev/core` (report schemas), `@trigger.dev/sdk` (chat reconnect) and the CLI's `mint-token` help text.
347 lines
11 KiB
TypeScript
347 lines
11 KiB
TypeScript
/**
|
|
* The project-wide PAT routes (`/projects/:ref/environments`, `/projects/:ref/runs`) are the door
|
|
* a delegated user-actor token could walk around its environment claim through: they list across a
|
|
* project, so org membership alone would answer for every environment. These tests drive both real
|
|
* routes against a real database with real signed tokens.
|
|
*/
|
|
|
|
import { postgresTest } from "@internal/testcontainers";
|
|
import type { PrismaClient } from "@trigger.dev/database";
|
|
import { signUserActorToken } from "@trigger.dev/rbac";
|
|
import { expect, vi } from "vitest";
|
|
|
|
const SESSION_SECRET = "test-session-secret-for-project-wide-scope";
|
|
|
|
const ctx = vi.hoisted(() => ({
|
|
prisma: undefined as unknown as PrismaClient,
|
|
patUserId: undefined as string | undefined,
|
|
presenterEnvironments: [] as Array<{ id: string; organizationId: string } | undefined>,
|
|
}));
|
|
|
|
vi.mock("~/db.server", () => {
|
|
const proxy = new Proxy(
|
|
{},
|
|
{ get: (_target, prop) => (ctx.prisma as unknown as Record<string, unknown>)[prop as string] }
|
|
);
|
|
return { prisma: proxy, $replica: proxy, sqlDatabaseSchema: undefined };
|
|
});
|
|
vi.mock("~/env.server", () => ({
|
|
env: { SESSION_SECRET: "test-session-secret-for-project-wide-scope" },
|
|
}));
|
|
vi.mock("~/services/logger.server", () => ({
|
|
logger: { debug: vi.fn(), error: vi.fn(), warn: vi.fn(), info: vi.fn() },
|
|
}));
|
|
vi.mock("~/services/personalAccessToken.server", () => ({
|
|
updateLastAccessedAtIfStale: vi.fn(),
|
|
// The plugin already verified the claims; test tokens carry no source PAT, so the
|
|
// liveness recheck is a no-op that hands the claims straight back.
|
|
resolveAndRecheckUserActorClaims: async (claims: unknown) => claims,
|
|
}));
|
|
vi.mock("~/services/authTelemetry.server", () => ({
|
|
authenticateBearerWithTelemetry: vi.fn(),
|
|
}));
|
|
vi.mock("~/services/tenantContext.server", () => ({
|
|
tenantContext: { enrich: vi.fn() },
|
|
tenantContextFromAuthEnvironment: vi.fn(),
|
|
}));
|
|
vi.mock("~/v3/services/worker/workerGroupTokenService.server", () => ({
|
|
WorkerGroupTokenService: class {},
|
|
}));
|
|
vi.mock("~/v3/services/common.server", () => ({
|
|
ServiceValidationError: class extends Error {},
|
|
}));
|
|
vi.mock("@internal/run-engine", () => ({
|
|
EngineServiceValidationError: class extends Error {},
|
|
}));
|
|
|
|
vi.mock("~/services/clickhouse/clickhouseFactoryInstance.server", () => ({
|
|
clickhouseFactory: { getClickhouseForOrganization: vi.fn() },
|
|
}));
|
|
|
|
// The run list itself isn't under test — which environment the presenter is handed is.
|
|
vi.mock("~/presenters/v3/ApiRunListPresenter.server", async () => {
|
|
const actual: any = await vi.importActual("~/presenters/v3/ApiRunListPresenter.server");
|
|
return {
|
|
ApiRunListSearchParams: actual.ApiRunListSearchParams,
|
|
ApiRunListPresenter: class {
|
|
async call(
|
|
_project: unknown,
|
|
_searchParams: unknown,
|
|
_apiVersion: unknown,
|
|
environment?: any
|
|
) {
|
|
ctx.presenterEnvironments.push(
|
|
environment
|
|
? { id: environment.id, organizationId: environment.organizationId }
|
|
: undefined
|
|
);
|
|
return { data: [] };
|
|
}
|
|
},
|
|
};
|
|
});
|
|
|
|
// The RBAC controller is the OSS fallback's behaviour: verify the token, ability from its own cap.
|
|
vi.mock("~/services/rbac.server", async () => {
|
|
const { buildJwtAbility, verifyUserActorToken } = await import("@trigger.dev/rbac");
|
|
const bearerOf = (request: Request) =>
|
|
request.headers
|
|
.get("Authorization")
|
|
?.replace(/^Bearer /, "")
|
|
.trim() ?? "";
|
|
|
|
return {
|
|
rbac: {
|
|
authenticateUserActor: async (request: Request, context: any) => {
|
|
const claims = await verifyUserActorToken(
|
|
"test-session-secret-for-project-wide-scope",
|
|
bearerOf(request)
|
|
);
|
|
if (!claims) return { ok: false, status: 401, error: "Invalid user-actor token" };
|
|
return {
|
|
ok: true,
|
|
userId: claims.userId,
|
|
claims,
|
|
subject: {
|
|
type: "userActor",
|
|
userId: claims.userId,
|
|
organizationId: context.organizationId ?? "",
|
|
},
|
|
ability: buildJwtAbility(claims.cap ?? ["read:all"]),
|
|
};
|
|
},
|
|
authenticatePat: async (_request: Request, context: any) => ({
|
|
ok: true,
|
|
tokenId: "tok_test",
|
|
userId: ctx.patUserId,
|
|
lastAccessedAt: null,
|
|
subject: {
|
|
type: "personalAccessToken",
|
|
tokenId: "tok_test",
|
|
organizationId: context.organizationId ?? "",
|
|
},
|
|
ability: buildJwtAbility(["admin"]),
|
|
}),
|
|
},
|
|
};
|
|
});
|
|
|
|
const { loader: environmentsLoader } =
|
|
await import("~/routes/api.v1.projects.$projectRef.environments");
|
|
const { loader: runsLoader } = await import("~/routes/api.v1.projects.$projectRef.runs");
|
|
|
|
function suffix() {
|
|
return Math.random().toString(36).slice(2, 10);
|
|
}
|
|
|
|
/** An org with one project and two environments (prod + staging), and a member user. */
|
|
async function seedProject(prisma: PrismaClient) {
|
|
const slug = `scope_${suffix()}`;
|
|
const user = await prisma.user.create({
|
|
data: { email: `${slug}@example.com`, authenticationMethod: "MAGIC_LINK" },
|
|
});
|
|
const organization = await prisma.organization.create({ data: { title: slug, slug } });
|
|
await prisma.orgMember.create({
|
|
data: { organizationId: organization.id, userId: user.id, role: "ADMIN" },
|
|
});
|
|
const project = await prisma.project.create({
|
|
data: { name: slug, slug, organizationId: organization.id, externalRef: `proj_${slug}` },
|
|
});
|
|
|
|
const environmentFor = (envSlug: "prod" | "stg") =>
|
|
prisma.runtimeEnvironment.create({
|
|
data: {
|
|
slug: envSlug,
|
|
type: envSlug === "prod" ? "PRODUCTION" : "STAGING",
|
|
projectId: project.id,
|
|
organizationId: organization.id,
|
|
apiKey: `tr_${envSlug}_${slug}`,
|
|
pkApiKey: `pk_${envSlug}_${slug}`,
|
|
shortcode: `${envSlug}${suffix()}`,
|
|
},
|
|
});
|
|
|
|
return {
|
|
user,
|
|
organization,
|
|
project,
|
|
envA: await environmentFor("prod"),
|
|
envB: await environmentFor("stg"),
|
|
};
|
|
}
|
|
|
|
function agentToken(userId: string, environmentId?: string, client = "dashboard-agent") {
|
|
return signUserActorToken(SESSION_SECRET, {
|
|
userId,
|
|
client,
|
|
...(environmentId ? { environmentId } : {}),
|
|
cap: ["read:runs", "read:environments"],
|
|
});
|
|
}
|
|
|
|
async function call(
|
|
loader: typeof environmentsLoader | typeof runsLoader,
|
|
opts: { projectRef: string; token: string; search?: string }
|
|
) {
|
|
const url = `https://api.trigger.dev/api/v1/projects/${opts.projectRef}/x${opts.search ?? ""}`;
|
|
try {
|
|
const response = await (loader as any)({
|
|
request: new Request(url, { headers: { Authorization: `Bearer ${opts.token}` } }),
|
|
params: { projectRef: opts.projectRef },
|
|
context: {},
|
|
});
|
|
return { status: response.status, body: await response.json() };
|
|
} catch (thrown) {
|
|
if (thrown instanceof Response) {
|
|
return { status: thrown.status, body: await thrown.json() };
|
|
}
|
|
throw thrown;
|
|
}
|
|
}
|
|
|
|
/** A PAT is prefixed `tr_pat_` so the route builder takes the PAT branch. */
|
|
const PAT = "tr_pat_testtoken";
|
|
|
|
postgresTest(
|
|
"a user-actor token scoped to one environment lists only that environment",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
const scoped = await call(environmentsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, seeded.envA.id),
|
|
});
|
|
|
|
expect(scoped.status).toBe(200);
|
|
expect(scoped.body.map((env: any) => env.id)).toEqual([seeded.envA.id]);
|
|
},
|
|
60_000
|
|
);
|
|
|
|
postgresTest(
|
|
"a user-actor token sees only its own environment's runs",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
ctx.presenterEnvironments = [];
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
const scoped = await call(runsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, seeded.envA.id),
|
|
});
|
|
|
|
expect(scoped.status).toBe(200);
|
|
expect(ctx.presenterEnvironments).toEqual([
|
|
{ id: seeded.envA.id, organizationId: seeded.organization.id },
|
|
]);
|
|
},
|
|
60_000
|
|
);
|
|
|
|
postgresTest(
|
|
"a user-actor token asking for another environment is refused, not overridden",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
ctx.presenterEnvironments = [];
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
const conflicting = await call(runsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, seeded.envA.id),
|
|
search: `?filter[env]=${seeded.envB.slug}`,
|
|
});
|
|
|
|
expect(conflicting.status).toBe(403);
|
|
expect(conflicting.body.code).toBe("forbidden_environment");
|
|
expect(ctx.presenterEnvironments).toEqual([]);
|
|
},
|
|
60_000
|
|
);
|
|
|
|
postgresTest(
|
|
"a claimless dashboard-agent token is refused",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
// The agent always mints per-environment, so a claimless one of its own is a bug, not a flow.
|
|
const claimless = await call(environmentsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, undefined),
|
|
});
|
|
|
|
expect(claimless.status).toBe(403);
|
|
expect(claimless.body.code).toBe("forbidden_environment");
|
|
},
|
|
60_000
|
|
);
|
|
|
|
postgresTest(
|
|
"a claimless user-actor token from another client still gets the project-wide answer",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
ctx.presenterEnvironments = [];
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
// The public PAT exchange mints claimless tokens, so narrowing one would be a breaking
|
|
// change: it reads the whole project as it always has.
|
|
const environments = await call(environmentsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, undefined, "mcp"),
|
|
});
|
|
|
|
expect(environments.status).toBe(200);
|
|
expect(environments.body.map((env: any) => env.id).sort()).toEqual(
|
|
[seeded.envA.id, seeded.envB.id].sort()
|
|
);
|
|
|
|
const runs = await call(runsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: await agentToken(seeded.user.id, undefined, "mcp"),
|
|
search: `?filter[env]=${seeded.envB.slug}`,
|
|
});
|
|
|
|
// No forced environment, and its own filter is honoured rather than refused.
|
|
expect(runs.status).toBe(200);
|
|
expect(ctx.presenterEnvironments).toEqual([undefined]);
|
|
},
|
|
60_000
|
|
);
|
|
|
|
postgresTest(
|
|
"a personal access token still gets the project-wide answer",
|
|
async ({ prisma }) => {
|
|
ctx.prisma = prisma;
|
|
ctx.presenterEnvironments = [];
|
|
const seeded = await seedProject(prisma);
|
|
ctx.patUserId = seeded.user.id;
|
|
|
|
const environments = await call(environmentsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: PAT,
|
|
});
|
|
|
|
expect(environments.status).toBe(200);
|
|
expect(environments.body.map((env: any) => env.id).sort()).toEqual(
|
|
[seeded.envA.id, seeded.envB.id].sort()
|
|
);
|
|
|
|
const runs = await call(runsLoader, {
|
|
projectRef: seeded.project.externalRef,
|
|
token: PAT,
|
|
search: `?filter[env]=${seeded.envB.slug}`,
|
|
});
|
|
|
|
// No forced environment: the request's own filter decides, as before.
|
|
expect(runs.status).toBe(200);
|
|
expect(ctx.presenterEnvironments).toEqual([undefined]);
|
|
},
|
|
60_000
|
|
);
|