4569657923
## What & why This is the system behind the Dashboard Agent — an assistant that answers questions about a project's runs, errors, queues, deploys and health, and can investigate failures end to end. The agent runs as a chat.agent task in its own Trigger project. It has no access to the main database or ClickHouse; all platform data is read through the public API using a delegated, read-only user token. Everything here is behind `canAccessDashboardAgent` and inert with the flag off. The UI that mounts the panel lands in #4529. ## Stack `#4418` (this, base) ← `#4529` UI ← `#4525` Watch ← `#4516` storybook gallery. The scenario/contract reference for the whole stack is `internal-packages/dashboard-agent/GUIDEBOOK.md` (it lands on the Watch branch): it states, per feature, what makes each thing happen and where that is decided. ## What's inside **Agent runtime and tools** — `internal-packages/dashboard-agent`: prompt, tool set (API reads, TRQL query, docs, navigation, evidence/investigations, repo source), conversation compaction, a prompt-prefix token budget pinned by snapshot test, and sampled LLM-judged turn evals. The package cannot import webapp server code, which is what makes the "no DB access" claim structural rather than a convention. **Contracts** — `internal-packages/dashboard-agent-contracts`: `trigger://` URIs, intents, and the block envelope every rendered card travels in. **Conversation store** — `internal-packages/dashboard-agent-db`: drizzle over postgres-js in its own `trigger_dashboard_agent` Postgres schema, plus one additive migration. **Auth boundary** — the user-actor token gains an optional environment claim; one guard (`userActorEnvironment.server.ts`) enforces it so routes don't each re-derive the rule. Token minting, cap ceiling, and the RBAC fallback path for self-hosted. **Transport** — webapp resource routes that mint the token and proxy each turn, and SDK-side mid-turn reconnect. **Public API the agent reads through** — orgs, projects, environments, runs, queue metrics, workers, a run's commit metadata, repo snapshot, reports, and `POST /api/v1/query`. **Reports** — the health report's layout is declared once and shared by the card, the markdown surface and the JSON/MCP surface, so the same report reads the same in the dashboard, the terminal and an editor. **Block renderers** — the report and investigation cards the flows above already emit (`app/components/dashboard-agent/`). The panel that hosts them, and the rest of the chat UI, is #4529. **Query safety and CSP** — see below. ## Key decisions - **The agent is a separate Trigger project, not webapp code.** It reads platform data over the public API with a delegated user-actor token whose `cap` ceilings it to read scopes. No Prisma, no ClickHouse, no webapp imports. - **The PAT-only auth helper now refuses user-actor tokens.** This is an intentional behavioral change: its callers consume only a bare userId and do not enforce delegated-token capabilities. Actor-aware routes continue through the scoped route builders instead. - **RBAC fallback builds a delegated token's ability from its own cap**, never the blanket ability a PAT gets (read-only when the token declares none). Without this, the agent's read-only cap would buy a write JWT on self-hosted. - **Org creation checks RBAC only for user-actor tokens, and only after the env gate**, so an install with `ORG_CREATION_API_ENABLED` off returns 404 rather than 403, and an ordinary PAT never consults an ability the route has no org to scope. Both orderings are pinned by test. - **The query path is read-only in depth.** TRQL rejects write statements at the grammar level (they don't parse, rather than being filtered), ClickHouse runs with `readonly=1`, and the org/project/env filters are injected server-side from the credential — the request body cannot widen scope. An unparseable query denies instead of falling through to the permissive resource. - **Document-wide img-src CSP.** Remote images are an outbound-request/exfiltration surface, so the policy permits only own-origin/data/blob, the required SSO avatar hosts, and the favicon endpoint. Operators can add exact origins through CSP_IMG_SRC_ALLOWLIST; wildcard hosts and bare schemes are intentionally not allowed. - **The chat transport reconnects on a mid-turn EOF** (`@trigger.dev/sdk`). A body that ends without a turn-complete is terminal only when the server says `X-Session-Settled: true`; otherwise the transport resubscribes from `lastEventId` with bounded backoff, and any record re-earns the budget. Previously a closed long-poll window or a proxy restart left the reply stuck as if still generating. - **Conversations live in their own datastore**, schema-scoped and foreign-key-free (it references `organizationId`/`userId` by id, because in cloud it is a different database). It is a display read-model for the History tab and transport resume; `chat.agent`'s object-store snapshot remains the model's source of truth. - **Deterministic first.** Reports and health checks contain no LLM — they are computed from the same data the dashboard shows, and the model only narrates and links them. That is what makes a number in an answer auditable. ## Testing - 63 new test files, run with `pnpm run test --filter webapp` and per-package vitest. Heaviest coverage on the auth boundary (`userActorPatOnlyBoundary`, `userActorTokenClaimsAndScopes`, `contextlessPatRoutes`, `rbacFallbackBranch`), TRQL read-only, the report layout, and the SDK reconnect. - The agent package has a separate eval lane (`pnpm run test:evals`, `vitest.eval.config.ts`) that hits the real model, so it never runs in `pnpm test`. - Live-tested against a local stack scenario by scenario; the GUIDEBOOK lists the condition each behaviour is expected under, which is what those runs were checked against. ## Changelog `.server-changes/dashboard-agent.md`, plus changesets for `@trigger.dev/core` (report schemas), `@trigger.dev/sdk` (chat reconnect) and the CLI's `mint-token` help text.
163 lines
5.6 KiB
TypeScript
163 lines
5.6 KiB
TypeScript
import { beforeEach, describe, expect, it, vi } from "vitest";
|
|
|
|
const mocks = vi.hoisted(() => ({
|
|
fetch: vi.fn(),
|
|
findEnvironmentBySlug: vi.fn<(...args: any[]) => Promise<any>>(),
|
|
}));
|
|
|
|
vi.mock("~/db.server", () => ({ $replica: {} }));
|
|
vi.mock("~/env.server", () => ({ env: { SESSION_SECRET: "test-session-secret" } }));
|
|
vi.mock("~/services/session.server", () => ({
|
|
requireUser: async () => ({ id: "usr_real", admin: false, isImpersonating: false }),
|
|
}));
|
|
vi.mock("~/v3/canAccessDashboardAgent.server", () => ({
|
|
canAccessDashboardAgent: async () => true,
|
|
}));
|
|
vi.mock("~/models/project.server", () => ({
|
|
findProjectBySlug: async () => ({
|
|
id: "proj_real",
|
|
organizationId: "org_real",
|
|
externalRef: "proj_ref_real",
|
|
}),
|
|
}));
|
|
vi.mock("~/models/runtimeEnvironment.server", () => ({
|
|
findEnvironmentBySlug: mocks.findEnvironmentBySlug,
|
|
}));
|
|
vi.mock("~/services/dashboardAgent.server", () => ({
|
|
dashboardAgentApiOrigin: () => "https://api.trigger.dev",
|
|
mintDashboardAgentUserActorToken: async () => "tr_uat_real",
|
|
resolveDashboardAgentRepoSnapshot: async () => null,
|
|
}));
|
|
vi.mock("~/services/logger.server", () => ({
|
|
logger: { debug: vi.fn(), error: vi.fn(), warn: vi.fn(), info: vi.fn() },
|
|
}));
|
|
|
|
import { action } from "~/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.dashboard-agent.in.$";
|
|
|
|
async function appendTurn(metadata: Record<string, unknown>): Promise<Record<string, unknown>> {
|
|
const request = new Request(
|
|
"https://app.trigger.dev/resources/orgs/acme/projects/api/env/dev/dashboard-agent/in/realtime/v1/sessions/chat_1/in/append",
|
|
{
|
|
method: "POST",
|
|
headers: { "content-type": "application/json" },
|
|
body: JSON.stringify({
|
|
kind: "message",
|
|
payload: { metadata, message: { parts: [{ type: "text", text: "hi" }] } },
|
|
}),
|
|
}
|
|
);
|
|
|
|
const response = await action({
|
|
request,
|
|
params: {
|
|
organizationSlug: "acme",
|
|
projectParam: "api",
|
|
envParam: "dev",
|
|
"*": "realtime/v1/sessions/chat_1/in/append",
|
|
},
|
|
context: {},
|
|
} as any);
|
|
|
|
expect(response.status).toBe(200);
|
|
expect(mocks.fetch).toHaveBeenCalledTimes(1);
|
|
const forwarded = JSON.parse(mocks.fetch.mock.calls[0][1].body as string);
|
|
return forwarded.payload.metadata as Record<string, unknown>;
|
|
}
|
|
|
|
describe("dashboard agent `in` proxy — client metadata", () => {
|
|
beforeEach(() => {
|
|
mocks.findEnvironmentBySlug.mockReset();
|
|
mocks.findEnvironmentBySlug.mockResolvedValue({
|
|
id: "env_real",
|
|
type: "DEVELOPMENT",
|
|
branchName: null,
|
|
});
|
|
mocks.fetch.mockReset();
|
|
mocks.fetch.mockResolvedValue(
|
|
new Response(JSON.stringify({ ok: true }), {
|
|
status: 200,
|
|
headers: { "content-type": "application/json" },
|
|
})
|
|
);
|
|
vi.stubGlobal("fetch", mocks.fetch);
|
|
});
|
|
|
|
it("keeps the whitelisted page context", async () => {
|
|
const metadata = await appendTurn({
|
|
currentPage: "/orgs/acme/projects/api/env/dev/runs",
|
|
pageContext: { kind: "runs" },
|
|
});
|
|
|
|
expect(metadata.currentPage).toBe("/orgs/acme/projects/api/env/dev/runs");
|
|
expect(metadata.pageContext).toEqual({ kind: "runs" });
|
|
});
|
|
|
|
it("ignores a client-sent copy of every server-owned field", async () => {
|
|
const metadata = await appendTurn({
|
|
currentPage: "/runs",
|
|
organizationId: "org_evil",
|
|
userId: "usr_evil",
|
|
projectId: "proj_evil",
|
|
projectRef: "proj_ref_evil",
|
|
environmentId: "env_evil",
|
|
environmentName: "prod",
|
|
environmentBranch: "evil-branch",
|
|
apiOrigin: "https://evil.example.com",
|
|
userActorToken: "tr_uat_evil",
|
|
repoSnapshot: { tarballUrl: "https://evil.example.com/x.tar.gz" },
|
|
});
|
|
|
|
expect(metadata.organizationId).toBe("org_real");
|
|
expect(metadata.userId).toBe("usr_real");
|
|
expect(metadata.projectId).toBe("proj_real");
|
|
expect(metadata.projectRef).toBe("proj_ref_real");
|
|
expect(metadata.environmentId).toBe("env_real");
|
|
expect(metadata.environmentName).toBe("dev");
|
|
expect(metadata.environmentBranch).toBeUndefined();
|
|
expect(metadata.apiOrigin).toBe("https://api.trigger.dev");
|
|
expect(metadata.userActorToken).toBe("tr_uat_real");
|
|
expect(metadata.repoSnapshot).toBeUndefined();
|
|
});
|
|
|
|
// The whole address the proxy hands the agent, for each of the four environment shapes. The
|
|
// name is shared by a parent and all its branches, so a branch is only addressable when its
|
|
// branch travels with the name.
|
|
it.each([
|
|
["production", { id: "env_prod", type: "PRODUCTION", branchName: null }, "prod", undefined],
|
|
["staging", { id: "env_stg", type: "STAGING", branchName: null }, "staging", undefined],
|
|
[
|
|
"a preview branch",
|
|
{ id: "env_preview_branch", type: "PREVIEW", branchName: "feat/checkout" },
|
|
"preview",
|
|
"feat/checkout",
|
|
],
|
|
[
|
|
"a development branch",
|
|
{ id: "env_dev_branch", type: "DEVELOPMENT", branchName: "katia/spike" },
|
|
"dev",
|
|
"katia/spike",
|
|
],
|
|
])("addresses %s by the environment it resolved", async (_name, env, expectedName, branch) => {
|
|
mocks.findEnvironmentBySlug.mockResolvedValue(env);
|
|
|
|
const metadata = await appendTurn({ currentPage: "/runs" });
|
|
|
|
expect(metadata.environmentId).toBe(env.id);
|
|
expect(metadata.environmentName).toBe(expectedName);
|
|
expect(metadata.environmentBranch).toBe(branch);
|
|
});
|
|
|
|
it("drops any field the server doesn't own", async () => {
|
|
const metadata = await appendTurn({
|
|
currentPage: "/runs",
|
|
evalOptOut: false,
|
|
cap: ["admin"],
|
|
somethingNew: "smuggled",
|
|
});
|
|
|
|
expect(metadata).not.toHaveProperty("evalOptOut");
|
|
expect(metadata).not.toHaveProperty("cap");
|
|
expect(metadata).not.toHaveProperty("somethingNew");
|
|
});
|
|
});
|