feat(webapp): dashboard parity for mollifier-buffered runs (#3757)
## Summary Dashboard surfaces handle buffered runs by falling back to the mollifier snapshot: - Run detail, span detail, streams view (`_app.../runs.\$runParam`, `resources.../spans.\$spanParam`, `resources.../streams.\$streamKey`). - Redirect routes (`@.runs.\$runParam`, `runs.\$runParam`, `projects.v3.\$projectRef.runs.\$runParam`). - Action routes — cancel / replay / idempotency-reset / debug — under `resources.taskruns/...` and `resources.../idempotencyKey.reset`. - Logs download. - Realtime subscription route + per-run resource (`realtime.v1.runs.\$runId`, `resources.../realtime.v1.*`). - `CancelRunDialog` gains an `onCancelSubmitted` callback so submit isn't raced by the Radix `DialogClose` wrapper. Stacked on the mutations PR. ## Test plan - [x] \`pnpm run typecheck --filter webapp\` passes - [x] \`pnpm run test --filter webapp test/mollifierRealtimeRunResource.test.ts\` passes - [x] \`pnpm run test --filter webapp test/mollifierRealtimeRunResourceBuffer.test.ts\` passes - [x] \`pnpm run test --filter webapp test/mollifierRealtimeSubscription.test.ts\` passes - [x] Manual smoke: trigger a buffered run, open it in the dashboard, replay/cancel from the UI --- ## Ship-gate follow-up fixes - **Auto-redirect to root span on direct nav** — loader sets `?span=` from root span (PG) or buffered snapshot spanId before 302'ing, so bookmark/share-link/direct-nav doesn't leave the panel collapsed. - **RunPresenter switches from `findFirstOrThrow` to `findFirst` + typed `RunNotInPgError`** — kills the per-poll `PrismaClient error` log spam for buffered runs without changing the route-loader's fallback flow. - **Span detail panel renders for buffered runs** — `SpanPresenter.call` now falls back to `findRunByIdWithMollifierFallback` + `buildSyntheticSpanRun` instead of returning undefined and triggering the "Event not found" toast loop. - **Logs download for buffered runs returns a gzipped placeholder line** — replaces the 404 with a content-encoded line explaining the run is queued. Same org-membership gate as the PG path. - **Admin Debug-Run button hidden for buffered runs + SpanRun circular type alias broken** (squashed) — buttons gate on a new `isBuffered` flag on the synthetic SpanRun. Required grounding SpanRun in `SpanPresenter.getRun` to break a circular type alias TS no longer tolerates once `isBuffered` is a literal field on the shape. - **Replay action requires user auth + org-membership** (🚩 Devin finding) — `action` was unauthenticated and the PG `findFirst` had no org filter, so any caller with a valid `runParam` could replay any run. Buffered fallback inherited the same gap. Fixed to mirror the cancel route. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
---
|
||||
area: webapp
|
||||
type: feature
|
||||
---
|
||||
|
||||
Mollifier dashboard surface: run-detail page renders buffered runs via synthetic trace, header, and span shapes; admin-only "Buffered" indicator and drainer LOG event in the trace tree.
|
||||
@@ -10,9 +10,18 @@ import { SpinnerWhite } from "~/components/primitives/Spinner";
|
||||
type CancelRunDialogProps = {
|
||||
runFriendlyId: string;
|
||||
redirectPath: string;
|
||||
// Fired on submit so the parent can close the Radix Dialog without
|
||||
// wrapping the submit button in `DialogClose` — that wrapper races
|
||||
// submit (close fires first, unmounts the form, and the cancel POST
|
||||
// never lands). Optional so existing call sites still type-check.
|
||||
onCancelSubmitted?: () => void;
|
||||
};
|
||||
|
||||
export function CancelRunDialog({ runFriendlyId, redirectPath }: CancelRunDialogProps) {
|
||||
export function CancelRunDialog({
|
||||
runFriendlyId,
|
||||
redirectPath,
|
||||
onCancelSubmitted,
|
||||
}: CancelRunDialogProps) {
|
||||
const navigation = useNavigation();
|
||||
|
||||
const formAction = `/resources/taskruns/${runFriendlyId}/cancel`;
|
||||
@@ -27,7 +36,11 @@ export function CancelRunDialog({ runFriendlyId, redirectPath }: CancelRunDialog
|
||||
</Paragraph>
|
||||
<FormButtons
|
||||
confirmButton={
|
||||
<Form action={`/resources/taskruns/${runFriendlyId}/cancel`} method="post">
|
||||
<Form
|
||||
action={`/resources/taskruns/${runFriendlyId}/cancel`}
|
||||
method="post"
|
||||
onSubmit={() => onCancelSubmitted?.()}
|
||||
>
|
||||
<Button
|
||||
type="submit"
|
||||
name="redirectUrl"
|
||||
|
||||
@@ -20,6 +20,20 @@ export class RunEnvironmentMismatchError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
// Thrown by `call()` when the run isn't in PG. The route loader catches
|
||||
// this and falls back to the mollifier buffer via `tryMollifiedRunFallback`.
|
||||
// Using a typed error (rather than Prisma's `findFirstOrThrow` exception)
|
||||
// keeps the buffered case off the PrismaClient error path — that path
|
||||
// emits a `PrismaClient error` log every time it fires, which on the
|
||||
// run-detail page polls becomes per-tick log spam and Sentry noise for
|
||||
// any run that legitimately lives in the buffer.
|
||||
export class RunNotInPgError extends Error {
|
||||
constructor(public readonly runFriendlyId: string) {
|
||||
super(`Run ${runFriendlyId} not in PG`);
|
||||
this.name = "RunNotInPgError";
|
||||
}
|
||||
}
|
||||
|
||||
export class RunPresenter {
|
||||
#prismaClient: PrismaClient;
|
||||
|
||||
@@ -42,7 +56,13 @@ export class RunPresenter {
|
||||
showDeletedLogs: boolean;
|
||||
showDebug: boolean;
|
||||
}) {
|
||||
const run = await this.#prismaClient.taskRun.findFirstOrThrow({
|
||||
// `findFirst` + explicit null check (not `findFirstOrThrow`) because
|
||||
// a missing PG row is the *expected* path for buffered runs — the
|
||||
// route catches `RunNotInPgError` and falls back to the synthesised
|
||||
// buffer view. `findFirstOrThrow` would log a `PrismaClient error`
|
||||
// every tick of the page poll, masking real DB issues with synthetic
|
||||
// not-found noise.
|
||||
const run = await this.#prismaClient.taskRun.findFirst({
|
||||
select: {
|
||||
id: true,
|
||||
createdAt: true,
|
||||
@@ -106,6 +126,10 @@ export class RunPresenter {
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
throw new RunNotInPgError(runFriendlyId);
|
||||
}
|
||||
|
||||
if (environmentSlug !== run.runtimeEnvironment.slug) {
|
||||
throw new RunEnvironmentMismatchError(
|
||||
`Run ${runFriendlyId} is not in environment ${environmentSlug}`
|
||||
|
||||
@@ -3,6 +3,8 @@ import { logger } from "~/services/logger.server";
|
||||
import { singleton } from "~/utils/singleton";
|
||||
import { ABORT_REASON_SEND_ERROR, createSSELoader, SendFunction } from "~/utils/sse";
|
||||
import { throttle } from "~/utils/throttle";
|
||||
import { getMollifierBuffer } from "~/v3/mollifier/mollifierBuffer.server";
|
||||
import { deserialiseMollifierSnapshot } from "~/v3/mollifier/mollifierSnapshot.server";
|
||||
import { tracePubSub } from "~/v3/services/tracePubSub.server";
|
||||
|
||||
const PING_INTERVAL = 5_000;
|
||||
@@ -37,17 +39,48 @@ export class RunStreamPresenter {
|
||||
},
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
// Fall back to the mollifier buffer when the run isn't in PG yet.
|
||||
// The buffered run has no execution events to stream, but we still
|
||||
// attach a trace-pubsub subscription using the snapshot's traceId
|
||||
// so that the moment the drainer materialises the row and execution
|
||||
// begins, those events flow to this open SSE connection. Closing
|
||||
// with 404 would force the dashboard to keep retrying.
|
||||
let traceId: string | null = run?.traceId ?? null;
|
||||
if (!traceId) {
|
||||
const buffer = getMollifierBuffer();
|
||||
if (buffer) {
|
||||
try {
|
||||
const entry = await buffer.getEntry(runFriendlyId);
|
||||
if (entry) {
|
||||
// Go through the webapp wrapper so this read-side module
|
||||
// shares a single deserialisation path with readFallback —
|
||||
// see the contract comment in syntheticRedirectInfo.server.ts.
|
||||
const snapshot = deserialiseMollifierSnapshot(entry.payload);
|
||||
if (typeof snapshot.traceId === "string") {
|
||||
traceId = snapshot.traceId;
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
logger.warn("RunStreamPresenter buffer fallback failed", {
|
||||
runFriendlyId,
|
||||
err: err instanceof Error ? err.message : String(err),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!traceId) {
|
||||
throw new Response("Not found", { status: 404 });
|
||||
}
|
||||
const resolvedRun = { traceId };
|
||||
|
||||
logger.info("RunStreamPresenter.start", {
|
||||
runFriendlyId,
|
||||
traceId: run.traceId,
|
||||
traceId: resolvedRun.traceId,
|
||||
});
|
||||
|
||||
// Subscribe to trace updates
|
||||
const { unsubscribe, eventEmitter } = await tracePubSub.subscribeToTrace(run.traceId);
|
||||
const { unsubscribe, eventEmitter } = await tracePubSub.subscribeToTrace(resolvedRun.traceId);
|
||||
|
||||
// Only send max every 1 second
|
||||
const throttledSend = throttle(
|
||||
@@ -105,7 +138,7 @@ export class RunStreamPresenter {
|
||||
cleanup: () => {
|
||||
logger.info("RunStreamPresenter.cleanup", {
|
||||
runFriendlyId,
|
||||
traceId: run.traceId,
|
||||
traceId: resolvedRun.traceId,
|
||||
});
|
||||
|
||||
// Remove message listener
|
||||
@@ -119,13 +152,13 @@ export class RunStreamPresenter {
|
||||
.then(() => {
|
||||
logger.info("RunStreamPresenter.cleanup.unsubscribe succeeded", {
|
||||
runFriendlyId,
|
||||
traceId: run.traceId,
|
||||
traceId: resolvedRun.traceId,
|
||||
});
|
||||
})
|
||||
.catch((error) => {
|
||||
logger.error("RunStreamPresenter.cleanup.unsubscribe failed", {
|
||||
runFriendlyId,
|
||||
traceId: run.traceId,
|
||||
traceId: resolvedRun.traceId,
|
||||
error: {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
|
||||
@@ -32,6 +32,8 @@ import {
|
||||
extractAIEmbedData,
|
||||
} from "~/components/runs/v3/ai";
|
||||
import { getEventRepositoryForStore } from "~/v3/eventRepository/index.server";
|
||||
import { findRunByIdWithMollifierFallback } from "~/v3/mollifier/readFallback.server";
|
||||
import { buildSyntheticSpanRun } from "~/v3/mollifier/syntheticSpanRun.server";
|
||||
|
||||
export type PromptSpanData = {
|
||||
slug: string;
|
||||
@@ -72,9 +74,21 @@ function extractPromptSpanData(properties: Record<string, unknown>): PromptSpanD
|
||||
};
|
||||
}
|
||||
|
||||
// SpanRun is grounded in the PG-path `getRun` method rather than
|
||||
// inferred from `call`'s return type. The buffered branch of `call`
|
||||
// routes through `buildSyntheticSpanRun`, and that helper is annotated
|
||||
// `Promise<SpanRun>` — if SpanRun were derived from `call` it would
|
||||
// close a loop TS no longer tolerates ("Type alias 'Result' circularly
|
||||
// references itself"). `getRun` is the canonical source for the shape
|
||||
// (the synthetic helper just rebuilds the same shape from a buffer
|
||||
// snapshot), and it doesn't recurse, so grounding here breaks the
|
||||
// cycle while keeping Span available off `call` (Span's path through
|
||||
// `#getSpan` has no synthetic indirection).
|
||||
export type SpanRun = NonNullable<
|
||||
Awaited<ReturnType<InstanceType<typeof SpanPresenter>["getRun"]>>
|
||||
>;
|
||||
type Result = Awaited<ReturnType<SpanPresenter["call"]>>;
|
||||
export type Span = NonNullable<NonNullable<Result>["span"]>;
|
||||
export type SpanRun = NonNullable<NonNullable<Result>["run"]>;
|
||||
type FindRunResult = NonNullable<
|
||||
Awaited<ReturnType<InstanceType<typeof SpanPresenter>["findRun"]>>
|
||||
>;
|
||||
@@ -84,12 +98,18 @@ export class SpanPresenter extends BasePresenter {
|
||||
public async call({
|
||||
userId,
|
||||
projectSlug,
|
||||
envSlug,
|
||||
spanId,
|
||||
runFriendlyId,
|
||||
linkedRunId,
|
||||
}: {
|
||||
userId: string;
|
||||
projectSlug: string;
|
||||
// Optional for backwards compatibility, required for the mollifier
|
||||
// buffer fallback when the parent run isn't yet in PG — we need to
|
||||
// resolve the env id to satisfy `findRunByIdWithMollifierFallback`'s
|
||||
// auth check.
|
||||
envSlug?: string;
|
||||
spanId: string;
|
||||
runFriendlyId: string;
|
||||
linkedRunId?: string;
|
||||
@@ -127,7 +147,32 @@ export class SpanPresenter extends BasePresenter {
|
||||
});
|
||||
|
||||
if (!parentRun) {
|
||||
return;
|
||||
// PG miss → fall back to the mollifier buffer. Without this the
|
||||
// right-side span detail panel on the run-detail page never
|
||||
// resolves for buffered runs: `call()` returns undefined, the
|
||||
// resource route redirects with an "Event not found" toast, the
|
||||
// run-detail page reloads, the toast fires again — a perpetual
|
||||
// spin until the drainer materialises the row. Synthesise a
|
||||
// SpanRun straight from the buffer snapshot, reusing
|
||||
// `buildSyntheticSpanRun` (the same helper the run-detail
|
||||
// loader's header fallback already uses).
|
||||
if (!envSlug) return;
|
||||
const envRow = await this._replica.runtimeEnvironment.findFirst({
|
||||
where: { project: { id: project.id }, slug: envSlug },
|
||||
select: { id: true, slug: true, type: true, organizationId: true },
|
||||
});
|
||||
if (!envRow) return;
|
||||
const buffered = await findRunByIdWithMollifierFallback({
|
||||
runId: runFriendlyId,
|
||||
environmentId: envRow.id,
|
||||
organizationId: envRow.organizationId,
|
||||
});
|
||||
if (!buffered) return;
|
||||
const synth = await buildSyntheticSpanRun({
|
||||
run: buffered,
|
||||
environment: { id: envRow.id, slug: envRow.slug, type: envRow.type },
|
||||
});
|
||||
return { type: "run" as const, run: synth };
|
||||
}
|
||||
|
||||
const { traceId } = parentRun;
|
||||
@@ -373,6 +418,7 @@ export class SpanPresenter extends BasePresenter {
|
||||
traceId: run.traceId,
|
||||
spanId: run.spanId,
|
||||
isCached: !!linkedRunId,
|
||||
isBuffered: false,
|
||||
machinePreset: machine?.name,
|
||||
taskEventStore: run.taskEventStore,
|
||||
externalTraceId,
|
||||
|
||||
@@ -3,7 +3,8 @@ import { z } from "zod";
|
||||
import { prisma } from "~/db.server";
|
||||
import { redirectWithErrorMessage } from "~/models/message.server";
|
||||
import { requireUser } from "~/services/session.server";
|
||||
import { impersonate, rootPath, v3RunPath } from "~/utils/pathBuilder";
|
||||
import { impersonate, rootPath, v3RunPath, v3RunSpanPath } from "~/utils/pathBuilder";
|
||||
import { findBufferedRunRedirectInfo } from "~/v3/mollifier/syntheticRedirectInfo.server";
|
||||
|
||||
const ParamsSchema = z.object({
|
||||
runParam: z.string(),
|
||||
@@ -32,6 +33,7 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
friendlyId: runParam,
|
||||
},
|
||||
select: {
|
||||
spanId: true,
|
||||
runtimeEnvironment: {
|
||||
select: {
|
||||
slug: true,
|
||||
@@ -51,16 +53,45 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
});
|
||||
|
||||
if (!run) {
|
||||
// Admin impersonation route — bypass org membership so admins can
|
||||
// open any buffered run by friendlyId, mirroring the existing PG
|
||||
// behaviour above (no membership filter on the find).
|
||||
const buffered = await findBufferedRunRedirectInfo({
|
||||
runFriendlyId: runParam,
|
||||
userId: user.id,
|
||||
skipOrgMembershipCheck: true,
|
||||
});
|
||||
if (buffered) {
|
||||
// Preselect the root span so the run-detail trace tree opens with
|
||||
// the buffered run's span highlighted, matching the sibling
|
||||
// redirect routes (runs.$runParam.ts, projects.v3.$projectRef…).
|
||||
const path = buffered.spanId
|
||||
? v3RunSpanPath(
|
||||
{ slug: buffered.organizationSlug },
|
||||
{ slug: buffered.projectSlug },
|
||||
{ slug: buffered.environmentSlug },
|
||||
{ friendlyId: runParam },
|
||||
{ spanId: buffered.spanId }
|
||||
)
|
||||
: v3RunPath(
|
||||
{ slug: buffered.organizationSlug },
|
||||
{ slug: buffered.projectSlug },
|
||||
{ slug: buffered.environmentSlug },
|
||||
{ friendlyId: runParam }
|
||||
);
|
||||
return redirect(impersonate(path));
|
||||
}
|
||||
return redirectWithErrorMessage(rootPath(), request, "Run doesn't exist", {
|
||||
ephemeral: false,
|
||||
});
|
||||
}
|
||||
|
||||
const path = v3RunPath(
|
||||
const path = v3RunSpanPath(
|
||||
{ slug: run.project.organization.slug },
|
||||
{ slug: run.project.slug },
|
||||
{ slug: run.runtimeEnvironment.slug },
|
||||
{ friendlyId: runParam }
|
||||
{ friendlyId: runParam },
|
||||
{ spanId: run.spanId }
|
||||
);
|
||||
|
||||
return redirect(impersonate(path));
|
||||
|
||||
+156
-18
@@ -88,10 +88,18 @@ import { useReplaceSearchParams } from "~/hooks/useReplaceSearchParams";
|
||||
import { useSearchParams } from "~/hooks/useSearchParam";
|
||||
import { type Shortcut, useShortcutKeys } from "~/hooks/useShortcutKeys";
|
||||
import { useHasAdminAccess } from "~/hooks/useUser";
|
||||
import { env } from "~/env.server";
|
||||
import { findProjectBySlug } from "~/models/project.server";
|
||||
import { findEnvironmentBySlug } from "~/models/runtimeEnvironment.server";
|
||||
import { NextRunListPresenter } from "~/presenters/v3/NextRunListPresenter.server";
|
||||
import { RunEnvironmentMismatchError, RunPresenter } from "~/presenters/v3/RunPresenter.server";
|
||||
import {
|
||||
RunEnvironmentMismatchError,
|
||||
RunNotInPgError,
|
||||
RunPresenter,
|
||||
} from "~/presenters/v3/RunPresenter.server";
|
||||
import { findRunByIdWithMollifierFallback } from "~/v3/mollifier/readFallback.server";
|
||||
import { buildSyntheticRunHeader } from "~/v3/mollifier/syntheticRunHeader.server";
|
||||
import { buildSyntheticTraceForBufferedRun } from "~/v3/mollifier/syntheticTrace.server";
|
||||
import { clickhouseFactory } from "~/services/clickhouse/clickhouseFactoryInstance.server";
|
||||
import { getImpersonationId } from "~/services/impersonation.server";
|
||||
import { logger } from "~/services/logger.server";
|
||||
@@ -277,9 +285,78 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Only fall back to the mollifier buffer on a genuine PG miss. Any
|
||||
// other error (DB timeout during trace queries, event-repository
|
||||
// failure, etc.) means the run WAS in PG but a downstream lookup
|
||||
// failed — falling back to the buffer here would either return a
|
||||
// stale synth entry if one happens to exist in the brief drainer-
|
||||
// materialisation race window, or quietly mask the real failure.
|
||||
// `RunNotInPgError` is the typed signal RunPresenter throws for the
|
||||
// route loader's specific case (`RunPresenter.server.ts:130`).
|
||||
if (!(error instanceof RunNotInPgError)) {
|
||||
throw error;
|
||||
}
|
||||
|
||||
// PG miss → try the mollifier buffer. When the gate diverts a trigger
|
||||
// the run sits in Redis until the drainer materialises it; without
|
||||
// this fallback the run-detail page 404s for the brief buffered window
|
||||
// even though the API has accepted the trigger and returned an id.
|
||||
const buffered = await tryMollifiedRunFallback({
|
||||
runFriendlyId: runParam,
|
||||
organizationSlug,
|
||||
projectSlug: projectParam,
|
||||
envSlug: envParam,
|
||||
userId,
|
||||
});
|
||||
|
||||
if (buffered) {
|
||||
// Preselect the root span on the initial page load when the URL
|
||||
// doesn't already carry `?span=`. The sibling redirect routes
|
||||
// (runs.$runParam.ts, @.runs.$runParam.ts,
|
||||
// projects.v3.$projectRef.runs.$runParam.ts) all do this, but
|
||||
// direct navigation to the canonical project-scoped URL never
|
||||
// hit those redirects — leaving the right detail panel collapsed.
|
||||
// Skip on `_data` requests (Remix data fetches): they're
|
||||
// client-driven follow-ups and the client URL is what matters,
|
||||
// not the loader's view of it.
|
||||
if (
|
||||
!url.searchParams.has("span") &&
|
||||
!url.searchParams.has("_data") &&
|
||||
buffered.run.spanId
|
||||
) {
|
||||
url.searchParams.set("span", buffered.run.spanId);
|
||||
throw redirect(url.pathname + "?" + url.searchParams.toString());
|
||||
}
|
||||
|
||||
const parent = await getResizableSnapshot(request, resizableSettings.parent.autosaveId);
|
||||
const tree = await getResizableSnapshot(request, resizableSettings.tree.autosaveId);
|
||||
|
||||
return json({
|
||||
run: buffered.run,
|
||||
trace: buffered.trace,
|
||||
maximumLiveReloadingSetting: env.MAXIMUM_LIVE_RELOADING_EVENTS,
|
||||
resizable: { parent, tree },
|
||||
runsList: null,
|
||||
});
|
||||
}
|
||||
|
||||
throw error;
|
||||
}
|
||||
|
||||
// Preselect the root span on the initial page load when the URL
|
||||
// doesn't already carry `?span=`. See the comment on the equivalent
|
||||
// block in the buffered fallback above — the sibling redirect routes
|
||||
// do this, but direct navigation to the canonical project-scoped URL
|
||||
// never hits them, leaving the right detail panel collapsed.
|
||||
if (
|
||||
!url.searchParams.has("span") &&
|
||||
!url.searchParams.has("_data") &&
|
||||
result.run.spanId
|
||||
) {
|
||||
url.searchParams.set("span", result.run.spanId);
|
||||
throw redirect(url.pathname + "?" + url.searchParams.toString());
|
||||
}
|
||||
|
||||
//resizable settings
|
||||
const parent = await getResizableSnapshot(request, resizableSettings.parent.autosaveId);
|
||||
const tree = await getResizableSnapshot(request, resizableSettings.tree.autosaveId);
|
||||
@@ -305,6 +382,39 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
});
|
||||
};
|
||||
|
||||
async function tryMollifiedRunFallback(args: {
|
||||
runFriendlyId: string;
|
||||
organizationSlug: string;
|
||||
projectSlug: string;
|
||||
envSlug: string;
|
||||
userId: string;
|
||||
}) {
|
||||
const project = await findProjectBySlug(args.organizationSlug, args.projectSlug, args.userId);
|
||||
if (!project) return null;
|
||||
const environment = await findEnvironmentBySlug(project.id, args.envSlug, args.userId);
|
||||
if (!environment) return null;
|
||||
|
||||
const buffered = await findRunByIdWithMollifierFallback({
|
||||
runId: args.runFriendlyId,
|
||||
environmentId: environment.id,
|
||||
organizationId: project.organizationId,
|
||||
});
|
||||
if (!buffered) return null;
|
||||
|
||||
return {
|
||||
run: buildSyntheticRunHeader({
|
||||
run: buffered,
|
||||
environment: {
|
||||
id: environment.id,
|
||||
organizationId: project.organizationId,
|
||||
type: environment.type,
|
||||
slug: environment.slug,
|
||||
},
|
||||
}),
|
||||
trace: buildSyntheticTraceForBufferedRun(buffered),
|
||||
};
|
||||
}
|
||||
|
||||
type LoaderData = SerializeFrom<typeof loader>;
|
||||
|
||||
export default function Page() {
|
||||
@@ -407,23 +517,17 @@ export default function Page() {
|
||||
/>
|
||||
</Dialog>
|
||||
{run.isFinished ? null : (
|
||||
<Dialog key={`cancel-${run.friendlyId}`}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant="danger/small" LeadingIcon={StopCircleIcon} shortcut={{ key: "C" }}>
|
||||
Cancel run…
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<CancelRunDialog
|
||||
runFriendlyId={run.friendlyId}
|
||||
redirectPath={v3RunSpanPath(
|
||||
organization,
|
||||
project,
|
||||
environment,
|
||||
{ friendlyId: run.friendlyId },
|
||||
{ spanId: run.spanId }
|
||||
)}
|
||||
/>
|
||||
</Dialog>
|
||||
<ControlledCancelRunDialog
|
||||
key={`cancel-${run.friendlyId}`}
|
||||
runFriendlyId={run.friendlyId}
|
||||
redirectPath={v3RunSpanPath(
|
||||
organization,
|
||||
project,
|
||||
environment,
|
||||
{ friendlyId: run.friendlyId },
|
||||
{ spanId: run.spanId }
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
</PageAccessories>
|
||||
</NavBar>
|
||||
@@ -587,6 +691,35 @@ function TraceView({
|
||||
);
|
||||
}
|
||||
|
||||
// Controlled wrapper around the cancel dialog. Owns the Radix open state
|
||||
// so the dialog closes itself once the cancel action transitions through
|
||||
// submission. We can't `<DialogClose asChild>`-wrap the submit button
|
||||
// because Radix's onClick handler swallows the button's name=value pair
|
||||
// that the form action depends on for `redirectUrl`.
|
||||
function ControlledCancelRunDialog({
|
||||
runFriendlyId,
|
||||
redirectPath,
|
||||
}: {
|
||||
runFriendlyId: string;
|
||||
redirectPath: string;
|
||||
}) {
|
||||
const [open, setOpen] = useState(false);
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogTrigger asChild>
|
||||
<Button variant="danger/small" LeadingIcon={StopCircleIcon} shortcut={{ key: "C" }}>
|
||||
Cancel run…
|
||||
</Button>
|
||||
</DialogTrigger>
|
||||
<CancelRunDialog
|
||||
runFriendlyId={runFriendlyId}
|
||||
redirectPath={redirectPath}
|
||||
onCancelSubmitted={() => setOpen(false)}
|
||||
/>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
function NoLogsView({ run, resizable }: Pick<LoaderData, "run" | "resizable">) {
|
||||
const plan = useCurrentPlan();
|
||||
const organization = useOrganization();
|
||||
@@ -616,6 +749,11 @@ function NoLogsView({ run, resizable }: Pick<LoaderData, "run" | "resizable">) {
|
||||
>
|
||||
<div className="grid h-full place-items-center">
|
||||
{daysSinceCompleted === undefined ? (
|
||||
// NoLogsView only renders when the loader returns no trace.
|
||||
// Buffered runs always carry a synthetic trace (see
|
||||
// buildSyntheticTraceForBufferedRun) so they never reach
|
||||
// this branch — the message here is the pre-mollifier
|
||||
// copy for runs with no completedAt and no logs.
|
||||
<InfoPanel variant="info" icon={InformationCircleIcon} title="We delete old logs">
|
||||
<Paragraph variant="small">
|
||||
We tidy up older logs to keep things running smoothly.
|
||||
|
||||
+6
-1
@@ -120,6 +120,7 @@ export const loader = async ({ request, params }: LoaderFunctionArgs) => {
|
||||
try {
|
||||
const result = await presenter.call({
|
||||
projectSlug: projectParam,
|
||||
envSlug: envParam,
|
||||
spanId: spanParam,
|
||||
runFriendlyId: runParam,
|
||||
userId,
|
||||
@@ -1021,6 +1022,10 @@ function RunBody({
|
||||
<Paragraph spacing variant="small" className="text-yellow-500">
|
||||
Admin only
|
||||
</Paragraph>
|
||||
<Property.Item>
|
||||
<Property.Label>Buffered</Property.Label>
|
||||
<Property.Value>{run.isBuffered ? "Yes" : "No"}</Property.Value>
|
||||
</Property.Item>
|
||||
<Property.Item>
|
||||
<Property.Label>Worker queue</Property.Label>
|
||||
<Property.Value>{run.workerQueue}</Property.Value>
|
||||
@@ -1096,7 +1101,7 @@ function RunBody({
|
||||
{run.isCached ? "Jump to original run" : "Focus on run"}
|
||||
</LinkButton>
|
||||
)}
|
||||
<AdminDebugRun friendlyId={run.friendlyId} />
|
||||
{!run.isBuffered && <AdminDebugRun friendlyId={run.friendlyId} />}
|
||||
</div>
|
||||
<div className="flex items-center">
|
||||
{run.logsDeletedAt === null ? (
|
||||
|
||||
@@ -9,6 +9,8 @@ import { formatDurationMilliseconds } from "@trigger.dev/core/v3/utils/durations
|
||||
import { getTaskEventStoreTableForRun } from "~/v3/taskEventStore.server";
|
||||
import { TaskEventKind } from "@trigger.dev/database";
|
||||
import { getEventRepositoryForStore } from "~/v3/eventRepository/index.server";
|
||||
import { getMollifierBuffer } from "~/v3/mollifier/mollifierBuffer.server";
|
||||
import { deserialiseMollifierSnapshot } from "~/v3/mollifier/mollifierSnapshot.server";
|
||||
|
||||
export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
const user = await requireUser(request);
|
||||
@@ -30,6 +32,67 @@ export async function loader({ params, request }: LoaderFunctionArgs) {
|
||||
});
|
||||
|
||||
if (!run || !run.organizationId) {
|
||||
// Buffered run? It hasn't executed, so there are no events to
|
||||
// stream — but a 404 is wrong: the run does exist, the customer's
|
||||
// "Download logs" button on the run-detail page generates this
|
||||
// exact URL, and a 404 reads as "your run vanished" rather than
|
||||
// "no logs yet". Verify the entry exists in the buffer (with the
|
||||
// user as a member of the entry's org), and if so stream a single
|
||||
// informational line in the same `<timestamp> <task> <level>
|
||||
// <message>` shape `formatRunEvent` uses below — so a downstream
|
||||
// log viewer / grep over the downloaded file produces a
|
||||
// meaningful explanation, not a 0-byte mystery.
|
||||
const buffer = getMollifierBuffer();
|
||||
if (buffer) {
|
||||
const entry = await buffer.getEntry(parsedParams.runParam);
|
||||
if (entry) {
|
||||
const member = await prisma.orgMember.findFirst({
|
||||
where: { userId: user.id, organizationId: entry.orgId },
|
||||
select: { id: true },
|
||||
});
|
||||
if (member) {
|
||||
let taskIdentifier: string | undefined;
|
||||
try {
|
||||
// Use the shared webapp wrapper rather than raw JSON.parse so
|
||||
// every read-side module shares a single deserialisation path
|
||||
// (see contract comment in `mollifierSnapshot.server.ts` and
|
||||
// `syntheticRedirectInfo.server.ts`). Keeps behaviour
|
||||
// consistent if the snapshot encoding ever changes.
|
||||
const snapshot = deserialiseMollifierSnapshot(entry.payload) as {
|
||||
taskIdentifier?: unknown;
|
||||
};
|
||||
if (typeof snapshot.taskIdentifier === "string") {
|
||||
taskIdentifier = snapshot.taskIdentifier;
|
||||
}
|
||||
} catch {
|
||||
// Fall through — taskIdentifier stays undefined.
|
||||
}
|
||||
const placeholderParts = [
|
||||
entry.createdAt.toISOString(),
|
||||
...(taskIdentifier ? [taskIdentifier] : []),
|
||||
"INFO",
|
||||
"Run is queued, has not started executing yet — no logs to download.",
|
||||
];
|
||||
const placeholder = placeholderParts.join(" ") + "\n";
|
||||
const placeholderReadable = new Readable({
|
||||
read() {
|
||||
this.push(placeholder);
|
||||
this.push(null);
|
||||
},
|
||||
});
|
||||
const gzipStream = createGzip();
|
||||
const compressed = placeholderReadable.pipe(gzipStream);
|
||||
return new Response(compressed as any, {
|
||||
status: 200,
|
||||
headers: {
|
||||
"Content-Type": "application/octet-stream",
|
||||
"Content-Disposition": `attachment; filename="${parsedParams.runParam}.log"`,
|
||||
"Content-Encoding": "gzip",
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return new Response("Not found", { status: 404 });
|
||||
}
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ import { redirectWithErrorMessage, redirectWithSuccessMessage } from "~/models/m
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { requireUserId } from "~/services/session.server";
|
||||
import { CancelTaskRunService } from "~/v3/services/cancelTaskRun.server";
|
||||
import { getMollifierBuffer } from "~/v3/mollifier/mollifierBuffer.server";
|
||||
|
||||
export const cancelSchema = z.object({
|
||||
redirectUrl: z.string(),
|
||||
@@ -42,15 +43,56 @@ export const action: ActionFunction = async ({ request, params }) => {
|
||||
},
|
||||
});
|
||||
|
||||
if (!taskRun) {
|
||||
if (taskRun) {
|
||||
const cancelRunService = new CancelTaskRunService();
|
||||
await cancelRunService.call(taskRun);
|
||||
return redirectWithSuccessMessage(submission.value.redirectUrl, request, `Canceled run`);
|
||||
}
|
||||
|
||||
// PG miss — try the mollifier buffer. The customer can hit cancel
|
||||
// on a buffered run from the dashboard during the burst window.
|
||||
// Snapshot a `mark_cancelled` patch; the drainer's
|
||||
// bifurcation routes the run to `engine.createCancelledRun` on
|
||||
// next pop.
|
||||
const buffer = getMollifierBuffer();
|
||||
const entry = buffer ? await buffer.getEntry(runParam) : null;
|
||||
if (!entry) {
|
||||
submission.error = { runParam: ["Run not found"] };
|
||||
return json(submission);
|
||||
}
|
||||
|
||||
const cancelRunService = new CancelTaskRunService();
|
||||
await cancelRunService.call(taskRun);
|
||||
// Dashboard auth: verify the requesting user is a member of the
|
||||
// buffered run's org. The API path scopes by env id from the
|
||||
// authenticated request; the dashboard route uses org-membership
|
||||
// because the URL doesn't carry an envId.
|
||||
const member = await prisma.orgMember.findFirst({
|
||||
where: { userId, organizationId: entry.orgId },
|
||||
select: { id: true },
|
||||
});
|
||||
if (!member) {
|
||||
submission.error = { runParam: ["Run not found"] };
|
||||
return json(submission);
|
||||
}
|
||||
|
||||
return redirectWithSuccessMessage(submission.value.redirectUrl, request, `Canceled run`);
|
||||
const result = await buffer!.mutateSnapshot(runParam, {
|
||||
type: "mark_cancelled",
|
||||
cancelledAt: new Date().toISOString(),
|
||||
cancelReason: "Canceled by user",
|
||||
});
|
||||
if (result === "applied_to_snapshot") {
|
||||
return redirectWithSuccessMessage(submission.value.redirectUrl, request, `Canceled run`);
|
||||
}
|
||||
// "not_found" or "busy" — both indicate the drainer raced us between
|
||||
// the getEntry check above and mutateSnapshot. On "not_found" the
|
||||
// entry was just popped and the PG row is in flight; on "busy" the
|
||||
// drainer is mid-materialisation. Either way the customer should
|
||||
// retry — by then the PG row exists and the regular cancel path at
|
||||
// the top of this action takes over.
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.redirectUrl,
|
||||
request,
|
||||
"Run is materialising — retry in a moment"
|
||||
);
|
||||
} catch (error) {
|
||||
if (error instanceof Error) {
|
||||
logger.error("Failed to cancel run", {
|
||||
|
||||
@@ -11,6 +11,12 @@ import { requireUser } from "~/services/session.server";
|
||||
import { sortEnvironments } from "~/utils/environmentSort";
|
||||
import { v3RunSpanPath } from "~/utils/pathBuilder";
|
||||
import { ReplayTaskRunService } from "~/v3/services/replayTaskRun.server";
|
||||
import { getMollifierBuffer } from "~/v3/mollifier/mollifierBuffer.server";
|
||||
import { findRunByIdWithMollifierFallback } from "~/v3/mollifier/readFallback.server";
|
||||
import {
|
||||
buildSyntheticReplayTaskRun,
|
||||
type SyntheticReplayTaskRun,
|
||||
} from "~/v3/mollifier/syntheticReplayTaskRun.server";
|
||||
import parseDuration from "parse-duration";
|
||||
import { findCurrentWorkerDeployment } from "~/v3/models/workerDeployment.server";
|
||||
import { queueTypeFromType } from "~/presenters/v3/QueueRetrievePresenter.server";
|
||||
@@ -33,7 +39,7 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
Object.fromEntries(new URL(request.url).searchParams)
|
||||
);
|
||||
|
||||
const run = await $replica.taskRun.findFirst({
|
||||
let run = await $replica.taskRun.findFirst({
|
||||
select: {
|
||||
payload: true,
|
||||
payloadType: true,
|
||||
@@ -88,6 +94,83 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
where: { friendlyId: runParam, project: { organization: { members: { some: { userId } } } } },
|
||||
});
|
||||
|
||||
let synthetic:
|
||||
| (Awaited<ReturnType<typeof findRunByIdWithMollifierFallback>> & { __synth: true })
|
||||
| undefined;
|
||||
if (!run) {
|
||||
// Buffered fallback: read the snapshot and look up the env list via
|
||||
// the snapshot's organizationId. Without this the Replay dialog
|
||||
// 404s for runs queued in the mollifier buffer, which dumps the
|
||||
// user back to the task list.
|
||||
const buffer = getMollifierBuffer();
|
||||
const entry = buffer ? await buffer.getEntry(runParam) : null;
|
||||
if (!entry) throw new Response("Not Found", { status: 404 });
|
||||
const member = await prisma.orgMember.findFirst({
|
||||
where: { userId, organizationId: entry.orgId },
|
||||
select: { id: true },
|
||||
});
|
||||
if (!member) throw new Response("Not Found", { status: 404 });
|
||||
const buffered = await findRunByIdWithMollifierFallback({
|
||||
runId: runParam,
|
||||
environmentId: entry.envId,
|
||||
organizationId: entry.orgId,
|
||||
});
|
||||
if (!buffered) throw new Response("Not Found", { status: 404 });
|
||||
synthetic = Object.assign(buffered, { __synth: true as const });
|
||||
// Scope the project lookup to the buffer entry's org as well as the
|
||||
// env id. The prior `orgMember.findFirst` above confirms the user
|
||||
// belongs to `entry.orgId`; pinning `organizationId` here means a
|
||||
// malformed entry whose envId resolves to a different org can't leak
|
||||
// that project's data through this loader. Mirrors the PG path's
|
||||
// `project.organization.members.some.userId` scoping (lines 42-95)
|
||||
// — the env filter and select shape are kept identical so the Replay
|
||||
// dialog renders the same dropdown either way.
|
||||
const orgProject = await $replica.project.findFirst({
|
||||
where: {
|
||||
organizationId: entry.orgId,
|
||||
environments: { some: { id: entry.envId } },
|
||||
},
|
||||
select: {
|
||||
slug: true,
|
||||
environments: {
|
||||
select: {
|
||||
id: true,
|
||||
type: true,
|
||||
slug: true,
|
||||
branchName: true,
|
||||
orgMember: { select: { user: true } },
|
||||
},
|
||||
where: {
|
||||
archivedAt: null,
|
||||
OR: [
|
||||
{ type: { in: ["PREVIEW", "STAGING", "PRODUCTION"] } },
|
||||
{ type: "DEVELOPMENT", orgMember: { userId } },
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
if (!orgProject) throw new Response("Not Found", { status: 404 });
|
||||
run = {
|
||||
payload: buffered.payload,
|
||||
payloadType: buffered.payloadType ?? "application/json",
|
||||
seedMetadata: buffered.seedMetadata ?? null,
|
||||
seedMetadataType: buffered.seedMetadataType ?? null,
|
||||
runtimeEnvironmentId: entry.envId,
|
||||
concurrencyKey: buffered.concurrencyKey ?? null,
|
||||
maxAttempts: buffered.maxAttempts ?? null,
|
||||
maxDurationInSeconds: buffered.maxDurationInSeconds ?? null,
|
||||
machinePreset: buffered.machinePreset ?? null,
|
||||
workerQueue: buffered.workerQueue ?? null,
|
||||
ttl: buffered.ttl ?? null,
|
||||
idempotencyKey: buffered.idempotencyKey ?? null,
|
||||
runTags: buffered.runTags,
|
||||
queue: buffered.queue ?? "task/",
|
||||
taskIdentifier: buffered.taskIdentifier ?? "",
|
||||
project: orgProject,
|
||||
} as unknown as typeof run;
|
||||
}
|
||||
|
||||
if (!run) {
|
||||
throw new Response("Not Found", { status: 404 });
|
||||
}
|
||||
@@ -164,6 +247,15 @@ export async function loader({ request, params }: LoaderFunctionArgs) {
|
||||
}
|
||||
|
||||
export const action: ActionFunction = async ({ request, params }) => {
|
||||
// Dashboard auth: identical pattern to resources.taskruns.$runParam.cancel.ts.
|
||||
// The loader above this action already gates with `requireUser`, but
|
||||
// Remix's action runs independently — without this call any request
|
||||
// with a valid runParam could submit a replay. The PG findFirst below
|
||||
// also adds the org-membership filter so a PAT can't replay another
|
||||
// org's run, and the buffered fallback verifies org membership via
|
||||
// orgMember.findFirst against the snapshot's orgId.
|
||||
const user = await requireUser(request);
|
||||
const userId = user.id;
|
||||
const { runParam } = ParamSchema.parse(params);
|
||||
|
||||
const formData = await request.formData();
|
||||
@@ -174,9 +266,18 @@ export const action: ActionFunction = async ({ request, params }) => {
|
||||
}
|
||||
|
||||
try {
|
||||
const taskRun = await prisma.taskRun.findFirst({
|
||||
const pgRun = await prisma.taskRun.findFirst({
|
||||
where: {
|
||||
friendlyId: runParam,
|
||||
project: {
|
||||
organization: {
|
||||
members: {
|
||||
some: {
|
||||
userId,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
include: {
|
||||
runtimeEnvironment: {
|
||||
@@ -192,6 +293,50 @@ export const action: ActionFunction = async ({ request, params }) => {
|
||||
},
|
||||
});
|
||||
|
||||
// Mollifier read-fallback: if the original isn't in PG yet,
|
||||
// synthesise a TaskRun from the buffered snapshot. The B4-extended
|
||||
// SyntheticRun carries every field ReplayTaskRunService reads. We
|
||||
// also need projectSlug + orgSlug + envSlug for the redirect path,
|
||||
// so look those up via the snapshot's runtimeEnvironmentId.
|
||||
let taskRun: SyntheticReplayTaskRun | null = pgRun ?? null;
|
||||
if (!taskRun) {
|
||||
const buffer = getMollifierBuffer();
|
||||
const entry = buffer ? await buffer.getEntry(runParam) : null;
|
||||
if (entry) {
|
||||
// Same org-membership gate as the PG path above. Without this
|
||||
// any authenticated user who knows a runId could replay the
|
||||
// buffered run across orgs.
|
||||
const member = await prisma.orgMember.findFirst({
|
||||
where: { userId, organizationId: entry.orgId },
|
||||
select: { id: true },
|
||||
});
|
||||
if (!member) {
|
||||
return redirectWithErrorMessage(
|
||||
submission.value.failedRedirect,
|
||||
request,
|
||||
"Run not found"
|
||||
);
|
||||
}
|
||||
const synthetic = await findRunByIdWithMollifierFallback({
|
||||
runId: runParam,
|
||||
environmentId: entry.envId,
|
||||
organizationId: entry.orgId,
|
||||
});
|
||||
if (synthetic) {
|
||||
const envRow = await prisma.runtimeEnvironment.findFirst({
|
||||
where: { id: entry.envId },
|
||||
select: {
|
||||
slug: true,
|
||||
project: { select: { slug: true, organization: { select: { slug: true } } } },
|
||||
},
|
||||
});
|
||||
if (envRow) {
|
||||
taskRun = buildSyntheticReplayTaskRun({ synthetic, envRow });
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!taskRun) {
|
||||
return redirectWithErrorMessage(submission.value.failedRedirect, request, "Run not found");
|
||||
}
|
||||
|
||||
@@ -7,6 +7,7 @@ import type {
|
||||
MollifierDrainerTerminalFailureHandler,
|
||||
} from "@trigger.dev/redis-worker";
|
||||
import { logger } from "~/services/logger.server";
|
||||
import { recordRunDebugLog } from "~/v3/eventRepository/index.server";
|
||||
import { PerformTaskRunAlertsService } from "~/v3/services/alerts/performTaskRunAlerts.server";
|
||||
import { startSpan } from "~/v3/tracing.server";
|
||||
import type { MollifierSnapshot } from "./mollifierSnapshot.server";
|
||||
@@ -162,8 +163,10 @@ export function createDrainerHandler(deps: {
|
||||
span.setAttribute("mollifier.run_friendly_id", input.runId);
|
||||
span.setAttribute("taskRunId", input.runId);
|
||||
|
||||
let triggerSucceeded = false;
|
||||
try {
|
||||
await deps.engine.trigger(input.payload as any, deps.prisma);
|
||||
triggerSucceeded = true;
|
||||
} catch (err) {
|
||||
// The retryable-PG class re-throws so the drainer's outer
|
||||
// worker loop can `buffer.requeue` (handled in
|
||||
@@ -212,6 +215,54 @@ export function createDrainerHandler(deps: {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
// Admin-only audit trail emitted once engine.trigger has
|
||||
// landed a PG row. `recordRunDebugLog` flips this to the
|
||||
// admin-gated debug kind (TaskEventKind.LOG in the PG store /
|
||||
// DEBUG_EVENT in the ClickHouse store) which the trace view +
|
||||
// logs download already strip for non-admins
|
||||
// (`eventRepository.server.ts:108`,
|
||||
// `resources.runs.$runParam.logs.download.ts:118`).
|
||||
//
|
||||
// Placement: emit as a zero-duration marker AT materialisation
|
||||
// time, not as a back-dated bar spanning the buffered window.
|
||||
// `engine.trigger` rewrites the run's root span at
|
||||
// materialisation (it adopts the synth root via traceId/spanId
|
||||
// carryover but updates start_time to "now"), so the trace
|
||||
// renderer treats materialisation time as t=0. A back-dated
|
||||
// event with startTime = bufferedAt would land before that t=0
|
||||
// and get clipped from the tree. Same pattern as the
|
||||
// `[engine] QUEUED` markers. The window itself is preserved
|
||||
// in metadata so admins can read it off the span detail pane.
|
||||
//
|
||||
// Best-effort: `recordRunDebugLog` has its own try/catch and
|
||||
// returns a result, so it never throws into the materialisation
|
||||
// path. Failures are logged but not surfaced because the
|
||||
// customer-visible run has already landed.
|
||||
if (triggerSucceeded) {
|
||||
const debugResult = await recordRunDebugLog(
|
||||
RunId.fromFriendlyId(input.runId),
|
||||
`Mollifier buffered ${dwellMs}ms before materialising`,
|
||||
{
|
||||
attributes: {
|
||||
runId: input.runId,
|
||||
metadata: {
|
||||
"mollifier.bufferedAt": input.createdAt.toISOString(),
|
||||
"mollifier.materialisedAt": new Date().toISOString(),
|
||||
"mollifier.dwellMs": dwellMs,
|
||||
"mollifier.attempts": input.attempts,
|
||||
},
|
||||
},
|
||||
parentId: snapshotSpanId,
|
||||
}
|
||||
);
|
||||
if (!debugResult.success && debugResult.code !== "RUN_NOT_FOUND") {
|
||||
logger.warn("mollifier drainer: failed to record admin debug log", {
|
||||
runId: input.runId,
|
||||
code: debugResult.code,
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
import type { TaskRun } from "@trigger.dev/database";
|
||||
import type { SyntheticRun } from "./readFallback.server";
|
||||
|
||||
export type SyntheticReplayTaskRun = TaskRun & {
|
||||
project: { slug: string; organization: { slug: string } };
|
||||
runtimeEnvironment: { slug: string };
|
||||
};
|
||||
|
||||
// Adapt a buffered-run snapshot into the TaskRun-shaped input that
|
||||
// `ReplayTaskRunService.call` expects. ReplayTaskRunService builds the
|
||||
// new run's traceparent as `00-${existingTaskRun.traceId}-${existingTaskRun.spanId}-01`
|
||||
// without guarding for undefined, so a synthetic with missing traceId
|
||||
// or spanId (older snapshots — both fields are documented optional on
|
||||
// `SyntheticRun`) would produce `00-undefined-undefined-01`, an invalid
|
||||
// W3C traceparent that OTel silently drops, severing the replay's trace
|
||||
// link to the original run.
|
||||
//
|
||||
// Returns null when those fields are missing — the caller surfaces this
|
||||
// as "Run not found" so the customer retries once the drainer has
|
||||
// materialised the PG row, where traceId/spanId are guaranteed present.
|
||||
export function buildSyntheticReplayTaskRun(args: {
|
||||
synthetic: SyntheticRun;
|
||||
envRow: {
|
||||
slug: string;
|
||||
project: { slug: string; organization: { slug: string } };
|
||||
};
|
||||
}): SyntheticReplayTaskRun | null {
|
||||
const { synthetic, envRow } = args;
|
||||
if (!synthetic.traceId || !synthetic.spanId) return null;
|
||||
return {
|
||||
// The double `as unknown as TaskRun` cast is load-bearing — a direct
|
||||
// `synthetic as TaskRun` won't compile. `SyntheticRun` carries the
|
||||
// subset of fields that `ReplayTaskRunService.call` actually reads
|
||||
// (the contract is enumerated on the SyntheticRun type comment in
|
||||
// readFallback.server.ts), but its shape is not structurally
|
||||
// assignable to the full Prisma `TaskRun` row: optional vs required
|
||||
// fields diverge, several PG columns (number, batchId variants,
|
||||
// status enum widening) are deliberately absent or narrower on the
|
||||
// synthetic. Routing it through `unknown` is the explicit "we know
|
||||
// this is a subset, we've audited which fields are read" signal,
|
||||
// and the traceId/spanId guard above prevents the only field
|
||||
// ReplayTaskRunService consumes that would corrupt downstream
|
||||
// behaviour (the OTel traceparent) when undefined.
|
||||
...(synthetic as unknown as TaskRun),
|
||||
project: {
|
||||
slug: envRow.project.slug,
|
||||
organization: { slug: envRow.project.organization.slug },
|
||||
},
|
||||
runtimeEnvironment: { slug: envRow.slug },
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
import type { SyntheticRun } from "./readFallback.server";
|
||||
|
||||
// Synthesise the run-detail page's `run` header shape (the NavBar +
|
||||
// status badge + Cancel-button gate) from a buffered run snapshot. The
|
||||
// shape matches `RunPresenter.getRun`'s `runData` — keep this in sync
|
||||
// when fields are added there.
|
||||
//
|
||||
// CANCELED and FAILED state is reflected back from
|
||||
// `SyntheticRun.cancelledAt` / `status` so terminal buffered runs show
|
||||
// the correct status in the NavBar + isFinished:true (which collapses
|
||||
// the Cancel button on the page header) before the drainer materialises
|
||||
// the PG row. This mirrors what `buildSyntheticSpanRun` does for the
|
||||
// right-side details panel — the SyntheticRun.cancelledAt contract
|
||||
// comment in readFallback.server.ts names this exact UI surface.
|
||||
//
|
||||
// FAILED status maps to `SYSTEM_FAILURE` to match the drainer's
|
||||
// non-retryable terminal path, which is what `buildSyntheticSpanRun`
|
||||
// uses too. Symmetric across the header + span-detail panel so an
|
||||
// admin doesn't see "Pending" + "FAILED" simultaneously on the same
|
||||
// run.
|
||||
export function buildSyntheticRunHeader(args: {
|
||||
run: SyntheticRun;
|
||||
environment: {
|
||||
id: string;
|
||||
organizationId: string;
|
||||
type: "PRODUCTION" | "DEVELOPMENT" | "STAGING" | "PREVIEW";
|
||||
slug: string;
|
||||
};
|
||||
}) {
|
||||
const { run, environment } = args;
|
||||
const isCancelled = run.status === "CANCELED";
|
||||
const isFailed = run.status === "FAILED";
|
||||
|
||||
return {
|
||||
// `id` mirrors RunPresenter.getRun's runData (the PG path), which
|
||||
// is the internal cuid — not the friendlyId. SyntheticRun.id is
|
||||
// already the cuid (RunId.fromFriendlyId(entry.runId) in
|
||||
// readFallback.server.ts) so the admin debug tooltip on the run
|
||||
// detail page shows the same format for buffered + materialised
|
||||
// runs.
|
||||
id: run.id,
|
||||
number: 1,
|
||||
friendlyId: run.friendlyId,
|
||||
traceId: run.traceId ?? "",
|
||||
spanId: run.spanId ?? "",
|
||||
status: isCancelled
|
||||
? ("CANCELED" as const)
|
||||
: isFailed
|
||||
? ("SYSTEM_FAILURE" as const)
|
||||
: ("PENDING" as const),
|
||||
isFinished: isCancelled || isFailed,
|
||||
startedAt: null,
|
||||
// Symmetric with `buildSyntheticSpanRun` and the
|
||||
// `ApiRetrieveRunPresenter` synth path. The run-detail route
|
||||
// derives `isCompleted` from `completedAt !== null` and gates SSE
|
||||
// live-reloading on it (`route.tsx:459`, `:551`); leaving
|
||||
// `completedAt` null for FAILED would keep a terminal buffered run
|
||||
// live-reloading forever. PG-resident SYSTEM_FAILURE rows always
|
||||
// have completedAt set, so fall back to createdAt (the buffer
|
||||
// entry has no separate failedAt — closest proxy for when the
|
||||
// terminal state landed).
|
||||
completedAt: run.cancelledAt ?? (isFailed ? run.createdAt : null),
|
||||
logsDeletedAt: null,
|
||||
rootTaskRun: null,
|
||||
parentTaskRun: null,
|
||||
environment: {
|
||||
id: environment.id,
|
||||
organizationId: environment.organizationId,
|
||||
type: environment.type,
|
||||
slug: environment.slug,
|
||||
userId: undefined,
|
||||
userName: undefined,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -189,6 +189,7 @@ export async function buildSyntheticSpanRun(args: {
|
||||
traceId: run.traceId ?? "",
|
||||
spanId: run.spanId ?? "",
|
||||
isCached: false,
|
||||
isBuffered: true,
|
||||
machinePreset: narrowMachinePreset(run.machinePreset),
|
||||
taskEventStore: "taskEvent",
|
||||
externalTraceId: undefined,
|
||||
|
||||
@@ -19,6 +19,22 @@ vi.mock("~/v3/services/alerts/performTaskRunAlerts.server", () => ({
|
||||
},
|
||||
}));
|
||||
|
||||
// The drainer calls `recordRunDebugLog` after a successful engine.trigger
|
||||
// to emit an admin-only LOG-kind event encoding the buffered window.
|
||||
// The real implementation imports the configured event repository (prisma
|
||||
// + clickhouse + env), which has heavy side-effects on first import.
|
||||
// Stub it to a vi.fn so the unit tests can assert call shape without
|
||||
// dragging the whole eventRepository graph into webapp test setup.
|
||||
// `vi.hoisted` is required because `vi.mock` factories are hoisted above
|
||||
// regular `const`s — referencing a top-level variable from inside the
|
||||
// factory otherwise fires `Cannot access 'X' before initialization`.
|
||||
const { recordRunDebugLogMock } = vi.hoisted(() => ({
|
||||
recordRunDebugLogMock: vi.fn(async () => ({ success: true as const })),
|
||||
}));
|
||||
vi.mock("~/v3/eventRepository/index.server", () => ({
|
||||
recordRunDebugLog: recordRunDebugLogMock,
|
||||
}));
|
||||
|
||||
import {
|
||||
createDrainerHandler,
|
||||
isRetryablePgError,
|
||||
@@ -450,4 +466,109 @@ describe("createDrainerHandler", () => {
|
||||
).rejects.toThrow("engine rejected the snapshot");
|
||||
expect(createFailedTaskRun).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("emits an admin-only LOG-kind event with the buffered window after engine.trigger succeeds", async () => {
|
||||
// The drainer's audit trail rides the existing TaskEventKind.LOG
|
||||
// filter pattern (`eventRepository.server.ts:108` + `logs.download.ts:118`)
|
||||
// — admins see the buffered window in the trace; non-admins don't.
|
||||
recordRunDebugLogMock.mockClear();
|
||||
const trigger = vi.fn(async () => ({ friendlyId: "run_z" }));
|
||||
const handler = createDrainerHandler({
|
||||
engine: { trigger } as any,
|
||||
prisma: {} as any,
|
||||
});
|
||||
|
||||
const bufferedAt = new Date(Date.now() - 4_000);
|
||||
await handler({
|
||||
runId: "run_z",
|
||||
envId: "env_a",
|
||||
orgId: "org_1",
|
||||
payload: { taskIdentifier: "t", spanId: "snapspan", traceId: "snaptrace" },
|
||||
attempts: 2,
|
||||
createdAt: bufferedAt,
|
||||
} as any);
|
||||
|
||||
expect(recordRunDebugLogMock).toHaveBeenCalledOnce();
|
||||
const [callRunId, message, options] = recordRunDebugLogMock.mock.calls[0] as [
|
||||
string,
|
||||
string,
|
||||
any,
|
||||
];
|
||||
// Internal cuid derived from the friendlyId, mirroring what
|
||||
// `findRunForEventCreation` queries on.
|
||||
expect(callRunId).toBe("z");
|
||||
expect(message).toMatch(/Mollifier buffered \d+ms before materialising/);
|
||||
// Emitted as a marker at materialisation time (no `startTime` /
|
||||
// `duration` overrides) — engine.trigger has just rewritten the
|
||||
// root span's start_time to "now", so back-dating the event would
|
||||
// clip it off-screen in the trace renderer. The historical window
|
||||
// is preserved in metadata so admins can still read it.
|
||||
expect(options.startTime).toBeUndefined();
|
||||
expect(options.duration).toBeUndefined();
|
||||
expect(options.parentId).toBe("snapspan");
|
||||
expect(options.attributes.metadata["mollifier.bufferedAt"]).toBe(bufferedAt.toISOString());
|
||||
expect(options.attributes.metadata["mollifier.attempts"]).toBe(2);
|
||||
expect(options.attributes.metadata["mollifier.dwellMs"]).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("does NOT emit the admin LOG event when engine.trigger fails non-retryably", async () => {
|
||||
// The audit trail is for runs that actually materialised. On a
|
||||
// terminal SYSTEM_FAILURE path the customer-visible outcome is the
|
||||
// failure row; emitting a "buffered for Xms" event next to it would
|
||||
// imply the buffered window completed normally.
|
||||
recordRunDebugLogMock.mockClear();
|
||||
const trigger = vi.fn(async () => {
|
||||
throw new Error("engine rejected the snapshot");
|
||||
});
|
||||
const createFailedTaskRun = vi.fn(async () => ({ id: "internal" }));
|
||||
const handler = createDrainerHandler({
|
||||
engine: { trigger, createFailedTaskRun } as any,
|
||||
prisma: {} as any,
|
||||
});
|
||||
|
||||
await handler({
|
||||
runId: "run_z",
|
||||
envId: "env_a",
|
||||
orgId: "org_1",
|
||||
payload: { taskIdentifier: "t", environment: envFixture },
|
||||
attempts: 0,
|
||||
createdAt: new Date(),
|
||||
} as any);
|
||||
|
||||
expect(recordRunDebugLogMock).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does NOT emit the admin LOG event on the cancel-bifurcation path", async () => {
|
||||
// Cancel-bifurcation writes a CANCELED row directly without calling
|
||||
// engine.trigger. There's no buffered-then-materialised window to
|
||||
// describe — the run never ran.
|
||||
recordRunDebugLogMock.mockClear();
|
||||
const friendlyId = RunId.generate().friendlyId;
|
||||
const createCancelledRun = vi.fn(async () => ({
|
||||
id: "internal",
|
||||
friendlyId,
|
||||
status: "CANCELED",
|
||||
}));
|
||||
const handler = createDrainerHandler({
|
||||
engine: { createCancelledRun } as any,
|
||||
prisma: {} as any,
|
||||
});
|
||||
|
||||
await handler({
|
||||
runId: friendlyId,
|
||||
envId: "env_a",
|
||||
orgId: "org_1",
|
||||
payload: {
|
||||
friendlyId,
|
||||
taskIdentifier: "t",
|
||||
environment: envFixture,
|
||||
cancelledAt: new Date().toISOString(),
|
||||
cancelReason: "Canceled by user",
|
||||
},
|
||||
attempts: 0,
|
||||
createdAt: new Date(),
|
||||
} as any);
|
||||
|
||||
expect(recordRunDebugLogMock).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
serialiseMollifierSnapshot,
|
||||
deserialiseMollifierSnapshot,
|
||||
} from "~/v3/mollifier/mollifierSnapshot.server";
|
||||
import { prettyPrintPacket } from "@trigger.dev/core/v3";
|
||||
|
||||
// Regression test for the Devin "Buffered replay loader passes
|
||||
// non-string payload to prettyPrintPacket" finding on PR #3757.
|
||||
//
|
||||
// Devin's claim is that the snapshot codec double-unwraps the
|
||||
// payload: `engine.trigger` carries it pre-serialised, then the
|
||||
// snapshot serialise/deserialise round-trip would JSON.parse it a
|
||||
// second time, leaving `buffered.payload` as a *parsed* object —
|
||||
// which `prettyPrintPacket` then mis-handles, producing malformed
|
||||
// payload display in the Replay dialog.
|
||||
//
|
||||
// This test pins the actual contract: the snapshot codec is a single
|
||||
// JSON.stringify / JSON.parse layer. The payload field stored on the
|
||||
// engine trigger input is a string (the SDK-serialised payload from
|
||||
// `payloadPacket.data`). A string round-trips through
|
||||
// JSON.stringify/JSON.parse unchanged — it does NOT get a second
|
||||
// unwrap. Therefore `buffered.payload` reaches the replay loader as
|
||||
// a string, exactly the shape `prettyPrintPacket` expects.
|
||||
describe("mollifier replay payload shape", () => {
|
||||
it("serialise/deserialise preserves the payload as a string", () => {
|
||||
// Shape mirrors what `triggerTask.server.ts:#buildEngineTriggerInput`
|
||||
// produces — `payload` is `args.payloadPacket.data`, already a JSON
|
||||
// string from the SDK's packet serialisation.
|
||||
const triggerInput = {
|
||||
friendlyId: "run_x",
|
||||
taskIdentifier: "hello-world",
|
||||
payload: JSON.stringify({ hello: "world", n: 42 }),
|
||||
payloadType: "application/json",
|
||||
traceId: "trace_x",
|
||||
spanId: "span_x",
|
||||
};
|
||||
|
||||
const serialised = serialiseMollifierSnapshot(triggerInput);
|
||||
const roundTripped = deserialiseMollifierSnapshot(serialised);
|
||||
|
||||
expect(typeof roundTripped.payload).toBe("string");
|
||||
expect(roundTripped.payload).toBe(triggerInput.payload);
|
||||
expect(roundTripped.payloadType).toBe("application/json");
|
||||
});
|
||||
|
||||
it("prettyPrintPacket on the round-tripped payload produces the expected pretty JSON", async () => {
|
||||
const original = { hello: "world", nested: { count: 3 } };
|
||||
const triggerInput = {
|
||||
payload: JSON.stringify(original),
|
||||
payloadType: "application/json",
|
||||
};
|
||||
|
||||
const roundTripped = deserialiseMollifierSnapshot(
|
||||
serialiseMollifierSnapshot(triggerInput),
|
||||
);
|
||||
|
||||
// This is exactly the call the replay loader makes:
|
||||
// prettyPrintPacket(run.payload, run.payloadType)
|
||||
// If Devin were right, the payload here would be a parsed object
|
||||
// and prettyPrintPacket would either double-encode or skip
|
||||
// formatting. In reality it's a string, so we get correct pretty
|
||||
// JSON.
|
||||
const pretty = await prettyPrintPacket(
|
||||
roundTripped.payload,
|
||||
roundTripped.payloadType as string,
|
||||
);
|
||||
|
||||
expect(pretty).toBe(JSON.stringify(original, null, 2));
|
||||
});
|
||||
|
||||
it("string payload survives the buffer-codec round-trip even with snapshot fields around it", () => {
|
||||
// Replicate the realistic snapshot shape (the engine.trigger input
|
||||
// has many sibling fields). Confirms there's no field-shape
|
||||
// interaction that would mutate payload.
|
||||
const triggerInput = {
|
||||
friendlyId: "run_x",
|
||||
environment: {
|
||||
id: "env",
|
||||
type: "DEVELOPMENT",
|
||||
project: { id: "p" },
|
||||
organization: { id: "o" },
|
||||
},
|
||||
taskIdentifier: "t",
|
||||
payload: '{"a":1}',
|
||||
payloadType: "application/json",
|
||||
context: { run: { id: "x" } },
|
||||
traceContext: { traceparent: "00-...-..." },
|
||||
traceId: "abc",
|
||||
spanId: "def",
|
||||
tags: ["one", "two"],
|
||||
depth: 2,
|
||||
isTest: false,
|
||||
};
|
||||
const out = deserialiseMollifierSnapshot(serialiseMollifierSnapshot(triggerInput));
|
||||
expect(typeof out.payload).toBe("string");
|
||||
expect(out.payload).toBe('{"a":1}');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,106 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("~/db.server", () => ({ prisma: {}, $replica: {} }));
|
||||
|
||||
import { buildSyntheticReplayTaskRun } from "~/v3/mollifier/syntheticReplayTaskRun.server";
|
||||
import type { SyntheticRun } from "~/v3/mollifier/readFallback.server";
|
||||
|
||||
const NOW = new Date("2026-05-21T10:00:00Z");
|
||||
|
||||
function makeSyntheticRun(overrides: Partial<SyntheticRun> = {}): SyntheticRun {
|
||||
return {
|
||||
id: "run_internal_1",
|
||||
friendlyId: "run_friendly_1",
|
||||
status: "QUEUED",
|
||||
cancelledAt: undefined,
|
||||
cancelReason: undefined,
|
||||
delayUntil: undefined,
|
||||
taskIdentifier: "hello-world",
|
||||
createdAt: NOW,
|
||||
payload: { message: "hi" },
|
||||
payloadType: "application/json",
|
||||
metadata: undefined,
|
||||
metadataType: undefined,
|
||||
seedMetadata: undefined,
|
||||
seedMetadataType: undefined,
|
||||
idempotencyKey: undefined,
|
||||
idempotencyKeyOptions: undefined,
|
||||
isTest: false,
|
||||
depth: 0,
|
||||
ttl: "10m",
|
||||
tags: [],
|
||||
runTags: [],
|
||||
lockedToVersion: undefined,
|
||||
resumeParentOnCompletion: false,
|
||||
parentTaskRunId: undefined,
|
||||
traceId: "trace_1",
|
||||
spanId: "span_1",
|
||||
parentSpanId: undefined,
|
||||
runtimeEnvironmentId: "env_a",
|
||||
engine: "V2",
|
||||
workerQueue: "worker-queue-1",
|
||||
queue: "task/hello-world",
|
||||
concurrencyKey: undefined,
|
||||
machinePreset: "small-1x",
|
||||
realtimeStreamsVersion: "v1",
|
||||
maxAttempts: 3,
|
||||
maxDurationInSeconds: 3600,
|
||||
replayedFromTaskRunFriendlyId: undefined,
|
||||
annotations: undefined,
|
||||
traceContext: undefined,
|
||||
scheduleId: undefined,
|
||||
batchId: undefined,
|
||||
parentTaskRunFriendlyId: undefined,
|
||||
rootTaskRunFriendlyId: undefined,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
const ENV_ROW = {
|
||||
slug: "dev",
|
||||
project: { slug: "hello-world", organization: { slug: "references" } },
|
||||
};
|
||||
|
||||
describe("buildSyntheticReplayTaskRun", () => {
|
||||
it("returns the adapted TaskRun shape when traceId and spanId are present", () => {
|
||||
const taskRun = buildSyntheticReplayTaskRun({
|
||||
synthetic: makeSyntheticRun(),
|
||||
envRow: ENV_ROW,
|
||||
});
|
||||
expect(taskRun).not.toBeNull();
|
||||
expect(taskRun!.traceId).toBe("trace_1");
|
||||
expect(taskRun!.spanId).toBe("span_1");
|
||||
expect(taskRun!.project.slug).toBe("hello-world");
|
||||
expect(taskRun!.project.organization.slug).toBe("references");
|
||||
expect(taskRun!.runtimeEnvironment.slug).toBe("dev");
|
||||
});
|
||||
|
||||
it("returns null when the snapshot has no traceId", () => {
|
||||
// ReplayTaskRunService builds `00-${traceId}-${spanId}-01` without
|
||||
// guarding for undefined. Falling through with a missing traceId
|
||||
// would emit `00-undefined-...-01`, an invalid W3C traceparent that
|
||||
// OTel silently drops, breaking the replayed run's trace linkage to
|
||||
// the original. The helper must refuse rather than degrade silently.
|
||||
const taskRun = buildSyntheticReplayTaskRun({
|
||||
synthetic: makeSyntheticRun({ traceId: undefined }),
|
||||
envRow: ENV_ROW,
|
||||
});
|
||||
expect(taskRun).toBeNull();
|
||||
});
|
||||
|
||||
it("returns null when the snapshot has no spanId", () => {
|
||||
const taskRun = buildSyntheticReplayTaskRun({
|
||||
synthetic: makeSyntheticRun({ spanId: undefined }),
|
||||
envRow: ENV_ROW,
|
||||
});
|
||||
expect(taskRun).toBeNull();
|
||||
});
|
||||
|
||||
it("returns null when both traceId and spanId are missing", () => {
|
||||
const taskRun = buildSyntheticReplayTaskRun({
|
||||
synthetic: makeSyntheticRun({ traceId: undefined, spanId: undefined }),
|
||||
envRow: ENV_ROW,
|
||||
});
|
||||
expect(taskRun).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,130 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("~/db.server", () => ({ prisma: {}, $replica: {} }));
|
||||
|
||||
import { buildSyntheticRunHeader } from "~/v3/mollifier/syntheticRunHeader.server";
|
||||
import type { SyntheticRun } from "~/v3/mollifier/readFallback.server";
|
||||
|
||||
const NOW = new Date("2026-05-21T10:00:00Z");
|
||||
const CANCELLED_AT = new Date("2026-05-21T10:00:30Z");
|
||||
|
||||
function makeSyntheticRun(overrides: Partial<SyntheticRun> = {}): SyntheticRun {
|
||||
return {
|
||||
id: "run_internal_1",
|
||||
friendlyId: "run_friendly_1",
|
||||
status: "QUEUED",
|
||||
cancelledAt: undefined,
|
||||
cancelReason: undefined,
|
||||
delayUntil: undefined,
|
||||
taskIdentifier: "hello-world",
|
||||
createdAt: NOW,
|
||||
payload: { message: "hi" },
|
||||
payloadType: "application/json",
|
||||
metadata: undefined,
|
||||
metadataType: undefined,
|
||||
seedMetadata: undefined,
|
||||
seedMetadataType: undefined,
|
||||
idempotencyKey: undefined,
|
||||
idempotencyKeyOptions: undefined,
|
||||
isTest: false,
|
||||
depth: 0,
|
||||
ttl: "10m",
|
||||
tags: [],
|
||||
runTags: [],
|
||||
lockedToVersion: undefined,
|
||||
resumeParentOnCompletion: false,
|
||||
parentTaskRunId: undefined,
|
||||
traceId: "trace_1",
|
||||
spanId: "span_1",
|
||||
parentSpanId: undefined,
|
||||
runtimeEnvironmentId: "env_a",
|
||||
engine: "V2",
|
||||
workerQueue: "worker-queue-1",
|
||||
queue: "task/hello-world",
|
||||
concurrencyKey: undefined,
|
||||
machinePreset: "small-1x",
|
||||
realtimeStreamsVersion: "v1",
|
||||
maxAttempts: 3,
|
||||
maxDurationInSeconds: 3600,
|
||||
replayedFromTaskRunFriendlyId: undefined,
|
||||
annotations: undefined,
|
||||
traceContext: undefined,
|
||||
scheduleId: undefined,
|
||||
batchId: undefined,
|
||||
parentTaskRunFriendlyId: undefined,
|
||||
rootTaskRunFriendlyId: undefined,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
const ENV = {
|
||||
id: "env_a",
|
||||
organizationId: "org_a",
|
||||
type: "DEVELOPMENT" as const,
|
||||
slug: "dev",
|
||||
};
|
||||
|
||||
describe("buildSyntheticRunHeader", () => {
|
||||
it("returns PENDING / non-final state for a queued buffered run", () => {
|
||||
const header = buildSyntheticRunHeader({ run: makeSyntheticRun(), environment: ENV });
|
||||
expect(header.status).toBe("PENDING");
|
||||
expect(header.isFinished).toBe(false);
|
||||
expect(header.completedAt).toBeNull();
|
||||
});
|
||||
|
||||
it("reflects CANCELED state from the snapshot so the NavBar and Cancel-button gate update before the drainer materialises", () => {
|
||||
const header = buildSyntheticRunHeader({
|
||||
run: makeSyntheticRun({ status: "CANCELED", cancelledAt: CANCELLED_AT }),
|
||||
environment: ENV,
|
||||
});
|
||||
// The Cancel button in route.tsx is gated on `!run.isFinished` and the
|
||||
// status badge reads `run.status`. Both must flip on buffered-cancel
|
||||
// or the user sees a "Pending" badge with a Cancel button on a run
|
||||
// that's already cancelled in the snapshot.
|
||||
expect(header.status).toBe("CANCELED");
|
||||
expect(header.isFinished).toBe(true);
|
||||
expect(header.completedAt).toEqual(CANCELLED_AT);
|
||||
});
|
||||
|
||||
it("populates completedAt for FAILED runs so the route stops live-reloading and renders as completed", () => {
|
||||
// The run-detail route derives `isCompleted` from
|
||||
// `run.completedAt !== null` and gates SSE live-reloading on it
|
||||
// (`route.tsx:459`, `:551`). Leaving completedAt null for FAILED
|
||||
// buffered runs would keep a terminal run live-reloading forever
|
||||
// while the badge already says SYSTEM_FAILURE. Symmetric with
|
||||
// buildSyntheticSpanRun + ApiRetrieveRunPresenter.
|
||||
const header = buildSyntheticRunHeader({
|
||||
run: makeSyntheticRun({ status: "FAILED" }),
|
||||
environment: ENV,
|
||||
});
|
||||
expect(header.status).toBe("SYSTEM_FAILURE");
|
||||
expect(header.isFinished).toBe(true);
|
||||
expect(header.completedAt).toEqual(NOW);
|
||||
});
|
||||
|
||||
it("forwards identity and environment fields from the snapshot", () => {
|
||||
const header = buildSyntheticRunHeader({ run: makeSyntheticRun(), environment: ENV });
|
||||
expect(header.friendlyId).toBe("run_friendly_1");
|
||||
// `id` mirrors RunPresenter.getRun (the PG path) which puts the
|
||||
// internal cuid in this field. SyntheticRun.id is the cuid; the
|
||||
// header must surface it (not the friendlyId).
|
||||
expect(header.id).toBe("run_internal_1");
|
||||
expect(header.traceId).toBe("trace_1");
|
||||
expect(header.spanId).toBe("span_1");
|
||||
expect(header.environment).toMatchObject({
|
||||
id: "env_a",
|
||||
organizationId: "org_a",
|
||||
type: "DEVELOPMENT",
|
||||
slug: "dev",
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back to empty strings when the snapshot has no trace/span ids", () => {
|
||||
const header = buildSyntheticRunHeader({
|
||||
run: makeSyntheticRun({ traceId: undefined, spanId: undefined }),
|
||||
environment: ENV,
|
||||
});
|
||||
expect(header.traceId).toBe("");
|
||||
expect(header.spanId).toBe("");
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user