Compare commits

...

8 Commits

Author SHA1 Message Date
Eric Allam 4a68e71583 Fix pnpm lock file 2024-04-19 19:20:01 +01:00
github-actions[bot] b657eb6555 chore: Update version for release (beta) (#1044)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-19 19:19:11 +01:00
Eric Allam 62c9a5b712 Fix restoring after waiting for task/batch 2024-04-19 18:59:02 +01:00
Matt Aitken f339b41ef3 Added some basic Defer migration details for CRON 2024-04-19 18:30:58 +01:00
Eric Allam ae40ce3995 Fix the management file 2024-04-19 16:14:57 +01:00
Eric Allam 374edef020 Updates the trigger, batchTrigger and their *AndWait variants to use the first parameter for the payload/items, and the second parameter for options (#1045)
Also always returns a `TaskRunResult` object from `triggerAndWait` instead of rethrowing subtask errors in the parent
2024-04-19 14:51:51 +01:00
Eric Allam b82db67b81 Add additional logging around cleaning up dev workers, and always kill them after 5 seconds if they haven't already exited 2024-04-19 11:14:20 +01:00
Eric Allam 26093896d2 v3: Fixes for using (batch)triggerAndWait with idempotency keys (#1043)
* Fixes various issues with triggerAndWait and batchTriggerAndWait

When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)

- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A run’s idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task

* Remove the default queue concurrency limit as we now have env and org concurrency limits

* Use the run friendlyId in the completion result id

* Added some error logging
2024-04-19 10:54:43 +01:00
131 changed files with 1648 additions and 561 deletions
+4
View File
@@ -71,7 +71,10 @@
"rare-roses-float",
"real-planets-stare",
"rotten-dryers-exercise",
"shaggy-spoons-taste",
"sharp-emus-compare",
"sharp-zebras-serve",
"shiny-coats-cry",
"silly-suits-switch",
"smart-needles-move",
"smart-olives-eat",
@@ -85,6 +88,7 @@
"tame-guests-know",
"tender-oranges-rhyme",
"tidy-balloons-suffer",
"tidy-dryers-sleep",
"tiny-doors-type",
"tiny-elephants-scream",
"tricky-bulldogs-heal"
+56
View File
@@ -0,0 +1,56 @@
---
"@trigger.dev/sdk": patch
"@trigger.dev/core": patch
---
Updates the `trigger`, `batchTrigger` and their `*AndWait` variants to use the first parameter for the payload/items, and the second parameter for options.
Before:
```ts
await yourTask.trigger({ payload: { foo: "bar" }, options: { idempotencyKey: "key_1234" } });
await yourTask.triggerAndWait({ payload: { foo: "bar" }, options: { idempotencyKey: "key_1234" } });
await yourTask.batchTrigger({ items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }] });
await yourTask.batchTriggerAndWait({ items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }] });
```
After:
```ts
await yourTask.trigger({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.triggerAndWait({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.batchTrigger([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
await yourTask.batchTriggerAndWait([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
```
We've also changed the API of the `triggerAndWait` result. Before, if the subtask that was triggered finished with an error, we would automatically "rethrow" the error in the parent task.
Now instead we're returning a `TaskRunResult` object that allows you to discriminate between successful and failed runs in the subtask:
Before:
```ts
try {
const result = await yourTask.triggerAndWait({ foo: "bar" });
// result is the output of your task
console.log("result", result);
} catch (error) {
// handle subtask errors here
}
```
After:
```ts
const result = await yourTask.triggerAndWait({ foo: "bar" });
if (result.ok) {
console.log(`Run ${result.id} succeeded with output`, result.output);
} else {
console.log(`Run ${result.id} failed with error`, result.error);
}
```
+13
View File
@@ -0,0 +1,13 @@
---
"@trigger.dev/sdk": patch
"trigger.dev": patch
"@trigger.dev/core": patch
---
When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A runs idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
+5
View File
@@ -0,0 +1,5 @@
---
"trigger.dev": patch
---
Add additional logging around cleaning up dev workers, and always kill them after 5 seconds if they haven't already exited
+10
View File
@@ -0,0 +1,10 @@
---
"trigger.dev": patch
"@trigger.dev/core": patch
---
Fixes an issue that caused failed tasks when resuming after calling `triggerAndWait` or `batchTriggerAndWait` in prod/staging (this doesn't effect dev).
The version of Node.js we use for deployed workers (latest 20) would crash with an out-of-memory error when the checkpoint was restored. This crash does not happen on Node 18x or Node21x, so we've decided to upgrade the worker version to Node.js21x, to mitigate this issue.
You'll need to re-deploy to production to fix the issue.
@@ -32,6 +32,7 @@ import {
import { TimeFrameFilter } from "./TimeFrameFilter";
import { Button } from "../primitives/Buttons";
import { useCallback } from "react";
import assertNever from "assert-never";
export function RunsFilters() {
const navigate = useNavigate();
@@ -182,8 +183,7 @@ export function FilterStatusIcon({
case "FAILED":
return <XCircleIcon className={cn(filterStatusClassNameColor(status), className)} />;
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -205,8 +205,7 @@ export function filterStatusTitle(status: FilterableStatus): string {
case "TIMEDOUT":
return "Timed out";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -228,8 +227,7 @@ export function filterStatusClassNameColor(status: FilterableStatus): string {
case "TIMEDOUT":
return "text-amber-300";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -11,6 +11,7 @@ import type { JobRunStatus } from "@trigger.dev/database";
import { cn } from "~/utils/cn";
import { Spinner } from "../primitives/Spinner";
import { z } from "zod";
import assertNever from "assert-never";
export function RunStatus({ status }: { status: JobRunStatus }) {
return (
@@ -51,8 +52,7 @@ export function RunStatusIcon({ status, className }: { status: JobRunStatus; cla
case "CANCELED":
return <NoSymbolIcon className={cn(runStatusClassNameColor(status), className)} />;
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -89,8 +89,7 @@ export function runStatusTitle(status: JobRunStatus): string {
case "INVALID_PAYLOAD":
return "Invalid payload";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -123,8 +122,7 @@ export function runStatusClassNameColor(status: JobRunStatus): string {
case "CANCELED":
return "text-charcoal-500";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -5,6 +5,7 @@ import {
XCircleIcon,
} from "@heroicons/react/20/solid";
import { WorkerDeploymentStatus } from "@trigger.dev/database";
import assertNever from "assert-never";
import { Spinner } from "~/components/primitives/Spinner";
import { cn } from "~/utils/cn";
@@ -54,8 +55,7 @@ export function DeploymentStatusIcon({
/>
);
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -74,8 +74,7 @@ export function deploymentStatusClassNameColor(status: WorkerDeploymentStatus):
case "FAILED":
return "text-error";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -97,8 +96,7 @@ export function deploymentStatusTitle(status: WorkerDeploymentStatus): string {
case "FAILED":
return "Failed";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -8,6 +8,7 @@ import {
} from "@heroicons/react/20/solid";
import type { TaskRunAttemptStatus as TaskRunAttemptStatusType } from "@trigger.dev/database";
import { TaskRunAttemptStatus } from "@trigger.dev/database";
import assertNever from "assert-never";
import { SnowflakeIcon } from "lucide-react";
import { Spinner } from "~/components/primitives/Spinner";
import { cn } from "~/utils/cn";
@@ -72,8 +73,7 @@ export function TaskRunAttemptStatusIcon({
case "COMPLETED":
return <CheckCircleIcon className={cn(runAttemptStatusClassNameColor(status), className)} />;
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -99,8 +99,7 @@ export function runAttemptStatusClassNameColor(status: ExtendedTaskAttemptStatus
case "COMPLETED":
return "text-success";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -126,8 +125,7 @@ export function runAttemptStatusTitle(status: ExtendedTaskAttemptStatus | null):
case "COMPLETED":
return "Completed";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -10,6 +10,7 @@ import {
XCircleIcon,
} from "@heroicons/react/20/solid";
import { TaskRunStatus } from "@trigger.dev/database";
import assertNever from "assert-never";
import { SnowflakeIcon } from "lucide-react";
import { Spinner } from "~/components/primitives/Spinner";
import { cn } from "~/utils/cn";
@@ -88,8 +89,7 @@ export function TaskRunStatusIcon({
return <FireIcon className={cn(runStatusClassNameColor(status), className)} />;
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -120,8 +120,7 @@ export function runStatusClassNameColor(status: TaskRunStatus): string {
case "CRASHED":
return "text-error";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
@@ -153,8 +152,7 @@ export function runStatusTitle(status: TaskRunStatus): string {
case "CRASHED":
return "Crashed";
default: {
const _exhaustiveCheck: never = status;
throw new Error(`Non-exhaustive match for value: ${status}`);
assertNever(status);
}
}
}
-1
View File
@@ -76,7 +76,6 @@ const EnvironmentSchema = z.object({
REDIS_PASSWORD: z.string().optional(),
REDIS_TLS_DISABLED: z.string().optional(),
DEFAULT_QUEUE_EXECUTION_CONCURRENCY_LIMIT: z.coerce.number().int().default(5),
DEFAULT_ENV_EXECUTION_CONCURRENCY_LIMIT: z.coerce.number().int().default(10),
DEFAULT_ORG_EXECUTION_CONCURRENCY_LIMIT: z.coerce.number().int().default(10),
DEFAULT_DEV_ENV_EXECUTION_ATTEMPTS: z.coerce.number().int().positive().default(1),
+128
View File
@@ -0,0 +1,128 @@
import {
TaskRunError,
TaskRunExecutionResult,
TaskRunFailedExecutionResult,
TaskRunSuccessfulExecutionResult,
} from "@trigger.dev/core/v3";
import {
BatchTaskRunItemStatus,
TaskRun,
TaskRunAttempt,
TaskRunAttemptStatus,
TaskRunStatus,
} from "@trigger.dev/database";
import { assertNever } from "assert-never";
import { logger } from "~/services/logger.server";
const SUCCESSFUL_STATUSES = [TaskRunStatus.COMPLETED_SUCCESSFULLY];
const FAILURE_STATUSES = [
TaskRunStatus.CANCELED,
TaskRunStatus.INTERRUPTED,
TaskRunStatus.COMPLETED_WITH_ERRORS,
TaskRunStatus.SYSTEM_FAILURE,
TaskRunStatus.CRASHED,
];
export type TaskRunWithAttempts = TaskRun & {
attempts: TaskRunAttempt[];
};
export function executionResultForTaskRun(
taskRun: TaskRunWithAttempts
): TaskRunExecutionResult | undefined {
if (SUCCESSFUL_STATUSES.includes(taskRun.status)) {
// find the last attempt that was successful
const attempt = taskRun.attempts.find((a) => a.status === TaskRunAttemptStatus.COMPLETED);
if (!attempt) {
logger.error("Task run is successful but no successful attempt found", {
taskRunId: taskRun.id,
taskRunStatus: taskRun.status,
taskRunAttempts: taskRun.attempts.map((a) => a.status),
});
return undefined;
}
return {
ok: true,
id: taskRun.friendlyId,
output: attempt.output ?? undefined,
outputType: attempt.outputType,
} satisfies TaskRunSuccessfulExecutionResult;
}
if (FAILURE_STATUSES.includes(taskRun.status)) {
if (taskRun.status === TaskRunStatus.CANCELED) {
return {
ok: false,
id: taskRun.friendlyId,
error: {
type: "INTERNAL_ERROR",
code: "TASK_RUN_CANCELLED",
},
} satisfies TaskRunFailedExecutionResult;
}
const attempt = taskRun.attempts.find((a) => a.status === TaskRunAttemptStatus.FAILED);
if (!attempt) {
logger.error("Task run is failed but no failed attempt found", {
taskRunId: taskRun.id,
taskRunStatus: taskRun.status,
taskRunAttempts: taskRun.attempts.map((a) => a.status),
});
return undefined;
}
const error = TaskRunError.safeParse(attempt.error);
if (!error.success) {
logger.error("Failed to parse error from failed task run attempt", {
taskRunId: taskRun.id,
taskRunStatus: taskRun.status,
taskRunAttempts: taskRun.attempts.map((a) => a.status),
error: attempt.error,
});
return {
ok: false,
id: taskRun.friendlyId,
error: {
type: "INTERNAL_ERROR",
code: "CONFIGURED_INCORRECTLY",
},
} satisfies TaskRunFailedExecutionResult;
}
return {
ok: false,
id: taskRun.friendlyId,
error: error.data,
} satisfies TaskRunFailedExecutionResult;
}
}
export function batchTaskRunItemStatusForRunStatus(status: TaskRunStatus): BatchTaskRunItemStatus {
switch (status) {
case TaskRunStatus.COMPLETED_SUCCESSFULLY:
return BatchTaskRunItemStatus.COMPLETED;
case TaskRunStatus.CANCELED:
case TaskRunStatus.INTERRUPTED:
case TaskRunStatus.COMPLETED_WITH_ERRORS:
case TaskRunStatus.SYSTEM_FAILURE:
case TaskRunStatus.CRASHED:
case TaskRunStatus.COMPLETED_WITH_ERRORS:
return BatchTaskRunItemStatus.FAILED;
case TaskRunStatus.PENDING:
case TaskRunStatus.WAITING_FOR_DEPLOY:
case TaskRunStatus.WAITING_TO_RESUME:
case TaskRunStatus.RETRYING_AFTER_FAILURE:
case TaskRunStatus.EXECUTING:
case TaskRunStatus.PAUSED:
return BatchTaskRunItemStatus.PENDING;
default:
assertNever(status);
}
}
@@ -0,0 +1,46 @@
import { BatchTaskRunExecutionResult } from "@trigger.dev/core/v3";
import { executionResultForTaskRun } from "~/models/taskRun.server";
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
import { BasePresenter } from "./basePresenter.server";
export class ApiBatchResultsPresenter extends BasePresenter {
public async call(
friendlyId: string,
env: AuthenticatedEnvironment
): Promise<BatchTaskRunExecutionResult | undefined> {
return this.traceWithEnv("call", env, async (span) => {
const batchRun = await this._prisma.batchTaskRun.findUnique({
where: {
friendlyId,
runtimeEnvironmentId: env.id,
},
include: {
items: {
include: {
taskRun: {
include: {
attempts: {
orderBy: {
createdAt: "desc",
},
},
},
},
},
},
},
});
if (!batchRun) {
return undefined;
}
return {
id: batchRun.friendlyId,
items: batchRun.items
.map((item) => executionResultForTaskRun(item.taskRun))
.filter(Boolean),
};
});
}
}
@@ -0,0 +1,33 @@
import { TaskRunExecutionResult } from "@trigger.dev/core/v3";
import { executionResultForTaskRun } from "~/models/taskRun.server";
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
import { BasePresenter } from "./basePresenter.server";
export class ApiRunResultPresenter extends BasePresenter {
public async call(
friendlyId: string,
env: AuthenticatedEnvironment
): Promise<TaskRunExecutionResult | undefined> {
return this.traceWithEnv("call", env, async (span) => {
const taskRun = await this._prisma.taskRun.findUnique({
where: {
friendlyId,
runtimeEnvironmentId: env.id,
},
include: {
attempts: {
orderBy: {
createdAt: "desc",
},
},
},
});
if (!taskRun) {
return undefined;
}
return executionResultForTaskRun(taskRun);
});
}
}
@@ -0,0 +1,34 @@
import { Span, SpanKind } from "@opentelemetry/api";
import { PrismaClientOrTransaction, prisma } from "~/db.server";
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
import { attributesFromAuthenticatedEnv, tracer } from "../../v3/tracer.server";
export abstract class BasePresenter {
constructor(protected readonly _prisma: PrismaClientOrTransaction = prisma) {}
protected async traceWithEnv<T>(
trace: string,
env: AuthenticatedEnvironment,
fn: (span: Span) => Promise<T>
): Promise<T> {
return tracer.startActiveSpan(
`${this.constructor.name}.${trace}`,
{ attributes: attributesFromAuthenticatedEnv(env), kind: SpanKind.SERVER },
async (span) => {
try {
return await fn(span);
} catch (e) {
if (e instanceof Error) {
span.recordException(e);
} else {
span.recordException(new Error(String(e)));
}
throw e;
} finally {
span.end();
}
}
);
}
}
@@ -130,6 +130,9 @@ export default function Page() {
)}
<Property label="Message">{event.message}</Property>
<Property label="Task ID">{event.taskSlug}</Property>
{event.idempotencyKey && (
<Property label="Idempotency key">{event.idempotencyKey}</Property>
)}
{event.taskPath && event.taskExportName && (
<Property label="Task">
<TaskPath
@@ -0,0 +1,45 @@
import type { LoaderFunctionArgs } from "@remix-run/server-runtime";
import { json } from "@remix-run/server-runtime";
import { z } from "zod";
import { ApiBatchResultsPresenter } from "~/presenters/v3/ApiBatchResultsPresenter.server";
import { ApiRunResultPresenter } from "~/presenters/v3/ApiRunResultPresenter.server";
import { authenticateApiRequest } from "~/services/apiAuth.server";
const ParamsSchema = z.object({
/* This is the batch friendly ID */
batchParam: z.string(),
});
export async function loader({ request, params }: LoaderFunctionArgs) {
// Authenticate the request
const authenticationResult = await authenticateApiRequest(request);
if (!authenticationResult) {
return json({ error: "Invalid or Missing API Key" }, { status: 401 });
}
const parsed = ParamsSchema.safeParse(params);
if (!parsed.success) {
return json({ error: "Invalid or missing run ID" }, { status: 400 });
}
const { batchParam } = parsed.data;
try {
const presenter = new ApiBatchResultsPresenter();
const result = await presenter.call(batchParam, authenticationResult.environment);
if (!result) {
return json({ error: "Batch not found" }, { status: 404 });
}
return json(result);
} catch (error) {
if (error instanceof Error) {
return json({ error: error.message }, { status: 500 });
} else {
return json({ error: JSON.stringify(error) }, { status: 500 });
}
}
}
@@ -1,12 +1,10 @@
import type { ActionFunctionArgs } from "@remix-run/server-runtime";
import { json } from "@remix-run/server-runtime";
import { PrismaErrorSchema, prisma } from "~/db.server";
import { z } from "zod";
import { prisma } from "~/db.server";
import { authenticateApiRequest } from "~/services/apiAuth.server";
import { CancelRunService } from "~/services/runs/cancelRun.server";
import { ApiRunPresenter } from "~/presenters/ApiRunPresenter.server";
import { ReplayTaskRunService } from "~/v3/services/replayTaskRun.server";
import { logger } from "~/services/logger.server";
import { ReplayTaskRunService } from "~/v3/services/replayTaskRun.server";
const ParamsSchema = z.object({
/* This is the run friendly ID */
@@ -0,0 +1,44 @@
import type { LoaderFunctionArgs } from "@remix-run/server-runtime";
import { json } from "@remix-run/server-runtime";
import { z } from "zod";
import { ApiRunResultPresenter } from "~/presenters/v3/ApiRunResultPresenter.server";
import { authenticateApiRequest } from "~/services/apiAuth.server";
const ParamsSchema = z.object({
/* This is the run friendly ID */
runParam: z.string(),
});
export async function loader({ request, params }: LoaderFunctionArgs) {
// Authenticate the request
const authenticationResult = await authenticateApiRequest(request);
if (!authenticationResult) {
return json({ error: "Invalid or Missing API Key" }, { status: 401 });
}
const parsed = ParamsSchema.safeParse(params);
if (!parsed.success) {
return json({ error: "Invalid or missing run ID" }, { status: 400 });
}
const { runParam } = parsed.data;
try {
const presenter = new ApiRunResultPresenter();
const result = await presenter.call(runParam, authenticationResult.environment);
if (!result) {
return json({ error: "Run either doesn't exist or is not finished" }, { status: 404 });
}
return json(result);
} catch (error) {
if (error instanceof Error) {
return json({ error: error.message }, { status: 500 });
} else {
return json({ error: JSON.stringify(error) }, { status: 500 });
}
}
}
+4 -1
View File
@@ -63,6 +63,7 @@ export type TraceAttributes = Partial<
| "batchId"
| "payload"
| "payloadType"
| "idempotencyKey"
>
>;
@@ -371,6 +372,7 @@ export class EventRepository {
id: event.spanId,
parentId: event.parentId ?? undefined,
runId: event.runId,
idempotencyKey: event.idempotencyKey,
data: {
message: event.message,
style: event.style,
@@ -459,7 +461,7 @@ export class EventRepository {
const links: SpanLink[] = [];
if (messagingEvent.success && messagingEvent.data) {
if ("id" in messagingEvent.data.message) {
if (messagingEvent.data.message && "id" in messagingEvent.data.message) {
if (messagingEvent.data.message.id.startsWith("run_")) {
links.push({
type: "run",
@@ -719,6 +721,7 @@ export class EventRepository {
links: links as unknown as Prisma.InputJsonValue,
payload: options.attributes.payload,
payloadType: options.attributes.payloadType,
idempotencyKey: options.attributes.idempotencyKey,
};
if (options.immediate) {
@@ -118,7 +118,7 @@ export class DevQueueConsumer {
completion: TaskRunExecutionResult,
execution: TaskRunExecution
) {
this._inProgressAttempts.delete(completion.id);
this._inProgressAttempts.delete(execution.attempt.id);
if (completion.ok) {
this._taskSuccesses++;
@@ -424,7 +424,7 @@ export class DevQueueConsumer {
orderBy: { number: "desc" },
},
tags: true,
batchItem: {
batchItems: {
include: {
batchTaskRun: true,
},
@@ -499,6 +499,7 @@ export class DevQueueConsumer {
createdAt: lockedTaskRun.createdAt,
tags: lockedTaskRun.tags.map((tag) => tag.name),
isTest: lockedTaskRun.isTest,
idempotencyKey: lockedTaskRun.idempotencyKey ?? undefined,
},
queue: {
id: queue.friendlyId,
@@ -520,9 +521,10 @@ export class DevQueueConsumer {
slug: this.env.project.slug,
name: this.env.project.name,
},
batch: lockedTaskRun.batchItem?.batchTaskRun
? { id: lockedTaskRun.batchItem.batchTaskRun.friendlyId }
: undefined,
batch:
lockedTaskRun.batchItems[0] && lockedTaskRun.batchItems[0].batchTaskRun
? { id: lockedTaskRun.batchItems[0].batchTaskRun.friendlyId }
: undefined,
};
const environmentRepository = new EnvironmentVariablesRepository();
+20 -20
View File
@@ -39,7 +39,6 @@ const SemanticAttributes = {
export type MarQSOptions = {
redis: RedisOptions;
defaultQueueConcurrency: number;
defaultEnvConcurrency: number;
defaultOrgConcurrency: number;
windowSize?: number;
@@ -92,7 +91,7 @@ export class MarQS {
public async getQueueConcurrencyLimit(env: AuthenticatedEnvironment, queue: string) {
const result = await this.redis.get(this.keys.queueConcurrencyLimitKey(env, queue));
return result ? Number(result) : this.options.defaultQueueConcurrency;
return result ? Number(result) : undefined;
}
public async getEnvConcurrencyLimit(env: AuthenticatedEnvironment) {
@@ -860,7 +859,6 @@ export class MarQS {
messageQueue,
String(this.options.visibilityTimeoutInMs ?? 300000), // 5 minutes
String(Date.now()),
String(this.options.defaultQueueConcurrency),
String(this.options.defaultEnvConcurrency),
String(this.options.defaultOrgConcurrency)
);
@@ -1015,16 +1013,22 @@ export class MarQS {
concurrencyLimitKey,
envConcurrencyLimitKey,
orgConcurrencyLimitKey,
String(this.options.defaultQueueConcurrency),
String(this.options.defaultEnvConcurrency),
String(this.options.defaultOrgConcurrency)
);
const queueCurrent = Number(capacities[0]);
const envLimit = Number(capacities[3]);
const orgLimit = Number(capacities[5]);
const queueLimit = capacities[1] ? Number(capacities[1]) : Math.min(envLimit, orgLimit);
const envCurrent = Number(capacities[2]);
const orgCurrent = Number(capacities[4]);
// [queue current, queue limit, env current, env limit, org current, org limit]
return {
queue: { current: Number(capacities[0]), limit: Number(capacities[1]) },
env: { current: Number(capacities[2]), limit: Number(capacities[3]) },
org: { current: Number(capacities[4]), limit: Number(capacities[5]) },
queue: { current: queueCurrent, limit: queueLimit },
env: { current: envCurrent, limit: envLimit },
org: { current: orgCurrent, limit: orgLimit },
};
}
@@ -1119,13 +1123,12 @@ local currentConcurrencyKey = KEYS[7]
local envCurrentConcurrencyKey = KEYS[8]
local orgCurrentConcurrencyKey = KEYS[9]
-- Args: childQueueName, visibilityQueue, currentTime, defaultConcurrencyLimit, defaultEnvConcurrencyLimit, defaultOrgConcurrencyLimit
-- Args: childQueueName, visibilityQueue, currentTime, defaultEnvConcurrencyLimit, defaultOrgConcurrencyLimit
local childQueueName = ARGV[1]
local visibilityTimeout = tonumber(ARGV[2])
local currentTime = tonumber(ARGV[3])
local defaultConcurrencyLimit = ARGV[4]
local defaultEnvConcurrencyLimit = ARGV[5]
local defaultOrgConcurrencyLimit = ARGV[6]
local defaultEnvConcurrencyLimit = ARGV[4]
local defaultOrgConcurrencyLimit = ARGV[5]
-- Check current org concurrency against the limit
local orgCurrentConcurrency = tonumber(redis.call('SCARD', orgCurrentConcurrencyKey) or '0')
@@ -1145,8 +1148,9 @@ end
-- Check current queue concurrency against the limit
local currentConcurrency = tonumber(redis.call('SCARD', currentConcurrencyKey) or '0')
local concurrencyLimit = tonumber(redis.call('GET', concurrencyLimitKey) or defaultConcurrencyLimit)
local concurrencyLimit = tonumber(redis.call('GET', concurrencyLimitKey) or '1000000')
-- Check condition only if concurrencyLimit exists
if currentConcurrency >= concurrencyLimit then
return nil
end
@@ -1304,10 +1308,9 @@ local concurrencyLimitKey = KEYS[4]
local envConcurrencyLimitKey = KEYS[5]
local orgConcurrencyLimitKey = KEYS[6]
-- Args defaultConcurrencyLimit, defaultEnvConcurrencyLimit, defaultOrgConcurrencyLimit
local defaultConcurrencyLimit = tonumber(ARGV[1])
local defaultEnvConcurrencyLimit = tonumber(ARGV[2])
local defaultOrgConcurrencyLimit = tonumber(ARGV[3])
-- Args defaultEnvConcurrencyLimit, defaultOrgConcurrencyLimit
local defaultEnvConcurrencyLimit = tonumber(ARGV[1])
local defaultOrgConcurrencyLimit = tonumber(ARGV[2])
local currentOrgConcurrency = tonumber(redis.call('SCARD', currentOrgConcurrencyKey) or '0')
local orgConcurrencyLimit = tonumber(redis.call('GET', orgConcurrencyLimitKey) or defaultOrgConcurrencyLimit)
@@ -1316,7 +1319,7 @@ local currentEnvConcurrency = tonumber(redis.call('SCARD', currentEnvConcurrency
local envConcurrencyLimit = tonumber(redis.call('GET', envConcurrencyLimitKey) or defaultEnvConcurrencyLimit)
local currentConcurrency = tonumber(redis.call('SCARD', currentConcurrencyKey) or '0')
local concurrencyLimit = tonumber(redis.call('GET', concurrencyLimitKey) or defaultConcurrencyLimit)
local concurrencyLimit = redis.call('GET', concurrencyLimitKey)
-- Return current capacity and concurrency limits for the queue, env, org
return { currentConcurrency, concurrencyLimit, currentEnvConcurrency, envConcurrencyLimit, currentOrgConcurrency, orgConcurrencyLimit }
@@ -1398,7 +1401,6 @@ declare module "ioredis" {
childQueueName: string,
visibilityTimeout: string,
currentTime: string,
defaultConcurrencyLimit: string,
defaultEnvConcurrencyLimit: string,
defaultOrgConcurrencyLimit: string,
callback?: Callback<[string, string]>
@@ -1447,7 +1449,6 @@ declare module "ioredis" {
concurrencyLimitKey: string,
envConcurrencyLimitKey: string,
orgConcurrencyLimitKey: string,
defaultConcurrencyLimit: string,
defaultEnvConcurrencyLimit: string,
defaultOrgConcurrencyLimit: string,
callback?: Callback<number[]>
@@ -1492,7 +1493,6 @@ function getMarQSClient() {
envQueuePriorityStrategy: new SimpleWeightedChoiceStrategy({ queueSelectionCount: 12 }),
workers: 1,
redis: redisOptions,
defaultQueueConcurrency: env.DEFAULT_QUEUE_EXECUTION_CONCURRENCY_LIMIT,
defaultEnvConcurrency: env.DEFAULT_ENV_EXECUTION_CONCURRENCY_LIMIT,
defaultOrgConcurrency: env.DEFAULT_ORG_EXECUTION_CONCURRENCY_LIMIT,
visibilityTimeoutInMs: 120 * 1000, // 2 minutes,
@@ -812,7 +812,7 @@ class SharedQueueTasks {
if (ok) {
const success: TaskRunSuccessfulExecutionResult = {
ok,
id: attempt.friendlyId,
id: attempt.taskRun.friendlyId,
output: attempt.output ?? undefined,
outputType: attempt.outputType,
};
@@ -820,7 +820,7 @@ class SharedQueueTasks {
} else {
const failure: TaskRunFailedExecutionResult = {
ok,
id: attempt.friendlyId,
id: attempt.taskRun.friendlyId,
error: attempt.error as TaskRunError,
};
return failure;
@@ -848,7 +848,7 @@ class SharedQueueTasks {
taskRun: {
include: {
tags: true,
batchItem: {
batchItems: {
include: {
batchTaskRun: true,
},
@@ -956,6 +956,7 @@ class SharedQueueTasks {
createdAt: taskRun.createdAt,
tags: taskRun.tags.map((tag) => tag.name),
isTest: taskRun.isTest,
idempotencyKey: taskRun.idempotencyKey ?? undefined,
},
queue: {
id: queue.friendlyId,
@@ -977,9 +978,10 @@ class SharedQueueTasks {
slug: attempt.runtimeEnvironment.project.slug,
name: attempt.runtimeEnvironment.project.name,
},
batch: taskRun.batchItem?.batchTaskRun
? { id: taskRun.batchItem.batchTaskRun.friendlyId }
: undefined,
batch:
taskRun.batchItems[0] && taskRun.batchItems[0].batchTaskRun
? { id: taskRun.batchItems[0].batchTaskRun.friendlyId }
: undefined,
worker: {
id: attempt.backgroundWorkerId,
contentHash: attempt.backgroundWorker.contentHash,
-1
View File
@@ -1,4 +1,3 @@
import { RedisOptions } from "ioredis";
import { z } from "zod";
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
@@ -352,6 +352,7 @@ function extractResourceProperties(attributes: KeyValue[]) {
queueId: extractStringAttribute(attributes, SemanticInternalAttributes.QUEUE_ID),
queueName: extractStringAttribute(attributes, SemanticInternalAttributes.QUEUE_NAME),
batchId: extractStringAttribute(attributes, SemanticInternalAttributes.BATCH_ID),
idempotencyKey: extractStringAttribute(attributes, SemanticInternalAttributes.IDEMPOTENCY_KEY),
};
}
@@ -1,9 +1,9 @@
import { BatchTriggerTaskRequestBody } from "@trigger.dev/core/v3";
import { nanoid } from "nanoid";
import { AuthenticatedEnvironment } from "~/services/apiAuth.server";
import { generateFriendlyId } from "../friendlyIdentifiers";
import { BaseService } from "./baseService.server";
import { TriggerTaskService } from "./triggerTask.server";
import { batchTaskRunItemStatusForRunStatus } from "~/models/taskRun.server";
export type BatchTriggerTaskServiceOptions = {
idempotencyKey?: string;
@@ -22,23 +22,23 @@ export class BatchTriggerTaskService extends BaseService {
return await this.traceWithEnv("call()", environment, async (span) => {
span.setAttribute("taskId", taskId);
const idempotencyKey = options.idempotencyKey ?? nanoid();
const existingBatch = await this._prisma.batchTaskRun.findUnique({
where: {
runtimeEnvironmentId_idempotencyKey: {
runtimeEnvironmentId: environment.id,
idempotencyKey,
},
},
include: {
items: {
include: {
taskRun: true,
const existingBatch = options.idempotencyKey
? await this._prisma.batchTaskRun.findUnique({
where: {
runtimeEnvironmentId_idempotencyKey: {
runtimeEnvironmentId: environment.id,
idempotencyKey: options.idempotencyKey,
},
},
},
},
});
include: {
items: {
include: {
taskRun: true,
},
},
},
})
: undefined;
if (existingBatch) {
span.setAttribute("batchId", existingBatch.friendlyId);
@@ -58,7 +58,7 @@ export class BatchTriggerTaskService extends BaseService {
data: {
friendlyId: generateFriendlyId("batch"),
runtimeEnvironmentId: environment.id,
idempotencyKey,
idempotencyKey: options.idempotencyKey,
taskIdentifier: taskId,
dependentTaskAttemptId: dependentAttempt?.id,
},
@@ -70,8 +70,6 @@ export class BatchTriggerTaskService extends BaseService {
let index = 0;
for (const item of body.items) {
const idempotencyKey = nanoid();
const run = await triggerTaskService.call(
taskId,
environment,
@@ -83,7 +81,6 @@ export class BatchTriggerTaskService extends BaseService {
},
},
{
idempotencyKey,
triggerVersion: options.triggerVersion,
traceContext: options.traceContext,
spanParentAsLink: options.spanParentAsLink,
@@ -96,6 +93,7 @@ export class BatchTriggerTaskService extends BaseService {
data: {
batchTaskRunId: batch.id,
taskRunId: run.id,
status: batchTaskRunItemStatusForRunStatus(run.status),
},
});
@@ -4,9 +4,9 @@ import { marqs } from "~/v3/marqs/index.server";
import { devPubSub } from "../marqs/devPubSub.server";
import { BaseService } from "./baseService.server";
import { socketIo } from "../handleSocketIo.server";
import { assertUnreachable } from "../utils/asserts.server";
import { CancelAttemptService } from "./cancelAttempt.server";
import { logger } from "~/services/logger.server";
import assertNever from "assert-never";
export const CANCELLABLE_STATUSES: Array<TaskRunStatus> = [
"PENDING",
@@ -148,7 +148,7 @@ export class CancelTaskRunService extends BaseService {
break;
}
default: {
assertUnreachable(attempt.status);
assertNever(attempt.status);
}
}
}
@@ -39,10 +39,12 @@ export class CompleteAttemptService extends BaseService {
env?: AuthenticatedEnvironment;
checkpoint?: CheckpointData;
}): Promise<"COMPLETED" | "RETRIED"> {
const taskRunAttempt = await findAttempt(this._prisma, completion.id);
const taskRunAttempt = await findAttempt(this._prisma, execution.attempt.id);
if (!taskRunAttempt) {
logger.error("[CompleteAttemptService] Task run attempt not found", { id: completion.id });
logger.error("[CompleteAttemptService] Task run attempt not found", {
id: execution.attempt.id,
});
// Update the task run to be failed
await this._prisma.taskRun.update({
@@ -76,7 +78,7 @@ export class CompleteAttemptService extends BaseService {
env?: AuthenticatedEnvironment
): Promise<"COMPLETED"> {
await this._prisma.taskRunAttempt.update({
where: { friendlyId: completion.id },
where: { id: taskRunAttempt.id },
data: {
status: "COMPLETED",
completedAt: new Date(),
@@ -144,7 +146,7 @@ export class CompleteAttemptService extends BaseService {
}
await this._prisma.taskRunAttempt.update({
where: { friendlyId: completion.id },
where: { id: taskRunAttempt.id },
data: {
status: "FAILED",
completedAt: new Date(),
@@ -12,7 +12,7 @@ export class ResumeTaskRunDependenciesService extends BaseService {
include: {
taskRun: {
include: {
batchItem: true,
batchItems: true,
dependency: {
include: {
dependentAttempt: true,
@@ -34,14 +34,16 @@ export class ResumeTaskRunDependenciesService extends BaseService {
return;
}
const { batchItem, dependency } = taskAttempt.taskRun;
const { batchItems, dependency } = taskAttempt.taskRun;
if (!batchItem && !dependency) {
if (!batchItems.length && !dependency) {
return;
}
if (batchItem) {
await this.#resumeBatchItem(batchItem, taskAttempt);
if (batchItems.length) {
for (const batchItem of batchItems) {
await this.#resumeBatchItem(batchItem, taskAttempt);
}
return;
}
@@ -34,18 +34,20 @@ export class TriggerTaskService extends BaseService {
return await this.traceWithEnv("call()", environment, async (span) => {
span.setAttribute("taskId", taskId);
const idempotencyKey = options.idempotencyKey ?? body.options?.idempotencyKey ?? nanoid();
const idempotencyKey = options.idempotencyKey ?? body.options?.idempotencyKey;
const existingRun = await this._prisma.taskRun.findUnique({
where: {
runtimeEnvironmentId_idempotencyKey: {
runtimeEnvironmentId: environment.id,
idempotencyKey,
},
},
});
const existingRun = idempotencyKey
? await this._prisma.taskRun.findUnique({
where: {
runtimeEnvironmentId_idempotencyKey: {
runtimeEnvironmentId: environment.id,
idempotencyKey,
},
},
})
: undefined;
if (existingRun) {
if (existingRun && existingRun.taskIdentifier === taskId) {
span.setAttribute("runId", existingRun.friendlyId);
return existingRun;
}
@@ -68,6 +70,7 @@ export class TriggerTaskService extends BaseService {
},
runIsTest: body.options?.test ?? false,
batchId: options.batchId,
idempotencyKey,
},
incomplete: true,
immediate: true,
@@ -1,3 +0,0 @@
export function assertUnreachable(x: never): never {
throw new Error("Didn't expect to get here");
}
+1
View File
@@ -96,6 +96,7 @@
"@uiw/react-codemirror": "^4.19.5",
"@upstash/ratelimit": "^1.0.1",
"@whatwg-node/fetch": "^0.9.14",
"assert-never": "^1.2.1",
"aws4fetch": "^1.0.18",
"class-variance-authority": "^0.5.2",
"clsx": "^1.2.1",
+1 -1
View File
@@ -35,7 +35,7 @@ export const myTask = task({
maxAttempts: 10,
},
run: async (payload: string) => {
const result = await otherTask.triggerAndWait({ payload: "some data" });
const result = await otherTask.triggerAndWait("some data");
//...do other stuff
},
});
+36 -7
View File
@@ -11,7 +11,6 @@ This guide highlights the differences and should help you migrate your project.
Here are some features you might be using in Defer that are coming this month to v3:
- [Scheduled tasks (including CRON)](/v3/tasks-scheduled) will be available in mid-April.
- Triggering a task with a delay (like `assignOptions` delay in Defer) will be available soon there is [an alternative](#delay) you can use for now.
You can view the full feature matrix [here](/v3/feature-matrix).
@@ -71,7 +70,7 @@ export async function runLongRunningTask() {
}
```
In Trigger.dev your logic goes in the `run` function of a task. You can then `trigger` and `batchTrigger` that task, with a payload and options.
In Trigger.dev your logic goes in the `run` function of a task. You can then `trigger` and `batchTrigger` that task, with a payload as the first argument.
```ts /app/actions/actions.ts
"use server";
@@ -79,7 +78,7 @@ In Trigger.dev your logic goes in the `run` function of a task. You can then `tr
import { longRunningTask } from "@/trigger/someTasks";
export async function runLongRunningTask() {
return await longRunningTask.trigger({ payload: { foo: "bar" } });
return await longRunningTask.trigger({ foo: "bar" });
}
```
@@ -243,7 +242,7 @@ export const longRunningTask = task({
import { longRunningTask } from "@/trigger/longRunningTask";
export async function runLongRunningTask() {
return await longRunningTask.trigger({ payload: { foo: "bar" } });
return await longRunningTask.trigger({ foo: "bar" });
}
```
@@ -256,6 +255,36 @@ export async function runLongRunningTask() {
#### Example 2: A CRON task
<Warning>
"Scheduled" tasks will be available by mid-April. This will allow you to replace Defer CRON tasks.
</Warning>
We call these [scheduled tasks](/v3/tasks-scheduled) in Trigger.dev.
In Defer you might have a function like this:
```ts
import { defer } from "@defer/client";
async function sendMondayNewletter() {
// business logic here
}
export default defer.cron(sendMondayNewletter, "0 0 * * 1");
```
In Trigger.dev the task looks like this:
```ts
import { schedules } from "@trigger.dev/sdk/v3";
//this task will run when any of the attached schedules trigger
export const sendMondayNewletter = schedules.task({
id: "send-monday-newsletter",
run: async (payload) => {
// business logic here
},
});
```
Then you need to attach a schedule to the task, either using the dashboard or in your code. You can attach unlimited schedules to a task.
<Card title="Attaching schedules" icon="clock" href="/v3/tasks-scheduled">
How to attach a schedule to a task
</Card>
+19 -30
View File
@@ -107,24 +107,19 @@ export async function POST(request: Request) {
if (data.branch === "main") {
//trigger the task, with a different queue
const handle = await generatePullRequest.trigger({
payload: data,
options: {
queue: {
//the "main-branch" queue will have a concurrency limit of 10
//this triggered run will use that queue
name: "main-branch",
concurrencyLimit: 10,
},
const handle = await generatePullRequest.trigger(data, {
queue: {
//the "main-branch" queue will have a concurrency limit of 10
//this triggered run will use that queue
name: "main-branch",
concurrencyLimit: 10,
},
});
return Response.json(handle);
} else {
//triggered with the default (concurrency of 1)
const handle = await generatePullRequest.trigger({
payload: data,
});
const handle = await generatePullRequest.trigger(data);
return Response.json(handle);
}
}
@@ -146,32 +141,26 @@ export async function POST(request: Request) {
if (data.isFreeUser) {
//free users can only have 1 PR generated at a time
const handle = await generatePullRequest.trigger({
payload: data,
options: {
queue: {
//every free user gets a queue with a concurrency limit of 1
name: "free-users",
concurrencyLimit: 1,
},
concurrencyKey: data.userId,
const handle = await generatePullRequest.trigger(data, {
queue: {
//every free user gets a queue with a concurrency limit of 1
name: "free-users",
concurrencyLimit: 1,
},
concurrencyKey: data.userId,
});
//return a success response with the handle
return Response.json(handle);
} else {
//trigger the task, with a different queue
const handle = await generatePullRequest.trigger({
payload: data,
options: {
queue: {
//every paid user gets a queue with a concurrency limit of 10
name: "paid-users",
concurrencyLimit: 10,
},
concurrencyKey: data.userId,
const handle = await generatePullRequest.trigger(data, {
queue: {
//every paid user gets a queue with a concurrency limit of 10
name: "paid-users",
concurrencyLimit: 10,
},
concurrencyKey: data.userId,
});
//return a success response with the handle
+1 -1
View File
@@ -37,7 +37,7 @@ import { helloWorldTask } from "./trigger/hello-world";
async function triggerHelloWorld() {
//This triggers the task and return a handle
const handle = await helloWorld.trigger({ payload: { message: "Hello world!" } });
const handle = await helloWorld.trigger({ message: "Hello world!" });
//You can use the handle to check the status of the task, cancel and retry it.
console.log("Task is running with handle", handle.id);
+36 -29
View File
@@ -48,7 +48,7 @@ export async function POST(request: Request) {
const data = await request.json();
//trigger your task
const handle = await emailSequence.trigger({ payload: { to: data.email, name: data.name } });
const handle = await emailSequence.trigger({ to: data.email, name: data.name });
//return a success response with the handle
return Response.json(handle);
@@ -67,7 +67,7 @@ export async function action({ request, params }: ActionFunctionArgs) {
const data = await request.json();
//trigger your task
const handle = await emailSequence.trigger({ payload: { to: data.email, name: data.name } });
const handle = await emailSequence.trigger({ to: data.email, name: data.name });
//return a success response with the handle
return json(handle);
@@ -91,9 +91,9 @@ export async function POST(request: Request) {
const data = await request.json();
//batch trigger your task
const batchHandle = await emailSequence.batchTrigger({
items: data.users.map((u) => ({ payload: { to: u.email, name: u.name } })),
});
const batchHandle = await emailSequence.batchTrigger(
data.users.map((u) => ({ payload: { to: u.email, name: u.name } }))
);
//return a success response with the handle
return Response.json(batchHandle);
@@ -112,9 +112,9 @@ export async function action({ request, params }: ActionFunctionArgs) {
const data = await request.json();
//batch trigger your task
const batchHandle = await emailSequence.batchTrigger({
items: data.users.map((u) => ({ payload: { to: u.email, name: u.name } })),
});
const batchHandle = await emailSequence.batchTrigger(
data.users.map((u) => ({ payload: { to: u.email, name: u.name } }))
);
//return a success response with the handle
return json(batchHandle);
@@ -137,7 +137,7 @@ import { myOtherTask } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const handle = await myOtherTask.trigger({ payload: "some data" });
const handle = await myOtherTask.trigger("some data");
//...do other stuff
},
@@ -154,7 +154,7 @@ import { myOtherTask } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const batchHandle = await myOtherTask.batchTrigger({ items: [{ payload: "some data" }] });
const batchHandle = await myOtherTask.batchTrigger([{ payload: "some data" }]);
//...do other stuff
},
@@ -168,16 +168,18 @@ This is where it gets interesting. You can trigger a task and then wait for the
<Accordion title="Don't use this in parallel, e.g. with `Promise.all()`">
Instead, use `batchTriggerAndWait()` if you can, or a for loop if you can't.
To control concurrency using batch triggers, you can set `queue.concurrencyLimit` on the child task.
To control concurrency using batch triggers, you can set `queue.concurrencyLimit` on the child task.
<CodeGroup>
```ts /trigger/batch.ts
export const batchTask = task({
id: "batch-task",
run: async (payload: string) => {
const results = await childTask.batchTriggerAndWait({
items: [{ payload: "item1" }, { payload: "item2" }],
});
const results = await childTask.batchTriggerAndWait([
{ payload: "item1" },
{ payload: "item2" },
]);
console.log("Results", results);
//...do stuff with the results
@@ -192,7 +194,7 @@ export const loopTask = task({
//this will be slower than the batch version
//as we have to resume the parent after each iteration
for (let i = 0; i < 2; i++) {
const result = await childTask.triggerAndWait({ payload: `item${i}` });
const result = await childTask.triggerAndWait(`item${i}`);
console.log("Result", result);
//...do stuff with the result
@@ -200,6 +202,7 @@ export const loopTask = task({
},
});
```
</CodeGroup>
</Accordion>
@@ -208,7 +211,7 @@ export const loopTask = task({
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
const result = await batchChildTask.triggerAndWait({ payload: "some-data" });
const result = await batchChildTask.triggerAndWait("some-data");
console.log("Result", result);
//...do stuff with the result
@@ -223,16 +226,18 @@ You can batch trigger a task and wait for all the results. This is useful for th
<Accordion title="Don't use this in parallel, e.g. with `Promise.all()`">
Instead, pass in all items at once and set an appropriate `maxConcurrency`. Alternatively, use sequentially with a for loop.
To control concurrency, you can set `queue.concurrencyLimit` on the child task.
To control concurrency, you can set `queue.concurrencyLimit` on the child task.
<CodeGroup>
```ts /trigger/batch.ts
export const batchTask = task({
id: "batch-task",
run: async (payload: string) => {
const results = await childTask.batchTriggerAndWait({
items: [{ payload: "item1" }, { payload: "item2" }],
});
const results = await childTask.batchTriggerAndWait([
{ payload: "item1" },
{ payload: "item2" },
]);
console.log("Results", results);
//...do stuff with the results
@@ -247,9 +252,10 @@ export const loopTask = task({
//this will be slower than a single batchTriggerAndWait()
//as we have to resume the parent after each iteration
for (let i = 0; i < 2; i++) {
const result = await childTask.batchTriggerAndWait({
items: [{ payload: `itemA${i}` }, { payload: `itemB${i}` }],
});
const result = await childTask.batchTriggerAndWait([
{ payload: `itemA${i}` },
{ payload: `itemB${i}` },
]);
console.log("Result", result);
//...do stuff with the result
@@ -257,6 +263,7 @@ export const loopTask = task({
},
});
```
</CodeGroup>
</Accordion>
@@ -265,9 +272,11 @@ export const loopTask = task({
export const batchParentTask = task({
id: "parent-task",
run: async (payload: string) => {
const results = await childTask.batchTriggerAndWait({
items: [{ payload: "item4" }, { payload: "item5" }, { payload: "item6" }],
});
const results = await childTask.batchTriggerAndWait([
{ payload: "item4" },
{ payload: "item5" },
{ payload: "item6" },
]);
console.log("Results", results);
//...do stuff with the result
@@ -326,9 +335,7 @@ import { createAvatar } from "@/trigger/create-avatar";
export async function create() {
try {
const handle = await createAvatar.trigger({
payload: {
userImage: "http://...",
},
userImage: "http://...",
});
return { handle };
+1 -3
View File
@@ -164,9 +164,7 @@ We've unified triggering in v3. You use `trigger()` or `batchTrigger()` which yo
async function yourBackendFunction() {
//call `trigger()` on any task
const handle = await openaiTask.trigger({
payload: {
prompt: "Tell me a programming joke",
},
prompt: "Tell me a programming joke",
});
}
```
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/airtable
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/airtable",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for airtable",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -25,8 +25,8 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"airtable": "^0.12.1",
"zod": "3.22.3"
},
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/github
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/github",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official GitHub integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -30,8 +30,8 @@
"@octokit/request-error": "^5.0.1",
"@octokit/webhooks": "^12.0.10",
"octokit": "^3.1.2",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"zod": "3.22.3"
},
"engines": {
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/linear
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/linear",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for @linear/sdk",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -26,8 +26,8 @@
},
"dependencies": {
"@linear/sdk": "^8.0.0",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"zod": "3.22.3"
},
"engines": {
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/slack
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/openai",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official OpenAI integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -42,8 +42,8 @@
},
"dependencies": {
"openai": "^4.16.1",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14"
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15"
},
"engines": {
"node": ">=18.0.0"
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/plain
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/plain",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official Plain.com integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -24,8 +24,8 @@
"build:tsup": "tsup"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@team-plain/typescript-sdk": "^2.7.0"
},
"engines": {
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/replicate
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/replicate",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for replicate",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -25,8 +25,8 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"replicate": "^0.18.1",
"zod": "3.22.3"
},
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/resend
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/resend",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official Resend.com integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -24,8 +24,8 @@
"build:tsup": "tsup"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"resend": "^2.1.0"
},
"engines": {
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/sendgrid
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/sendgrid",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for @sendgrid/mail",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -26,8 +26,8 @@
},
"dependencies": {
"@sendgrid/mail": "^7.7.0",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14"
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15"
},
"engines": {
"node": ">=16.8.0"
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/shopify
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/shopify",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for @shopify/shopify-api",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -26,8 +26,8 @@
},
"dependencies": {
"@shopify/shopify-api": "^8.0.2",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"zod": "3.22.3"
},
"engines": {
+8
View File
@@ -1,5 +1,13 @@
# @trigger.dev/slack
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/slack",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official Slack integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -25,7 +25,7 @@
},
"dependencies": {
"@slack/web-api": "^6.8.1",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"zod": "3.22.3"
},
"engines": {
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/stripe
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/stripe",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for stripe",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -25,8 +25,8 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"stripe": "^12.14.0",
"zod": "3.22.3"
},
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/supabase
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/supabase",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Trigger.dev integration for @supabase/supabase-js",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -26,8 +26,8 @@
},
"dependencies": {
"@supabase/supabase-js": "^2.26.0",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"supabase-management-js": "^1.0.0",
"zod": "3.22.3"
},
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/typeform
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
- @trigger.dev/integration-kit@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/typeform",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The official Typeform integration for Trigger.dev",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -24,8 +24,8 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/integration-kit": "workspace:^3.0.0-beta.15",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@typeform/api-client": "^1.8.0",
"zod": "3.22.3"
},
+8
View File
@@ -1,5 +1,13 @@
# @trigger.dev/astro
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+2 -2
View File
@@ -1,7 +1,7 @@
{
"name": "@trigger.dev/astro",
"description": "An Astro-native integration for Trigger.dev background jobs platform",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"files": [
@@ -20,7 +20,7 @@
"build:tsup": "tsup"
},
"peerDependencies": {
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14"
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15"
},
"devDependencies": {
"astro": "^3.0.12",
+23
View File
@@ -1,5 +1,28 @@
# trigger.dev
## 3.0.0-beta.15
### Patch Changes
- 26093896d: When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A runs idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
- b82db67b8: Add additional logging around cleaning up dev workers, and always kill them after 5 seconds if they haven't already exited
- 62c9a5b71: Fixes an issue that caused failed tasks when resuming after calling `triggerAndWait` or `batchTriggerAndWait` in prod/staging (this doesn't effect dev).
The version of Node.js we use for deployed workers (latest 20) would crash with an out-of-memory error when the checkpoint was restored. This crash does not happen on Node 18x or Node21x, so we've decided to upgrade the worker version to Node.js21x, to mitigate this issue.
You'll need to re-deploy to production to fix the issue.
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- Updated dependencies [62c9a5b71]
- @trigger.dev/core@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "trigger.dev",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "A Command-Line Interface for Trigger.dev (v3) projects",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
@@ -85,7 +85,7 @@
"@opentelemetry/sdk-trace-base": "^1.22.0",
"@opentelemetry/sdk-trace-node": "^1.22.0",
"@opentelemetry/semantic-conventions": "^1.22.0",
"@trigger.dev/core": "workspace:^3.0.0-beta.14",
"@trigger.dev/core": "workspace:^3.0.0-beta.15",
"@types/degit": "^2.8.3",
"chalk": "^5.2.0",
"chokidar": "^3.5.3",
+1 -1
View File
@@ -1,4 +1,4 @@
FROM node:20-bookworm-slim@sha256:d4cdfc305abe5ea78da7167bf78263c22596dc332f2654b662890777ea166224 AS base
FROM node:21-bookworm-slim@sha256:fb82287cf66ca32d854c05f54251fca8b572149163f154248df7e800003c90b5 AS base
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -627,6 +627,15 @@ class TaskRunProcess {
});
this._isBeingKilled = kill;
// Set a timeout to kill the child process if it hasn't been killed within 5 seconds
setTimeout(() => {
if (this._child && !this._child.killed) {
logger.debug(`[${this.execution.run.id}] killing task run process after timeout`);
this._child.kill();
}
}, 5000);
}
async executeTaskRun(payload: TaskRunExecutionPayload): Promise<TaskRunExecutionResult> {
@@ -709,6 +718,8 @@ class TaskRunProcess {
break;
}
case "READY_TO_DISPOSE": {
logger.debug(`[${this.execution.run.id}] task run process is ready to dispose`);
this.#kill();
break;
@@ -791,6 +802,8 @@ class TaskRunProcess {
#kill() {
if (this._child && !this._child.killed) {
logger.debug(`[${this.execution.run.id}] killing task run process`);
this._child?.kill();
}
}
@@ -117,7 +117,7 @@ const handler = new ZodMessageHandler({
execution,
result: {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.TASK_ALREADY_RUNNING,
@@ -139,7 +139,7 @@ const handler = new ZodMessageHandler({
execution,
result: {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.COULD_NOT_FIND_EXECUTOR,
@@ -110,7 +110,7 @@ const zodIpc = new ZodIpcConnection({
execution,
result: {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.TASK_ALREADY_RUNNING,
@@ -131,7 +131,7 @@ const zodIpc = new ZodIpcConnection({
execution,
result: {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.COULD_NOT_FIND_EXECUTOR,
@@ -182,7 +182,7 @@ const zodIpc = new ZodIpcConnection({
execution: _execution,
result: {
ok: false,
id: _execution.attempt.id,
id: _execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.GRACEFUL_EXIT_TIMEOUT,
+10
View File
@@ -1,5 +1,15 @@
# create-trigger
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- Updated dependencies [62c9a5b71]
- @trigger.dev/core@3.0.0-beta.15
- @trigger.dev/yalt@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/cli",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "The Trigger.dev CLI",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
+2
View File
@@ -1,5 +1,7 @@
# @trigger.dev/core-apps
## 3.0.0-beta.15
## 3.0.0-beta.14
## 3.0.0-beta.13
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "@trigger.dev/core-apps",
"description": "Backend core code used across apps",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"private": true,
"license": "MIT",
"main": "./dist/index.js",
+2
View File
@@ -1,5 +1,7 @@
# @trigger.dev/core-backend
## 3.0.0-beta.15
## 3.0.0-beta.14
## 3.0.0-beta.13
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/core-backend",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Core code used across `@trigger.dev/sdk` and Trigger.dev server",
"license": "MIT",
"main": "./dist/index.js",
+75
View File
@@ -1,5 +1,80 @@
# internal-platform
## 3.0.0-beta.15
### Patch Changes
- 374edef02: Updates the `trigger`, `batchTrigger` and their `*AndWait` variants to use the first parameter for the payload/items, and the second parameter for options.
Before:
```ts
await yourTask.trigger({ payload: { foo: "bar" }, options: { idempotencyKey: "key_1234" } });
await yourTask.triggerAndWait({
payload: { foo: "bar" },
options: { idempotencyKey: "key_1234" },
});
await yourTask.batchTrigger({
items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }],
});
await yourTask.batchTriggerAndWait({
items: [{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }],
});
```
After:
```ts
await yourTask.trigger({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.triggerAndWait({ foo: "bar" }, { idempotencyKey: "key_1234" });
await yourTask.batchTrigger([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
await yourTask.batchTriggerAndWait([{ payload: { foo: "bar" } }, { payload: { foo: "baz" } }]);
```
We've also changed the API of the `triggerAndWait` result. Before, if the subtask that was triggered finished with an error, we would automatically "rethrow" the error in the parent task.
Now instead we're returning a `TaskRunResult` object that allows you to discriminate between successful and failed runs in the subtask:
Before:
```ts
try {
const result = await yourTask.triggerAndWait({ foo: "bar" });
// result is the output of your task
console.log("result", result);
} catch (error) {
// handle subtask errors here
}
```
After:
```ts
const result = await yourTask.triggerAndWait({ foo: "bar" });
if (result.ok) {
console.log(`Run ${result.id} succeeded with output`, result.output);
} else {
console.log(`Run ${result.id} failed with error`, result.error);
}
```
- 26093896d: When using idempotency keys, triggerAndWait and batchTriggerAndWait will still work even if the existing runs have already been completed (or even partially completed, in the case of batchTriggerAndWait)
- TaskRunExecutionResult.id is now the run friendlyId, not the attempt friendlyId
- A single TaskRun can now have many batchItems, in the case of batchTriggerAndWait while using idempotency keys
- A runs idempotencyKey is now added to the ctx as well as the TaskEvent and displayed in the span view
- When resolving batchTriggerAndWait, the runtimes no longer reject promises, leading to an error in the parent task
- 62c9a5b71: Fixes an issue that caused failed tasks when resuming after calling `triggerAndWait` or `batchTriggerAndWait` in prod/staging (this doesn't effect dev).
The version of Node.js we use for deployed workers (latest 20) would crash with an out-of-memory error when the checkpoint was restored. This crash does not happen on Node 18x or Node21x, so we've decided to upgrade the worker version to Node.js21x, to mitigate this issue.
You'll need to re-deploy to production to fix the issue.
## 3.0.0-beta.14
### Patch Changes
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/core",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Core code used across the Trigger.dev SDK and platform",
"license": "MIT",
"main": "./dist/index.js",
+37
View File
@@ -1,6 +1,7 @@
import { context, propagation } from "@opentelemetry/api";
import { ZodFetchOptions, zodfetch } from "../zodfetch";
import {
BatchTaskRunExecutionResult,
BatchTriggerTaskRequestBody,
BatchTriggerTaskResponse,
CanceledRunResponse,
@@ -11,6 +12,7 @@ import {
ListSchedulesResult,
ReplayRunResponse,
ScheduleObject,
TaskRunExecutionResult,
TriggerTaskRequestBody,
TriggerTaskResponse,
UpdateScheduleOptions,
@@ -18,6 +20,7 @@ import {
import { taskContextManager } from "../tasks/taskContextManager";
import { getEnvVar } from "../utils/getEnv";
import { SafeAsyncLocalStorage } from "../utils/safeAsyncLocalStorage";
import { APIError } from "../apiErrors";
export type TriggerOptions = {
spanParentAsLink?: boolean;
@@ -46,6 +49,40 @@ export class ApiClient {
this.baseUrl = baseUrl.replace(/\/$/, "");
}
async getRunResult(runId: string): Promise<TaskRunExecutionResult | undefined> {
try {
return await zodfetch(
TaskRunExecutionResult,
`${this.baseUrl}/api/v1/runs/${runId}/result`,
{
method: "GET",
headers: this.#getHeaders(false),
},
zodFetchOptions
);
} catch (error) {
if (error instanceof APIError) {
if (error.status === 404) {
return undefined;
}
}
throw error;
}
}
async getBatchResults(batchId: string): Promise<BatchTaskRunExecutionResult | undefined> {
return await zodfetch(
BatchTaskRunExecutionResult,
`${this.baseUrl}/api/v1/batches/${batchId}/results`,
{
method: "GET",
headers: this.#getHeaders(false),
},
zodFetchOptions
);
}
triggerTask(taskId: string, body: TriggerTaskRequestBody, options?: TriggerOptions) {
return zodfetch(
TriggerTaskResponse,
+13 -1
View File
@@ -108,8 +108,20 @@ export class NoopTaskLogger implements TaskLogger {
function safeJsonProcess(value?: Record<string, unknown>): Record<string, unknown> | undefined {
try {
return JSON.parse(JSON.stringify(value));
return JSON.parse(JSON.stringify(value, jsonErrorReplacer));
} catch {
return value;
}
}
function jsonErrorReplacer(key: string, value: unknown) {
if (value instanceof Error) {
return {
name: value.name,
message: value.message,
stack: value.stack,
};
}
return value;
}
@@ -8,10 +8,7 @@ import { RuntimeManager } from "./manager";
import { unboundedTimeout } from "../utils/timers";
export class DevRuntimeManager implements RuntimeManager {
_taskWaits: Map<
string,
{ resolve: (value: TaskRunExecutionResult) => void; reject: (err?: any) => void }
> = new Map();
_taskWaits: Map<string, { resolve: (value: TaskRunExecutionResult) => void }> = new Map();
_batchWaits: Map<
string,
@@ -41,8 +38,8 @@ export class DevRuntimeManager implements RuntimeManager {
return pendingCompletion;
}
const promise = new Promise<TaskRunExecutionResult>((resolve, reject) => {
this._taskWaits.set(params.id, { resolve, reject });
const promise = new Promise<TaskRunExecutionResult>((resolve) => {
this._taskWaits.set(params.id, { resolve });
});
return await promise;
@@ -65,16 +62,12 @@ export class DevRuntimeManager implements RuntimeManager {
if (pendingCompletion) {
this._pendingCompletionNotifications.delete(runId);
if (pendingCompletion.ok) {
resolve(pendingCompletion);
} else {
reject(pendingCompletion);
}
resolve(pendingCompletion);
return;
}
this._taskWaits.set(runId, { resolve, reject });
this._taskWaits.set(runId, { resolve });
});
})
);
@@ -97,11 +90,7 @@ export class DevRuntimeManager implements RuntimeManager {
return;
}
if (completion.ok) {
wait.resolve(completion);
} else {
wait.reject(completion);
}
wait.resolve(completion);
this._taskWaits.delete(execution.run.id);
}
@@ -1,4 +1,3 @@
import { setTimeout } from "node:timers/promises";
import { clock } from "../clock-api";
import {
BatchTaskRunExecutionResult,
@@ -8,19 +7,16 @@ import {
TaskRunExecution,
TaskRunExecutionResult,
} from "../schemas";
import { unboundedTimeout } from "../utils/timers";
import { ZodIpcConnection } from "../zodIpc";
import { RuntimeManager } from "./manager";
import { unboundedTimeout } from "../utils/timers";
export type ProdRuntimeManagerOptions = {
waitThresholdInMs?: number;
};
export class ProdRuntimeManager implements RuntimeManager {
_taskWaits: Map<
string,
{ resolve: (value: TaskRunExecutionResult) => void; reject: (err?: any) => void }
> = new Map();
_taskWaits: Map<string, { resolve: (value: TaskRunExecutionResult) => void }> = new Map();
_batchWaits: Map<
string,
@@ -91,15 +87,19 @@ export class ProdRuntimeManager implements RuntimeManager {
}
async waitForTask(params: { id: string; ctx: TaskRunContext }): Promise<TaskRunExecutionResult> {
const promise = new Promise<TaskRunExecutionResult>((resolve, reject) => {
this._taskWaits.set(params.id, { resolve, reject });
const promise = new Promise<TaskRunExecutionResult>((resolve) => {
this._taskWaits.set(params.id, { resolve });
});
await this.ipc.send("WAIT_FOR_TASK", {
friendlyId: params.id,
});
return await promise;
const result = await promise;
clock.reset();
return result;
}
async waitForBatch(params: {
@@ -114,7 +114,7 @@ export class ProdRuntimeManager implements RuntimeManager {
const promise = Promise.all(
params.runs.map((runId) => {
return new Promise<TaskRunExecutionResult>((resolve, reject) => {
this._taskWaits.set(runId, { resolve, reject });
this._taskWaits.set(runId, { resolve });
});
})
);
@@ -126,6 +126,8 @@ export class ProdRuntimeManager implements RuntimeManager {
const results = await promise;
clock.reset();
return {
id: params.id,
items: results,
@@ -139,11 +141,7 @@ export class ProdRuntimeManager implements RuntimeManager {
return;
}
if (completion.ok) {
wait.resolve(completion);
} else {
wait.reject(completion);
}
wait.resolve(completion);
this._taskWaits.delete(execution.run.id);
}
+1
View File
@@ -74,6 +74,7 @@ export const TaskRun = z.object({
tags: z.array(z.string()),
isTest: z.boolean().default(false),
createdAt: z.coerce.date(),
idempotencyKey: z.string().optional(),
});
export type TaskRun = z.infer<typeof TaskRun>;
@@ -41,4 +41,5 @@ export const SemanticInternalAttributes = {
RETRY_DELAY: "retry.delay",
RETRY_COUNT: "retry.count",
LINK_TITLE: "$link.title",
IDEMPOTENCY_KEY: "ctx.run.idempotencyKey",
};
@@ -68,6 +68,7 @@ export class TaskContextManager {
[SemanticInternalAttributes.ORGANIZATION_SLUG]: this.ctx.organization.slug,
[SemanticInternalAttributes.ORGANIZATION_NAME]: this.ctx.organization.name,
[SemanticInternalAttributes.BATCH_ID]: this.ctx.batch?.id,
[SemanticInternalAttributes.IDEMPOTENCY_KEY]: this.ctx.run.idempotencyKey,
};
}
+4 -4
View File
@@ -116,7 +116,7 @@ export class TaskExecutor {
return {
ok: true,
id: execution.attempt.id,
id: execution.run.id,
output: finalOutput.data,
outputType: finalOutput.dataType,
} satisfies TaskRunExecutionResult;
@@ -125,7 +125,7 @@ export class TaskExecutor {
return {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.TASK_OUTPUT_ERROR,
@@ -150,7 +150,7 @@ export class TaskExecutor {
recordSpanException(span, handleErrorResult.error ?? runError);
return {
id: execution.attempt.id,
id: execution.run.id,
ok: false,
error: handleErrorResult.error
? parseError(handleErrorResult.error)
@@ -164,7 +164,7 @@ export class TaskExecutor {
return {
ok: false,
id: execution.attempt.id,
id: execution.run.id,
error: {
type: "INTERNAL_ERROR",
code: TaskRunErrorCodes.HANDLE_ERROR_ERROR,
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "TaskRun" ALTER COLUMN "idempotencyKey" DROP NOT NULL;
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "TaskEvent" ADD COLUMN "idempotencyKey" TEXT;
@@ -0,0 +1,2 @@
-- AlterTable
ALTER TABLE "BatchTaskRun" ALTER COLUMN "idempotencyKey" DROP NOT NULL;
@@ -0,0 +1,2 @@
-- DropIndex
DROP INDEX "BatchTaskRunItem_taskRunId_key";
+6 -4
View File
@@ -1588,7 +1588,7 @@ model TaskRun {
status TaskRunStatus @default(PENDING)
idempotencyKey String
idempotencyKey String?
taskIdentifier String
isTest Boolean @default(false)
@@ -1625,7 +1625,7 @@ model TaskRun {
concurrencyKey String?
batchItem BatchTaskRunItem?
batchItems BatchTaskRunItem[]
dependency TaskRunDependency?
CheckpointRestoreEvent CheckpointRestoreEvent[]
@@ -1816,6 +1816,8 @@ model TaskEvent {
runId String
runIsTest Boolean @default(false)
idempotencyKey String?
taskSlug String
taskPath String?
taskExportName String?
@@ -1913,7 +1915,7 @@ model BatchTaskRun {
status BatchTaskRunStatus @default(PENDING)
idempotencyKey String
idempotencyKey String?
taskIdentifier String
checkpointEvent CheckpointRestoreEvent? @relation(fields: [checkpointEventId], references: [id], onDelete: Cascade, onUpdate: Cascade)
@@ -1948,7 +1950,7 @@ model BatchTaskRunItem {
batchTaskRunId String
taskRun TaskRun @relation(fields: [taskRunId], references: [id], onDelete: Cascade, onUpdate: Cascade)
taskRunId String @unique
taskRunId String
taskRunAttempt TaskRunAttempt? @relation(fields: [taskRunAttemptId], references: [id], onDelete: SetNull, onUpdate: Cascade)
taskRunAttemptId String?
+2
View File
@@ -1,5 +1,7 @@
# @trigger.dev/eslint-plugin
## 3.0.0-beta.15
## 3.0.0-beta.14
## 3.0.0-beta.13
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/eslint-plugin",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "ESLint plugin with trigger.dev best practices",
"keywords": [
"eslint",
+8
View File
@@ -1,5 +1,13 @@
# @trigger.dev/express
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+3 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/express",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "Official Express adapter for Trigger.dev",
"license": "MIT",
"main": "./dist/index.js",
@@ -23,7 +23,7 @@
"./package.json": "./package.json"
},
"devDependencies": {
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15",
"@trigger.dev/tsconfig": "workspace:*",
"@types/debug": "^4.1.7",
"@types/express": "^4.17.13",
@@ -39,7 +39,7 @@
"build:tsup": "tsup"
},
"peerDependencies": {
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14"
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15"
},
"dependencies": {
"debug": "^4.3.4",
+8
View File
@@ -1,5 +1,13 @@
# @trigger.dev/hono
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- @trigger.dev/sdk@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes
+2 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@trigger.dev/hono",
"version": "3.0.0-beta.14",
"version": "3.0.0-beta.15",
"description": "A Trigger.dev adapter for Hono.dev",
"license": "MIT",
"main": "./dist/index.js",
@@ -32,7 +32,7 @@
},
"peerDependencies": {
"hono": "3.x",
"@trigger.dev/sdk": "workspace:^3.0.0-beta.14"
"@trigger.dev/sdk": "workspace:^3.0.0-beta.15"
},
"devDependencies": {
"@trigger.dev/tsconfig": "workspace:*",
+9
View File
@@ -1,5 +1,14 @@
# @trigger.dev/integration-kit
## 3.0.0-beta.15
### Patch Changes
- Updated dependencies [374edef02]
- Updated dependencies [26093896d]
- Updated dependencies [62c9a5b71]
- @trigger.dev/core@3.0.0-beta.15
## 3.0.0-beta.14
### Patch Changes

Some files were not shown because too many files have changed in this diff Show More