044d38e390
* Auto-yield run execution to help prevent duplicate task executions * Add auto-yield config to endpoints * Refactor run execution with buffer and limits Introduced constants RUN_CHUNK_EXECUTION_BUFFER and MAX_RUN_CHUNK_EXECUTION_LIMIT. Adjusted PerformRunExecutionV2Service to use the new constants to fine-tune execution timings and buffers. * Add endpoint probing functionality Added new `RESPONSE_TIMEOUT_STATUS_CODES` in `consts.ts` to manage timeout responses. Additional functions `detectResponseIsTimeout(response: Response)` was added in `endpoint.server.ts` to detect if a response was a timeout based on the status codes from `RESPONSE_TIMEOUT_STATUS_CODES`. Update actions to use new endpoint probing endpoint service. This allows for the early probing of endpoints to determine if they're up and running. A new class `ProbeEndpointService` was created in `probeEndpoint.server.ts` which makes HTTP requests to a given endpoint and updates its properties based on the result. Finally, `detectResponseIsTimeout(response)` is used in `performRunExecutionV2.server.ts` for marking the execution as succeeded when facing a timeout. * Refactored probe method in EndpointApi class The probe method of the EndpointApi class has been refactored to remove the error handling part and it now takes a timeout sent from the client directly. The corresponding changes were also made in the ProbeEndpointService and TriggerClient objects to reflect the alterations in the probe method. The error handling related to the timeout has been removed and the responsibility of handling the timeout has been shifted to the client. Thus, the probe method has been greatly simplified. The `probeEndpoint.server.ts` file was also changed to accommodate the change in behavior of the probe result. In the `triggerClient.ts` the timeout for probe is now read from the incoming request object. For backward compatibility, if no timeout is provided in the request, the default value of 15 minutes is used. * Remove performRunExecution v1 enqueue function * Better document limits and add docs on increasing function timeouts * Upgrade webapp docker container to use 18.18.2 * force clients to yield when a run is executing in a gracefully shutting down worker * Renamed task `key` to `cacheKey` and added more task documentation * Index the `@trigger.dev/sdk` version on Endpoints
322 lines
10 KiB
Plaintext
322 lines
10 KiB
Plaintext
---
|
||
title: "Tasks"
|
||
description: "Tasks are individual building blocks of a Run."
|
||
---
|
||
|
||
A [Task](/documentation/concepts/tasks) is a cached unit of work in a Job Run that are logged to the Trigger.dev UI.
|
||
|
||
<Warning>
|
||
Any interaction with an external service (database or API) should be wrapped in a Task. Failing to
|
||
do so could result in repeated work when runs are resumed.
|
||
</Warning>
|
||
|
||
## Why do you need tasks?
|
||
|
||
Tasks are a key building block of how Trigger.dev works, and failing to use them will result in unpredictable results. Tasks allow bits of work inside a Job Run to be cached and the results of those tasks to be reused.
|
||
|
||
This is **very important** because for a Job Run to be resumable (e.g. after a serverless function timeout, or because of a call to `io.wait()`), we need to call the `Job.run` function multiple times. If we didn't cache the results of Tasks, then we would be repeating work on each run.
|
||
|
||
```ts
|
||
client.defineJob({
|
||
id: "new-user",
|
||
name: "Run when a new user signs up",
|
||
version: "1.0.0",
|
||
trigger: eventTrigger({
|
||
name: "new.user",
|
||
schema: z.object({
|
||
userId: z.string(),
|
||
}),
|
||
}),
|
||
integrations: {
|
||
resend,
|
||
},
|
||
run: async (payload, io, ctx) => {
|
||
// This code will run twice. Once when the run first starts, and once after the wait
|
||
const user = await prisma.user.findUniqueOrThrow({
|
||
where: { id: payload.userId },
|
||
select: { email: true, name: true },
|
||
});
|
||
|
||
// This code will run once, because the resend integration creates a task with the "welcome-email" cacheKey
|
||
await io.resend.sendEmail("welcome-email", {
|
||
to: user.email,
|
||
from: "jane@acme.inc",
|
||
subject: "Welcome!",
|
||
html: welcomeEmail(user.name),
|
||
});
|
||
|
||
// This code will run once, because io.wait creates a task with the "wait" cacheKey
|
||
await io.wait("wait", 60 * 60 * 3); // wait for 3 hours
|
||
|
||
// This code will run once, because we're manually creating a task with the "my-task" cacheKey
|
||
const response = await io.runTask(
|
||
"my-task",
|
||
async () => {
|
||
return await longRunningCode(payload.userId);
|
||
},
|
||
{ name: "My Task" }
|
||
);
|
||
|
||
return response;
|
||
},
|
||
});
|
||
```
|
||
|
||
As well as powering the resumable nature of Trigger.dev, Tasks also provide:
|
||
|
||
- **Retryable** – If a Task fails, it can be retried. You can configure how (or if) a Task is retried. Full details in the [io.runTask() SDK reference](/sdk/io/runtask).
|
||
- **Logging** – Tasks are logged, so you can see what happened in a Run. Find out more about [viewing runs](/documentation/guides/viewing-runs).
|
||
|
||
## Task Cache Keys
|
||
|
||
The first param of all Tasks is a `cacheKey`. This is a unique identifier for the Task inside that Run. It is used for storing the cached result of a task. It is also used to identify the Task in the [Viewing Runs Dashboard](/documentation/guides/viewing-runs).
|
||
|
||
<Warning>It's important that cacheKey's are unique inside an individual Job Run.</Warning>
|
||
|
||
## Creating Tasks
|
||
|
||
There are **3** ways of using tasks in your code:
|
||
|
||
- Using the [io.runTask()](/sdk/io/runtask) function
|
||
- Using one of our [Integration packages](/documentation/concepts/integrations) and calling the `io.integration.runTask()` wrapper
|
||
- Using one of our [Integration packages](/documentation/concepts/integrations) and calling a task wrapper function, such as [io.slack.postMessage()](/integrations/apis/slack)
|
||
|
||
### Using `io.runTask()`
|
||
|
||
The `io.runTask()` function allows you to run a Task manually. It takes a `cacheKey` and a function to run. The function will only be run if the Task is not already cached.
|
||
|
||
```ts
|
||
const response = await io.runTask("my-task", async (task) => {
|
||
return await longRunningCode(payload.userId);
|
||
});
|
||
```
|
||
|
||
The callback function is passed a `task` object, which can be useful for providing an idempotency key to an external service. For example, Stripe:
|
||
|
||
<Info>
|
||
Our [Stripe Integration](integrations/apis/stripe) handles this for you automatically, this is
|
||
just for documentation purposes
|
||
</Info>
|
||
|
||
```ts
|
||
const stripe = new Stripe(process.env.STRIPE_SECRET_KEY, {
|
||
apiVersion: "2020-08-27",
|
||
});
|
||
|
||
await io.runTask("create-customer", async (task) => {
|
||
await stripe.customers.create(
|
||
{
|
||
email: "eric@trigger.dev",
|
||
},
|
||
{
|
||
idempotencyKey: task.idempotencyKey,
|
||
}
|
||
);
|
||
});
|
||
```
|
||
|
||
`runTask` also takes an optional 3rd argument, which allows you to customize how the Task is displayed and run. For example, you can supply a name and some properties to be displayed in the Viewing Runs Dashboard:
|
||
|
||
```ts
|
||
const response = await io.runTask(
|
||
"my-task",
|
||
async (task) => {
|
||
return await longRunningCode(payload.userId);
|
||
},
|
||
{
|
||
name: "My Task",
|
||
properties: [
|
||
{
|
||
label: "User ID",
|
||
value: payload.userId,
|
||
},
|
||
],
|
||
icon: "user",
|
||
}
|
||
);
|
||
```
|
||
|
||
See the [io.runTask() SDK reference](/sdk/io/runtask) for more information.
|
||
|
||
### Using `io.integration.runTask()`
|
||
|
||
All of our [Integration packages](/documentation/concepts/integrations) expose a `runTask()` function. The main differences between this and `io.runTask()` are:
|
||
|
||
- Adds an additional callback parameter which provides the underlying authenticated integration client
|
||
- Automatically sets the `icon` property on the Task.
|
||
- Configures sensible defaults for retries and error handling.
|
||
|
||
An example here demonstrates using the GitHub integration's `runTask` function to create a project card when a new user signs up:
|
||
|
||
```ts
|
||
import { Github } from "@trigger.dev/github";
|
||
|
||
const github = new Github({
|
||
id: "github",
|
||
});
|
||
|
||
client.defineJob({
|
||
id: "create-project-card",
|
||
name: "Create Project Card",
|
||
version: "1.0.0",
|
||
trigger: eventTrigger({
|
||
name: "new.user",
|
||
}),
|
||
integrations: {
|
||
github,
|
||
},
|
||
run: async (payload, io, ctx) => {
|
||
await io.github.runTask(
|
||
"create-card",
|
||
async (client, task) => {
|
||
// client is an authenticated GitHub client (https://github.com/octokit/octokit.js)
|
||
return client.rest.projects.createCard({
|
||
column_id: process.env.GITHUB_PROJECT_COLUMN_ID,
|
||
note: `New User ${payload.user.name} signed up!`,
|
||
});
|
||
},
|
||
{ name: "Create card" }
|
||
);
|
||
},
|
||
});
|
||
```
|
||
|
||
### Using an Integration Task Wrapper Function
|
||
|
||
Our [Integration packages](/documentation/concepts/integrations) also expose a number of task wrapper functions. These are functions that wrap a common task for that integration. For example, the [Slack integration](/integrations/apis/slack) exposes a `postMessage()` function:
|
||
|
||
```ts
|
||
import { Slack } from "@trigger.dev/slack";
|
||
|
||
const slack = new Slack({
|
||
id: "slack",
|
||
});
|
||
|
||
client.defineJob({
|
||
id: "send-welcome-message",
|
||
name: "Send welcome message",
|
||
version: "1.0.0",
|
||
trigger: eventTrigger({
|
||
name: "new.user",
|
||
}),
|
||
integrations: {
|
||
slack,
|
||
},
|
||
run: async (payload, io, ctx) => {
|
||
await io.slack.postMessage("send-message", {
|
||
channel: process.env.SLACK_CHANNEL_ID,
|
||
text: `New user ${payload.user.name} signed up!`,
|
||
});
|
||
},
|
||
});
|
||
```
|
||
|
||
All task wrapper functions take a `cacheKey` as the first argument, because they are Tasks under the hood. Think of them as a convenience wrapper around `io.runTask()`.
|
||
|
||
We strive to document all of the task wrapper functions in our [Integration packages](/documentation/concepts/integrations). For example, checkout our [GitHub integration task](/integrations/apis/github-tasks) docs.
|
||
|
||
## Subtasks
|
||
|
||
You can break up a task into multiple subtasks. This is useful for breaking up a long-running task into smaller chunks, while consolidating the logging into a single task in the dashboard with children.
|
||
|
||
<Info>We currently support nesting up to 5 levels</Info>
|
||
|
||
```ts
|
||
const response = await io.runTask("parent-task", async (task) => {
|
||
await io.runTask("child-1", async () => {
|
||
// do something
|
||
});
|
||
|
||
await io.runTask("child-2", async () => {
|
||
// do something
|
||
});
|
||
});
|
||
```
|
||
|
||
Task cacheKey's are automatically scoped to the parent task. So for example, you can reuse a cacheKey inside a parent task and it will not conflict with another top-level task.
|
||
|
||
```ts
|
||
const response = await io.runTask("parent-task", async (task) => {
|
||
await io.runTask("child-1", async () => {
|
||
// do something
|
||
});
|
||
|
||
await io.runTask("child-2", async () => {
|
||
// do something
|
||
});
|
||
});
|
||
|
||
// This will not conflict with the child-1 task above
|
||
const response = await io.runTask("child-1", async (task) => {
|
||
// do something
|
||
});
|
||
```
|
||
|
||
### Extracting Common Tasks
|
||
|
||
Subtasks allow you to DRY up any repeating task code into a single function. For example, if you have a common task that sends a welcome email, you can extract that into a function:
|
||
|
||
```ts
|
||
const sendWelcomeEmail = async (cacheKey: string, io: IO, resend: Resend, userId: string) => {
|
||
return await io.runTask(cacheKey, async () => {
|
||
const user = await io.runTask("fetch-user", async () => {
|
||
return prisma.user.findUniqueOrThrow({
|
||
where: { id: userId },
|
||
select: { email: true, name: true },
|
||
});
|
||
});
|
||
|
||
await io.resend.sendEmail("📧", {
|
||
to: user.email,
|
||
from: "eric@trigger.dev",
|
||
subject: "Welcome!",
|
||
html: welcomeEmail(user.name),
|
||
});
|
||
});
|
||
};
|
||
|
||
client.defineJob({
|
||
id: "new-user",
|
||
name: "Run when a new user signs up",
|
||
version: "1.0.0",
|
||
trigger: eventTrigger({
|
||
name: "new.user",
|
||
schema: z.object({
|
||
userId: z.string(),
|
||
}),
|
||
}),
|
||
integrations: {
|
||
resend,
|
||
},
|
||
run: async (payload, io, ctx) => {
|
||
await sendWelcomeEmail("🫡", io, io.resend, payload.userId);
|
||
},
|
||
});
|
||
```
|
||
|
||
<Note>
|
||
Always make sure you are allow passing a unique cacheKey to the `runTask` function, so the tasks
|
||
inside the function are not accidentally reused.
|
||
</Note>
|
||
|
||
## Limitations
|
||
|
||
A single task has an upper-bound on it's execution duration, which must be less than the serverless function execution timeout of your deployed platform. For more information see our [Limits docs](/documentation/concepts/limits#maximum-task-duration)
|
||
|
||
## References
|
||
|
||
<CardGroup cols={2}>
|
||
<Card title="Resumability" icon="clock" href="/documentation/concepts/resumability">
|
||
Runs can be very long-running. Learn how we handle this.
|
||
</Card>
|
||
<Card title="Integrations" icon="grid-2" href="/documentation/concepts/integrations">
|
||
Integrations utilize Tasks.
|
||
</Card>
|
||
<Card title="`io` SDK Reference" icon="wrench" href="/sdk/io">
|
||
The `io` object allows you to easily run a Task yourself.
|
||
</Card>
|
||
<Card title="Viewing Runs Dashboard" icon="globe" href="/documentation/guides/viewing-runs">
|
||
View all Runs for a Job, all the way down to individual Tasks.
|
||
</Card>
|
||
</CardGroup>
|