github-featured/triggerdotdev--trigger.dev
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

Docs: Batch Trigger upgrades (#1505)

Browse Source 
* WIP batch docs

* More batch trigger v2 docs
This commit is contained in:
Eric Allam
2024-11-28 15:55:45 +00:00
committed by GitHub
parent 0acd052061
commit f4a18feca0
8 changed files with 635 additions and 208 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/guides/examples/scrape-hacker-news.mdx
+14 -23
View File
@@ -8,7 +8,16 @@ import LocalDevelopment from "/snippets/local-development-extensions.mdx";
import ScrapingWarning from "/snippets/web-scraping-warning.mdx";
<div className="w-full h-full aspect-video">
<iframe width="100%" height="100%" src="https://www.youtube.com/embed/6azvzrZITKY?si=muKtsBiS9TJGGKWg" title="YouTube video player" frameborder="0" allow="accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen/>
<iframe
width="100%"
height="100%"
src="https://www.youtube.com/embed/6azvzrZITKY?si=muKtsBiS9TJGGKWg"
title="YouTube video player"
frameborder="0"
allow="accelerometer; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
referrerpolicy="strict-origin-when-cross-origin"
allowfullscreen
/>
</div>
## Overview
@@ -125,12 +134,9 @@ export const summarizeHackerNews = schedules.task({
.batchTriggerAndWait(
articles.map((article) => ({
payload: { title: article.title!, link: article.link! },
idempotencyKey: article.link,
}))
)
.then((batch) =>
batch.runs.filter((run) => run.ok).map((run) => run.output)
);
.then((batch) => batch.runs.filter((run) => run.ok).map((run) => run.output));
// Send email using Resend
await resend.emails.send({
@@ -165,11 +171,7 @@ export const scrapeAndSummarizeArticle = task({
// Prevent all assets from loading, images, stylesheets etc
await page.setRequestInterception(true);
page.on("request", (request) => {
if (
["script", "stylesheet", "image", "media", "font"].includes(
request.resourceType()
)
) {
if (["script", "stylesheet", "image", "media", "font"].includes(request.resourceType())) {
request.abort();
} else {
request.continue();
@@ -218,16 +220,7 @@ To prevent the main example from becoming too cluttered, we'll create a separate
Notice how this file is imported into the main task code and passed to Resend to send the email.
```tsx summarize-hn-email.tsx
import {
Html,
Head,
Body,
Container,
Section,
Heading,
Text,
Link,
} from "@react-email/components";
import { Html, Head, Body, Container, Section, Heading, Text, Link } from "@react-email/components";
interface Article {
title: string;
@@ -235,9 +228,7 @@ interface Article {
summary: string | null;
}
export const HNSummaryEmail: React.FC<{ articles: Article[] }> = ({
articles,
}) => (
export const HNSummaryEmail: React.FC<{ articles: Article[] }> = ({ articles }) => (
<Html>
<Head />
<Body style={{ fontFamily: "Arial, sans-serif", padding: "20px" }}>
docs/idempotency.mdx
+53 -13
View File
@@ -5,11 +5,17 @@ description: "An API call or operation is “idempotent” if it has the same re
We currently support idempotency at the task level, meaning that if you trigger a task with the same `idempotencyKey` twice, the second request will not create a new task run.
<Warning>
In version 3.3.0 and later, the `idempotencyKey` option is not available when using
`triggerAndWait` or `batchTriggerAndWait`, due to a bug that would sometimes cause the parent task
to become stuck. We are working on a fix for this issue.
</Warning>
## `idempotencyKey` option
You can provide an `idempotencyKey` to ensure that a task is only triggered once with the same key. This is useful if you are triggering a task within another task that might be retried:
```typescript
```ts
import { idempotencyKeys, task } from "@trigger.dev/sdk/v3";
export const myTask = task({
@@ -18,13 +24,14 @@ export const myTask = task({
maxAttempts: 4,
},
run: async (payload: any) => {
// By default, idempotency keys generated are unique to the run, to prevent retries from duplicating child tasks
// This idempotency key will be unique to this task run, meaning the childTask will only be triggered once across all retries
const idempotencyKey = await idempotencyKeys.create("my-task-key");
// childTask will only be triggered once with the same idempotency key
await childTask.triggerAndWait(payload, { idempotencyKey });
await childTask.trigger({ foo: "bar" }, { idempotencyKey });
// Do something else, that may throw an error and cause the task to be retried
throw new Error("Something went wrong");
},
});
```
@@ -33,7 +40,7 @@ You can use the `idempotencyKeys.create` SDK function to create an idempotency k
We automatically inject the run ID when generating the idempotency key when running inside a task by default. You can turn it off by passing the `scope` option to `idempotencyKeys.create`:
```typescript
```ts
import { idempotencyKeys, task } from "@trigger.dev/sdk/v3";
export const myTask = task({
@@ -42,21 +49,18 @@ export const myTask = task({
maxAttempts: 4,
},
run: async (payload: any) => {
// This idempotency key will be the same for all runs of this task
// This idempotency key will be globally unique, meaning only a single task run will be triggered with this key
const idempotencyKey = await idempotencyKeys.create("my-task-key", { scope: "global" });
// childTask will only be triggered once with the same idempotency key
await childTask.triggerAndWait(payload, { idempotencyKey });
// This is the same as the above
await childTask.triggerAndWait(payload, { idempotencyKey: "my-task-key" });
await childTask.trigger({ foo: "bar" }, { idempotencyKey });
},
});
```
If you are triggering a task from your backend code, you can use the `idempotencyKeys.create` SDK function to create an idempotency key.
```typescript
```ts
import { idempotencyKeys, tasks } from "@trigger.dev/sdk/v3";
// You can also pass an array of strings to create a idempotency key
@@ -66,7 +70,7 @@ await tasks.trigger("my-task", { some: "data" }, { idempotencyKey });
You can also pass a string to the `idempotencyKey` option, without first creating it with `idempotencyKeys.create`.
```typescript
```ts
import { myTask } from "./trigger/myTasks";
// You can also pass an array of strings to create a idempotency key
@@ -77,7 +81,7 @@ await myTask.trigger({ some: "data" }, { idempotencyKey: myUser.id });
You can pass the `idempotencyKey` when calling `batchTrigger` as well:
```typescript
```ts
import { tasks } from "@trigger.dev/sdk/v3";
await tasks.batchTrigger("my-task", [
@@ -88,11 +92,47 @@ await tasks.batchTrigger("my-task", [
]);
```
## `idempotencyKeyTTL` option
By default idempotency keys are stored for 30 days. You can change this by passing the `idempotencyKeyTTL` option when triggering a task:
```ts
import { idempotencyKeys, task, wait } from "@trigger.dev/sdk/v3";
export const myTask = task({
id: "my-task",
retry: {
maxAttempts: 4,
},
run: async (payload: any) => {
const idempotencyKey = await idempotencyKeys.create("my-task-key");
// The idempotency key will expire after 60 seconds
await childTask.trigger({ foo: "bar" }, { idempotencyKey, idempotencyKeyTTL: "60s" });
await wait.for({ seconds: 61 });
// The idempotency key will have expired, so the childTask will be triggered again
await childTask.trigger({ foo: "bar" }, { idempotencyKey });
// Do something else, that may throw an error and cause the task to be retried
throw new Error("Something went wrong");
},
});
```
You can use the following units for the `idempotencyKeyTTL` option:
- `s` for seconds (e.g. `60s`)
- `m` for minutes (e.g. `5m`)
- `h` for hours (e.g. `2h`)
- `d` for days (e.g. `3d`)
## Payload-based idempotency
We don't currently support payload-based idempotency, but you can implement it yourself by hashing the payload and using the hash as the idempotency key.
```typescript
```ts
import { idempotencyKeys, task } from "@trigger.dev/sdk/v3";
import { createHash } from "node:crypto";
docs/limits.mdx
+48 -39
View File
@@ -3,15 +3,15 @@ title: "Limits"
description: "There are some hard and soft limits that you might hit."
---
import RateLimitHitUseBatchTrigger from '/snippets/rate-limit-hit-use-batchtrigger.mdx';
import RateLimitHitUseBatchTrigger from "/snippets/rate-limit-hit-use-batchtrigger.mdx";
## Concurrency limits
| Pricing tier | Limit |
|:---------------- |:-------------------- |
| Free | 5 concurrent runs |
| Hobby | 25 concurrent runs |
| Pro | 100+ concurrent runs |
| Pricing tier | Limit |
| :----------- | :------------------- |
| Free | 5 concurrent runs |
| Hobby | 25 concurrent runs |
| Pro | 100+ concurrent runs |
If you need more than 100 concurrent runs on the Pro tier, you can request more by contacting us via [email](https://trigger.dev/contact) or [Discord](https://trigger.dev/discord).
@@ -20,28 +20,28 @@ If you need more than 100 concurrent runs on the Pro tier, you can request more
Generally speaking each SDK call is an API call.
| Limit | Details |
|:----- |:------------------------- |
| :---- | :------------------------ |
| API | 1,500 requests per minute |
<RateLimitHitUseBatchTrigger/>
<RateLimitHitUseBatchTrigger />
## Queued tasks
The number of queued tasks by environment.
| Limit | Details |
|:------- |:------------------ |
| :------ | :----------------- |
| Dev | At most 500 |
| Staging | At most 10 million |
| Prod | At most 10 million |
## Schedules
| Pricing tier | Limit |
|:---------------- |:-------------------- |
| Free | 5 per project |
| Hobby | 100 per project |
| Pro | 1,000+ per project |
| Pricing tier | Limit |
| :----------- | :----------------- |
| Free | 5 per project |
| Hobby | 100 per project |
| Pro | 1,000+ per project |
When attaching schedules to tasks we strongly recommend you add them [in our dashboard](/tasks/scheduled#attaching-schedules-in-the-dashboard) if they're "static". That way you can control them easily per environment.
@@ -49,15 +49,29 @@ If you add them [dynamically using code](/management/schedules/create) make sure
If you're creating schedules for your user you will definitely need to request more schedules from us.
## Task payloads and outputs
| Limit | Details |
| :--------------------- | :-------------------------------------------- |
| Single trigger payload | Must not exceed 3MB |
| Batch trigger payload | The total of all payloads must not exceed 1MB |
| Task outputs | Must not exceed 10MB |
Payloads and outputs that exceed 512KB will be offloaded to object storage and a presigned URL will be provided to download the data when calling `runs.retrieve`. You don't need to do anything to handle this in your tasks however, as we will transparently upload/download these during operation.
## Batch size
A single batch can have a maximum of 500 items.
<SoftLimit />
## Log retention
| Pricing tier | Limit |
|:---------------- |:--------- |
| Free | 1 day |
| Hobby | 7 days |
| Pro | 30 days |
| Pricing tier | Limit |
| :----------- | :------ |
| Free | 1 day |
| Hobby | 7 days |
| Pro | 30 days |
## Log size
@@ -66,25 +80,30 @@ We limit the size of logs to prevent oversized data potentially causing issues.
<Expandable title="log limits">
#### Attribute Limits
- Span Attribute Count Limit: 256
- Log Attribute Count Limit: 256
- Span Attribute Value Length Limit: 1028 characters
- Log Attribute Value Length Limit: 1028 characters
#### Event and Link Limits
- Span Event Count Limit: 10
- Link Count Limit: 2
- Attributes per Link Limit: 10
- Attributes per Event Limit: 10
#### I/O Packet Length Limit
128 KB (131,072 bytes)
#### Attribute Clipping Behavior
- Attributes exceeding the value length limit (1028 characters) are discarded.
- If the total number of attributes exceeds 256, additional attributes are not included.
#### Attribute Value Size Calculation
- Strings: Actual length of the string
- Numbers: 8 bytes
- Booleans: 4 bytes
@@ -93,25 +112,15 @@ We limit the size of logs to prevent oversized data potentially causing issues.
</Expandable>
## Task payloads and outputs
| Limit | Details |
|:--- |:--- |
| Single trigger payload | Must not exceed 10MB |
| Batch trigger payload | The total of all payloads must not exceed 10MB |
| Task outputs | Must not exceed 10MB |
Payloads and outputs that exceed 512KB will be offloaded to object storage and a presigned URL will be provided to download the data when calling `runs.retrieve`. You don't need to do anything to handle this in your tasks however, as we will transparently upload/download these during operation.
## Alerts
An alert destination is a single email address, Slack channel, or webhook URL that you want to send alerts to. If you're on the Pro and need more than 100 alert destinations, you can request more by contacting us via [email](https://trigger.dev/contact) or [Discord](https://trigger.dev/discord).
| Pricing tier | Limit |
|:---------------- |:----------------------- |
| Free | 1 alert destination |
| Hobby | 3 alert destinations |
| Pro | 100+ alert destinations |
| Pricing tier | Limit |
| :----------- | :---------------------- |
| Free | 1 alert destination |
| Hobby | 3 alert destinations |
| Pro | 100+ alert destinations |
## Machines
@@ -121,8 +130,8 @@ See the [machine configurations](/machines#machine-configurations) for more deta
## Team members
| Pricing tier | Limit |
|:---------------- |:----------------- |
| Free | 5 team members |
| Hobby | 5 team members |
| Pro | 25+ team members |
| Pricing tier | Limit |
| :----------- | :--------------- |
| Free | 5 team members |
| Hobby | 5 team members |
| Pro | 25+ team members |
docs/mint.json
+2 -1
View File
@@ -217,7 +217,8 @@
"realtime/streams",
"realtime/react-hooks",
"realtime/subscribe-to-run",
"realtime/subscribe-to-runs-with-tag"
"realtime/subscribe-to-runs-with-tag",
"realtime/subscribe-to-batch"
]
},
{
docs/realtime/subscribe-to-batch.mdx
+49
View File
@@ -0,0 +1,49 @@
---
title: runs.subscribeToBatch
sidebarTitle: subscribeToBatch
description: Subscribes to all changes for runs in a batch.
---
import RunObject from "/snippets/realtime/run-object.mdx";
<RequestExample>
```ts Example
import { runs } from "@trigger.dev/sdk/v3";
for await (const run of runs.subscribeToBatch("batch_1234")) {
console.log(run);
}
```
</RequestExample>
This function subscribes to all changes for runs in a batch. It returns an async iterator that yields the a run object whenever a run in the batch is updated. The iterator does not complete on it's own, you must manually `break` the loop when you want to stop listening for updates.
### Authentication
This function supports both server-side and client-side authentication. For server-side authentication, use your API key. For client-side authentication, you must generate a public access token with one of the following scopes:
- `read:batch:<batchId>`
- `read:runs` will provide access to all runs (not recommended for production use)
To generate a public access token, use the `auth.createPublicToken` function:
```ts
import { auth } from "@trigger.dev/sdk/v3";
// Somewhere in your backend code
const publicToken = await auth.createPublicToken({
scopes: {
read: {
batch: ["batch_1234"],
},
},
});
```
### Response
The AsyncIterator yields an object with the following properties:
<RunObject />
docs/runs.mdx
+128 -20
View File
@@ -98,19 +98,21 @@ At this point, the run will have either an output (if successful) or an error (i
When triggering a task, you can provide an idempotency key to ensure the task is executed only once, even if triggered multiple times. This is useful for preventing duplicate executions in distributed systems.
```javascript
yourTask.trigger({ foo: "bar" }, { idempotencyKey: "unique-key" });
```ts
await yourTask.trigger({ foo: "bar" }, { idempotencyKey: "unique-key" });
```
- If a run with the same idempotency key is already in progress, the new trigger will be ignored.
- If the run has already finished, the previous output or error will be returned.
See our [Idempotency docs](/idempotency) for more information.
### Canceling runs
You can cancel an in-progress run using the API or the dashboard:
```ts
runs.cancel(runId);
await runs.cancel(runId);
```
When a run is canceled:
@@ -128,7 +130,7 @@ When a run is canceled:
You can set a TTL when triggering a run:
```ts
yourTask.trigger({ foo: "bar" }, { ttl: "10m" });
await yourTask.trigger({ foo: "bar" }, { ttl: "10m" });
```
If the run hasn't started within the specified TTL, it will automatically expire. This is useful for time-sensitive tasks. Note that dev runs automatically have a 10-minute TTL.
@@ -140,7 +142,7 @@ If the run hasn't started within the specified TTL, it will automatically expire
You can schedule a run to start after a specified delay:
```ts
yourTask.trigger({ foo: "bar" }, { delay: "1h" });
await yourTask.trigger({ foo: "bar" }, { delay: "1h" });
```
This is useful for tasks that need to be executed at a specific time in the future.
@@ -152,7 +154,7 @@ This is useful for tasks that need to be executed at a specific time in the futu
You can create a new run with the same payload as a previous run:
```ts
runs.replay(runId);
await runs.replay(runId);
```
This is useful for re-running a task with the same input, especially for debugging or recovering from failures. The new run will use the latest version of the task.
@@ -175,37 +177,143 @@ Similar to `triggerAndWait()`, the `batchTriggerAndWait()` function lets you bat
### Runs API
The runs API provides methods to interact with and manage runs:
#### runs.list()
List runs in a specific environment. You can filter the runs by status, created at, task identifier, version, and more:
```ts
// List all runs
runs.list();
import { runs } from "@trigger.dev/sdk/v3";
// Get a specific run by ID
runs.retrieve(runId);
// Get the first page of runs, returning up to 20 runs
let page = await runs.list({ limit: 20 });
// Replay a run
runs.replay(runId);
for (const run of page.data) {
console.log(run);
}
// Reschedule a run
runs.reschedule(runId, delay);
// Cancel a run
runs.cancel(runId);
// Keep getting the next page until there are no more runs
while (page.hasNextPage()) {
page = await page.getNextPage();
// Do something with the next page of runs
}
```
These methods allow you to access detailed information about runs and their attempts, including payloads, outputs, parent runs, and child runs.
You can also use an Async Iterator to get all runs:
```ts
import { runs } from "@trigger.dev/sdk/v3";
for await (const run of runs.list({ limit: 20 })) {
console.log(run);
}
```
You can provide multiple filters to the `list()` function to narrow down the results:
```ts
import { runs } from "@trigger.dev/sdk/v3";
const response = await runs.list({
status: ["QUEUED", "EXECUTING"], // Filter by status
taskIdentifier: ["my-task", "my-other-task"], // Filter by task identifier
from: new Date("2024-04-01T00:00:00Z"), // Filter by created at
to: new Date(),
version: "20241127.2", // Filter by deployment version,
tag: ["tag1", "tag2"], // Filter by tags
batch: "batch_1234", // Filter by batch ID
schedule: "sched_1234", // Filter by schedule ID
});
```
#### runs.retrieve()
Fetch a single run by it's ID:
```ts
import { runs } from "@trigger.dev/sdk/v3";
const run = await runs.retrieve(runId);
```
You can provide the type of the task to correctly type the `run.payload` and `run.output`:
```ts
import { runs } from "@trigger.dev/sdk/v3";
import type { myTask } from "./trigger/myTask";
const run = await runs.retrieve<typeof myTask>(runId);
console.log(run.payload.foo); // string
console.log(run.output.bar); // string
```
If you have just triggered a run, you can pass the entire response object to `retrieve()` and the response will already be typed:
```ts
import { runs, tasks } from "@trigger.dev/sdk/v3";
import type { myTask } from "./trigger/myTask";
const response = await tasks.trigger<typeof myTask>({ foo: "bar" });
const run = await runs.retrieve(response);
console.log(run.payload.foo); // string
console.log(run.output.bar); // string
```
#### runs.cancel()
Cancel a run:
```ts
import { runs } from "@trigger.dev/sdk/v3";
await runs.cancel(runId);
```
#### runs.replay()
Replay a run:
```ts
import { runs } from "@trigger.dev/sdk/v3";
await runs.replay(runId);
```
#### runs.reschedule()
Updates a delayed run with a new delay. Only valid when the run is in the DELAYED state.
```ts
import { runs } from "@trigger.dev/sdk/v3";
await runs.reschedule(runId, { delay: "1h" });
```
### Real-time updates
You can subscribe to run updates in real-time using the `subscribeToRun()` function:
Subscribe to changes to a specific run in real-time:
```ts
import { runs } from "@trigger.dev/sdk/v3";
for await (const run of runs.subscribeToRun(runId)) {
console.log(run);
}
```
Similar to `runs.retrieve()`, you can provide the type of the task to correctly type the `run.payload` and `run.output`:
```ts
import { runs } from "@trigger.dev/sdk/v3";
import type { myTask } from "./trigger/myTask";
for await (const run of runs.subscribeToRun<typeof myTask>(runId)) {
console.log(run.payload.foo); // string
console.log(run.output?.bar); // string | undefined
}
```
For more on real-time updates, see the [Realtime](/realtime) documentation.
### Triggering runs for undeployed tasks
docs/triggering.mdx
+337 -108
View File
@@ -3,44 +3,38 @@ title: "Triggering"
description: "Tasks need to be triggered in order to run."
---
## Trigger functions
Trigger tasks **from your backend**:
| Function | This works | What it does |
| :----------------------- | :--------- | :-------------------------------------------------------------------------------------------------------------------------- |
| `tasks.trigger()` | Anywhere | Triggers a task and gets a handle you can use to fetch and manage the run. [Read more](#tasks-trigger) |
| `tasks.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to fetch and manage the runs. [Read more](#tasks-batchtrigger) |
| `tasks.triggerAndPoll()` | Anywhere | Triggers a task and then polls the run until it’s complete. [Read more](#tasks-triggerandpoll) |
| Function | What it does | |
| :----------------------- | :----------------------------------------------------------------------------------------------- | ----------------------------- |
| `tasks.trigger()` | Triggers a task and returns a handle you can use to fetch and manage the run. | [Docs](#tasks-trigger) |
| `tasks.batchTrigger()` | Triggers a single task in a batch and returns a handle you can use to fetch and manage the runs. | [Docs](#tasks-batchtrigger) |
| `tasks.triggerAndPoll()` | Triggers a task and then polls the run until it’s complete. | [Docs](#tasks-triggerandpoll) |
| `batch.trigger()` | Similar to `tasks.batchTrigger` but allows running multiple different tasks | [Docs](#batch-trigger) |
Trigger tasks **from inside a run**:
Trigger tasks **from inside a another task**:
| Function | This works | What it does |
| :------------------------------- | :---------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `yourTask.trigger()` | Anywhere | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. [Read more](#yourtask-trigger) |
| `yourTask.batchTrigger()` | Anywhere | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. [Read more](#yourtask-batchtrigger) |
| `yourTask.triggerAndWait()` | Inside task | Triggers a task and then waits until it's complete. You get the result data to continue with. [Read more](#yourtask-triggerandwait) |
| `yourTask.batchTriggerAndWait()` | Inside task | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. [Read more](#yourtask-batchtriggerandwait) |
Additionally, [scheduled tasks](/tasks/scheduled) get **automatically** triggered on their schedule and webhooks when receiving a webhook.
## Scheduled tasks
You should attach one or more schedules to your `schedules.task()` to trigger it on a recurring schedule. [Read the scheduled tasks docs](/tasks/scheduled).
## Authentication
When you trigger a task from your backend code, you need to set the `TRIGGER_SECRET_KEY` environment variable. You can find the value on the API keys page in the Trigger.dev dashboard. [More info on API keys](/apikeys).
| Function | What it does | |
| :------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |
| `yourTask.trigger()` | Triggers a task and gets a handle you can use to monitor and manage the run. It does not wait for the result. | [Docs](#yourtask-trigger) |
| `yourTask.batchTrigger()` | Triggers a task multiple times and gets a handle you can use to monitor and manage the runs. It does not wait for the results. | [Docs](#yourtask-batchtrigger) |
| `yourTask.triggerAndWait()` | Triggers a task and then waits until it's complete. You get the result data to continue with. | [Docs](#yourtask-triggerandwait) |
| `yourTask.batchTriggerAndWait()` | Triggers a task multiple times in parallel and then waits until they're all complete. You get the resulting data to continue with. | [Docs](#yourtask-batchtriggerandwait) |
| `batch.triggerAndWait()` | Similar to `batch.trigger` but will wait on the triggered tasks to finish and return the results. | [Docs](#batch-triggerandwait) |
| `batch.triggerByTask()` | Similar to `batch.trigger` but allows passing in task instances instead of task IDs. | [Docs](#batch-triggerbytask) |
| `batch.triggerByTaskAndWait()` | Similar to `batch.triggerbyTask` but will wait on the triggered tasks to finish and return the results. | [Docs](#batch-triggerbytaskandwait) |
## Triggering from your backend
You can trigger any task from your backend code using the `tasks.trigger()` or `tasks.batchTrigger()` SDK functions.
When you trigger a task from your backend code, you need to set the `TRIGGER_SECRET_KEY` environment variable. You can find the value on the API keys page in the Trigger.dev dashboard. [More info on API keys](/apikeys).
<Note>
Do not trigger tasks directly from your frontend. If you do, you will leak your private
Trigger.dev API key.
If you are using Next.js Server Actions [you'll need to be careful with
bundling](/guides/frameworks/nextjs#triggering-your-task-in-next-js).
</Note>
You can use Next.js Server Actions but [you need to be careful with bundling](/guides/frameworks/nextjs#triggering-your-task-in-next-js).
### tasks.trigger()
Triggers a single run of a task with the payload you pass in, and any options you specify, without needing to import the task.
@@ -51,9 +45,7 @@ Triggers a single run of a task with the payload you pass in, and any options yo
application.
</Note>
<CodeGroup>
```ts Next.js API route
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
// 👆 **type-only** import
@@ -74,45 +66,37 @@ export async function POST(request: Request) {
}
```
```ts Remix
You can pass in options to the task using the second argument:
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
// 👆 **type-only** import
export async function action({ request, params }: ActionFunctionArgs) {
if (request.method !== "POST") {
throw new Response("Method Not Allowed", { status: 405 });
}
//app/email/route.ts
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass the task type to `trigger()` as a generic argument, giving you full type checking
const handle = await tasks.trigger<typeof emailSequence>("email-sequence", {
to: data.email,
name: data.name,
});
const handle = await tasks.trigger<typeof emailSequence>(
"email-sequence",
{
to: data.email,
name: data.name,
},
{ delay: "1h" } // 👈 Pass in the options here
);
//return a success response with the handle
return json(handle);
return Response.json(handle);
}
```
</CodeGroup>
### tasks.batchTrigger()
Triggers multiple runs of a task with the payloads you pass in, and any options you specify, without needing to import the task.
Triggers multiple runs of a single task with the payloads you pass in, and any options you specify, without needing to import the task.
<Note>
By using `tasks.batchTrigger()`, you can pass in the task type as a generic argument, giving you
full type checking. Make sure you use a `type` import so that your task code is not imported into
your application.
</Note>
<CodeGroup>
```ts Next.js API route
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
// 👆 **type-only** import
@@ -133,44 +117,62 @@ export async function POST(request: Request) {
}
```
```ts Remix
You can pass in options to the `batchTrigger` function using the second argument:
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
export async function action({ request, params }: ActionFunctionArgs) {
if (request.method !== "POST") {
throw new Response("Method Not Allowed", { status: 405 });
}
//app/email/route.ts
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass the task type to `batchTrigger()` as a generic argument, giving you full type checking
const batchHandle = await tasks.batchTrigger<typeof emailSequence>(
"email-sequence",
data.users.map((u) => ({ payload: { to: u.email, name: u.name } }))
data.users.map((u) => ({ payload: { to: u.email, name: u.name } })),
{ idempotencyKey: "my-idempotency-key" } // 👈 Pass in the options here
);
//return a success response with the handle
return json(batchHandle);
return Response.json(batchHandle);
}
```
</CodeGroup>
You can also pass in options for each run in the batch:
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
//app/email/route.ts
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass the task type to `batchTrigger()` as a generic argument, giving you full type checking
const batchHandle = await tasks.batchTrigger<typeof emailSequence>(
"email-sequence",
data.users.map((u) => ({ payload: { to: u.email, name: u.name }, options: { delay: "1h" } })) // 👈 Pass in options to each item like so
);
//return a success response with the handle
return Response.json(batchHandle);
}
```
### tasks.triggerAndPoll()
Triggers a single run of a task with the payload you pass in, and any options you specify, and then polls the run until it's complete.
<Note>
By using `tasks.triggerAndPoll()`, you can pass in the task type as a generic argument, giving you
full type checking. Make sure you use a `type` import so that your task code is not imported into
your application.
</Note>
<Warning>
We don't recommend using `triggerAndPoll()`, especially inside a web request, as it will block the
request until the run is complete. Please see our [Realtime docs](/realtime) for a better way to
handle this.
</Warning>
<CodeGroup>
```ts Next.js API route
```ts Your backend
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
@@ -194,71 +196,83 @@ export async function POST(request: Request) {
}
```
```ts Remix
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
### batch.trigger()
export async function action({ request, params }: ActionFunctionArgs) {
if (request.method !== "POST") {
throw new Response("Method Not Allowed", { status: 405 });
}
Triggers multiple runs of different tasks with the payloads you pass in, and any options you specify. This is useful when you need to trigger multiple tasks at once.
```ts Your backend
import { batch } from "@trigger.dev/sdk/v3";
import type { myTask1, myTask2 } from "~/trigger/myTasks";
export async function POST(request: Request) {
//get the JSON from the request
const data = await request.json();
// Pass the task type to `triggerAndPoll()` as a generic argument, giving you full type checking
const result = await tasks.triggerAndPoll<typeof emailSequence>(
"email-sequence",
{
to: data.email,
name: data.name,
},
{ pollIntervalMs: 5000 }
);
// Pass a union of the tasks to `trigger()` as a generic argument, giving you full type checking
const result = await batch.trigger<typeof myTask1 | typeof myTask2>([
// Because we're using a union, we can pass in multiple tasks by ID
{ id: "my-task-1", payload: { some: data.some } },
{ id: "my-task-2", payload: { other: data.other } },
]);
//return a success response with the result
return json(result);
return Response.json(result);
}
```
</CodeGroup>
## Triggering from inside another task
<Note>
The above code is just a demonstration of the API and is not recommended to use in an API route
this way as it will block the request until the task is complete.
</Note>
The following functions should only be used when running inside a task, for one of the following reasons:
## Triggering from inside a run
Task instance methods are available on the `Task` object you receive when you define a task. We recommend you use these methods inside another task to trigger subtasks.
- You need to **wait** for the result of the triggered task.
- You need to import the task instance. Importing a task instance from your backend code is not recommended, as it can pull in a lot of unnecessary code and dependencies.
### yourTask.trigger()
Triggers a single run of a task with the payload you pass in, and any options you specify. It does NOT wait for the result.
Triggers a single run of a task with the payload you pass in, and any options you specify.
If called from within a task, you can use the `AndWait` version to pause execution until the triggered run is complete.
<Note>
If you need to call `trigger()` on a task in a loop, use
[`batchTrigger()`](#yourTask-batchtrigger) instead which will trigger up to 500 runs in a single
call.
</Note>
If you need to call `trigger()` on a task in a loop, use [`batchTrigger()`](/triggering#task-batchtrigger) instead which will trigger up to 100 tasks in a single call.
```ts /trigger/my-task.ts
import { myOtherTask } from "~/trigger/my-other-task";
```ts ./trigger/my-task.ts
import { myOtherTask, runs } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const handle = await myOtherTask.trigger("some data");
const handle = await myOtherTask.trigger({ foo: "some data" });
//...do other stuff
const run = await runs.retrieve(handle);
// Do something with the run
},
});
```
To pass options to the triggered task, you can use the second argument:
```ts ./trigger/my-task.ts
import { myOtherTask, runs } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const handle = await myOtherTask.trigger({ foo: "some data" }, { delay: "1h" });
const run = await runs.retrieve(handle);
// Do something with the run
},
});
```
### yourTask.batchTrigger()
Triggers multiple runs of a task with the payloads you pass in, and any options you specify. It does NOT wait for the result.
Triggers multiple runs of a single task with the payloads you pass in, and any options you specify.
```ts /trigger/my-task.ts
import { myOtherTask } from "~/trigger/my-other-task";
import { myOtherTask, batch } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
@@ -266,6 +280,43 @@ export const myTask = task({
const batchHandle = await myOtherTask.batchTrigger([{ payload: "some data" }]);
//...do other stuff
const batch = await batch.retrieve(batchHandle.id);
},
});
```
If you need to pass options to `batchTrigger`, you can use the second argument:
```ts /trigger/my-task.ts
import { myOtherTask, batch } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const batchHandle = await myOtherTask.batchTrigger([{ payload: "some data" }], {
idempotencyKey: "my-task-key",
});
//...do other stuff
const batch = await batch.retrieve(batchHandle.id);
},
});
```
You can also pass in options for each run in the batch:
```ts /trigger/my-task.ts
import { myOtherTask, batch } from "~/trigger/my-other-task";
export const myTask = task({
id: "my-task",
run: async (payload: string) => {
const batchHandle = await myOtherTask.batchTrigger([
{ payload: "some data", options: { delay: "1h" } },
]);
//...do other stuff
const batch = await batch.retrieve(batchHandle.id);
},
});
```
@@ -504,6 +555,152 @@ export const batchParentTask = task({
error.
</Warning>
### batch.triggerAndWait()
You can batch trigger multiple different tasks and wait for all the results:
```ts /trigger/batch.ts
import { batch, task } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
// 👇 Pass a union of all the tasks you want to trigger
const results = await batch.triggerAndWait<typeof childTask1 | typeof childTask2>([
{ id: "child-task-1", payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task `id`
{ id: "child-task-2", payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task `id`
]);
for (const result of results) {
if (result.ok) {
// 👇 Narrow the type of the result based on the taskIdentifier
switch (result.taskIdentifier) {
case "child-task-1":
console.log("Child task 1 output", result.output); // 👈 result.output is typed as a string
break;
case "child-task-2":
console.log("Child task 2 output", result.output); // 👈 result.output is typed as a number
break;
}
} else {
console.error("Error", result.error); // 👈 result.error is the error that caused the run to fail
}
}
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});
```
### batch.triggerByTask()
You can batch trigger multiple different tasks by passing in the task instances. This function is especially useful when you have a static set of tasks you want to trigger:
```ts /trigger/batch.ts
import { batch, task, runs } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
const results = await batch.triggerByTask([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
// 👇 results.runs is a tuple, allowing you to get type safety without needing to narrow
const run1 = await runs.retrieve(results.runs[0]); // 👈 run1 is typed as the output of childTask1
const run2 = await runs.retrieve(results.runs[1]); // 👈 run2 is typed as the output of childTask2
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});
```
### batch.triggerByTaskAndWait()
You can batch trigger multiple different tasks by passing in the task instances, and wait for all the results. This function is especially useful when you have a static set of tasks you want to trigger:
```ts /trigger/batch.ts
import { batch, task, runs } from "@trigger.dev/sdk/v3";
export const parentTask = task({
id: "parent-task",
run: async (payload: string) => {
const { runs } = await batch.triggerByTaskAndWait([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
if (runs[0].ok) {
console.log("Child task 1 output", runs[0].output); // 👈 runs[0].output is typed as the output of childTask1
}
if (runs[1].ok) {
console.log("Child task 2 output", runs[1].output); // 👈 runs[1].output is typed as the output of childTask2
}
// 💭 A nice alternative syntax is to destructure the runs array:
const {
runs: [run1, run2],
} = await batch.triggerByTaskAndWait([
{ task: childTask1, payload: { foo: "World" } }, // 👈 The payload is typed correctly based on the task instance
{ task: childTask2, payload: { bar: 42 } }, // 👈 The payload is typed correctly based on the task instance
]);
if (run1.ok) {
console.log("Child task 1 output", run1.output); // 👈 run1.output is typed as the output of childTask1
}
if (run2.ok) {
console.log("Child task 2 output", run2.output); // 👈 run2.output is typed as the output of childTask2
}
},
});
export const childTask1 = task({
id: "child-task-1",
run: async (payload: { foo: string }) => {
return `Hello ${payload}`;
},
});
export const childTask2 = task({
id: "child-task-2",
run: async (payload: { bar: number }) => {
return bar + 1;
},
});
```
## Triggering from your frontend
If you want to trigger a task directly from a frontend application, you can use our [React
hooks](/frontend/react-hooks#trigger-hooks).
## Options
All of the above functions accept an options object:
@@ -623,7 +820,39 @@ export const myTask = task({
const idempotencyKey = await idempotencyKeys.create("my-task-key");
// childTask will only be triggered once with the same idempotency key
await childTask.triggerAndWait(payload, { idempotencyKey });
await childTask.trigger(payload, { idempotencyKey });
// Do something else, that may throw an error and cause the task to be retried
},
});
```
For more information, see our [Idempotency](/idempotency) documentation.
<Warning>
In version 3.3.0 and later, the `idempotencyKey` option is not available when using
`triggerAndWait` or `batchTriggerAndWait`, due to a bug that would sometimes cause the parent task
to become stuck. We are working on a fix for this issue.
</Warning>
### `idempotencyKeyTTL`
Idempotency keys automatically expire after 30 days, but you can set a custom TTL for an idempotency key when triggering a task:
```typescript
import { idempotencyKeys, task } from "@trigger.dev/sdk/v3";
export const myTask = task({
id: "my-task",
retry: {
maxAttempts: 4,
},
run: async (payload: any) => {
// By default, idempotency keys generated are unique to the run, to prevent retries from duplicating child tasks
const idempotencyKey = await idempotencyKeys.create("my-task-key");
// childTask will only be triggered once with the same idempotency key
await childTask.trigger(payload, { idempotencyKey, idempotencyKeyTTL: "1h" });
// Do something else, that may throw an error and cause the task to be retried
},
@@ -827,4 +1056,4 @@ export const myTask = task({
### Batch Triggering
When using `batchTrigger` or `batchTriggerAndWait`, the total size of all payloads cannot exceed 10MB. This means if you are doing a batch of 100 runs, each payload should be less than 100KB.
When using triggering a batch, the total size of all payloads cannot exceed 1MB. This means if you are doing a batch of 100 runs, each payload should be less than 100KB. The max batch size is 500 runs.
docs/v3-openapi.yaml
+4 -4
View File
@@ -753,7 +753,7 @@ paths:
import { runs, configure } from "@trigger.dev/sdk/v3";
configure({
secretKey: "tr_pat_1234" // always use an environment variable for this
accessToken: "tr_pat_1234" // always use an environment variable for this
});
// Get the first page of runs
@@ -781,7 +781,7 @@ paths:
import { runs, configure } from "@trigger.dev/sdk/v3";
configure({
secretKey: "tr_pat_1234" // always use an environment variable for this
accessToken: "tr_pat_1234" // always use an environment variable for this
});
const response = await runs.list("proj_1234", {
@@ -1503,7 +1503,7 @@ components:
```typescript
import { configure } from "@trigger.dev/sdk/v3";
configure({ secretKey: "tr_dev_1234" });
configure({ accessToken: "tr_dev_1234" });
```
personalAccessToken:
@@ -1517,7 +1517,7 @@ components:
```typescript
import { configure } from "@trigger.dev/sdk/v3";
configure({ secretKey: "tr_pat_1234" });
configure({ accessToken: "tr_pat_1234" });
```
schemas:
TriggerTaskResponse: