Eric Allam
cf13fbdf32
* Add triggerAndWait().unwrap() to more easily get at the output or throw the subtask error * Add taskId and runId to SubtaskUnwrapError * WIP docs update beta -> latest * trigger.dev init now adds @trigger.dev/build to devDependencies * How it works doc * Restructure some docs and update the cli commands * Config file docs, plus aptGet and ffmpeg extensions * Update to latest from beta docs * Add runtime to templates * Add --runtime option to the init CLI command * A bunch more doc updates after feedback * Document triggerAndWait with unwrap and result types * beta -> latest in the webapp * CLI update check no longer references beta * Add major release * Leave changeset beta, back to normal package release * Fixed default dirs option in init command * exclude windows-yarn variation of cli e2e tests because it’s buggy * Remove cache to try and fix yarn e2e test workflow errors
632 lines
20 KiB
Plaintext
632 lines
20 KiB
Plaintext
---
|
|
title: "The trigger.config.ts file"
|
|
sidebarTitle: "Configuration"
|
|
description: "This file is used to configure your project and how it's built."
|
|
---
|
|
|
|
import BundlePackages from "/snippets/bundle-packages.mdx";
|
|
|
|
The `trigger.config.ts` file is used to configure your Trigger.dev project. It is a TypeScript file at the root of your project that exports a default configuration object. Here's an example:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//Your project ref (you can see it on the Project settings page in the dashboard)
|
|
project: "proj_gtcwttqhhtlasxgfuhxs",
|
|
//The paths for your trigger folders
|
|
dirs: ["./trigger"],
|
|
retries: {
|
|
//If you want to retry a task in dev mode (when using the CLI)
|
|
enabledInDev: false,
|
|
//the default retry settings. Used if you don't specify on a task.
|
|
default: {
|
|
maxAttempts: 3,
|
|
minTimeoutInMs: 1000,
|
|
maxTimeoutInMs: 10000,
|
|
factor: 2,
|
|
randomize: true,
|
|
},
|
|
},
|
|
});
|
|
```
|
|
|
|
The config file handles a lot of things, like:
|
|
|
|
- Specifying where your trigger tasks are located using the `dirs` option.
|
|
- Setting the default retry settings.
|
|
- Configuring OpenTelemetry instrumentations.
|
|
- Customizing the build process.
|
|
- Adding global task lifecycle functions.
|
|
|
|
<Note>
|
|
The config file is bundled with your project, so code imported in the config file is also bundled,
|
|
which can have an effect on build times and cold start duration. One important qualification is
|
|
anything defined in the `build` config is automatically stripped out of the config file, and
|
|
imports used inside build config with be tree-shaken out.
|
|
</Note>
|
|
|
|
## Lifecycle functions
|
|
|
|
You can add lifecycle functions to get notified when any task starts, succeeds, or fails using `onStart`, `onSuccess` and `onFailure`:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
onSuccess: async (payload, output, { ctx }) => {
|
|
console.log("Task succeeded", ctx.task.id);
|
|
},
|
|
onFailure: async (payload, error, { ctx }) => {
|
|
console.log("Task failed", ctx.task.id);
|
|
},
|
|
onStart: async (payload, { ctx }) => {
|
|
console.log("Task started", ctx.task.id);
|
|
},
|
|
init: async (payload, { ctx }) => {
|
|
console.log("I run before any task is run");
|
|
},
|
|
});
|
|
```
|
|
|
|
Read more about task lifecycle functions in the [tasks overview](/tasks-overview).
|
|
|
|
## Instrumentations
|
|
|
|
We use OpenTelemetry (OTEL) for our run logs. This means you get a lot of information about your tasks with no effort. But you probably want to add more information to your logs. For example, here's all the Prisma calls automatically logged:
|
|
|
|
|
|
|
|
Here we add Prisma and OpenAI instrumentations to your `trigger.config.ts` file.
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { PrismaInstrumentation } from "@prisma/instrumentation";
|
|
import { OpenAIInstrumentation } from "@traceloop/instrumentation-openai";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
instrumentations: [new PrismaInstrumentation(), new OpenAIInstrumentation()],
|
|
});
|
|
```
|
|
|
|
There is a [huge library of instrumentations](https://opentelemetry.io/ecosystem/registry/?language=js) you can easily add to your project like this.
|
|
|
|
Some ones we recommend:
|
|
|
|
| Package | Description |
|
|
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
|
|
| `@opentelemetry/instrumentation-undici` | Logs all fetch calls (inc. Undici fetch) |
|
|
| `@opentelemetry/instrumentation-fs` | Logs all file system calls |
|
|
| `@opentelemetry/instrumentation-http` | Logs all HTTP calls |
|
|
| `@prisma/instrumentation` | Logs all Prisma calls, you need to [enable tracing](https://github.com/prisma/prisma/tree/main/packages/instrumentation) |
|
|
| `@traceloop/instrumentation-openai` | Logs all OpenAI calls |
|
|
|
|
## Runtime
|
|
|
|
We currently only officially support the `node` runtime, but you can try our experimental `bun` runtime by setting the `runtime` option in your config file:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
runtime: "bun",
|
|
});
|
|
```
|
|
|
|
See our [Bun guide](/guides/bun) for more information.
|
|
|
|
## Default machine
|
|
|
|
You can specify the default machine for all tasks in your project:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
defaultMachine: "large-1x",
|
|
});
|
|
```
|
|
|
|
See our [machines documentation](/machines) for more information.
|
|
|
|
## Log level
|
|
|
|
You can set the log level for your project:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
logLevel: "debug",
|
|
});
|
|
```
|
|
|
|
The `logLevel` only determines which logs are sent to the Trigger.dev instance when using the `logger` API. All `console` based logs are always sent.
|
|
|
|
## Build configuration
|
|
|
|
You can customize the build process using the `build` option:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
// Don't bundle these packages
|
|
external: ["header-generator"],
|
|
},
|
|
});
|
|
```
|
|
|
|
<Note>
|
|
The `trigger.config.ts` file is included in the bundle, but with the `build` configuration
|
|
stripped out. These means any imports only used inside the `build` configuration are also removed
|
|
from the final bundle.
|
|
</Note>
|
|
|
|
### External
|
|
|
|
All code is bundled by default, but you can exclude some packages from the bundle using the `external` option:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
external: ["header-generator"],
|
|
},
|
|
});
|
|
```
|
|
|
|
When a package is excluded from the bundle, it will be added to a dynamically generated package.json file in the build directory. The version of the package will be the same as the version found in your `node_modules` directory.
|
|
|
|
Each entry in the external should be a package name, not necessarily the import path. For example, if you want to exclude the `ai` package, but you are importing `ai/rsc`, you should just include `ai` in the `external` array:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
external: ["ai"],
|
|
},
|
|
});
|
|
```
|
|
|
|
<Note>
|
|
Any packages that install or build a native binary should be added to external, as native binaries
|
|
cannot be bundled. For example, `re2`, `sharp`, and `sqlite3` should be added to external.
|
|
</Note>
|
|
|
|
### JSX
|
|
|
|
You can customize the `jsx` options that are passed to `esbuild` using the `jsx` option:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
jsx: {
|
|
// Use the Fragment component instead of React.Fragment
|
|
fragment: "Fragment",
|
|
// Use the h function instead of React.createElement
|
|
factory: "h",
|
|
// Turn off automatic runtime
|
|
automatic: false,
|
|
},
|
|
},
|
|
});
|
|
```
|
|
|
|
By default we enabled [esbuild's automatic JSX runtime](https://esbuild.github.io/content-types/#auto-import-for-jsx) which means you don't need to import `React` in your JSX files. You can disable this by setting `automatic` to `false`.
|
|
|
|
See the [esbuild JSX documentation](https://esbuild.github.io/content-types/#jsx) for more information.
|
|
|
|
### Conditions
|
|
|
|
You can add custom [import conditions](https://esbuild.github.io/api/#conditions) to your build using the `conditions` option:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
conditions: ["react-server"],
|
|
},
|
|
});
|
|
```
|
|
|
|
These conditions effect how imports are resolved during the build process. For example, the `react-server` condition will resolve `ai/rsc` to the server version of the `ai/rsc` export.
|
|
|
|
Custom conditions will also be passed to the `node` runtime when running your tasks.
|
|
|
|
### Extensions
|
|
|
|
Build extension allow you to hook into the build system and customize the build process or the resulting bundle and container image (in the case of deploying). You can use pre-built extensions by installing the `@trigger.dev/build` package into your `devDependencies`, or you can create your own.
|
|
|
|
#### additionalFiles
|
|
|
|
Import the `additionalFiles` build extension and use it in your `trigger.config.ts` file:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { additionalFiles } from "@trigger.dev/build/extensions/core";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [
|
|
additionalFiles({ files: ["wrangler/wrangler.toml", "./assets/**", "./fonts/**"] }),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
This will copy the files specified in the `files` array to the build directory. The `files` array can contain globs. The output paths will match the path of the file, relative to the root of the project.
|
|
|
|
<Note>The root of the project is the directory that contains the trigger.config.ts file</Note>
|
|
|
|
#### `additionalPackages`
|
|
|
|
Import the `additionalPackages` build extension and use it in your `trigger.config.ts` file:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { additionalPackages } from "@trigger.dev/build/extensions/core";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [additionalPackages({ packages: ["wrangler"] })],
|
|
},
|
|
});
|
|
```
|
|
|
|
This allows you to include additional packages in the build that are not automatically included via imports. This is useful if you want to install a package that includes a CLI tool that you want to invoke in your tasks via `exec`. We will try to automatically resolve the version of the package but you can specify the version by using the `@` symbol:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [additionalPackages({ packages: ["wrangler@1.19.0"] })],
|
|
},
|
|
});
|
|
```
|
|
|
|
#### `emitDecoratorMetadata`
|
|
|
|
If you need support for the `emitDecoratorMetadata` typescript compiler option, import the `emitDecoratorMetadata` build extension and use it in your `trigger.config.ts` file:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { emitDecoratorMetadata } from "@trigger.dev/build/extensions/typescript";
|
|
|
|
export default defineConfig({
|
|
project: "<project ref>",
|
|
build: {
|
|
extensions: [emitDecoratorMetadata()],
|
|
},
|
|
});
|
|
```
|
|
|
|
This is usually required if you are using certain ORMs, like TypeORM, that require this option to be enabled. It's not enabled by default because there is a performance cost to enabling it.
|
|
|
|
<Note>
|
|
emitDecoratorMetadata works by hooking into the esbuild bundle process and using the TypeScript
|
|
compiler API to compile files where we detect the use of decorators. This means you must have
|
|
`emitDecoratorMetadata` enabled in your `tsconfig.json` file, as well as `typescript` installed in
|
|
your `devDependencies`.
|
|
</Note>
|
|
|
|
#### Prisma
|
|
|
|
If you are using Prisma, you should use the prisma build extension.
|
|
|
|
- Automatically handles copying prisma files to the build directory.
|
|
- Generates the prisma client during the deploy process
|
|
- Optionally will migrate the database during the deploy process
|
|
- Support for TypedSQL and multiple schema files.
|
|
|
|
You can use it for a simple Prisma setup like this:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { prismaExtension } from "@trigger.dev/build/extensions/prisma";
|
|
|
|
export default defineConfig({
|
|
build: {
|
|
extensions: [
|
|
prismaExtension({
|
|
version: "5.19.0", // optional, we'll automatically detect the version if not provided
|
|
schema: "prisma/schema.prisma",
|
|
}),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
<Note>
|
|
This does not have any effect when running the `dev` command, only when running the `deploy`
|
|
command.
|
|
</Note>
|
|
|
|
If you want to also run migrations during the build process, you can pass in the `migrate` option:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { prismaExtension } from "@trigger.dev/build/extensions/prisma";
|
|
|
|
export default defineConfig({
|
|
project: "<project ref>",
|
|
build: {
|
|
extensions: [
|
|
prismaExtension({
|
|
schema: "prisma/schema.prisma",
|
|
migrate: true,
|
|
directUrlEnvVarName: "DATABASE_URL_UNPOOLED", // optional - the name of the environment variable that contains the direct database URL if you are using a direct database URL
|
|
}),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
If you have multiple `generator` statements defined in your schema file, you can pass in the `clientGenerator` option to specify the `prisma-client-js` generator, which will prevent other generators from being generated:
|
|
|
|
<CodeGroup>
|
|
|
|
```prisma schema.prisma
|
|
datasource db {
|
|
provider = "postgresql"
|
|
url = env("DATABASE_URL")
|
|
directUrl = env("DATABASE_URL_UNPOOLED")
|
|
}
|
|
|
|
// We only want to generate the prisma-client-js generator
|
|
generator client {
|
|
provider = "prisma-client-js"
|
|
}
|
|
|
|
generator kysely {
|
|
provider = "prisma-kysely"
|
|
output = "../../src/kysely"
|
|
enumFileName = "enums.ts"
|
|
fileName = "types.ts"
|
|
}
|
|
```
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { prismaExtension } from "@trigger.dev/build/extensions/prisma";
|
|
|
|
export default defineConfig({
|
|
project: "<project ref>",
|
|
build: {
|
|
extensions: [
|
|
prismaExtension({
|
|
schema: "prisma/schema.prisma",
|
|
clientGenerator: "client",
|
|
}),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
</CodeGroup>
|
|
|
|
If you are using [TypedSQL](https://www.prisma.io/typedsql), you'll need to enable it via the `typedSql` option:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
project: "<project ref>",
|
|
build: {
|
|
extensions: [
|
|
prismaExtension({
|
|
schema: "prisma/schema.prisma",
|
|
typedSql: true,
|
|
}),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
<Note>
|
|
The `prismaExtension` will inject the `DATABASE_URL` environment variable into the build process
|
|
when running the `deploy` command. This means the CLI needs to have `process.env.DATABASE_URL` set
|
|
at the time of calling the `deploy` command. You can do this via a `.env` file and passing the
|
|
`--env-file .env` option to the deploy command or via shell environment variables. This goes for direct database URLs as well.
|
|
|
|
These environment variables are only used during the build process and are not embedded in the final image.
|
|
|
|
</Note>
|
|
|
|
#### syncEnvVars
|
|
|
|
The `syncEnvVars` build extension replaces the deprecated `resolveEnvVars` export. Check out our [syncEnvVars documentation](/deploy-environment-variables#sync-env-vars-from-another-service) for more information.
|
|
|
|
#### audioWaveform
|
|
|
|
Previously, we installed [Audio Waveform](https://github.com/bbc/audiowaveform) in the build image. That's been moved to a build extension:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { audioWaveform } from "@trigger.dev/build/extensions/audioWaveform";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [audioWaveform()], // uses verson 1.1.0 of audiowaveform by default
|
|
},
|
|
});
|
|
```
|
|
|
|
#### ffmpeg
|
|
|
|
You can add the `ffmpeg` build extension to your build process:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { ffmpeg } from "@trigger.dev/build/extensions/core";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [ffmpeg()],
|
|
},
|
|
});
|
|
```
|
|
|
|
By default, this will install the version of `ffmpeg` that is available in the Debian package manager. If you need a specific version, you can pass in the version as an argument:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { ffmpeg } from "@trigger.dev/build/extensions/core";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [ffmpeg({ version: "6.0-4" })],
|
|
},
|
|
});
|
|
```
|
|
|
|
This extension will also add the `FFMPEG_PATH` and `FFPROBE_PATH` to your environment variables, making it easy to use popular ffmpeg libraries like `fluent-ffmpeg`.
|
|
|
|
#### esbuild plugins
|
|
|
|
You can easily add existing or custom esbuild plugins to your build process using the `esbuildPlugin` extension:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { esbuildPlugin } from "@trigger.dev/build/extensions";
|
|
import { sentryEsbuildPlugin } from "@sentry/esbuild-plugin";
|
|
|
|
export default defineConfig({
|
|
project: "<project ref>",
|
|
build: {
|
|
extensions: [
|
|
esbuildPlugin(
|
|
sentryEsbuildPlugin({
|
|
org: process.env.SENTRY_ORG,
|
|
project: process.env.SENTRY_PROJECT,
|
|
authToken: process.env.SENTRY_AUTH_TOKEN,
|
|
}),
|
|
// optional - only runs during the deploy command, and adds the plugin to the end of the list of plugins
|
|
{ placement: "last", target: "deploy" }
|
|
),
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
#### aptGet
|
|
|
|
You can install system packages into the deployed image using using the `aptGet` extension:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
import { aptGet } from "@trigger.dev/build/extensions/core";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [aptGet({ packages: ["ffmpeg"] })],
|
|
},
|
|
});
|
|
```
|
|
|
|
If you want to install a specific version of a package, you can specify the version like this:
|
|
|
|
```ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [aptGet({ packages: ["ffmpeg=6.0-4"] })],
|
|
},
|
|
});
|
|
```
|
|
|
|
#### Custom extensions
|
|
|
|
You can create your own extensions to further customize the build process. Extensions are an object with a `name` and zero or more lifecycle hooks (`onBuildStart` and `onBuildComplete`) that allow you to modify the `BuildContext` object that is passed to the build process through adding layers. For example, this is how the `aptGet` extension is implemented:
|
|
|
|
```ts
|
|
import { BuildExtension } from "@trigger.dev/core/v3/build";
|
|
|
|
export type AptGetOptions = {
|
|
packages: string[];
|
|
};
|
|
|
|
export function aptGet(options: AptGetOptions): BuildExtension {
|
|
return {
|
|
name: "aptGet",
|
|
onBuildComplete(context) {
|
|
if (context.target === "dev") {
|
|
return;
|
|
}
|
|
|
|
context.logger.debug("Adding apt-get layer", {
|
|
pkgs: options.packages,
|
|
});
|
|
|
|
context.addLayer({
|
|
id: "apt-get",
|
|
image: {
|
|
pkgs: options.packages,
|
|
},
|
|
});
|
|
},
|
|
};
|
|
}
|
|
```
|
|
|
|
Instead of creating this function and worrying about types, you can define an extension inline in your `trigger.config.ts` file:
|
|
|
|
```ts trigger.config.ts
|
|
import { defineConfig } from "@trigger.dev/sdk/v3";
|
|
|
|
export default defineConfig({
|
|
//..other stuff
|
|
build: {
|
|
extensions: [
|
|
{
|
|
name: "aptGet",
|
|
onBuildComplete(context) {
|
|
if (context.target === "dev") {
|
|
return;
|
|
}
|
|
|
|
context.logger.debug("Adding apt-get layer", {
|
|
pkgs: ["ffmpeg"],
|
|
});
|
|
|
|
context.addLayer({
|
|
id: "apt-get",
|
|
image: {
|
|
pkgs: ["ffmpeg"],
|
|
},
|
|
});
|
|
},
|
|
},
|
|
],
|
|
},
|
|
});
|
|
```
|
|
|
|
We'll be expanding the documentation on how to create custom extensions in the future, but for now you are encouraged to look at the existing extensions in the `@trigger.dev/build` package for inspiration, which you can see in our repo [here](https://github.com/triggerdotdev/trigger.dev/tree/main/packages/build/src/extensions)
|