* chore(runner): move max duration logic into parent process
* chore(rsc): remove type-marker package.json
* add changeset
* chore(core): remove irrelevant test after our changes
* chore(core): clarify we don't care about the timeout promise
* fix(webapp): display correct concurrency override base value
* fix(webapp): persist concurrency overrides on deploy
* fix(webapp): use correct override base value type
* fix(webapp): override input is bounded by env concurrency
* Enable skipping image push during deployment finalization step
* Add endpoint to generate registry credentials for a deployment
* Add a --force-local-build flag to the deployment command to skip remote build
* Do not show the new flag in the help output
* Add changeset
* Remove registry login logs from onLog, not useful
* Rename var
* Update platform package to the latest version
* Limit local dev concurrency using the dev environment concurrency limit
Previously it was limited to max of 25, no matter the environment limit
* Have global dev limit
The Postgres query to get the latest run ids for the test page was very
slow when there were a lot of runs and/or versions.
This now uses the standard runs list we use everywhere else.
* feat(queues): add ability to override concurrency limit via API and dashboard
* Updates the modal layout and tweaks copy
* Improves the dropdown menu item
* Popover supports both Button and LinkButton
* Right align the columns and fix the dropdown menu item styles
* Organize imports,
* Fix spinner icon in dropdown menu
* Remove unused props
* Adds a tooltip to the Concurrency override badge
* Fixes console error with popover menu
* typo
* Fixes incorrect className
* Minimal buttons to view runs
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
* WIP using ClickHouse for the tags filter list
* WIP on tags listing
* Webapp: exclude test files when typechecking
* Tags filtering working with CH
* Remove unused import
* The AI filter should only look at the last past 30d of tags
* Do the text query in ClickHouse
* Deal with encoded characters better
* More encoding fixes
* Fix for wrong items being checked
* Put applied tags back
* Add the env.id to the dependencies array
* chore(billing): improve logs to distinguish between failure modes
* fix(engine): default to paid placement on billing errors
* chore(engine): set plan type according to paying field when missing
- Move S3 credentials from secrets.objectStore to s3.auth
- Update external PostgreSQL config to use databaseUrl/directUrl approach
- Add existingSecret support for PostgreSQL with secretKeys
- Add TLS configuration for external Redis
- Add existingSecret support for Redis, ClickHouse and S3
- Add complete external S3 configuration example
- Improve secure credential management documentation
These changes align the production example file with the current values.yaml
structure, making it easier to configure external services with better
support for secret management.
* Allow shortcuts hook to work if undefined
* Conditionally show shortcut button if only 1 result
* LinkButton can accept conditionally shown shortcuts
* docs: deploying using the github integration
* Add hint in the gh actions docs page
* Remove extra space
* Add a couple of hints to the build config fields
* feat(build-server): add option to specify pre-build command
Adds an option to specify a pre-build command in the build settings. Can
be useful for projects that need a step before the build, e.g., to
generate a prisma client.
Also, remove the install directory in favor of simplicity. Both
pre-build and install commands are run from the root of the repo. Users
that need to run the commands in a different dir can just prepend to the
command, e.g., `cd apps/web && pnpm run primsa:migrate`
* Fix spelling
* Show hint if preview branches are disabled in the project
* Enable preview deployments only if the preview environemtn is enabled
* Fix prisma reference
* fix(otel): prevent unpaired unicode surrogate pairs from causing insert errors
* only check parts of the string that are not going to get truncated
remove unnecessary taks
* fix(charts): use bitnamilegacy repo for MinIO Object Browser
The Bitnami `minio-object-browser` image has been removed from Docker
Hub. This patch updates the Trigger Helm chart to reference the
`bitnamilegacy/minio-object-browser` repository under the
`s3.console.image` section, ensuring that the legacy MinIO browser pod
can still be deployed without image pull errors.
All other MinIO components continue using the `bitnamilegacy/minio`
image for consistency across Bitnami Legacy dependencies.
* chore(helm): bump Helm chart version to 4.0.4
The Depot build init with `depot.build.v1.BuildService.createBuild` fails surprisingly often due to transient errors, causing the whole deployment to fail. This PR adds a simple retry mechanism with backoff using p-retry. This should improve the failure rate.
* chore(docker): use bitnami legacy repo
* chore(helm): use bitnami legacy repo
* Make Helm webapp chart images configurable
Adds configurability for init and token syncer container images through
new values in the Helm chart configuration
* chore(helm): refactor utility image config
* chore(helm): bump chart version to 4.0.3
---------
Co-authored-by: LeoKaynan <leokaynan@hotmail.com>
* feat(webapp): rate limit magic-link login attempts
Adds a simple rate limiter to the login with magic link flow. Similar implementation to the MFA rate limits.
* Fix error message
* Add an env var feature flags for login rate limiting
* Use BoolEnv instead of `0`/`1`
* Parse xff properly
* feat(server): add two admin endpoints for queue and environment concurrency debugging and repairing
feat(run-engine): ability to repair runs in QUEUED, SUSPENDED, and FINISHED execution status
* Handle FINISHED snapshot in the repair
* fix: use higher entropy invite tokens
We currently use CUIDs for invite tokens, which are generated using
a pattern and are not cryptographically secure. This PR switches to
a higher entropy string generated with `nanoid`.
* Dedupe the invite emails in the application
* fix: org scoping in the select plan flow
Adds proper org scoping in the loader and action in the plans page.
* Fix billing alerts scope
* Fix org usage page scope
* Fix token revoking flow scope check
* Throw error for failed PAT revokes instead of silent failure
* fix(webapp): toast message issue after gh app installation
Fixes an issue with displaying toasts messages in the project settings
page. The github callback cookie was interfering with the flash cookie used
for toast messages.
* Do not set a tracking branch in the staging env by default
* Add canceledAt to the deployment db schema
* Expose an api endpoint to cancel deployments
* Show the canceled status description in the dashboard
* Enable canceling deployments from the dashboard
* Show cancelation reason in the deployment details
* Make verifyProjectMembership a function for consistency
* Apply some good 🐰 suggestions
* Add installing status to the deployment db schema
* Replace the deployments /start endpoint with /progress
* Show the installing status in the dashboard
* Add installing status to the api schema and cli
* Add changeset
Depot builds have short-lived tokens and their TTL is not exposed in the SDK. As queued deployments can stay in the queue for an arbitrary amount of time, deferring the remote build creation helps avoid expired Depot token issues.
* Cancel run events which then propogate cancellation status to span ancestors
* WIP
* convert closing cached run spans to new system
* converted expired complete span event to new method
* move v3 over to new methods
* Convert getDetailedTraceSummary to use the new ancestor override stuff
* remove debug logs
* Don't return UNSPECIFIED task events in getRunEvents
* fix the call site for cancelling run event in v3
* Add changeset
* remove methods
This PR adapts the deployment initialization endpoint to handle build server deployments with older CLI versions gracefully.
When we introduced automatic deployments via the build server, we slightly changed the deployment flow
mainly in the initialization and starting step: now deployments are first initialized in the `PENDING` status
and updated to `BUILDING` once the build server dequeues the build job.
Newer versions of the `deploy` command in the CLI will automatically attach to the existing deployment
and continue with the build process. For older versions, we can't change the command's client-side behavior,
so we need to handle this case here in the initialization endpoint. As we control the env variables which
the git meta is extracted from in the build server, we can use those to pass the existing deployment ID
to this endpoint. This doesn't affect the git meta on the deployment as it is set prior to this step using the
/start endpoint. It's a rather hacky solution, but it will do for now as it enables us to avoid degrading the
build server experience for users with older CLI versions. We'll eventually be able to remove this workaround
once we stop supporting 3.x CLI versions.
* Fix `current` badge inconsistency in the deployment details page
* Add custom hook for auto revalidation based on an interval and/or focus change
* Use the autoRevalidate hook for live reloading of the deployments page
* Extract autoReloadPollIntervalMs to an env var
* Replace the sse-based autoreload in bulk actions and queues page with the simpler autoRevalidate hook
* Enable setting the initial status on deployment creation
* Expose endpoint to start deployments
* Extend build timeout on deployment start
* Use separate timeout value for queued deployments
* Add startedAt to the deployment schema
* Show the new startedAt instead of createdAt in the dashboard
* Show github user tag also in the deployment details page
* Show `pending` deployment status as `queued` in the dashboard
* Apply some good 🐰 suggestions
* Add missing return
* Improve styling of onboarding pricing plans
* type only import
* Improve the onboarding plan page so it scrolls on smaller screens
* Adds additional pricing for bolt-ons for the Pro plan
* fix text wrapping issue
* add bg color back in
* Change the color to indigo
* Pro tier pricing information now matches the marketing site
* Update the button styles to secondary
* WIP adding separate links to Parent and Root runs
* TextLink now supports optional shortcuts
* Adds shortcut keys to the root and parent links + the shortcut help panel
* Adds new icons for root and parent
* root friendlyId works
* Updates icons for jump to root and parent
* Copy tweak
* Improve how the Free tier shows no preview branches
* Improve the wording in the tooltip
* Align the x icon better
* Show price for additional preview branches
* Change the shortcut key
* Fixes button alignment
* Adds nested dependencies task hello-world
* Fixes typo “Cancelled”
* Removes taskIdentifier, not needed
* Removes unused taskIdentifier
This PR enables setting project build settings in the settings page:
root directory, install command and trigger config file path.
For most cases there should be no need to set these explicitly.
* Mo-Stashed changes
* fix(core): prettyPrintingPacket will now do a structuredClone on non-circular references instead of outputting [Circular]
This also fixes an issue with replaying of runs that include
non-circular references
* Add external build data and image platform to the get deployment endpoint
* If provided, attach to an existing deployment in the deploy command
* Check status for existing deployments
* Add changeset
* feat(engine): Improve execution stalls troubleshooting, align dev and prod behavior, adding heartbeats.yield utility
* A few improvements via the 🐇 review
* Allow treating EXECUTION stalls as OOM errors, improve the error message, add more information to the docs, improve resource monitor and add it to the docs
* Add changeset
* Add a CLI command to list and view env vars
* Add changeset
* Restrict pemissions on env files created with `env pull`
* Escape env vars when exporting to file
* Switch changeset to patch
* Fix settigns page delete project width issue
* Apply a couple of touch-ups to the project settings page
* Add UI flow to connect gh repos
* Enabling adding another gh account in the ui
* Enable connecting a repo to a project
* Enable updating git settings
* Enable disconnecting gh repos from a project
* Remove prisma migration drifts
* Hide git settings when github app is disabled
* Fix migration order
* Avoid using `location` to avoid SSR issues
* Make branch tracking optional
* Disable save buttons when there are no field changes
* Disable delete project button unless the input matches the project slug
* Show connected repo connectedAt date
* Check that tracking branch exists when updating git settings
* Show tracking branch hint in the deployments page
* Fix positioning issue of the pagination pane in the deployments page
* Use mono font for branch names
* Add link to git settings
* Show tracking branch hint for the preview env too
* Add a confirmation prompt on repo disconnect
* Add link to configure repo access in gh
* Add rel prop to github links
* Automatically open repo connection modal after app installation
* Apply some fixes suggested by mr rabbit
* Fix flash cookie issue
* Extract project settings actions into a service
* Extract project settings loader into a presenter service
* Introduce neverthrow for error handling
* Try out neverthrow for error handling in the project setting flows
* Move env gh branch resolution to the presenter service
* feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf
* cleaned up signal handling and resolved some valid 🐇 issues
* Add schemas for gh app installations
* Implement gh app installation flow
* Make the gh app configs optional
* Add additional org check on gh app installation callback
* Save account handle and repo default branch on install
* Do repo hard deletes in favor of simplicity
* Disable github app by default
* Fix gh env schema union issue
* Use octokit's iterator for paginating repos
* Parse gh app install callback with a discriminated union
* Remove duplicate env vars
* Use bigint for github integer IDs
* Sanitize redirect paths in the gh installation and auth flow
* Regenerate migration after rebase on main to fix ordering
* Handle gh install updates separately from new installs
* feat(supervisor): dynamic queue consumer pool
* add changeset
* fix: correctly handle zero median and even samples
* feat(supervisor): consumer pool metrics
* fix tests
* more tests and fixes
* decrease default scaling cooldowns
* don't treat initial pool size as scale up
* handle scale down when queue length drops to zero
* remove changeset, supervisor changes only
* add damping factor env var
* feat(webapp): upgrade otel packages and add more metrics
* add env var to disable additional detectors
* expose more prisma metrics
* chore(webapp): drop node 16 support
* Added MCP to the intro
* Added human-in-the-loop and new build extensions
* Waitpoint notes
* Moved openai guardrails example to Python
* Added a connection limit note
* Added Supabase + Prisma note to supabase auth
* Updates based on what the rabbit said
* Moved the supavisor section to the prismaExtension docs
* docs: update triggering.mdx fixing wrong argument numeration for options
Update the docs triggering.mdx: text indicating options argument position inside `tasks.trigger` function.
Options object is a third argument instead of stated second argument.
* update triggering.mdx to change options argument position in batchTrigger
* feat(mcp): add wait_for_run_to_complete tool so agents don't spam the get_run_details call after triggering
This also fixes the search docs MCP tool
* Install mcp using the latest tag, not the specific version
* more eager dequeuing, queue cooloff periods, return workerQueueLength when dequeueing
* Cache worker group authentication and remove old self-hosted worker code (only managed is currently supported)
* add additional spans during dequeue
* Add env vars and additional spans
* Initial work on upgrading to 6.14.0
Set the output to node_modules still to make it easier
* Use ./generated Prisma folder, update types to fix issues
* Docker compose restart Clickhouse
* Prisma instrumentation update
* Docker
* Removed database dockerignore file, add generated prisma client to the top-level one
* Delete v3-catalog package.json
* Resolved pnpm lock file
* Log errors for very slow queries
* Create schema and migration for organization access tokens
* Add helpers for creating and authenticating OATs
* Adapt the auth service to also accept OATs
* Accept OATs in the whoami v2 endpoint
* Enable deployments with the CLI using OATs
* Avoid reading env variables directly in the token utils
* Remove duplicate cli token utils
* Validate ENCRYPTION_KEY length when parsing env vars
* Make token utils a server-only module
* Disallow revoking already revoked OATs
* Simplify generics in authenticateRequest
* Use 32 bytes mock encryption key in the test setup
* Update dummy encryption key values in tests and templates
* Add a column in the OATs table to differentiate between user and system generated
* Simplify args for v3ProjectPath
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Add index on org id and createdAt
* Avoid storing the encrypted oat token and its obfuscated version in the DB at all
It is a safer approach. Also we do not need to ever read the decrypted token value after creation.
* Fix prisma update condition
* Add token type to the OAT table index
* Accept OATs in the mcp auth flow
* Simplify env auth flow around the /projects endpoints
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Add MCP Server documentation to documentation site
This change introduces comprehensive documentation for the Trigger.dev MCP Server, covering its installation, usage, and available tools. The addition is necessary to provide users with detailed guidance on leveraging the MCP Server for managing Trigger.dev projects, tasks, and deployment processes.
- Updated `docs.json` to include a new section for the MCP Server documentation, improving navigation and accessibility.
- Added `mcp-introduction.mdx` as an introductory guide to the MCP Server, detailing its purpose and installation steps.
- Included `mcp-tools.mdx`, a document outlining each tool available in the MCP Server, complete with descriptions and example usages.
- The new content is aimed at improving user understanding and efficiency in using Trigger.dev's MCP Server capabilities.
* Refactor MCP installation documentation
The changes were necessary to streamline the MCP installation guidance and align our documentation with the latest client support capabilities. Removing the manual installation section and redundant information helps reduce confusion for users.
- Updated installation instructions to lead with auto-configuration for supported clients.
- Cleaned up the options and examples sections to enhance clarity.
- Reformatted client configuration details for better readability.
- Removed outdated links in the "Next Steps" section, focusing on more direct resources such as available MCP tools.
* mcp docs edits
* Add debugMode to get_run_details
* add cancel_run
* Add list deployments tools
* Add list preview branches tool
* Docs: adds agent rules page
* Adds redirect for the CLI path to work
* Adds subagent install image
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* fix(v3): prevent saturated queues from dominating dequeue attempts by adding a cooloff period of successive failed dequeues
* Add env vars and add a marqs shutdown on SIGTERM/INT
* fix: span events are optional, as logs don't have them
* remove CI: true env var to prevent massive tool responses
* fix github resolver relative path
* prevent the dev command from asking for user input when not interactive
* Remove lodash.get because it's deprecated
* span output can be object other than an object, like a string
* prevent large traces from causing get_run_details failures
* Decreased max log lines, made it a tool input property
* Some install mcp tweaks
* Remove variables from dequeue log message
* Continue snapshot throw json
* Waiting for deploy error removed
* Realtime ECONNRESET is expected
* Redis worker logErrors changes, removed ids
* Preview branch without a branch shouldn't log an error, it's a user provided issue
* "Task run is not in a cancellable state" isn't an error, it's expected
* "CreateCheckpointService: Child run already resumed" is expected
* "CreateCheckpointService: Batch already resumed" is expected
* "Failed to insert events, will attempt bisection" changed to info, we have errors for complete failures
* Ignore "PrismaClient error"
* Don't log Redis worker DLQ errors if we're ignoring
* "Failed to parse machine config" is fine, sometimes a config is null or undefined
* "Failed to parse machine config" for v3
* MetadataTooLargeError shouldn't log an error
* Don't log an error when the snapshot shouldn't be created, it's normal for this to happen
* Slack alerts, skip `account_inactive` errors
* v3 finalize run with no locked isn't an error
* Another false error
* Finalize run CRASHED runs were logging errors
* All slack alert errors are warnings except invalid blocks
* Adds new features table to top of v4 upgrade guide
* Adds wait idempotency to wait-until, wait-for, and wait-for-token pages
* Adds new priority docs page and updates the v4 upgrade guide
* Adds new task lifecycle hooks
* Removes the message about requiring tasks to be exported
* Adds new global lifecycle hooks section
* Moves sections from upgrade guide into the table
* Adds hidden task page
* Improves the global lifecycle hooks section
* Updates middleware and locals section
* Adds new useWaitToken page to the react hooks section
* Adds a new ai.tool section
* Moves Docker (legacy) page into self-hosting section
* Removes known issues from v4 upgrade guide
* Replace “toolTask” with “ai.tool” in the Streams page example
* Renames guide to “Migrating from v3” and adds redirect
* Remove references to v4
* Removes changelog from migration guide
* The installation guide now references `@latest update`
* Changes all references from `/sdk/v3` to `/sdk`
* Updates @v4-beta to @latest
* Fixed broken link
* Fixes broken link
* Adds an upgrade to v4 using AI section
* Fixes 2 broken links
* Adds an entry for targetting preview branches
* Updates the run statuses
* Adds boolean helpers section to the runs and realtime pages
* Updates the concurrency page
* Updates the test page to include the new options
* Adds SDK and curl options for the preview branch targeting
* Updates new bulk actions page
* Remove the releasing concurrency section
* Got rid of some more @v4-beta mentions
* Improved rate limit docs
* Improved migrating docs
* Removed commented sections of the docs
* useWaitToken hook
* Fixed the description
* Fix for missing test image
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Co-authored-by: Dan <8297864+D-K-P@users.noreply.github.com>
* fix: external traces now respect parent sampling, and prevent broken traces when there is no external trace context
* Add changeset
* improve trace flag handling and better internal host checking
* the traceFlags are now being properly passed through as a number
* add tier scheduling support to supervisor
* add billing info to dequeued message w/o cache
* add cache with best effort invalidation
* fix invalidate circular dep
* add changeset
* use new plan type on runs as fallback during dequeue
* tidy up
* be more explicit with plan type fallback
* remove additional billing check from hot path
* switch to placement tags
* update changeset
* update platform package
* start using new entitlement response
* ensure skipChecks optimization validates at batch level
* add optional items to add to queue manager limits
* make the bool env helper only accept boolean defaults
* remove redundant private field
* update placement tag helper to prevent unsupported tags
Small change that shows the github user in the deployments page in cases
where it was triggered by the github app integration.
These deployments are not triggered using a PAT, so we cannot associate
them with a Trigger user ID.
This PR adapts the deployment command to also evaluate build server env
variables when creating the git meta. The build server will initially
only support deployments triggered by the github app, but we might add
other git providers in the future.
Sticking to the GH actions naming convention for the env variables set
by the build server in favor of consistency.
* fix: sentry memory leak by disabling includeLocalVariables
* Enhance heap snapshot consistency and labeling
To facilitate more accurate and consistent heap memory snapshots, a new function forceConsistentGC was added before taking a snapshot. This ensures the garbage collector (GC) runs multiple times, stabilizing the heap state for more reliable analysis. This is particularly helpful when debugging memory-related issues.
Updates to the memory-leak-detector script now allow labeling of snapshot runs using the --label flag. This helps in distinguishing different runs for easier tracking and comparison of memory usage across test sessions. Additionally, the --expose-gc flag ensures that the GC can be manually triggered during test runs, leading to more consistent memory states and potentially uncovering hidden memory leaks.
* Refactor forceConsistentGC for improved readability
The function forceConsistentGC was refactored to enhance code readability and consistency. The main improvements include:
- Updated syntax for consistent string quotation and spacing.
- Simplified garbage collection by removing specific major/minor GC calls, as the distinction isn't necessary.
- Implemented minor changes to arrow function formatting for consistency.
These changes neither impact the program logic nor the function behavior but help maintain code quality standards and readability.
* Fix memory leak by removing request.clone() usage
Identified that the memory leak in the project was linked to the usage of request.clone() within the `@sentry/remix` package's callRouteAction handler. Although initially suspected as a Sentry issue, the problem appears to arise from the handling of request.clone() in Remix version 2.1.0. By removing the call to request.clone(), the memory leak has been resolved.
- Introduced garbage collection execution before snapshot to manage memory allocation effectively.
- Improved error handling and timeout mechanisms in the memory leak detector to enhance its resilience during runtime.
- Expanded testing for both GET and POST requests to monitor and validate potential memory leaks better. The POST requests involve sending large payloads to stress-test the system.
- The modification particularly focuses on enhancing robust memory tracking and providing detailed progress reporting during request phases.
* patch @sentry/remix to prevent memory leaks
* Fix pnpm lock
* undo some unrelated changes
* The logger now supports metadata
* Added metadata to ServiceValidationError in some critical places
* Don't ack the heartbeat if there's a mismatch, it might prevent a brand new one
* Don't ack the heartbeat inside stalled. By returning it will be acked IF the deduplication key matches
* Only start an attempt if not finished. Send message to worker if pending executing
* Fix the exit process reason tet
* Fixed cancelling test since bug fix
The old behaviour was wrong for pending executing in the test
* set correct run status on snapshot after dequeue
* set run status back to PENDING when we requeue
* remove retrying after failure status from v4 and fix tests
* fix one last test
* Adds a background image dashboard wrapper
* Dashboard background image
* Adds a background image to the welcome onboarding page
* Background is constructed of 3 images
* Adds the background to the create org page
* Adds a background to the choose plan page
* Change the default button spinner color to white
* Adds the background to the create new project page
* Updates the invite team member page
* Adds background image to received invite page
* Reordered react hooks + frontend sections
* Updated the overview and nav
* Separated out SWR hooks
* Restructured metadata sections
* Improved backend docs
* Fixed broken link
* Fixed broken links
* Updated the structure
* Restructured overview
* Updated examples cards
* Improved overview
* Updated how it works
* Updated auth
* Added type safety to the run object page
* made the subscribe description clearer
* Fixed links in triggering
* Fixed link
* Removed examples footers
* Copy tweak
* Consolidated metadata and subscribe pages
* moved metadata task examples to metadata
* Fixed links
* Fixing links like zelda
* Removed dead import
* Clearer titles
* Cap R for Realtime
* Fixes broken link
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Map new allowedMasterQueues → allowedWorkerQueues
* ClickHouse worker_queue on task runs
* Added the Region to the run inspector
* Pass a region in when triggering
* Added a changeset
* Added triggering regions docs
* Added region to the ctx
* Fix for backfiller masterQueue/workerQueue
* Add manual monorepo setup guide for Trigger.dev
This commit introduces a new "Manual setup" guide for setting up Trigger.dev in projects, specifically focused on monorepo configurations. The guide outlines two primary approaches for monorepos: creating a dedicated tasks package and integrating tasks directly within apps. This detailed documentation aims to help developers manually configure their projects, bypassing automated steps, and understanding the setup better.
- Provides step-by-step instructions for both 'Tasks as a package' and 'Tasks in apps' approaches.
- Includes example configurations for various package managers, environment setups, and runtime options.
- Enhances user understanding of Trigger.dev's configuration requirements in complex monorepo environments.
The purpose of adding this guide is to enable developers to seamlessly integrate Trigger.dev into their monorepos, whether they choose to abstract tasks into packages or embed them within individual applications. This flexibility supports diverse project structures while maintaining consistency with Trigger.dev's operational prerequisites.
* Correct tasks usage in documentation
The 'tasks as package' section in the documentation had an incorrect example under 'Use tasks in your apps' which needed correction to align with the actual package usage.
- Fixed incorrect import of tasks by updating to the correct import from '@repo/tasks/trigger'.
- Updated the syntax to use 'tasks.trigger' with type parameters, following the new pattern established for triggering tasks with TypeScript.
- Added error handling to catch and log errors during task execution, returning a meaningful error message instead of just failing silently.
This update ensures developers have an accurate reference when implementing tasks in their applications, especially given the breaking changes in TypeScript compatibility due to recent package updates.
* Update package configuration for Zod 3-4 compatibility
The recent Zod package updates from version 3 to 4 introduce breaking changes in TypeScript compatibility that require adjustments in our project configuration files. The primary updates involve adding type annotations and modifying import statements to support the shift without breaking existing functionality.
- Updated `package.json` and initialization files to align with new compatibility requirements.
- Modified server task examples to explicitly use type imports for improved error handling and consistency.
- Adjusted workspace and project settings to maintain compatibility and improve configuration clarity.
- Provided references to current examples for better implementation guidance.
* More manual setup guide steps
* Fix bun docs link
* Initial Regions page
* Fix for bad attribute name
* Switching regions is working. Some style improvements
* Use a dialog for confirmation, not great yet
* Added allowedMasterQueues
* Improves flag icons
* Improves the region switch modal with more info
* Adds “suggest a region” table row
* New icons for the buttons
* Improved the tooltip information
* New “small” badge style
* Make the default badge live in its column
* Better DO icon size
* Remove unused export of regions options
* Show upgrade message for free users to get static IPs
* Admins can view all regions and switch at will
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Update zod package to version 3.25.76 across all modules
Update the zod library from version 3.23.8 to 3.25.76 in multiple package files to ensure compatibility and take advantage of new features or bug fixes introduced in recent releases. Keeping all modules synchronized with the latest version of zod helps maintain consistency across the project and reduces potential compatibility issues.
- Modified zod version in apps/supervisor, webapp, and various internal packages.
- Updated zod references in pnpm-lock.yaml to reflect the new version.
- Ensure dependencies that rely on zod are using the updated version to avoid mismatches.
* Add changeset
* Add payload schema handling for task indexing
This change introduces support for handling payload schemas during task indexing. By incorporating the `payloadSchema` attribute into various components, we ensure that each task's payload structure is clearly defined and can be validated before processing.
- Updated the TaskManifest and task metadata structures to include an optional `payloadSchema` attribute. This addition allows for more robust validation and handling of task payloads.
- Enhanced several core modules to export and utilize the new `getSchemaToJsonSchema` function, providing easier conversion of schema types to JSON schemas.
- Modified the database schema to store the `payloadSchema` attribute, ensuring that the payload schema information is persisted.
- The change helps in maintaining consistency in data handling and improves the integrity of task data across the application.
* Refactor: Remove getSchemaToJsonSchema in favor of schemaToJsonSchema
The `getSchemaToJsonSchema` function was removed and replaced with `schemaToJsonSchema` across the codebase. This update introduces a new `@trigger.dev/schema-to-json` package to handle conversions of schema validation libraries to JSON Schema format, centralizing the functionality and improving maintainability.
- Removed `getSchemaToJsonSchema` exports and references.
- Added new schema conversion utility `@trigger.dev/schema-to-json`.
- Updated `trigger-sdk` package to utilize `schemaToJsonSchema` for payloads.
- Extensive testing coverage included to ensure conversion accuracy across various schema libraries including Zod, Yup, ArkType, Effect, and TypeBox.
- The update ensures consistent and reliable schema conversions, facilitating future enhancements and supporting additional schema libraries.
* Add support for Zod 4 in schema-to-json
This change enhances the schema-to-json package by adding support for Zod version 4, which introduces the native `toJsonSchema` method. This method facilitates a direct conversion of Zod schemas to JSON Schema format, improving performance and reducing reliance on the `zod-to-json-schema` library.
- Updated README to reflect Zod 4 support with native method and retained support for Zod 3 via existing library.
- Modified package.json to allow installation of both Zod 3 and 4 versions.
- Implemented handling for Zod 4 schemas in `src/index.ts` using their native method.
- Added a test case to verify the proper conversion of Zod 4 schemas to JSON Schema.
- Included a script for updating the package version based on the root package.json.
- Introduced a specific TypeScript config for source files.
* Revise schema-to-json for bundle safety and tests
The package @trigger.dev/schema-to-json has been revised to ensure bundle safety by removing direct dependencies on schema libraries such as Zod, Yup, and Effect. This change minimizes bundle size and enhances tree-shaking by allowing external conversion libraries to be utilized only at runtime if necessary. As a result, the README was updated to reflect this usage pattern.
- Introduced `initializeSchemaConverters` function to load necessary conversion libraries at runtime, keeping the base package slim.
- Adjusted test suite to initialize converters before tests, ensuring accurate testing of schema conversion capabilities.
- Updated `schemaToJsonSchema` function to dynamically check for availability of conversion libraries, improving flexibility without increasing the package size.
- Added configuration files for Vitest to support the new testing framework, reflecting the transition from previous test setups.
These enhancements ensure that only the schema libraries actively used in an application are bundled, optimizing performance and resource usage.
* Refine JSON Schema typing across packages
The changes introduce stricter typing for JSON Schema-related definitions, specifically replacing vague types with more precise ones, such as using `z.record(z.unknown())` instead of `z.any()` and `Record<string, unknown>` in place of `any`. This is part of an effort to better align with common practices and improve type safety in the packages.
- Updated the `payloadSchema` in several files to use `z.record(z.unknown())`, enhancing the type strictness and consistency with JSON Schema Draft 7 recommendations.
- Added `@types/json-schema` as a dependency, utilizing its definitions for improved type clarity and adherence to best practices in TypeScript.
- Modified various comments to explicitly mention JSON Schema Draft 7, ensuring developers are aware of the JSON Schema version being implemented.
- These adjustments are informed by research into how popular libraries and tools handle JSON Schema typing, aiming to integrate best practices for improved maintainability and interoperability.
* Add JSON Schema examples using various libraries
The change introduces extensive examples of using JSON Schemas in the 'references/hello-world' project within the 'trigger.dev' repository. These examples utilize libraries like Zod, Yup, and TypeBox for JSON Schema conversion and validation. The new examples demonstrate different use cases, including automatic conversion with schemaTask, manual schema provision, and schema conversion at build time. We also updated the dependencies in 'package.json' to include the necessary libraries for schema conversion and validation.
- Included examples of processing tasks with JSON Schema using libraries such as Zod, Yup, TypeBox, and ArkType.
- Showcased schema conversion techniques and type-safe JSON Schema creation.
- Updated 'package.json' to ensure all necessary dependencies for schema operations are available.
- Created illustrative scripts that cover task management from user processing to complex schema implementations.
* Refactor SDK to encapsulate schema-to-json package
The previous implementation required users to directly import and initialize functions from the `@trigger.dev/schema-to-json` package, which was not the intended user experience. This change refactors the SDK so that all necessary functions and types from `@trigger.dev/schema-to-json` are encapsulated within the `@trigger.dev/*` packages.
- The examples in `usage.ts` have been updated to clearly mark `@trigger.dev/schema-to-json` as an internal-only package.
- Re-export JSON Schema types and conversions in the SDK to improve developer experience (DX).
- Removed unnecessary direct dependencies on `@trigger.dev/schema-to-json` from user-facing code, ensuring initialization and conversion logic is handled internally.
- Replaced instances where users were required to manually perform schema conversions with automatic handling within the SDK for simplification and better maintainability.
* Add JSONSchema type for payloadSchema in tasks
The change was necessary to improve type safety by using a proper JSONSchema type definition instead of a generic Record<string, unknown>. This enhances the developer experience and ensures that task payloads conform to the JSON Schema Draft 7 specification. The JSONSchema type is now re-exported from the SDK for user convenience, hiding internal complexity and maintaining a seamless developer experience.
- Added JSONSchema type based on Draft 7 specification
- Updated task metadata and options to use JSONSchema type
- Hid internal schema conversion logic from users by re-exporting types from SDK
- Improved bundle safety and dependency management
* Add JSON schema testing and revert package dependencies
This commit introduces a comprehensive set of JSON schema testing within the monorepo, specifically adding a new test project in `references/json-schema-test`. This includes a variety of schema definitions and tasks utilizing multiple validation libraries to ensure robust type-checking and runtime validation.
Additionally, the dependency versions for `@effect/schema` have been adjusted from `^0.76.5` to `^0.75.5` to maintain compatibility across the project components. This ensures consistent behavior and compatibility with existing code bases without introducing breaking changes or unexpected behavior due to version discrepancies.
Key updates include:
- Added new test project with extensive schema validation tests.
- Ensured type safety across various task implementations.
- Reverted dependency versions to ensure compatibility.
- Created multiple schema tasks using libraries like Zod, Yup, and others for thorough testing.
* Refactor JSON Schema test files for clarity
Whitespace and formatting changes were applied across the `json-schema-test` reference project to enhance code readability and cohesion. This included removing unnecessary trailing spaces and ensuring consistent indentation patterns, which improves maintainability and readability by following the project's code style guidelines.
- Renamed JSONSchema type annotations to adhere to TypeScript conventions, ensuring that all schema definitions properly satisfy the JSONSchema interface.
- Restructured some object declarations for improved clarity, especially within complex schema definitions.
- These adjustments are crucial for better future maintainability, reducing potential developer errors when interacting with these test schemas.
* Fixed some stuff
* WIP
* we now convert schema to jsonSchema on the CLI side via the indexing
* Remove the json-schema-test reference project
* Improve schema-to-json peer deps and fix effect schema
* Explain the casting and match the version numbers
* Fixed a bunch more schema stuff
* Don't clean files that might be written to
* Don't use a custom version of vitest in the new package
* fix attw in schema-to-json
* prevent infinite retry loop for unicode errors
* structured logs for prisma events
* preserve all prisma event fields
* either use structured logs or stdout, never both
* split runs repo tests
* decrease test shards to 8
* First draft billing alerts page
* Budget alert form working
* Don't let free plan users change the billing alert amount
* Fix missing key in map in the form
* Disable queues/org from admin API endpoint
* Don't allow resuming if runsEnabled is false
* Refer to "Billing alerts" not "Plans"
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Form missing dependencies fix
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Deal with thrown errors, fix for duplicating email fields
* Added a RuntimeEnvironment organizationId index
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Cleanup context and execution creation, cache stuff, add parent and root task run ids
* more efficient by using friendly IDs instead of doing joins
* metadata.root/parent now reference current run when run has no root/parent
* Adding changeset
* try to make test less flaky
* Clean imports
* Another attempt to fix the flaky test
* Fix usage by still passing durationMs and costInCents to the execution, just not the run.ctx
* Improve dynamic flush scheduler for otel data
The changes introduce a more flexible and adaptive dynamic flush scheduler to address production issues where the system wasn't flushing data fast enough, causing memory growth and crashes. This issue arises from the existing scheduler handling only a single flush at a time, limiting concurrency and failing to cope with the influx of logs.
- Added configuration options for setting minimum and maximum concurrency levels, maximum batch size, and memory pressure threshold. These parameters ensure that flush operations adjust dynamically based on workload and pressure.
- Implemented `pLimit` to facilitate concurrent flush operations, with adjustments made according to batch queue length and memory pressure.
- Metrics reporting improvements were added to monitor the dynamic behavior of the flush scheduler, aiding in identifying performance issues and optimizing the operation accordingly.
* Implement load shedding for TaskEvent records
This change introduces load shedding mechanisms to manage TaskEvent
records, particularly those of kind LOG, when the system experiences
high volumes and is unable to flush to the database in a timely
manner. The addition aims to prevent overwhelming the system and
ensure critical tasks are prioritized.
- Added configuration options for `loadSheddingThreshold` and
`loadSheddingEnabled` in multiple modules to activate load shedding.
- Introduced `isDroppableEvent` function to allow specific events to
be dropped when load shedding is enabled.
- Ensured metrics are updated to reflect dropped events and load
shedding status, providing visibility into system performance
during high load conditions.
- Updated loggers to inform about load shedding state changes,
ensuring timely awareness of load management activities.
* Fix undefined 'queuePressure' variable in DynamicFlushScheduler
The 'queuePressure' variable was being used without being defined
in the DynamicFlushScheduler class, causing potential runtime
errors. This commit adds the missing definition and ensures that
the variable is correctly calculated based on the 'totalQueuedItems'
and 'memoryPressureThreshold'.
- Addressed code inconsistencies and improved formatting.
- Defined 'queuePressure' in the 'adjustConcurrency' method
to prevent potential undefined errors.
- Enhanced readability by maintaining consistent spacing and
format across the file, contributing to the stability and
maintainability of the code.
- Adjusted batch size logic based on the newly defined 'queuePressure'
variable.
* Refactor concurrency adjustment logic in scheduler
The concurrency adjustment logic in the dynamic flush scheduler has been refactored to improve clarity and maintainability. This change moves the calculation of pressure metrics outside of the conditional blocks to ensure they are always determined prior to decision-making.
- The queue pressure and time since last flush calculations were moved up in the code to be independent of the 'backOff' condition.
- This refactor sets up the groundwork for more reliable concurrency scaling and better performance monitoring capabilities. The overall logic of adjusting concurrency based on system pressure metrics remains unchanged.
This adjustment addresses ongoing issues with the scheduler that were not resolved by previous changes.
* Some tweaks
* Sentry WIP
* Configure sentry for uploading and releasing during the publish webapp step
* Delete source maps after uploading
* Forward logger.error calls to sentry through Logger.onError
* Couple tweaks to the dockerfile
* Premerge run batch before sending it to clickhouse
* Pre-order batch items before sending to clickhouse in favor of performance
* existing
* Emit event on batch flushes
* When merging batches, keep the last occurrence items with the same version
* Add a couple of tests
* Initial burst changes to queue page
* Added tooltip and changed wording around
* View runs from Queues page
* Fix for ugly Version filter "Current" badge
The normal database needs to be mocked because an import somewhere is
using it which causes issues with the test container dbs and hanging at
the end.
Strategy copied from the trigger test file.
* Set the default replication concurrency to 2 for self-hosters
100 was a bit crazy
* Made the run repository an interface, deferring just to CH for now
* Added run repository feature flag, allowing passing a default when getting a flag
* Switch run repository using a feature flag
* Added spans
* Pass the default repository in, so we can try Postgres in the tests
* Fallback to Postgres if ClickHouse errors
* Update feature flags API endpoint
Added experimental_devProcessCwdInBuildDir config option to opt-in to new process.cwd behavior when executing tasks in the dev CLI. Currently process.cwd maps to the "root" of your trigger.dev project (the directory that contains your trigger.config.ts file). Setting experimental_devProcessCwdInBuildDir to true changes process.cwd to instead be the temporary build directory inside of the .trigger directory.
Removes the `releaseConcurrencyOnWaitpoint` option on queues and the `releaseConcurrency` option on various wait functions. Replaced with the following default behavior:
- Concurrency is never released when a run is first blocked via a waitpoint, at either the env or queue level.
- Concurrency is always released when a run is checkpointed and shutdown, at both the env and queue level.
Additionally, environment concurrency limits now have a new "Burst Factor", defaulting to 2.0x. The "Burst Factor" allows the environment-wide concurrency limit to be higher than any individual queue's concurrency limit. For example, if you have an environment concurrency limit of 100, and a Burst Factor of 2.0x, then you can execute up to 200 runs concurrently, but any one task/queue can still only execute 100 runs concurrently.
We've done some work cleaning up the run statuses. The new statuses are:
-`PENDING_VERSION`: Task is waiting for a version update because it cannot execute without additional information (task, queue, etc.)
-`QUEUED`: Task is waiting to be executed by a worker
-`DEQUEUED`: Task has been dequeued and is being sent to a worker to start executing.
-`EXECUTING`: Task is currently being executed by a worker
-`WAITING`: Task has been paused by the system, and will be resumed by the system
-`COMPLETED`: Task has been completed successfully
-`CANCELED`: Task has been canceled by the user
-`FAILED`: Task has failed to complete, due to an error in the system
-`CRASHED`: Task has crashed and won't be retried, most likely the worker ran out of resources, e.g. memory or storage
-`SYSTEM_FAILURE`: Task has failed to complete, due to an error in the system
-`DELAYED`: Task has been scheduled to run at a specific time
-`EXPIRED`: Task has expired and won't be executed
-`TIMED_OUT`: Task has reached it's maxDuration and has been stopped
We've removed the following statuses:
-`WAITING_FOR_DEPLOY`: This is no longer used, and is replaced by `PENDING_VERSION`
-`FROZEN`: This is no longer used, and is replaced by `WAITING`
-`INTERRUPTED`: This is no longer used
-`REATTEMPTING`: This is no longer used, and is replaced by `EXECUTING`
We've also added "boolean" helpers to runs returned via the API and from Realtime:
-`isQueued`: Returns true when the status is `QUEUED`, `PENDING_VERSION`, or `DELAYED`
-`isExecuting`: Returns true when the status is `EXECUTING`, `DEQUEUED`. These count against your concurrency limits.
-`isWaiting`: Returns true when the status is `WAITING`. These do not count against your concurrency limits.
-`isCompleted`: Returns true when the status is any of the completed statuses.
-`isCanceled`: Returns true when the status is `CANCELED`
-`isFailed`: Returns true when the status is any of the failed statuses.
-`isSuccess`: Returns true when the status is `COMPLETED`
This change adds the ability to easily detect which runs are being counted against your concurrency limit by filtering for both `EXECUTING` or `DEQUEUED`.
Now each worker gets it's own pLimit concurrency limiter, and we will only ever dequeue items where there is concurrency capacity, preventing incorrectly retried jobs due to visibility timeout expiry
Update base images to latest compatible versions. The `node-22` runtime now uses v22.16.0 and `bun` uses the latest v1.2.18 release. The default `node` runtime is unchanged and points at v21.7.3.
Fixes an issue with realtime when re-subscribing to a run, that would temporarily display stale data and the changes. Now when re-subscribing to a run only the latest changes will be vended
The main trigger.dev webapp, which powers it's API and dashboard and makes up the docker image that is produced as an OSS image, is a Remix 2.1.0 app that uses an express server, written in TypeScript. The following subsystems are either included in the webapp or are used by the webapp in another part of the monorepo:
- `@trigger.dev/database` exports a Prisma 5.4.1 client that is used extensively in the webapp to access a PostgreSQL instance. The schema file is [schema.prisma](mdc:internal-packages/database/prisma/schema.prisma)
- `@trigger.dev/database` exports a Prisma 6.14.0 client that is used extensively in the webapp to access a PostgreSQL instance. The schema file is [schema.prisma](mdc:internal-packages/database/prisma/schema.prisma)
- `@trigger.dev/core` is a published package and is used to share code between the `@trigger.dev/sdk` and the webapp. It includes functionality but also a load of Zod schemas for data validation. When importing from `@trigger.dev/core` in the webapp, we never import the root `@trigger.dev/core` path, instead we favor one of the subpath exports that you can find in [package.json](mdc:packages/core/package.json)
- `@internal/run-engine` has all the code needed to trigger a run and take it through it's lifecycle to completion.
- `@trigger.dev/redis-worker` is a custom redis based background job/worker system that's used in the webapp and also used inside the run engine.
@@ -31,7 +31,10 @@ We originally the Trigger.dev "Run Engine" not as a single system, but just spre
- The batch trigger API endpoint is [api.v1.tasks.batch.ts](mdc:apps/webapp/app/routes/api.v1.tasks.batch.ts)
- Setup code for the prisma client is in [db.server.ts](mdc:apps/webapp/app/db.server.ts)
- The run engine is configured in [runEngine.server.ts](mdc:apps/webapp/app/v3/runEngine.server.ts)
- All the "services" that are found in app/v3/services/**/*.server.ts
- All the "services" that are found in app/v3/services/\*_/_.server.ts
- The code for the TaskEvent data, which is the otel data sent from tasks to our servers, is in both the [eventRepository.server.ts](mdc:apps/webapp/app/v3/eventRepository.server.ts) and also the [otlpExporter.server.ts](mdc:apps/webapp/app/v3/otlpExporter.server.ts). The otel endpoints which are hit from production and development otel exporters is [otel.v1.logs.ts](mdc:apps/webapp/app/routes/otel.v1.logs.ts) and [otel.v1.traces.ts](mdc:apps/webapp/app/routes/otel.v1.traces.ts)
- We use "presenters" to move more complex loader code into a class, and you can find those are app/v3/presenters/**/*.server.ts
- We use "presenters" to move more complex loader code into a class, and you can find those are app/v3/presenters/\*_/_.server.ts
- All the "services" that are found in app/v3/services/\*_/_.server.ts
- The code for the TaskEvent data, which is the otel data sent from tasks to our servers, is in both the [eventRepository.server.ts](mdc:apps/webapp/app/v3/eventRepository.server.ts) and also the [otlpExporter.server.ts](mdc:apps/webapp/app/v3/otlpExporter.server.ts). The otel endpoints which are hit from production and development otel exporters is [otel.v1.logs.ts](mdc:apps/webapp/app/routes/otel.v1.logs.ts) and [otel.v1.traces.ts](mdc:apps/webapp/app/routes/otel.v1.traces.ts)
- We use "presenters" to move more complex loader code into a class, and you can find those are app/v3/presenters/\*_/_.server.ts
@@ -431,28 +431,6 @@ export async function POST(request: Request) {
}
```
### tasks.triggerAndPoll()
Triggers a task and polls until completion. Not recommended for web requests as it blocks until the run completes. Consider using Realtime docs for better alternatives.
```ts
import { tasks } from "@trigger.dev/sdk/v3";
import type { emailSequence } from "~/trigger/emails";
export async function POST(request: Request) {
const data = await request.json();
const result = await tasks.triggerAndPoll<typeof emailSequence>(
"email-sequence",
{
to: data.email,
name: data.name,
},
{ pollIntervalMs: 5000 }
);
return Response.json(result);
}
```
### batch.trigger()
Triggers multiple runs of different tasks at once, useful when you need to execute multiple tasks simultaneously.
@@ -84,17 +84,17 @@ branch are tagged into a release periodically.
2. Once the app is running click the magic link button and enter your email. You will automatically be logged in, since you are running locally. Create an Org and your first project in the dashboard.
## Manual testing using v3-catalog
## Manual testing using hello-world
We use the `<root>/references/v3-catalog` subdirectory as a staging ground for testing changes to the SDK (`@trigger.dev/sdk` at `<root>/packages/trigger-sdk`), the Core package (`@trigger.dev/core` at `<root>packages/core`), the CLI (`trigger.dev` at `<root>/packages/cli-v3`) and the platform (The remix app at `<root>/apps/webapp`). The instructions below will get you started on using the `v3-catalog` for local development of Trigger.dev (v3).
We use the `<root>/references/hello-world` subdirectory as a staging ground for testing changes to the SDK (`@trigger.dev/sdk` at `<root>/packages/trigger-sdk`), the Core package (`@trigger.dev/core` at `<root>packages/core`), the CLI (`trigger.dev` at `<root>/packages/cli-v3`) and the platform (The remix app at `<root>/apps/webapp`). The instructions below will get you started on using the `hello-world` for local development of Trigger.dev.
### First-time setup
First, make sure you are running the webapp according to the instructions above. Then:
1. Visit http://localhost:3030 in your browser and create a new V3 project called "v3-catalog".
1. Visit http://localhost:3030 in your browser and create a new V3 project called "hello-world".
2. In Postgres go to the "Projects" table and for the project you create change the `externalRef` to `yubjwjsfkxnylobaqvqz`.
2. In Postgres go to the "Projects" table and for the project you create change the `externalRef` to `proj_rrkpdguyagvsoktglnod`.
3. Build the CLI
@@ -105,10 +105,10 @@ pnpm run build --filter trigger.dev
pnpm i
```
4. Change into the `<root>/references/v3-catalog` directory and authorize the CLI to the local server:
4. Change into the `<root>/references/hello-world` directory and authorize the CLI to the local server:
```sh
cd references/v3-catalog
cd references/hello-world
cp .env.example .env
pnpm exec trigger login -a http://localhost:3030
```
@@ -118,7 +118,7 @@ This will open a new browser window and authorize the CLI against your local use
You can optionally pass a `--profile` flag to the `login` command, which will allow you to use the CLI with separate accounts/servers. We suggest using a profile called `local` for your local development:
```sh
cd references/v3-catalog
cd references/hello-world
pnpm exec trigger login -a http://localhost:3030 --profile local
# later when you run the dev or deploy command:
pnpm exec trigger dev --profile local
@@ -137,84 +137,29 @@ The following steps should be followed any time you start working on a new featu
pnpm run dev --filter trigger.dev --filter "@trigger.dev/*"
```
3. Open another terminal window, and change into the `<root>/references/v3-catalog` directory.
3. Open another terminal window, and change into the `<root>/references/hello-world` directory.
4. You'll need to run the following commands to setup prisma and migrate the database:
4. Run the `dev` command, which will register all the local tasks with the platform and allow you to start testing task execution:
```sh
pnpm exec prisma migrate deploy
pnpm run generate:prisma
```
5. Run the `dev` command, which will register all the local tasks with the platform and allow you to start testing task execution:
```sh
# in <root>/references/v3-catalog
# in <root>/references/hello-world
pnpm exec trigger dev
```
If you want additional debug logging, you can use the `--log-level debug` flag:
```sh
# in <root>/references/v3-catalog
# in <root>/references/hello-world
pnpm exec trigger dev --log-level debug
```
6. If you make any changes in the CLI/Core/SDK, you'll need to `CTRL+C` to exit the `dev` command and restart it to pickup changes. Any changes to the files inside of the `v3-catalog/src/trigger` dir will automatically be rebuilt by the `dev` command.
6. If you make any changes in the CLI/Core/SDK, you'll need to `CTRL+C` to exit the `dev` command and restart it to pickup changes. Any changes to the files inside of the `hello-world/src/trigger` dir will automatically be rebuilt by the `dev` command.
7. Navigate to the `v3-catalog` project in your local dashboard at localhost:3030 and you should see the list of tasks.
7. Navigate to the `hello-world` project in your local dashboard at localhost:3030 and you should see the list of tasks.
8. Go to the "Test" page in the sidebar and select a task. Then enter a payload and click "Run test". You can tell what the payloads should be by looking at the relevant task file inside the `/references/v3-catalog/src/trigger` folder. Many of them accept an empty payload.
8. Go to the "Test" page in the sidebar and select a task. Then enter a payload and click "Run test". You can tell what the payloads should be by looking at the relevant task file inside the `/references/hello-world/src/trigger` folder. Many of them accept an empty payload.
9. Feel free to add additional files in `v3-catalog/src/trigger` to test out specific aspects of the system, or add in edge cases.
## Running end-to-end webapp tests (deprecated)
To run the end-to-end tests, follow the steps below:
1. Set up environment variables (copy example envs into the correct place)
pnpm run build --filter @references/nextjs-test^...
pnpm --filter @trigger.dev/database generate
# Move trigger-cli bin to correct place
pnpm install --frozen-lockfile
# Install playwrite browsers (ONE TIME ONLY)
npx playwright install
```
3. Set up the database
```sh
pnpm run docker
pnpm run db:migrate
pnpm run db:seed
```
4. Run the end-to-end tests
```sh
pnpm run test:e2e
```
### Cleanup
The end-to-end tests use a `setup` and `teardown` script to seed the database with test data. If the test runner doesn't exit cleanly, then the database can be left in a state where the tests can't run because the `setup` script will try to create data that already exists. If this happens, you can manually delete the `users` and `organizations` from the database using prisma studio:
```sh
# With the database running (i.e. pnpm run docker)
pnpm run db:studio
```
9. Feel free to add additional files in `hello-world/src/trigger` to test out specific aspects of the system, or add in edge cases.
Trigger.dev is an opensource platform and SDK which allows you to create long-running background jobs. Write normal async code, deploy, and never hit a timeout.
Trigger.dev is the open-source platform for building AI workflows in TypeScript. Long-running tasks with retries, queues, observability, and elastic scaling.
### Key features:
## The platform designed for building AI agents
- JavaScript and TypeScript SDK
- No timeouts
- Retries (with exponential backoff)
- Queues and concurrency controls
- Schedules and crons
- Full Observability; logs, live trace views, advanced filtering
- React hooks to interact with the Trigger API from your React app
- Pipe LLM streams straight to your users through the Realtime API
- Trigger tasks and display the run status and metadata anywhere in your app
- Custom alerts, get notified by email, Slack or webhooks
- No infrastructure to manage
- Elastic (scaling)
- Works with your existing tech stack
Build [AI agents](https://trigger.dev/product/ai-agents) using all the frameworks, services and LLMs you're used to, deploy them to Trigger.dev and get durable, long-running tasks with retries, queues, observability, and elastic scaling out of the box.
## In your codebase
- **Long-running without timeouts**: Execute your tasks with absolutely no timeouts, unlike AWS Lambda, Vercel, and other serverless platforms.
- **Durability, retries & queues**: Build rock solid agents and AI applications using our durable tasks, retries, queues and idempotency.
- **True runtime freedom**: Customize your deployed tasks with system packages – run browsers, Python scripts, FFmpeg and more.
- **Human-in-the-loop**: Programmatically pause your tasks until a human can approve, reject or give feedback.
- **Realtime apps & streaming**: Move your background jobs to the foreground by subscribing to runs or streaming AI responses to your app.
- **Observability & monitoring**: Each run has full tracing and logs. Configure error alerts to catch bugs fast.
## Key features:
- **[JavaScript and TypeScript SDK](https://trigger.dev/docs/tasks/overview)** - Build background tasks using familiar programming models
- **[Long-running tasks](https://trigger.dev/docs/runs/max-duration)** - Handle resource-heavy tasks without timeouts
- **[Durable cron schedules](https://trigger.dev/docs/tasks/scheduled#scheduled-tasks-cron)** - Create and attach recurring schedules of up to a year
- **[Trigger.dev Realtime](https://trigger.dev/docs/realtime/overview)** - Trigger, subscribe to, and get real-time updates for runs, with LLM streaming support
- **[Build extensions](https://trigger.dev/docs/config/extensions/overview#build-extensions)** - Hook directly into the build system and customize the build process. Run Python scripts, FFmpeg, browsers, and more.
- **[React hooks](https://trigger.dev/docs/frontend/react-hooks#react-hooks)** - Interact with the Trigger.dev API on your frontend using our React hooks package
- **[Batch triggering](https://trigger.dev/docs/triggering#tasks-batchtrigger)** - Use batchTrigger() to initiate multiple runs of a task with custom payloads and options
- **[Structured inputs / outputs](https://trigger.dev/docs/tasks/schemaTask#schematask)** - Define precise data schemas for your tasks with runtime payload validation
- **[Waits](https://trigger.dev/docs/wait)** - Add waits to your tasks to pause execution for a specified duration
- **[Preview branches](https://trigger.dev/docs/deployment/preview-branches)** - Create isolated environments for testing and development. Integrates with Vercel and git workflows
- **[Waitpoints](https://trigger.dev/docs/wait-for-token#wait-for-token)** - Add human-in-the-loop judgment at critical decision points without disrupting workflow
- **[Concurrency & queues](https://trigger.dev/docs/queue-concurrency#concurrency-and-queues)** - Set concurrency rules to manage how multiple tasks execute
- **[Multiple environments](https://trigger.dev/docs/how-it-works#dev-mode)** - Support for DEV, PREVIEW, STAGING, and PROD environments
- **[No infrastructure to manage](https://trigger.dev/docs/how-it-works#trigger-dev-architecture)** - Auto-scaling infrastructure that eliminates timeouts and server management
- **[Automatic retries](https://trigger.dev/docs/errors-retrying)** - If your task encounters an uncaught error, we automatically attempt to run it again
- **[Checkpointing](https://trigger.dev/docs/how-it-works#the-checkpoint-resume-system)** - Tasks are inherently durable, thanks to our checkpointing feature
- **[Versioning](https://trigger.dev/docs/versioning)** - Atomic versioning allows you to deploy new versions without affecting running tasks
- **[Machines](https://trigger.dev/docs/machines)** - Configure the number of vCPUs and GBs of RAM you want the task to use
- **[Observability & monitoring](https://trigger.dev/product/observability-and-monitoring)** - Monitor every aspect of your tasks' performance with comprehensive logging and visualization tools
- **[Logging & tracing](https://trigger.dev/docs/logging)** - Comprehensive logging and tracing for all your tasks
- **[Tags](https://trigger.dev/docs/tags#tags)** - Attach up to ten tags to each run, allowing you to filter via the dashboard, realtime, and the SDK
- **[Run metadata](https://trigger.dev/docs/runs/metadata#run-metadata)** - Attach metadata to runs which updates as the run progresses and is available to use in your frontend for live updates
- **[Bulk actions](https://trigger.dev/docs/bulk-actions)** - Perform actions on multiple runs simultaneously, including replaying and cancelling
- **[Real-time alerts](https://trigger.dev/docs/troubleshooting-alerts#alerts)** - Choose your preferred notification method for run failures and deployments
## Write tasks in your codebase
Create tasks where they belong: in your codebase. Version control, localhost, test and review like you're already used to.
```ts
import{task}from"@trigger.dev/sdk/v3";
import{task}from"@trigger.dev/sdk";
//1. You need to export each task
exportconsthelloWorld=task({
@@ -58,13 +90,13 @@ Use our SDK to write tasks in your codebase. There's no infrastructure to manage
## Environments
We support `Development`, `Staging`, and `Production` environments, allowing you to test your tasks before deploying them to production.
We support `Development`, `Staging`,`Preview`, and `Production` environments, allowing you to test your tasks before deploying them to production.
## Full visibility of every job run
View every task in every run so you can tell exactly what happened. We provide a full trace view of every task run so you can see what happened at every step.
@@ -73,14 +105,19 @@ The quickest way to get started is to create an account and project in our [web
### Useful links:
- [Quick start](https://trigger.dev/docs/quick-start) - get up and running in minutes
- [How it works](https://trigger.dev/docs/v3/how-it-works) - understand how Trigger.dev works under the hood
- [How it works](https://trigger.dev/docs/how-it-works) - understand how Trigger.dev works under the hood
- [Guides and examples](https://trigger.dev/docs/guides/introduction) - walk-through guides and code examples for popular frameworks and use cases
## Self-hosting
If you prefer to self-host Trigger.dev, you can follow our [self-hosting guide](https://trigger.dev/docs/v3/open-source-self-hosting#overview).
If you prefer to self-host Trigger.dev, you can follow our [self-hosting guides](https://trigger.dev/docs/self-hosting/overview):
We also have a dedicated self-hosting channel in our [Discord server](https://trigger.dev/discord) for support.
- [Docker self-hosting guide](https://trigger.dev/docs/self-hosting/docker) - use Docker Compose to spin up a Trigger.dev instance
- [Kubernetes self-hosting guide](https://trigger.dev/docs/self-hosting/kubernetes) - use our official Helm chart to deploy Trigger.dev to your Kubernetes cluster
## Support and community
We have a large active community in our official [Discord server](https://trigger.dev/discord) for support, including a dedicated channel for self-hosting.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
github-actions[bot]