Commit Graph

6543 Commits

Author SHA1 Message Date
Eric Allam 07f2466999 feat(webapp): Display schemas in test and auto-generate test payloads using AI 2025-09-08 14:01:39 +01:00
Eric Allam 89b1d8ba13 fix all lifecycle hooks (#2480) 2025-09-05 11:17:38 +01:00
Eric Allam 97015ba8c8 fix(docs): update sentry error tracking guide for v4 (#2479) 2025-09-05 11:13:13 +01:00
nicktrn 99660112bd feat(supervisor): add configurable resource requests (#2474) 2025-09-04 17:09:34 +01:00
Matt Aitken e6586d3c1a Remove deprecated releaseConcurrency from wait.forToken() docs (#2477) 2025-09-04 14:57:17 +01:00
Eric Allam 00d32ed4ee feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf (#2472)
* feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf

* cleaned up signal handling and resolved some valid 🐇 issues
2025-09-04 09:53:15 +01:00
nicktrn a1e9738faa fix(webapp): prevent duplicate preview env image tags (#2475)
* fix(webapp): prevent duplicate preview env image tags

* use deploy shortcode instead of new nanoid

* replace regexps in tests
2025-09-04 09:37:22 +01:00
nicktrn 59c17e04e9 feat(run-engine): worker queue resolver (#2476) 2025-09-04 09:27:06 +01:00
Eric Allam ed23615aa4 perf(webapp): add event loop utilization metric (#2471)
* perf(webapp): add event loop utilization metric

* add event loop utilization logging as well
2025-09-03 11:39:03 +01:00
Saadi Myftija 436d951b65 feat(webapp): github app installation flow (#2463)
* Add schemas for gh app installations

* Implement gh app installation flow

* Make the gh app configs optional

* Add additional org check on gh app installation callback

* Save account handle and repo default branch on install

* Do repo hard deletes in favor of simplicity

* Disable github app by default

* Fix gh env schema union issue

* Use octokit's iterator for paginating repos

* Parse gh app install callback with a discriminated union

* Remove duplicate env vars

* Use bigint for github integer IDs

* Sanitize redirect paths in the gh installation and auth flow

* Regenerate migration after rebase on main to fix ordering

* Handle gh install updates separately from new installs
2025-09-02 16:35:33 +02:00
nicktrn cf9398b56e fix(run-engine): retry SIGTERM errors (#2468) 2025-09-02 15:18:16 +01:00
Eric Allam 9b1877bef2 fix(run-engine): retry non-zero exit code errors (#2467)
We’re also now saving the retryConfig from the BackgroundWorkerTask on 
TaskRun.lockedRetryConfig when the run is first locked to the version
2025-09-02 14:12:14 +01:00
Eric Allam 0b2b73fc52 chore(docs): improve version pinning advise to use installed trigger.dev CLI (#2466) 2025-09-02 14:10:01 +01:00
nicktrn 0d1eac9406 feat(supervisor): dynamic queue consumer pool (#2461)
* feat(supervisor): dynamic queue consumer pool

* add changeset

* fix: correctly handle zero median and even samples

* feat(supervisor): consumer pool metrics

* fix tests

* more tests and fixes

* decrease default scaling cooldowns

* don't treat initial pool size as scale up

* handle scale down when queue length drops to zero

* remove changeset, supervisor changes only

* add damping factor env var
2025-09-02 13:52:42 +01:00
Willow (GHOST) ddbae6b6b4 fix(docker): clickhouse healthcheck (#2462) 2025-09-01 20:20:17 +01:00
nicktrn 2b095b1072 chore(webapp): upgrade otel packages and add more metrics (#2458)
* feat(webapp): upgrade otel packages and add more metrics

* add env var to disable additional detectors

* expose more prisma metrics

* chore(webapp): drop node 16 support
2025-09-01 16:24:12 +01:00
Dan 847ea866b6 Various docs improvements (#2456)
* Added MCP to the intro

* Added human-in-the-loop and new build extensions

* Waitpoint notes

* Moved openai guardrails example to Python

* Added a connection limit note

* Added Supabase + Prisma note to supabase auth

* Updates based on what the rabbit said

* Moved the supavisor section to the prismaExtension docs
2025-08-29 16:48:04 +01:00
James Ritchie fee31f2dc0 chore(webapp): update region message to say we are GDPR compliant (#2455)
* Updates the Region message to say we are GDPR compliant

* Hide the message if not on cloud
2025-08-29 14:33:46 +01:00
github-actions[bot] 10baa396d8 Release v4.0.2 (#2447)
* chore: Update version for release

* Release v4.0.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
@trigger.dev/python@4.0.2 @trigger.dev/build@4.0.2 @trigger.dev/core@4.0.2 @trigger.dev/react-hooks@4.0.2 @trigger.dev/redis-worker@4.0.2 @trigger.dev/rsc@4.0.2 @trigger.dev/schema-to-json@4.0.2 @trigger.dev/sdk@4.0.2 trigger.dev@4.0.2
2025-08-29 12:58:22 +01:00
James Ritchie 27bc2ad232 chore(webapp): show a regions info panel to explain about data residency (#2428) 2025-08-29 11:35:54 +01:00
James Ritchie 3a3e863daf chore(webapp): bulk action cancel as default (#2437)
* Switches “replay” and “cancel” options in the list

* Fixed incorrect cancel icon being used

* Fixes the alignment of the radiobuttons
2025-08-29 11:30:46 +01:00
Eric Allam f5f14c4206 chore(webapp): fix @trigger.dev/database imports so it doesn't end up in the frontend bundle (#2452) 2025-08-29 11:27:01 +01:00
Eric Allam 892adfe81c fix(mcp): don't require a trigger.config.ts file to be present when installing rules (#2453) 2025-08-29 11:20:45 +01:00
nicktrn 73e7378459 feat(webapp): add ecs otel resource attributes (#2454)
* add attributes to runlock dequeue span

* feat(webapp): add ecs otel resource attributes
2025-08-29 11:05:47 +01:00
Eric Allam b0b0df69be fix(run-engine): prevent stalled SUSPENDED snapshots from endlessly retrying (#2448) 2025-08-28 14:25:50 +01:00
Saadi Myftija e629810e96 fix: dev env connection issue (#2449)
This PR fixes an issue introduced in #2391 where the dev runtime env was not being resolved correctly.
2025-08-28 14:05:10 +01:00
Ognjen Simic e6f6d93e59 docs: update triggering.mdx fixing wrongly stated argument position for options (#2283)
* docs: update triggering.mdx fixing wrong argument numeration for options

Update the docs triggering.mdx: text indicating options argument position inside `tasks.trigger` function.

Options object is a third argument instead of stated second argument.

* update triggering.mdx to change options argument position in batchTrigger
2025-08-28 10:16:42 +01:00
Eric Allam 6f6ca01584 feat(mcp): add wait_for_run_to_complete tool so agents don't spam the get_run_details call after triggering (#2445)
* feat(mcp): add wait_for_run_to_complete tool so agents don't spam the get_run_details call after triggering

This also fixes the search docs MCP tool

* Install mcp using the latest tag, not the specific version
2025-08-28 10:13:46 +01:00
James Ritchie 11cbd1b5a0 Docs: Fixes incorrect retrieving runs from batchTrigger (#2427)
* Docs: fix for retrieving runs in batchTrigger

* Removes old v3-to-v4 migration guide (we have a redirect)

* Corrects tag limit from 5 to 10
2025-08-28 10:11:42 +01:00
Eric Allam 82401ec4fe fix(run-engine): dequeue performance improvements and improved telemetry (#2446)
* more eager dequeuing, queue cooloff periods, return workerQueueLength when dequeueing

* Cache worker group authentication and remove old self-hosted worker code (only managed is currently supported)

* add additional spans during dequeue

* Add env vars and additional spans
2025-08-27 21:22:10 +01:00
Matt Aitken 24a915133e Prisma 6.14.0 upgrade (#2444)
* Initial work on upgrading to 6.14.0

Set the output to node_modules still to make it easier

* Use ./generated Prisma folder, update types to fix issues

* Docker compose restart Clickhouse

* Prisma instrumentation update

* Docker

* Removed database dockerignore file, add generated prisma client to the top-level one

* Delete v3-catalog package.json

* Resolved pnpm lock file

* Log errors for very slow queries
2025-08-27 16:52:58 +01:00
Eric Allam 688b108ec3 chore(references): remove v3-catalog (#2443) 2025-08-27 14:35:55 +01:00
Saadi Myftija 1cc62230ab feat: introduce organization access tokens (#2391)
* Create schema and migration for organization access tokens

* Add helpers for creating and authenticating OATs

* Adapt the auth service to also accept OATs

* Accept OATs in the whoami v2 endpoint

* Enable deployments with the CLI using OATs

* Avoid reading env variables directly in the token utils

* Remove duplicate cli token utils

* Validate ENCRYPTION_KEY length when parsing env vars

* Make token utils a server-only module

* Disallow revoking already revoked OATs

* Simplify generics in authenticateRequest

* Use 32 bytes mock encryption key in the test setup

* Update dummy encryption key values in tests and templates

* Add a column in the OATs table to differentiate between user and system generated

* Simplify args for v3ProjectPath

Co-authored-by: Matt Aitken <matt@mattaitken.com>

* Add index on org id and createdAt

* Avoid storing the encrypted oat token and its obfuscated version in the DB at all

It is a safer approach. Also we do not need to ever read the decrypted token value after creation.

* Fix prisma update condition

* Add token type to the OAT table index

* Accept OATs in the mcp auth flow

* Simplify env auth flow around the /projects endpoints

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-08-27 13:23:45 +02:00
Eric Allam 96243efe16 docs: MCP Server 2.0 (#2385)
* Add MCP Server documentation to documentation site

This change introduces comprehensive documentation for the Trigger.dev MCP Server, covering its installation, usage, and available tools. The addition is necessary to provide users with detailed guidance on leveraging the MCP Server for managing Trigger.dev projects, tasks, and deployment processes.

- Updated `docs.json` to include a new section for the MCP Server documentation, improving navigation and accessibility.
- Added `mcp-introduction.mdx` as an introductory guide to the MCP Server, detailing its purpose and installation steps.
- Included `mcp-tools.mdx`, a document outlining each tool available in the MCP Server, complete with descriptions and example usages.
- The new content is aimed at improving user understanding and efficiency in using Trigger.dev's MCP Server capabilities.

* Refactor MCP installation documentation

The changes were necessary to streamline the MCP installation guidance and align our documentation with the latest client support capabilities. Removing the manual installation section and redundant information helps reduce confusion for users.

- Updated installation instructions to lead with auto-configuration for supported clients.
- Cleaned up the options and examples sections to enhance clarity.
- Reformatted client configuration details for better readability.
- Removed outdated links in the "Next Steps" section, focusing on more direct resources such as available MCP tools.

* mcp docs edits

* Add debugMode to get_run_details

* add cancel_run

* Add list deployments tools

* Add list preview branches tool

* Docs: adds agent rules page

* Adds redirect for the CLI path to work

* Adds subagent install image

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-08-22 15:37:35 +01:00
Eric Allam ad26cded99 fix(v3): prevent saturated queues from dominating dequeue attempts by adding a cooloff period of successive failed dequeues (#2439)
* fix(v3): prevent saturated queues from dominating dequeue attempts by adding a cooloff period of successive failed dequeues

* Add env vars and add a marqs shutdown on SIGTERM/INT
2025-08-22 14:37:21 +01:00
Eric Allam 5e4756f8b7 fix(v3): eagerly dequeue messages from a queue when that queue is added to or removed from (v4 backport) (#2438)
* fix(v3): eagerly dequeue messages from a queue when that queue is added to or removed from (v4 backport)

* fixed configuration
2025-08-22 13:23:34 +01:00
Eric Allam c16cc577b0 fix(v3): adopt v4 dequeue strategy to increase perf (#2436) 2025-08-22 07:40:59 +01:00
github-actions[bot] 5b40c44ca6 Release v4.0.1
* chore: Update version for release

* Release v4.0.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
@trigger.dev/python@4.0.1 @trigger.dev/build@4.0.1 @trigger.dev/core@4.0.1 @trigger.dev/redis-worker@4.0.1 @trigger.dev/rsc@4.0.1 @trigger.dev/schema-to-json@4.0.1 @trigger.dev/sdk@4.0.1 trigger.dev@4.0.1 @trigger.dev/react-hooks@4.0.1
2025-08-21 17:41:58 +01:00
Eric Allam c7e3811794 fix(webapp): correctly associate event-loop-blocked spans with parent span context (#2435) 2025-08-21 16:21:26 +01:00
Eric Allam bf2120d5a7 fix(runengine): remove dev worker queues from metrics (#2434) 2025-08-21 14:11:22 +01:00
Eric Allam 895e24a0d7 fix(cli): various mcp server fixes (#2430)
* fix: span events are optional, as logs don't have them

* remove CI: true env var to prevent massive tool responses

* fix github resolver relative path

* prevent the dev command from asking for user input when not interactive

* Remove lodash.get because it's deprecated

* span output can be object other than an object, like a string

* prevent large traces from causing get_run_details failures

* Decreased max log lines, made it a tool input property

* Some install mcp tweaks
2025-08-21 13:57:14 +01:00
Matt Aitken 58ff034325 Add attemptDequeue span to marqs (#2433)
* Add attemptDequeue span to marqs

More telemetry to figure out why dequeuing is slow

* More spans

* use "receive" for all operation names
2025-08-21 13:09:08 +01:00
Dan 84750f68a4 Updated main README (#2432)
* readme wip

* Added new commands

* Simplified readme

* Added docs links and improved about

* Added nice header and updated links

* Consistent headers

* Copy update

* Updated badges

* Added features / agents section

* Updated list

* Added links

* Updated links

* Minor readme tweaks

* Updated banner

* removed /v3/

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-08-21 12:28:52 +01:00
Matt Aitken 1087e5ede0 When replaying a v3 run on v4, don't pass the region (#2431) 2025-08-21 10:15:26 +01:00
Eric Allam f086626a41 feat(cli): MCP server 2.0 (#2384) 2025-08-20 16:04:50 +01:00
Matt Aitken 8483900832 Updated queue/concurrency docs for v4 (#2418) 2025-08-20 11:47:56 +01:00
Matt Aitken 4264fc066e chore(webapp): More false errors turned to logs (#2422)
* Remove variables from dequeue log message

* Continue snapshot throw json

* Waiting for deploy error removed

* Realtime ECONNRESET is expected

* Redis worker logErrors changes, removed ids

* Preview branch without a branch shouldn't log an error, it's a user provided issue

* "Task run is not in a cancellable state" isn't an error, it's expected

* "CreateCheckpointService: Child run already resumed" is expected

* "CreateCheckpointService: Batch already resumed" is expected

* "Failed to insert events, will attempt bisection" changed to info, we have errors for complete failures

* Ignore "PrismaClient error"

* Don't log Redis worker DLQ errors if we're ignoring

* "Failed to parse machine config" is fine, sometimes a config is null or undefined

* "Failed to parse machine config" for v3

* MetadataTooLargeError shouldn't log an error
2025-08-20 11:44:26 +01:00
nicktrn da79c2b185 chore(docker): add registry namespace var to .env.example (#2426) 2025-08-20 11:03:30 +01:00
nicktrn 677b7aed9c chore(docs): update bun guide (#2425)
* chore(docs): remove missing snippet references

* chore(docs): update bun guide
2025-08-20 11:03:23 +01:00
nicktrn 9092ca863f feat(supervisor): add ecr support to docker client (#2424) 2025-08-20 10:49:19 +01:00