Commit Graph

1499 Commits

Author SHA1 Message Date
github-actions[bot] 7b509849e9 chore: Update version for release (v4-beta) (#1996)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.12

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <eallam@icloud.com>
2025-05-03 21:26:12 +01:00
Eric Allam 13f750a18b v4: implement onCancel callbacks (#2022)
* v4: implement onCancel callbacks

* chat example

* Add changeset

* A few improvements
2025-05-03 20:50:11 +01:00
Matt Aitken 423209661b Give helpful hint if you pass a bad directory when deploying (#2013) 2025-05-02 10:54:26 +01:00
nicktrn 0df7af4403 Add idle dequeue interval to supervisor (#2007)
* supervisor uses new idle interval after empty dequeues and errors

* make supervisor heartbeat interval configurable
2025-05-01 10:53:19 +01:00
nicktrn c0807ad0d0 Display terminal links in cursor (#1998)
* add terminal link as cli module so we can more easily patch it

* apply cursor patch

* add license info

* remove terminal-link package and add deprecation notice

* remove old patch

* remove terminal-link from sdk

* changeset
2025-05-01 08:53:11 +01:00
Matt Aitken 7fe111abe9 AI dashboard widget + CLI AI generate fix suggestions (#1925)
* Made a better AI icon and improved how it looks on the create new schedules inspector

* WIP adding kapa ai to the app

* WIP adding a new Ask AI button to the side menu

* WIP using the react example from the docs

* Align the AI button in the bottom bar

* Kapa widget now works

* Trigger the Kapa modal from the custom button

* Fix imports

* Adds Ask AI shortcut to Shortcuts panel

* Adds a new enter shortcut key

* Adds a prop so you can optionally hide the shortcut key

* Latest

* Moved the Kapa/Help stuff into a component, out of root

* WIP using onModalClose

* Creates a wrapper provider to block shortcuts while kapa modal is open (has bugs)

* Fixes button alignment

* Hide the shortcut key at the button layer

* Fix for enable/disable shortcut keys globally

* Kapa is working

* You can bring up the shortcut keys without opening the help panel

* TODO remove listeners

* remove imports and fix invalid tailwind class

* style kapa widget as best i can

* Remove Kapa event listeners

* Allow passing in a query

* Open the AI widget if there’s a URL param

* Much cleaner implementation for Kapa

* Trying to auto-open the Kapa widget when the page loads

* Delay opening the widget because it was causing issues

* Improved Kapa widget colors

* Added an AI help link to the CLI

* We don’t need this anymore

* Exit with 1, indicating an error. This is important for CI

* Removed old auto-opening code

* Added some code comments to explain some of the annoying stuff

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-04-30 11:11:42 +01:00
github-actions[bot] 525285da7f chore: Update version for release (v4-beta) (#1994)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.11

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-04-29 16:48:57 +01:00
Eric Allam 07365cd405 realtime: fix streams missing chunks when streams are longer than 5 minutes and receive 408 Request Timeout errors. Also now support multiple client streams being sent to a single stream key (#1993)
* realtime: fix streams missing chunks when streams are longer than 5 minutes and receive 408 Request Timeout errors. Also now support multiple client streams being sent to a single stream key

* Safely release the reader lock
2025-04-29 12:42:38 +01:00
github-actions[bot] 5691480dac Release 4.0.0-v4-beta.10 (#1984)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.10

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-28 13:34:34 +01:00
nicktrn 09a859d056 Fix managed run controller edge cases (#1987)
* parse disconnect description

* decrease snapshot poller logs

* remove another useless log

* attempt number change detection

* ensure no executions or child processes are being reused

* prevent snapshot change handler from running after execution stopped

* prevent creating zombie intervals on reset

* add changeset

* add enabled poller check before getting latest snapshot

* make attempt number update more explicit, improve logs

* remove deprecated heartbeat service

* ..also remove the import
2025-04-28 13:22:30 +01:00
Eric Allam e837500486 feat: v4 deadlock detection (#1970)
* Locked task runs will now require queues and tasks to be in the locked version

* Client errors caught in a run function now will skip retrying

* Extracted out the trigger queues logic

* extract validation, idempotency keys, payloads to concerns

* Extracted out a bunch of more stuff and getting trigger tests to work

* Add queue and locked version tests

* Deadlock detection WIP

* more deadlock detection

* Only detect deadlocks when the parent run is waiting on the child run

* Improve the error experience around deadlocks

* A couple tweaks to make CodeRabbit happy and fixing the tests in CI

* Fixed failing test

* Changeset

* wip

* Make sure to scope queries to the runtime env
2025-04-25 12:56:54 +01:00
github-actions[bot] 60fcf5fdac Release 4.0.0-v4-beta.9 (#1980)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.9

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-25 12:43:32 +01:00
Eric Allam 30f93521c8 fix: default machine config indexing now works (#1979) 2025-04-24 13:38:03 +01:00
Matt Aitken 672a6b8be7 RedisWorker clean up orphaned items when dequeuing (#1975)
* Remove setting the invisibility timeout because it’s already done in the dequeue Lua script

* Remove orphaned queue items when dequeuing

* Added an ack to the visibility timeout test
2025-04-24 11:15:42 +01:00
github-actions[bot] 56ec0523b5 Release 4.0.0-v4-beta.8 (#1972)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.8

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-23 19:30:49 +01:00
nicktrn 73214d760e Prevent large outputs from overwriting each other (#1971)
* large outputs now get unique keys again

* replace other attempt id occurences

* add changeset
2025-04-23 19:21:21 +01:00
github-actions[bot] cfb77159d7 Release 4.0.0-v4-beta.7 (#1954)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.7

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-23 13:34:26 +01:00
nicktrn cedd932f99 Fix queued snapshot status handler (#1963)
* handle queued status change gracefully

* add resource monitor to references

* add resource monitor example

* improve runtime manager debug logs

* fix resource monitor example

* add changeset
2025-04-22 15:32:16 +01:00
Matt Aitken e379fc15be Fix redis worker debounce (#1966)
* Added some Redis worker debounce tests (one failing that reproduces a bug)

* Added some tests for acking

* Added a deduplicationKey to prevent acking when items are queued

* The worker passes the deduplicationKey back in for acking

* Improved logs and removed events from test
2025-04-22 15:24:39 +01:00
Dmitry Sabanin 2c19693cfc Fixes runLimiter check on #dequeueRuns (#1953)
* Fixes runLimiter check on #dequeueRuns

This was causing issues in the dev environment with logs being spammed with `[DevSupervisor] dequeueRuns. Run limit reached, trying again later` and making the worker unresponsive and lose connection.

* Create polite-impalas-care.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-04-19 21:45:27 +01:00
github-actions[bot] 64ca9dc9a8 Release 4.0.0-v4-beta.6 (#1951)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.6

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-18 16:50:22 +01:00
Eric Allam 39e535772f fix: realtime streams from tasks now retry if they receive a 408 timeout error from the realtime/trigger.dev server (#1952) 2025-04-18 16:20:13 +01:00
nicktrn aa74260646 Dev command engine URL now configurable via the webapp (#1948)
* return dev engine url from config call

* set and use engine url from dev config reply
2025-04-18 11:11:11 +01:00
nicktrn f683fe8d2d Multiple queue consumers per supervisor (#1947)
* multiple queue consumer in the same supervisor instance

* disable pre

* Revert "disable pre"

This reverts commit 4b154398982758d09667531e3a64761da83664eb.
2025-04-18 11:07:49 +01:00
github-actions[bot] 4a8303212c Release 4.0.0-v4-beta.5 (#1943)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.5

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-04-17 17:25:07 +01:00
github-actions[bot] 59eea654ac Release 4.0.0-v4-beta.4 (#1939)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.4

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-04-17 17:05:52 +01:00
Himanshu Sharma e0fd06490a maintain proper context in metadata.root and parent getters (#1917)
* fix(TRI-5140): maintain proper context in metadata.root and parent getters

* Create two-tigers-dream.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2025-04-17 15:04:01 +01:00
Matt Aitken 33e7b6865a Set env vars to be "secret" (#1923)
* WIP on secret env vars

* Editing individual env var values is working

* Sort the env vars by the key

* Deleting values

* Allowing setting secret env vars

* Added medium switch style

* Many style changes to the env var form

* “Copy text” -> “Copy”

* Draw a divider between hidden buttons

* Env var tweaks

* Don’t show Dev:you anymore

* Grouping the same env var keys together

* Styles improved

* Improved styling of edit panel

* Fix bun detection, dev flushing, and init command (#1914)

* update nypm to support text-based bun lockfiles

* add nypm changeset

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add changesets

* chore: remove pre.json after exiting pre mode

* init command to install v4-beta packages

* Revert "chore: remove pre.json after exiting pre mode"

This reverts commit f5694fde9314114c74a220c2213d19667bca1a6c.

* make init default to cli version for all packages

* Release 4.0.0-v4-beta.1 (#1916)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Both run engines will only lock to versions they can handle (#1922)

* run engine v1 will only lock to v1 deployments

* run engine v2 will only lock to managed v2 deployments

* test: create background worker and deployment with correct engine version

* Add links to and from deployments (#1921)

* link from deployments tasks to filtered runs view

* jump to deployment

* don't add version links for dev (yet)

* Fix current worker deployment getter (#1924)

* only return last v1 deployment in the shared queue consumer

* be explicit about only returning managed deployments

* Add a docs page for the human-in-the-loop example project (#1919)

* Add a docs page for the human-in-the-loop example project

* Order guides, example projects and example tasks alphabetically in the docs list

* Managed run controller revamp (#1927)

* update nypm to support text-based bun lockfiles

* fix retry spans

* only download debug logs if admin

* add nypm changeset

* pull out env override logic

* use runner env gather helper

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add logger

* add execution heartbeat service

* add snapshot poller service

* fix poller

* add changesets

* create socket in constructor

* enable strictPropertyInitialization

* deprecate dequeue from version

* start is not async

* dependency injection in prep for tests

* add warm start count to all controller logs

* add restore count

* pull out run execution logic

* temp disable pre

* add a controller log when starting an execution

* refactor execution and squash some bugs

* cleanup completed docker containers by default

* execution fixes and logging improvements

* don't throw afet abort cleanup

* poller should use private interval

* rename heartbeat service file

* rename HeartbeatService to IntervalService

* restore old heartbeat service but deprecate it

* use the new interval service everywhere

* Revert "temp disable pre"

This reverts commit e03f4179de6a731c17253b68a6e00bcb7ac1736b.

* add changeset

* replace all run engine find uniques with find first

* Release 4.0.0-v4-beta.2 (#1928)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Remove batch ID carryover for non-batch waits (#1930)

* add failing test case

* do not carry over previous batch id when blocking with waitpoint

* delete irrelevant test

* Delete project (#1913)

* Delete project

- Don’t schedule tasks if the project is deleted
- Delete queues from the master queues
- Add the old delete project UI back in

* Mark the project as deleted last

* Fix for overriding local variable

* Added a todo for deleting env queues

* Remove todo

* Improve usage flushing (#1931)

* add flush to global usage api

* enable controller debug logs

* initialize usage manager after env overrides

* add previous run id to more debug logs

* add changeset

* For secret env vars, don’t return the value

* Added a new env var repository function for getting secrets with redactions

* Test task for env vars

* Delete heartbeat file, merge mess up

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Saadi Myftija <saadi.myftija@gmail.com>
2025-04-17 14:39:09 +01:00
github-actions[bot] 543540581d Release 4.0.0-v4-beta.3 (#1932)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.3

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-16 19:57:42 +01:00
nicktrn 8b182ce8cb Fix stalled run detection (#1934)
* drive run heartbeats from child process

* track last heartbeat

* add changeset

* use v4-specific heartbeat interval env var and decrease to 20s

* delete old heartbeat service

* remove heartbeat handler during execution cleanup
2025-04-16 19:48:51 +01:00
nicktrn 1f6a283ddf Improve usage flushing (#1931)
* add flush to global usage api

* enable controller debug logs

* initialize usage manager after env overrides

* add previous run id to more debug logs

* add changeset
2025-04-16 14:34:20 +01:00
github-actions[bot] bcef9696e8 Release 4.0.0-v4-beta.2 (#1928)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-15 16:19:04 +01:00
nicktrn 3f97cf4a6b Managed run controller revamp (#1927)
* update nypm to support text-based bun lockfiles

* fix retry spans

* only download debug logs if admin

* add nypm changeset

* pull out env override logic

* use runner env gather helper

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add logger

* add execution heartbeat service

* add snapshot poller service

* fix poller

* add changesets

* create socket in constructor

* enable strictPropertyInitialization

* deprecate dequeue from version

* start is not async

* dependency injection in prep for tests

* add warm start count to all controller logs

* add restore count

* pull out run execution logic

* temp disable pre

* add a controller log when starting an execution

* refactor execution and squash some bugs

* cleanup completed docker containers by default

* execution fixes and logging improvements

* don't throw afet abort cleanup

* poller should use private interval

* rename heartbeat service file

* rename HeartbeatService to IntervalService

* restore old heartbeat service but deprecate it

* use the new interval service everywhere

* Revert "temp disable pre"

This reverts commit e03f4179de6a731c17253b68a6e00bcb7ac1736b.

* add changeset

* replace all run engine find uniques with find first
2025-04-15 16:08:51 +01:00
github-actions[bot] 1d920d5b3f Release 4.0.0-v4-beta.1 (#1916)
* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
2025-04-11 20:19:03 +01:00
nicktrn cc553d3392 Fix bun detection, dev flushing, and init command (#1914)
* update nypm to support text-based bun lockfiles

* add nypm changeset

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add changesets

* chore: remove pre.json after exiting pre mode

* init command to install v4-beta packages

* Revert "chore: remove pre.json after exiting pre mode"

This reverts commit f5694fde9314114c74a220c2213d19667bca1a6c.

* make init default to cli version for all packages
2025-04-11 20:11:13 +01:00
github-actions[bot] b949b5d257 chore: Update version for release (v4-beta) (#1755)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-09 16:07:14 +01:00
nicktrn 98a3cfbfa1 Clear runner state of previous executions before warm starts (#1905)
* attach all run controller logs to the run

* make controller-level pre-suspend wait duration configurable

* snapshot status should remain EXECUTING for short retry delays

* add suspended process error

* ensure clean slate before waiting for next run

* treat immediate retries as warm starts

* fix for finished runs waiting forever
2025-04-09 15:53:42 +01:00
nicktrn 72d98778eb Fix node options and OOM retries (#1897)
* don't hardcode max old space size in deployment images

* flags: treat underscores as hyphens

* append attempt number to runner name if >1

* improve retry spans for oom errors
2025-04-08 13:07:45 +01:00
Matt Aitken a51b9b9f98 Fix: custom queue releaseConcurrencyOnWaitpoint. (#1896)
* Make sure releaseConcurrencyOnWaitpoint is set on taskQueue

* Allow task queue to have releaseConcurrencyOnWaitpoint (required to fix custom queues)

* Test for releaseConcurrencyOnWaitpoint

* Added a warning if you use the same queue twice with different settings

* Improved the error some more
2025-04-08 12:32:22 +01:00
nicktrn a9d5646d75 Very short waits count towards compute usage (#1888)
* add duration wait threshold

* improve wording

* Updates the wait overview, wait.for and wait.until pages copy to better explain compute costs

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-04-07 14:52:33 +01:00
Matt Aitken 32d2992802 Security update of nanoid from 3.3.4 to 3.3.8 (#1887) 2025-04-07 13:56:37 +01:00
Himanshu Sharma e546028e9b upgrade nanoid to latest version (#1886) 2025-04-07 11:31:06 +01:00
nicktrn 58567bfe44 Add v4 timeline metrics to prod runs (#1884)
* set correct worker name on create

* dev: kill run process on failed cancel attempt

* add missing prod timeline events

* add more debug logs after api call failures

* import events are always visible

* set dequeued at on new runners

* skip fork event test

* fix timeline test for import event
2025-04-04 18:39:13 +01:00
Matt Aitken 36159be544 Warm start UI (#1882)
* Better trace icon

* Better Waitpoint token icon

* Fix for bad jsx

* Warm and cold start icons

* Tooltips now use a <Portal> so they appear on top

* Warm start components

* Added warm start markers to the Run page and inspector

* Fix for getting the correct value from the metadata

* Better trace icon, with fallback to a passed in one

* Removed unused isWarmStart function
2025-04-04 14:20:28 +01:00
nicktrn 7644d92efa Fix restored runner ID and various small enhancements (#1880)
* make worker nodetype configurable

* make max dequeue count configurable from supervisor

* deterministic runner ids

* update runner id on restore

* disable resource monitor

* explicit run controller type imports

* enhance debug logs
2025-04-04 12:35:23 +01:00
Eric Allam 7b9ab1781a re2: add ability to selectively disable release concurrency queue consumers via env var (#1881)
* run engine: add ability to selectively disable release concurrency queue consumers via env var

Also added some additional logging

* actually pass the disableConsumers option through to the release concurrency system

* Fixed CLI e2e failing tests
2025-04-04 12:28:39 +01:00
Eric Allam 25c1abac8e v4: eagerly fork child processes before warm start initiates (#1879)
* v4: eagerly fork child process before warm start

Also fixes an issue where the attempt span events weren't coming through in the partial spans

* Making flushing a safer operation

* Add changeset

* Improve attempt metrics when in a warm start, and pass the STYLE_VARIANT for a warm or cold start on attempt spans
2025-04-04 11:07:36 +01:00
Eric Allam 18ad897171 fix: silence erroneous prisma package.json warnings (#1875) 2025-04-02 22:39:29 +01:00
nicktrn 7591b6e619 Fix v4 restore race condition (#1870) 2025-04-02 18:05:43 +01:00
Eric Allam 8e5d5cf690 fix: only pass engine version header when inside a task (#1871)
This will allow migrating to the v4 SDK in your application backend before deploying v4 to trigger, and everything will just work
2025-04-02 17:40:07 +01:00