Commit Graph

3375 Commits

Author SHA1 Message Date
nicktrn 2b586c87a0 Fix controller waitpoint resolution, suspendable state, and snapshot race conditions (#2006)
* remove dead code

* rename managed to shared runtime manager

* rename to resolve waitpoint for clarity

* add resolver id helper

* store and correctly resolve waipoints that come in early

* fix ipc message type change

* branded type for resolver ids

* add fixme comments

* remove more unused ipc schemas

* fix entitlement validation when client doesn't exist

* restore hello world reference workspace imports

* runtime manager debug logs

* prefix engine run logs

* managed run logger accepts nested props

* runtime suspendable state and improved logs

* require suspendable state for checkpoints, fix snapshot processing queue

* add terminal link as cli module so we can more easily patch it

* apply cursor patch

* add license info

* remove terminal-link package and add deprecation notice

* remove old patch

* remove terminal-link from sdk

* rename snapshot module

* add cli test tsconfig

* add run logger base type

* add snapshot manager tests

* fix cli builds

* improve QUEUED_EXECUTING test

* changeset

* make testcontainers wait until container has stopped

* require unit tests for publishing again

* avoid mutation during iteration when resolving pending waitpoints

* improve debug logs and make them less noisy

* always update poller snapshot id for accurate logs

* detach task run process handlers

* check for env overrides in a few more places and add verbose logs

* log when poller is still executing when we stop it

* add supervisor to publish workflow

* always print full deploy logs in CI

* Revert "avoid mutation during iteration when resolving pending waitpoints"

This reverts commit 87b0ce1e5b.

* disable pre

* print prerelease script errors

* Revert "disable pre"

This reverts commit 9403409637.

* misc fixes

* better debug logs

* add snapshots since methods and route

* prep for snapshots since

* improve deprecated execution detection

* update supervisor and schema

* properly log http server errors

* detect restore after failed snapshot fetch

* run and snapshot id can be overridden

* fix restore detection

* fix deprecation checks, move into snapshot manager

* less logs

* rename snapshot manager stop

* restore detection was moved into snapshot manager

* fix notifier logs

* make runtime manager status a debug log

* no need to attach runtime status twice

* findUnique -> findFirst

* sort snapshots by created at everywhere
2025-05-07 13:02:32 +01:00
nicktrn 053389d9b3 Fix testcontainers cleanup (#2023)
* update testcontainers packages

* increase cleanup timeout and add better logs

* small tweaks

* decrease docker network size so we can have more of them

* add a test flow to check this all works

* await all engine.quit calls

* add docker diagnostics

* reduce number of test runs

* improve network attachment output

* add setup logs

* log redis setup errors

* add cleanup log type

* stop redis container if setup fails

* disable ipv6

* tidy things up a bit

* fix unit tests workflow

* decrease container cleanup timeout

* fix types

* fix webapp typecheck
2025-05-06 08:56:34 +01:00
Eric Allam 13f750a18b v4: implement onCancel callbacks (#2022)
* v4: implement onCancel callbacks

* chat example

* Add changeset

* A few improvements
2025-05-03 20:50:11 +01:00
nicktrn 15816e9955 Fix long retry delays (#2020)
* make testcontainers wait until container has stopped

* require unit tests for publishing again

* add failing test case

* make it pass

* add retry threshold ms env var and use it
2025-05-02 16:57:21 +01:00
nicktrn a10e2a8b16 Small fixes in prep for v4 self-hosting (#2018)
* fix entitlement validation when client doesn't exist

* add supervisor to publish workflow

* fix v4 publish workflow name
2025-05-02 14:33:20 +01:00
James Ritchie 73f363f9d5 Changes the spinner to white now we have purple primary buttons (#2014) 2025-05-02 10:50:22 +01:00
James Ritchie 4456184f37 Fix task and schedule icons squishing (#2017) 2025-05-02 10:47:57 +01:00
James Ritchie 2448242be3 Small improvement to the loading animation so it moves left to right only (#2012) 2025-05-01 19:24:03 +01:00
James Ritchie 9c3aeb3efb Adds friendly formatting for max duration values (#2011) 2025-05-01 19:22:56 +01:00
James Ritchie df0bce3a8f Code blocks have an optional text-wrap toggle (#2009)
* Code blocks have an optional text-wrap button

* Wrap “words”, not “all”

* wrapping is default false

* Change the wording in the tooltip
2025-05-01 17:44:17 +01:00
James Ritchie 0fe85eaec4 Highlight queues when concurrency limit reached and new table col (#2008)
* Combines the Running and Concurrency limit cols into 1

* Display a badge when a queue is at the concurrency limit

* Colors the Running/Limit column text amber if the concurrency limit is hit

* Turns the “Running” big number amber and shows “At concurrency limit” text

* BigNumber now handles big values using formatNumber and formatNumberCompact

Also includes some responsive improvements to make sure things wrap when it gets tight

* Adds a new col for showing how the queue is limited

* Reinstates a threshold for making very big numbers compact

* Added border to make the search bar not float
2025-05-01 17:40:54 +01:00
Matt Aitken d2dde0a844 formatDateTimeISO() fix (#2010)
* Fix for the DateTime ISO function

* Fix implementation and added unit tests

* Preserve milliseconds properly
2025-05-01 17:35:55 +01:00
James Ritchie 553a03580f Show a tooltip when hovering over any DateTime (#2003)
* Adds a tooltip to display the UTC, timezone and local time for DateTime

* Made it safer for SSR

* Makes the copy function into a hook and separate button + adds copy dateTime button to the date tooltip

* Adds an extra-small size copy button

* Show the offset (UTC +1) next to your local time

* Optionally don’t show the date tooltip. Defaults to true (always show)

* Tidy imports

* Refactor the tooltip content to remove duplicated markup

* Coderabbit suggestion to include support for minutes for some timezones like Nepal
2025-05-01 10:00:42 +01:00
Matt Aitken af019473cd Get data for the last day of the month for usage graphs (#1999) 2025-05-01 09:53:06 +01:00
Matt Aitken cb47200d34 Added search to the Queues page (#2000)
* Added search to the queues page

* Fix for empty state copy
2025-05-01 09:33:03 +01:00
James Ritchie 98fd8045e9 Tooltip to explain dev schedules not counting towards your limit (#2004) 2025-05-01 09:32:13 +01:00
James Ritchie 44184a61c7 Updates the secret key text to be more explicit what you have to do to start triggering tasks (#2005) 2025-05-01 09:31:37 +01:00
James Ritchie cf9460afee Post new sign up reasons to slack, not plain (#1995)
* Post new sign up reasons to slack, not plain

* Makes the Slack env vars optional

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2025-04-30 14:02:02 +01:00
Matt Aitken 7fe111abe9 AI dashboard widget + CLI AI generate fix suggestions (#1925)
* Made a better AI icon and improved how it looks on the create new schedules inspector

* WIP adding kapa ai to the app

* WIP adding a new Ask AI button to the side menu

* WIP using the react example from the docs

* Align the AI button in the bottom bar

* Kapa widget now works

* Trigger the Kapa modal from the custom button

* Fix imports

* Adds Ask AI shortcut to Shortcuts panel

* Adds a new enter shortcut key

* Adds a prop so you can optionally hide the shortcut key

* Latest

* Moved the Kapa/Help stuff into a component, out of root

* WIP using onModalClose

* Creates a wrapper provider to block shortcuts while kapa modal is open (has bugs)

* Fixes button alignment

* Hide the shortcut key at the button layer

* Fix for enable/disable shortcut keys globally

* Kapa is working

* You can bring up the shortcut keys without opening the help panel

* TODO remove listeners

* remove imports and fix invalid tailwind class

* style kapa widget as best i can

* Remove Kapa event listeners

* Allow passing in a query

* Open the AI widget if there’s a URL param

* Much cleaner implementation for Kapa

* Trying to auto-open the Kapa widget when the page loads

* Delay opening the widget because it was causing issues

* Improved Kapa widget colors

* Added an AI help link to the CLI

* We don’t need this anymore

* Exit with 1, indicating an error. This is important for CI

* Removed old auto-opening code

* Added some code comments to explain some of the annoying stuff

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-04-30 11:11:42 +01:00
Eric Allam 07365cd405 realtime: fix streams missing chunks when streams are longer than 5 minutes and receive 408 Request Timeout errors. Also now support multiple client streams being sent to a single stream key (#1993)
* realtime: fix streams missing chunks when streams are longer than 5 minutes and receive 408 Request Timeout errors. Also now support multiple client streams being sent to a single stream key

* Safely release the reader lock
2025-04-29 12:42:38 +01:00
Matt Aitken 83d0e8710d Fix for Schedules list page slow loading (#1992)
* Fix for Schedules list page slow loading

Getting BackgroundWorkerTask was very slow (Prisma was getting every single one…)

* Same fix for the upserting of schedules in the dashboard
2025-04-29 12:40:56 +01:00
Matt Aitken f579afb4cc V4 dequeue performance (return faster) (#1989)
* Loop through tenants only once, return as soon as we have enough.

* Hello world deadlock task set back to triggerAndWait
2025-04-28 14:49:38 +01:00
Saadi Myftija 7bdbbdc433 Handle rate limit errors from Slack on new alert creation (#1986) 2025-04-28 11:07:59 +01:00
Matt Aitken 8b0e4f2367 Fix for completing batches (when there are idempotent runs) (#1985) 2025-04-25 18:41:20 +01:00
Eric Allam e837500486 feat: v4 deadlock detection (#1970)
* Locked task runs will now require queues and tasks to be in the locked version

* Client errors caught in a run function now will skip retrying

* Extracted out the trigger queues logic

* extract validation, idempotency keys, payloads to concerns

* Extracted out a bunch of more stuff and getting trigger tests to work

* Add queue and locked version tests

* Deadlock detection WIP

* more deadlock detection

* Only detect deadlocks when the parent run is waiting on the child run

* Improve the error experience around deadlocks

* A couple tweaks to make CodeRabbit happy and fixing the tests in CI

* Fixed failing test

* Changeset

* wip

* Make sure to scope queries to the runtime env
2025-04-25 12:56:54 +01:00
Eric Allam 3a62ee36d9 v3: prevent envs with excess queues from degrading dequeue performance (#1982)
* v3: WIP maximum queues per env implementation

* Add a queue distribution test

* Add edge case tests

* Set the marqs maximumQueuePerEnvCount
2025-04-24 18:02:49 +01:00
Matt Aitken e9c6cc71f0 New batch id length is longer (#1944) 2025-04-24 13:38:15 +01:00
James Ritchie 32ff569844 Fixes bug on Safari preventing Queues and Schedules tables from scrolling (#1973) 2025-04-24 11:40:47 +01:00
James Ritchie 853a7ef7e4 Updates button copy and layout tweaks (#1941) 2025-04-23 16:33:46 +01:00
Saadi Myftija 2dbc085bb3 Fix alert creation bug related to the Slack integration (#1967)
* Fix alert creation bug related to the Slack integration

* Handle token_expired error from slack gracefully
2025-04-23 11:11:17 +01:00
Saadi Myftija 26aad58abd Add fuzzy matching to the tasks filter (#1960)
* Add a custom react hook to enable fuzzy searching in item lists

* Use fuzzy filtering in the tasks view list

* Use fuzzy filtering in the test tasks list

* Remove the old tasks filtering react hook
2025-04-22 16:34:56 +01:00
Saadi Myftija b62c17bec1 Show more recent payloads in the test task page (#1961) 2025-04-22 16:24:15 +01:00
Saadi Myftija fab565256e Allow empty payloads in test runs (#1962)
* Allow empty payloads in test runs

* Expose a basic linter configs in the code editor component

* Touch up the linting error elements in the code editor component
2025-04-22 16:23:52 +01:00
Eric Allam 006951aa53 Set a maximum number of times the dequeue loop can run (#1950) 2025-04-18 13:36:52 +01:00
nicktrn aa74260646 Dev command engine URL now configurable via the webapp (#1948)
* return dev engine url from config call

* set and use engine url from dev config reply
2025-04-18 11:11:11 +01:00
Matt Aitken bbf397e661 Completed batch waitpoints when we completed the BatchTaskRun (#1945)
* Completed batch waitpoints when we completed the BatchTaskRun

* Try complete the batch faster now it’s being used operationally

* Fix for tests that were using the old engine.unblockRunForCreatedBatch() function
2025-04-17 21:01:58 +01:00
James Ritchie ad088456d4 Changes button spinner to white and Replay Run button changes to secondary variant (#1912)
* Uses the white variant of the spinner instead of dark on the replay run button

* Changes the replay run button on the run page to secondary
2025-04-17 15:09:29 +01:00
James Ritchie 2aacf76444 Adds a new entry in the help and feedback modal to request more feedback (#1936) 2025-04-17 15:06:41 +01:00
Matt Aitken 5597291380 Schedules list performance (#1938)
* Order the schedules: most recent first

* TaskSchedule dashboard speed indexes
2025-04-17 15:05:50 +01:00
Matt Aitken 5c9610b807 The mini task page graphs were missing Timed out and Expired (#1915) 2025-04-17 15:05:37 +01:00
Matt Aitken 33e7b6865a Set env vars to be "secret" (#1923)
* WIP on secret env vars

* Editing individual env var values is working

* Sort the env vars by the key

* Deleting values

* Allowing setting secret env vars

* Added medium switch style

* Many style changes to the env var form

* “Copy text” -> “Copy”

* Draw a divider between hidden buttons

* Env var tweaks

* Don’t show Dev:you anymore

* Grouping the same env var keys together

* Styles improved

* Improved styling of edit panel

* Fix bun detection, dev flushing, and init command (#1914)

* update nypm to support text-based bun lockfiles

* add nypm changeset

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add changesets

* chore: remove pre.json after exiting pre mode

* init command to install v4-beta packages

* Revert "chore: remove pre.json after exiting pre mode"

This reverts commit f5694fde9314114c74a220c2213d19667bca1a6c.

* make init default to cli version for all packages

* Release 4.0.0-v4-beta.1 (#1916)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.1

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Both run engines will only lock to versions they can handle (#1922)

* run engine v1 will only lock to v1 deployments

* run engine v2 will only lock to managed v2 deployments

* test: create background worker and deployment with correct engine version

* Add links to and from deployments (#1921)

* link from deployments tasks to filtered runs view

* jump to deployment

* don't add version links for dev (yet)

* Fix current worker deployment getter (#1924)

* only return last v1 deployment in the shared queue consumer

* be explicit about only returning managed deployments

* Add a docs page for the human-in-the-loop example project (#1919)

* Add a docs page for the human-in-the-loop example project

* Order guides, example projects and example tasks alphabetically in the docs list

* Managed run controller revamp (#1927)

* update nypm to support text-based bun lockfiles

* fix retry spans

* only download debug logs if admin

* add nypm changeset

* pull out env override logic

* use runner env gather helper

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add logger

* add execution heartbeat service

* add snapshot poller service

* fix poller

* add changesets

* create socket in constructor

* enable strictPropertyInitialization

* deprecate dequeue from version

* start is not async

* dependency injection in prep for tests

* add warm start count to all controller logs

* add restore count

* pull out run execution logic

* temp disable pre

* add a controller log when starting an execution

* refactor execution and squash some bugs

* cleanup completed docker containers by default

* execution fixes and logging improvements

* don't throw afet abort cleanup

* poller should use private interval

* rename heartbeat service file

* rename HeartbeatService to IntervalService

* restore old heartbeat service but deprecate it

* use the new interval service everywhere

* Revert "temp disable pre"

This reverts commit e03f4179de6a731c17253b68a6e00bcb7ac1736b.

* add changeset

* replace all run engine find uniques with find first

* Release 4.0.0-v4-beta.2 (#1928)

* chore: Update version for release (v4-beta)

* Release 4.0.0-v4-beta.2

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>

* Remove batch ID carryover for non-batch waits (#1930)

* add failing test case

* do not carry over previous batch id when blocking with waitpoint

* delete irrelevant test

* Delete project (#1913)

* Delete project

- Don’t schedule tasks if the project is deleted
- Delete queues from the master queues
- Add the old delete project UI back in

* Mark the project as deleted last

* Fix for overriding local variable

* Added a todo for deleting env queues

* Remove todo

* Improve usage flushing (#1931)

* add flush to global usage api

* enable controller debug logs

* initialize usage manager after env overrides

* add previous run id to more debug logs

* add changeset

* For secret env vars, don’t return the value

* Added a new env var repository function for getting secrets with redactions

* Test task for env vars

* Delete heartbeat file, merge mess up

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Saadi Myftija <saadi.myftija@gmail.com>
2025-04-17 14:39:09 +01:00
Matt Aitken 1b758bd894 Only prevent triggering runs if the parent is completed AND it’s a triggerAndWait (#1935) 2025-04-17 14:25:20 +01:00
Matt Aitken 10f78cbad4 Delete project (#1913)
* Delete project

- Don’t schedule tasks if the project is deleted
- Delete queues from the master queues
- Add the old delete project UI back in

* Mark the project as deleted last

* Fix for overriding local variable

* Added a todo for deleting env queues

* Remove todo
2025-04-16 13:31:22 +01:00
nicktrn 3f97cf4a6b Managed run controller revamp (#1927)
* update nypm to support text-based bun lockfiles

* fix retry spans

* only download debug logs if admin

* add nypm changeset

* pull out env override logic

* use runner env gather helper

* handle dev flushing failures gracefully

* fix path normalization for init.ts

* add logger

* add execution heartbeat service

* add snapshot poller service

* fix poller

* add changesets

* create socket in constructor

* enable strictPropertyInitialization

* deprecate dequeue from version

* start is not async

* dependency injection in prep for tests

* add warm start count to all controller logs

* add restore count

* pull out run execution logic

* temp disable pre

* add a controller log when starting an execution

* refactor execution and squash some bugs

* cleanup completed docker containers by default

* execution fixes and logging improvements

* don't throw afet abort cleanup

* poller should use private interval

* rename heartbeat service file

* rename HeartbeatService to IntervalService

* restore old heartbeat service but deprecate it

* use the new interval service everywhere

* Revert "temp disable pre"

This reverts commit e03f4179de6a731c17253b68a6e00bcb7ac1736b.

* add changeset

* replace all run engine find uniques with find first
2025-04-15 16:08:51 +01:00
nicktrn 1816115f0f Fix current worker deployment getter (#1924)
* only return last v1 deployment in the shared queue consumer

* be explicit about only returning managed deployments
2025-04-14 14:55:27 +01:00
nicktrn 5f4c6071d8 Add links to and from deployments (#1921)
* link from deployments tasks to filtered runs view

* jump to deployment

* don't add version links for dev (yet)
2025-04-14 14:43:52 +01:00
nicktrn faf5d01b2c Both run engines will only lock to versions they can handle (#1922)
* run engine v1 will only lock to v1 deployments

* run engine v2 will only lock to managed v2 deployments

* test: create background worker and deployment with correct engine version
2025-04-13 21:51:42 +01:00
nicktrn 63a221abd0 Only include v4 debug logs for admins (#1910)
* fix retry spans

* only download debug logs if admin
2025-04-11 09:07:51 +01:00
Matt Aitken 030c8ae510 Don't show queue time in the run timeline by default (#1909)
* WIP on hiding the queued time

* Fix for position/sizes of spans

* Fix for duration squashing

* Tweaked some styles on the timeline

* Fix for retry spans appearing in the wrong place

* Added “Q” shortcut key

* Fix for in progress span durations
2025-04-10 16:37:39 +01:00
Matt Aitken db97f80646 Fix for queue indexing not updating concurrencyLimit/removing it (#1906) 2025-04-09 15:53:21 +01:00