Commit Graph

2994 Commits

Author SHA1 Message Date
James Ritchie 2940b07430 Added a description panel to the Concurrency admin page 2024-08-14 11:33:15 +01:00
James Ritchie c46e3c97b5 Fix for schedule page not scrolling 2024-08-14 11:23:16 +01:00
Matt Aitken 2b0d68a0b6 Include the old message data when replacing, so we get the projectId etc. 2024-08-13 17:09:29 +01:00
Matt Aitken f1ffc2e6c8 Added messageReplaced to concurrency tracking (when freezing) 2024-08-13 16:36:39 +01:00
Matt Aitken c0b9394099 Call subscriber.messageDequeued in dequeueMessageInSharedQueue 2024-08-13 15:31:25 +01:00
Matt Aitken 937b61913d Added more logging to TaskRunConcurrencyTracker and some more try/catches 2024-08-13 14:54:50 +01:00
nicktrn 5cf90da722 v3: fix unfreezable state crashes for runs with multiple waits (#1253)
* support named capture groups

* write crash errors to attempt.error

* make restored pod names unique per checkpoint

* use last eight characters of checkpoint id instead

* add more chaos monkey env vars

* Ignore unfreezable states

* prevent excessive queue config parsing errors

* handle dependency resume edge case

* better entry point logging

* ignore checkpoint cancellation timeouts

* add missing idempotency keys to wait for dep replays

* remove checkpoints between attempts

* fix retry container names on kubernetes

* add changeset

* fix types

* bring back internal duration timers
2024-08-13 11:44:40 +01:00
Matt Aitken da6ce3c8d5 Concurrency page and more accurate tracking (#1252)
* Initial TaskRunConcurrencyTracker implementation

* MARQS calls a subscriber to events

* When enqueuing add the extra required metadata

* Track concurrency per environment for tasks too

* Admin page for global concurrency

* Use the new concurrency tracker on the tasks page

* Useful performance test task

* getAllTaskIdentifiers()

* New page for concurrency

* BackgroundWorkerTask index for quick lookup of task identifiers

* Added a way to get concurrency for environments

* Added upgrade/request more concurrency button

* Queued task column working

* Use defer and suspense

* Added queue column to the concurrency environments table

* Some comments added for clarity

* Fixed bad log message

* Sidemenu: move lower and rename to “Concurrency limits”

* Only show the environments, not tasks. Renamed to “Concurrency limits”
2024-08-13 11:43:46 +01:00
James Ritchie 07054f13d1 Fixed v2 usage meter displaying when on paid plan (#1255)
* Fixed v2 usage meter displaying when on paid plan

* Show the free usage panel only for v3 projects
2024-08-13 11:14:28 +01:00
Matt Aitken bb38261947 FinalizeRunService (#1250)
* WIP notes on each location where we’ll use finalize

* Initial FinalizeTaskRunService

* ExpireEnqueuedRunService uses FinalizeTaskRunService

* FailedTaskRunService uses FinalizeTaskRunService

* Allow passing in an include when finalizing the run

* CrashTaskRunService using FinalizeTaskRunService

* Remove comments

* Status is optional

* CancelAttemptService using FinalizeTaskRunService

* Import tidy

* CancelTaskRunService using FinalizeTaskRunService

* Import tidying

* CompleteAttemptService system failure switched to FinalizeTaskRunService

* Added more logging to Finalizing

* CompleteAttemptStatus COMPLETED_SUCCESSFULLY

* CompletedAttempt “SYSTEM_FAILURE”

* CompletedService final pair

* Use satisfies so we can derive types from the groups

* Only allow final states to be used with this service

* BaseService tx support, minor improvements
2024-08-08 15:32:06 +01:00
James Ritchie 0d4e3e70c1 Lots of small docs improvements (#1251)
* tasks no longer inside a group in the side menu (and added “cron”)

* Delay using a timezone

* Added React Not Defined error to the troubleshooting page

* Improved the React common problem

* Link to v2 docs

* New Development section and entry in Common Problems

* Concurrently running the terminal

* Fixed the .env weirdness

* Added section on creating PATs for Github actions

* Improved the Machine spec and limits page

* Quick start steps now have nice images

* Added a diagram for the lifecycle functions

* Added note about onFailure

* CRON -> cron/Cron

* Updated the cli-dev steps for the concurrently package

* Fixed capital letter

* Updated diagram text

* Removed dead page
2024-08-08 15:03:05 +01:00
nicktrn bf1482fdb9 Save crash errors on the attempt 2024-08-08 11:42:46 +01:00
Matt Aitken e2d3358271 Added error from the run attempt with nice display (#1249) 2024-08-07 15:50:45 +01:00
James Ritchie fafda21e3b V2 end of life messaging, paid plan badge, and project menu improvements (#1245)
* Updated the v2 project message to include end of life

* Removed the v3 badge. Showing a v2 badge instead

* Removed the v3 badge variant

* Show a badge in the side menu if you’re paying

* Popover menu section header and button state colours now reflects the design UI

* Added a link to the blog in the end-of-life message

* Removed missing variant from badge
2024-08-06 10:50:23 +01:00
nicktrn dc2d68107f Revert "Ignore unfreezable states"
This reverts commit 7d9cec8652.
2024-08-06 10:11:28 +01:00
Matt Aitken b65e7f7a7a Show a different message if your logs are deleted but you’re within your retention period.
This happens if you’ve upgraded but the logs were already deleted.
2024-08-06 10:10:40 +01:00
nicktrn 7d9cec8652 Ignore unfreezable states 2024-08-06 08:56:06 +01:00
Matt Aitken a591e1118f Improved the experience when you have no run logs (#1244)
* Remove the old log truncation

* Added TaskRUn logsDeletedAt column

* Accurate timestamps for the run inspector

* EnsureProperty type when you want to make a single property not nullable

* No logs and upgrade messages working

* Button can be autofocused

* Replay dialog code editor is autofocused

* Fix for wrapping of span duration
2024-08-05 18:06:23 +01:00
nicktrn 4d2ea86d6c Use accurate datetime for run timeline 2024-08-05 15:16:43 +01:00
Eric Allam 366ae5ca7d Create a shared queue span every 10 seconds 2024-08-02 17:31:52 +01:00
Eric Allam 157973a720 Force recording shared queue consumer spans 2024-08-02 17:30:27 +01:00
Eric Allam 1353d66ca1 Additional logging to help track down possible issue with MarQS selection algo 2024-08-02 12:27:50 +01:00
Matt Aitken 1c5ee5d94c Added TaskRun completedAt column (#1243)
* Added TaskRun completedAt column

* When finalising a run set the completedAt date

* A note to discuss whether we need to set the completedAt to null

* Remove the note in the sharedQueueConsumer
2024-08-02 10:31:38 +01:00
nicktrn 8cae1d0875 v3: fix triggering tasks with custom queues (#1242)
* add missing dotenv requires in catalog scripts

* pass task queue options to all task trigger functions

* remove unnecessary include

* fallback to background worker task queue options

* add changeset
2024-08-01 11:26:34 +01:00
Matt Aitken 09413a62a4 Improved run page and replaying (#1240)
* PropertyTable component changed to use sub components

* Separate run span component

* Early WIP on tabs that use search query

* Shortcut key tabs for the span panel

* The span timeline is working

* When runs get expired, update the OTEL event with an error

* Improved the expired error message

* Reveal env vars when editing

* Tightened things up a bit

* Added detail tab properties

* Progress dashed line

* Move the env label next to the Run number title

* Top level cancel/replay buttons

* Replay with a different payload and environment

* Fix for non json payloads

* Hide the clear/copy buttons

* UI improvements with large payloads

* Close the panels when you replay/cancel

* Added the new timeline to spans

* Use the u-turn left icon for replay

* Added an index for spanId on TaskRun

* Remove replay/cancel buttons the span view

* Replay shortcut works inside the code editor

* Split the log/span inspector between Overview and Detail as well

* More improvements to the inspector

* Context and output improvements

* Focus on run working

* Added version to run.ctx

* Added some padding to the detail view

* Added context tab with shortcut

* Only load the replay data when the dialog is open

* Replaying uses the tags from the original run

* Links are now text links

* Removed version links for now because we don’t have dropdown filters for them yet

* Tabs are now outside of the scrollview

* The inspector is now 30% of the width by default

* Allow replaying and editing SuperJSON payloads

* Deleted unused CodeGroup file

* Increase the tags limit to 5, do the limiting on the server

* The admin tooltip now always shows basic org, project and user info

* Fix for schedule inspector disabled state layout

* Remove new unused span metadata and context

* Removed unused import
2024-07-31 15:11:50 +01:00
nicktrn 9882d66f87 v3: pre-pull deployments for faster startups from the first run (#1236)
* pre-pull deployed images on all workers

* automatically clean up pre-pull resources

* rename pre-pull image to deployment

* add changeset
2024-07-26 13:12:58 +01:00
nicktrn e27d5cd861 Prevent registry proxy from trying to send responses twice 2024-07-26 08:44:03 +01:00
nicktrn e64cd5ee86 Fix usage page for previous months 2024-07-25 18:34:14 +01:00
James Ritchie ac088c1a73 Fixed safari bug not respecting recharts height 2024-07-24 12:11:01 +01:00
James Ritchie 12d2bd6cdd Added links to featurebase in the docs and app feedback panel (#1213)
* Added links to the new roadmap and feature request pages. Also redirected the old feature-matrix page to docs home

* Added a new request feature link to the help and feedback panel

* Added Book a call with the founders option in the feedback panel
2024-07-24 11:59:48 +01:00
Matt Aitken 8614ea2107 Tags docs and a couple of minor fixes (#1235)
* runs.list and runs.retrieve OpenAPI docs updated

* Added docs for how to use runs.retrieve and runs.list with usage data

* SDK reference for triggering with tags

* Added Tags docs page to the sidenav, it’s a copy of usage right now

* Fix for type “users” -> “uses”

* Docs for the tags feature

* Fix for bad import

* Added docs how propagating tags to child runs

* Better error messages when Zod parsing fails for triggering and batch triggering

* Tags are now a string or array of strings, this makes it a lot easier to use
2024-07-24 11:22:03 +01:00
Matt Aitken 8dfd47eb41 Fix for the tags making the run table rows taller 2024-07-23 19:17:02 +01:00
Matt Aitken 55264657d7 Run tags (#1232)
* TaskRunTag migration. Removed unused TaskTag

* Show tags for a run in the run list

* API endpoint for searching tags

* Let’s not expose an API endpoint for tags at the moment

* Filter by tags working from the URL

* Tag async filter working

* We don’t need a “None” option because it’s multi-select

* When triggering a run you can add tags

* SDK runs.retrieve and runs.list with tag support

* Tidied imports

* Changed the run list query so we show all the tags even if a run only matches one of them

* Fix for dealing with weird characters in tags

* Run tags changeset

* Creating a project has a proper loading state (and blocks multiple)

* Improved the error message for tag length

* Added a tags icon for display on the run screen

* Convenient functions for creating and getting run tags

* tags.set() from inside the run function

* Replay a run passes tags through

* Less ridiculous tags for the catalog example

* Allow passing just a string for the tags

* Order by id because there’s an index on the primary key already

* Trim the tags earlier so we don’t accidentally error if a blank string is passed

* Renamed some thing from setTags to addTags

* Use findFirst for the tags project lookup

* Added an index for "TaskRunTag"("name", "id")

It massively improves the performance of run filtering based on tags

* Use `array_agg` for the run list tags so pagination works and we get a single result for each run

* Tidied imports

* More comprehensive test of tags with all triggering functions

* Support tags with tasks.trigger, tasks.batchTrigger and tasks poll variants

* Added tasks.batchTrigger tags

* Sort the tags in the UI so they’re always in the same order

* Added tooltips in the run table for delay, ttl and tags

* Added costInCents, baseCostInCents and durationMs to runs.retrieve and runs.list

* Added support for displaying a split tag if you use key_value or key:value format

* Fix code comment

* Tweaked the JSDoc to make the prefixing clearer

* Added tasks.triggerAndWait to the tags test task
2024-07-23 16:48:09 +01:00
Matt Aitken 6293290fb9 Better prediction of usage (uses Linear regression) 2024-07-20 10:50:46 +01:00
James Ritchie da592d867a Removed logo 2024-07-19 15:20:40 +01:00
Eric Allam d7089e12a9 Replace findUnique with findFirst when doing a relation aggregate 2024-07-19 10:18:56 +01:00
Eric Allam 162d864517 Fixed prisma dataLoader slow query when getting an attempt payload 2024-07-19 09:42:48 +01:00
Matt Aitken 8ba9987944 Declarative schedules (#1226)
* Added type (STATIC or DYNAMIC) to TaskSchedule. Defaults to dynamic

* WIP with dev indexing of static schedules

* Added a code comment

* First stab at deleting unused static schedules

* Dashboard changes for the static schedules

* Generate the description. Upsert the instances when editing. Fix for the friendlyId

* Don’t allow deleting of static schedules

* Don’t allow enabling/disabling of static schedules

* Added filtering for schedule types

* Syncing of schedule for deployed tasks

* Static schedules are now created for each environment

* Added a second static schedule for testing

* Add the type to the schedule task run payload and the object you get back from the SDK

* Changed static/dynamic to declarative/imperative

* Timezone example

* Changeset

* Updated scheduled docs to include declarative

* When you test a schedule it set the type to “IMPERATIVE”

* Improved the tooltip

* Fix for queue time continuing to rise when a run is canceled/expired etc

* Update the info panel on a selected declarative schedule

* Check if there are no instances. This should never happen but log an error if it does

* Throw errors and push them through to the CLI dev command

* Fail deployments if creating the background tasks or schedules fails

* Format the deployment error so it gets displayed

* Changed the maxed out schedules error message to remove bit about support
2024-07-18 20:24:54 +01:00
nicktrn e0cdebbe09 Disable excessive websocket heartbeat logs 2024-07-17 13:28:40 +01:00
nicktrn dbda820a71 v3: fix websocket timeouts (#1220)
* prevent uncaught websocket exceptions

* dev cli sends regular pings to keep connection alive

* add changeset

* fix for prod message forwarding

* add server-side websocket ping

* Revert "dev cli sends regular pings to keep connection alive"

This reverts commit 9a6a7a21433b9f0665f72be015d0ba5a8d83eba4.

* add dev cli websocket debug logs, always print errors

* actually start the ping service

* update changeset

* catch remaining dev cli message handler errors
2024-07-17 09:05:15 +01:00
Matt Aitken f91a118958 Fix for dev env vars getting deleted when another user edits theirs 2024-07-16 10:57:35 +01:00
Matt Aitken 0847bee5b8 Fix for stripping out TRIGGER_ env vars 2024-07-15 17:11:15 +01:00
Matt Aitken 6152b2f3dc Don’t error when setting TRIGGER_SECRET_KEY or TRIGGER_API_URL env vars, just filter them out 2024-07-15 16:39:13 +01:00
Matt Aitken ec8381e2d0 Fix for staging environment variable API endpoints
Ugly but effective fix for the fact we can pass the slug in as “staging” when we use “stg” internally
2024-07-15 16:33:00 +01:00
Matt Aitken 76a15fd278 The run filter for tasks query was very slow if you had a lot of synced tasks 2024-07-15 15:59:11 +01:00
nicktrn 824a6205f7 Fix deployments stuck in deploying state 2024-07-15 12:00:16 +01:00
Matt Aitken a0984d5292 Allow alerts from external and unlinked Slack channels 2024-07-12 11:14:57 +01:00
Matt Aitken c61d8ed890 Removed badge from the Tasks in the side menu
It was showing the v2 job count. Mostly this didn't actually show anything but one user reported they saw a number here and it's definitely a bug.
2024-07-12 10:51:54 +01:00
Eric Allam 76a0f61dee v3: improve schedule reliability by ensuring next tick is always scheduled 2024-07-11 15:35:34 +01:00
Jacob Paris d934feb02a feature: add no-trigger-core-import (#1175)
* feature: add no-trigger-core-import

* fix: handle aliases and limit to exported modules

* fix: use longest export map name

* fix: support export type and interface

* fix: simplify message when export map is needed

* feature: fine grained modules for core

* fix: add prerequisite lint plugins

* fix: why do I have to fix these components again???

* fix: tests

* Add eslint config and vscode settings

* Create yellow-roses-arrive.md

* Remove the vscode linting

This is temporary so no one fixes these until we merge the remix branch

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-07-11 15:14:55 +01:00