Commit Graph

3609 Commits

Author SHA1 Message Date
Dan 5164935f9c Added X-Frame-Options and CSP (#1690) 2025-02-28 11:06:27 +00:00
James Ritchie e1db12b8ba Expandable code blocks (#1683)
* Removed unnecessary classes from Dialog

* Display large modal with code and copy button

* Fixes button position if chrome is shown

* Removes the duplication of the Highlight component
2025-02-28 09:17:56 +00:00
Eric Allam 187200a1bc Feat: Improved run start timeline visibility (#1732)
* Record cold start and execution metrics on attempt executions. Add cold start metrics as span events on attempt spans and display them in the run dashboard

* Add deployed tasks run timeline metrics

* Add Dequeued event to run timeline and cleanup the run timeline code

* Adds variants to storybook

* WIP adding new span styles

* Added offset progress bar animation

* More storybook states

* Adds support for the full vertical span to show the same state

* Adds error state to timelineLine

* Added additional state

* Added more line styling

* Added progress state to dequeued

* Added another state to storybook

* Fixed classname error

* Updated styles for the span timeline points

* Fixes alignment of timeline follow cursor indicator

* Adds help text tooltip to timeline span type titles

* Fixes type error

* Tweaked wording of tooltips

* Fixed type error (check this)

* Moved isAdmin to a higher level

* removed unused admin props

* Removed unused Admin filter

* Fixed border styling

* made the opacity of the timeline states 30% less

* Undo type cast

* Added a diminished style that’s used for spans (grey progress bar)

* Adds new storybook state

* Fixed timeline state

* Removed state if span isn’t the first

* Changed the timestamp span icon

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2025-02-27 20:47:48 +00:00
Eric Allam 26f9a1e478 Feat: two phase deployment, version pinning (#1739)
* WIP two-phase deployments

* Fix the help text

* Rename TRIGGER_WORKER_VERSION to TRIGGER_VERSION

* Add changeset

* A few naming fixes
2025-02-27 20:44:23 +00:00
Eric Allam 0e5ec8bfbc Only allow a single dev queue consumer to dequeue at once (#1737)
* Only allow a single dev queue consumer to dequeue at once

* Expire dev queue consumer connections keys

* Add missing return
2025-02-27 15:08:47 +00:00
Eric Allam 62e3238ecc Add configurable redis TTL on realtime streams (#1725) 2025-02-21 16:07:12 +00:00
nicktrn 80f2fc8bbe Add scheduled at timestamp to created run containers (#1722)
* add scheduled at timestamp to k8s provider

* add scheduled at timestamp to docker provider
2025-02-20 17:27:38 +00:00
Eric Allam bf43fd4f8e Use priority offsets in MarQS and schedule future messages using redis worker (#1720)
* Improve the MarQS priority system by moving future messages into the LRE worker and using a priority timestamp offset to define priority in messages

* Add a test to make sure priority offsets don't unfairly favor environments

* requeuing should clear concurrency sets

* Heartbeats should only reschedule existing heartbeat jobs

* Fix type error
2025-02-20 14:06:32 +00:00
nicktrn 7186b1e428 Add support for deferred checkpoints (#1721)
* add ability to delay checkpoints

* optional checkpoint delays for dependency waits

* prevent checkpoint creation for resumed batches

* unpause after checkpoint rejected by platform

* prevent checkpoint creation for resumed task waits

* fix checkpoint cancellation
2025-02-20 12:47:09 +00:00
Eric Allam 23b4c2a214 Add MarQS requeueMessage, an atomic version of replaceMessage (#1717)
* New MarQS method requeueMessage, an atomic version of replace message

* Remove redundant call to remove the message from the env queue in requeueMessage
2025-02-19 15:11:43 +00:00
Eric Allam 7b1159eb45 MarQS reserve concurrency system & queue priority for resuming/retrying (#1715)
* run engine v1: orgs are no longer considered for concurrency

* Add reserve concurrency concept to allow waiting to resume parent tasks to release concurrency at the env level for child tasks to use (or else there is a deadlock). WIP recursive tasks

* child tasks inherit the queue timestamp from their parent tasks to prioritize completing child tasks based on when their parent started

* handle reserve concurrency with recursive deadlocks

* Finish docs update for concurrency

* Some fixes from badge conflict resolution

* WIP priority queues

* Implement MarQS priority queues

* Fix the migrations
2025-02-19 11:43:34 +00:00
Erin Allison b4d1619732 Remove V2 countdown from sidebar (#1712)
Signed-off-by: Erin Allison <erin@eallison.us>
Signed-off-by: Erin Allison <eallison@andrettikarting.com>
Co-authored-by: James Ritchie <james@trigger.dev>
2025-02-18 10:59:12 +00:00
nicktrn 8f3f373002 Improve reliability of completion submission (#1711)
* add timeout support to sendWithAck

* coordinator will retry completion submission

* actually retry

* increase default retries

* something went wrong there, add this back in

* add changeset
2025-02-14 15:14:42 +00:00
nicktrn 36fea7d645 Add support for manual checkpoints (#1709)
* don't crash run on lazy attempt errors and count on retry

* remove abort controller after checkpoint in all cases

* duration checkpoints

* manual checkpoints

* add busybox to coordinator image

* add async toggle to process in background

* remove early returns

* add changeset for manual checkpoint schema
2025-02-14 11:35:10 +00:00
Matt Aitken 440d413ce8 Alert Webhook improvements (#1703)
* WIP with webhook SDK function and types

* JSDocs added to the schema

* Webhooks are working

* Expanded the alert docs

* Remove duplicate export of waitUntil.js

* Use uncrypto

* Don’t rate limit webhooks

* Create slow-olives-fix.md
2025-02-13 13:44:36 +00:00
Eric Allam a2c70b450a Upgrade local dev to use electric beta.15 (#1699) 2025-02-13 11:02:01 +00:00
nicktrn a38f713e32 Fix run container exits after OOM retries (#1701)
* remove unused imports

* tell run to exit before force requeue

* handle exit for case where we already retried after oom

* improve retry span and add machine props

* don't try to exit run in dev
2025-02-12 18:32:58 +00:00
nicktrn 0924b37283 Provider changes to support image cache (#1700)
* add env var for additional pull secrets

* make static images configurable

* optional image prefixes

* optional labels with sample rates

* add missing core paths

* remove excessive logs
2025-02-12 16:57:55 +00:00
Eric Allam d622aba86d Fix missing logs on child runs by using the root task run createdAt if it exists (#1697) 2025-02-12 14:03:48 +00:00
Matt Aitken d214aa9093 Don't create an attempt if the run is final, batchTriggerAndWait bad continue fix (#1698)
* WIP fix for ResumeAttemptService selecting the wrong attempt (which has no error or output)

* Don’t create an attempt if the run is already in a final status

* Don’t get all the columns for the query. Improved the logging.

* Added a log to the batch example

* Filter out the undefined values
2025-02-12 14:02:29 +00:00
Eric Allam bd0cc541c5 Create new partitioned TaskEvent table, and switch to it gradually as new runs are created (#1696)
* Create new partitioned TaskEvent table, and switch to it gradually as new runs are created

* Add env var for partition window in seconds

* Make startCreatedAt required in task event store
2025-02-12 09:12:08 +00:00
Matt Aitken efd2d21762 Detect ffmpeg OOM errors, added manual OutOfMemoryError (#1694)
* Detect ffmpeg OOM errors, added manual OutOfMemoryError

* Create eighty-spies-knock.md
2025-02-12 00:02:01 +00:00
Matt Aitken 6017c522e1 Batch queue runs that are waiting for deploy (#1693) 2025-02-11 19:06:38 +00:00
Matt Aitken d2ee232fae Revert "Complete the original attempt span if retrying due to an OOM"
This reverts commit 5f652c6212.
2025-02-11 11:33:07 +00:00
Matt Aitken 5f652c6212 Complete the original attempt span if retrying due to an OOM 2025-02-11 11:01:21 +00:00
Matt Aitken 036a5061e1 Kubernetes OOMs appear as non-zero sigkills, adding support for treating these as OOMs 2025-02-11 10:32:01 +00:00
Matt Aitken bb65b2614a OOM retrying on larger machines (#1691)
* OOM retrying on larger machines

* Create forty-windows-shop.md

* Update forty-windows-shop.md

* Only retry again if the machine is different from the original
2025-02-10 17:43:59 +00:00
Matt Aitken 4dd42cd6d4 Retry heartbeat timeouts by putting back in the queue (#1689)
* If there’s a heartbeat error and no attempts we put it back in the queue to try again

* When nacking, return whether it was put back in the queue or not

* Try and nack, if it fails then fail the run

* Consolidated switch statement

* Fail executing/retrying runs
2025-02-10 13:41:32 +00:00
Matt Aitken bc7d44582d If there’s no queue concurrency limit, default to the env concurrency limit (not 1m) (#1687) 2025-02-09 21:34:40 +00:00
James Ritchie 1f1a6b0405 Fixed runs page filter bug (#1681) 2025-02-07 17:32:30 +00:00
Eric Allam 9f843d0a0f Remove v2 event list (#1680) 2025-02-07 13:51:34 +00:00
Eric Allam 9dfd6a5a0a Rate limit alerts by channel for task run alerts using generic cell rate algo (#1679) 2025-02-07 13:36:37 +00:00
Eric Allam 5210d3a8bb Add internal spans to event repo (#1678) 2025-02-07 13:36:15 +00:00
Eric Allam b5867627be Use idempotency key index when finding existing runs by idempotency key in batch (#1677) 2025-02-07 13:36:03 +00:00
Eric Allam 0b555faf01 Force upgrade v1 batches to v3 (#1676)
* Force upgrade v1 batches to v3

* Actually lets process v1 batches async
2025-02-06 22:37:26 +00:00
Eric Allam dcf1ab6f38 Remove schedule constraints from TaskRun to prevent db load issues when a schedule or instance is deleted (#1674) 2025-02-06 19:16:48 +00:00
Matt Aitken c49af774ba Retry batch item completion (#1675)
* Added isPrismaRetriableError()

* Retry completeBatchTaskRunItem if they fail because of a retriable Prisma error

* Retry using Redis worker

* Handle more retriable errors. Add special condition in for race condition error

* Added Postgres connection_timeout with default 20s

* Added a simple batchTriggerAndWait example
2025-02-06 19:16:30 +00:00
Eric Allam f00ed9b13e Add the sha256 digest to the deployment image reference (#1673) 2025-02-06 19:14:34 +00:00
Eric Allam 3ec6983e45 Move run ttl and delays from graphile to redis worker (#1672) 2025-02-06 14:25:37 +00:00
Eric Allam a2c2d920b7 Use redis worker for run heartbeats and alerts (#1669)
* Move the task run heartbeats to RedisWorker

* Move alerts to redis worker, improving redis worker

* Fix typecheck errors

* Use single threaded tests for redis worker

* Enable/disable the redis workers independently

* Remove preview release from PR checks
2025-02-06 10:21:57 +00:00
nicktrn b946b9f38e Add optional env var and label to disable image pre-pulls (#1667)
* add env var to disable pre-pulls

* add label to disable pre-pulls on specific nodes
2025-02-05 16:49:57 +00:00
Eric Allam e61573600b Reduce contention on batchTaskRun when setting expected count (#1662) 2025-02-04 13:43:50 +00:00
Eric Allam 53e6d478ed Fix deploy timeout issues (#1661)
* deploy v2 streaming WIP

* finalize deployment now SSE and won't timeout

* handle zodfetchSSE connection errors

* Create tender-cycles-melt.md
2025-02-04 12:44:14 +00:00
nicktrn 7fa45ad337 Fix pre-pulling of deployment images onto worker nodes (#1660)
* lint

* get pre-pull data from finalized deployment
2025-02-04 10:07:07 +00:00
Eric Allam 6f52b00560 Set taskRunAttemptId on batch task run items on completion (#1658) 2025-02-03 15:25:12 +00:00
Eric Allam b377b577ca db efficiency improvements (#1657)
- remove calls to findUnique, especially when including/selecting relations
- add some missing indexes
- add spans to $transaction calls to help track down long running txs
2025-02-03 13:59:58 +00:00
Eric Allam b67ea9a2f5 improve batch completion system for run engine v1 (#1656)
* Automatically retry TriggerTaskService when hitting a unique constraint error on idempotency key

* improve batch completion system for run engine v1

* Rename batch stuff to v3 so it's not confusing

* Handle unique constraint error on BatchTaskRunItem creation and allow different limits for batchTrigger and batchTriggerAndWait
2025-02-03 11:29:51 +00:00
Eric Allam 3641647add Add the keyPrefix option when redis is not in cluster mode (#1655) 2025-01-31 10:50:52 +00:00
Eric Allam 813d73d1a3 Fix issue with heavy contention on TaskQueue updating concurrency limit (#1653)
* Fix issue with heavy contention on TaskQueue updating concurrency limit

* Remove concurrency limit when creating background worker tasks
2025-01-30 14:01:11 +00:00
Eric Allam 3eede3c339 Better handle redis error events (#1652) 2025-01-29 17:27:52 +00:00