Eric Allam
8d5c86fea0
* WIP * Make release concurrency system extremely simple, everything just releases all the time * update the deadlock detection to use the new lockedQueueReleaseConcurrencyOnWaitpoint column * WIP new release concurrency system * Remove releaseConcurrency and releaseConcurrencyOnWaitpoint Also removed deadlock detection, and added environment burst concurrency * Added new DEQUEUED status Cleaned up the API run statuses, including now detecting new clients and not breaking older clients by adding an API version header to all requests * Introduce the new "current dequeued concurrency set" * Remove QUEUED_EXECUTING because we no longer "eagerly" release before checkpointing * Remove waitpoint test for QUEUED_EXECUTING * Add isWaiting * Add changeset * Use createdAt for ordering realtime runs instead of number * Clarify the envCurrentDequeuedKey usage * mock the db.server file to fix the tests * Updated changset "EXECUTED" -> "EXECUTING" --------- Co-authored-by: Matt Aitken <matt@mattaitken.com>
3.0 KiB
@trigger.dev/sdk
| @trigger.dev/sdk |
|---|
| patch |
Removes the releaseConcurrencyOnWaitpoint option on queues and the releaseConcurrency option on various wait functions. Replaced with the following default behavior:
- Concurrency is never released when a run is first blocked via a waitpoint, at either the env or queue level.
- Concurrency is always released when a run is checkpointed and shutdown, at both the env and queue level.
Additionally, environment concurrency limits now have a new "Burst Factor", defaulting to 2.0x. The "Burst Factor" allows the environment-wide concurrency limit to be higher than any individual queue's concurrency limit. For example, if you have an environment concurrency limit of 100, and a Burst Factor of 2.0x, then you can execute up to 200 runs concurrently, but any one task/queue can still only execute 100 runs concurrently.
We've done some work cleaning up the run statuses. The new statuses are:
PENDING_VERSION: Task is waiting for a version update because it cannot execute without additional information (task, queue, etc.)QUEUED: Task is waiting to be executed by a workerDEQUEUED: Task has been dequeued and is being sent to a worker to start executing.EXECUTING: Task is currently being executed by a workerWAITING: Task has been paused by the system, and will be resumed by the systemCOMPLETED: Task has been completed successfullyCANCELED: Task has been canceled by the userFAILED: Task has failed to complete, due to an error in the systemCRASHED: Task has crashed and won't be retried, most likely the worker ran out of resources, e.g. memory or storageSYSTEM_FAILURE: Task has failed to complete, due to an error in the systemDELAYED: Task has been scheduled to run at a specific timeEXPIRED: Task has expired and won't be executedTIMED_OUT: Task has reached it's maxDuration and has been stopped
We've removed the following statuses:
WAITING_FOR_DEPLOY: This is no longer used, and is replaced byPENDING_VERSIONFROZEN: This is no longer used, and is replaced byWAITINGINTERRUPTED: This is no longer usedREATTEMPTING: This is no longer used, and is replaced byEXECUTING
We've also added "boolean" helpers to runs returned via the API and from Realtime:
isQueued: Returns true when the status isQUEUED,PENDING_VERSION, orDELAYEDisExecuting: Returns true when the status isEXECUTING,DEQUEUED. These count against your concurrency limits.isWaiting: Returns true when the status isWAITING. These do not count against your concurrency limits.isCompleted: Returns true when the status is any of the completed statuses.isCanceled: Returns true when the status isCANCELEDisFailed: Returns true when the status is any of the failed statuses.isSuccess: Returns true when the status isCOMPLETED
This change adds the ability to easily detect which runs are being counted against your concurrency limit by filtering for both EXECUTING or DEQUEUED.