* Delete the proxy app (was v2)
* Delete RunPresenterElectric
* Select the best proj/org/env
* Storing current proj/env in DB. Initial selection logic working with tasks page
* 2sm needed to be in the Tailwind merge list
* Move the task stream route (although we don’t actually use the env for now)
* Alerts moved from /v3
* API keys page moved from /v3
* Concurrency page moved from /v3
* WIP on side menu sections
* Improved the accordion animation
* Moved schedules from /v3
* More pages moved
* Move pages working
* Run page working
* Schedules working
* Moved deployments
* Alert pages moved
* Delete electric hooks, not used
* Started setting up blank states
* Test page working
* Removed “Select task” from the test page
* Some work on deployment page
* Style tweaks
* Redirect from project root to approriate env
* Improved env selector styling
* Fix for jsx errors
* Better min width on env selector
* Improved the env switching logic
* Added deployments to env routing
* Redirect deployments to the correct env
* Redirect run from proj to env
* JSX icon fix
* Only allow single env schedules from now on
* Remove env var count from the API keys page
* Move improvements and redirects
* Project settings moved
* Fix for scroll area on test page
* Tweaked the test design
* Made recent payloads column narrower
* Improved the test layout some more
* Added org icon, new project selector menu
* WIP on org switching menu
* Org switching is working
* New menu working well, removed old side menu items
* Buttons can now have a component name or an actual component for their icons
* Removed the Projects page, instead redirect appropriately
* Fix for broken blank states
* Minor run table improvements
* Removed unused switcher log and logic
* Concurrency page fix for invalid html, improved layout
* Minor improvements
* Moved the side menu to the project level
* Improved account styling
* Moved org settings pages (with redirects)
* Add current plan to billing side menu link
* Upgrade to get staging from env dropdown
* New env badge on concurrency limits page
* Show Run Engine version in span presenter
* New promote icon
* Concurrency limits page is the sum of engine v1 + v2 queues
* Fix for missing batch import
* Added currentConcurrencyOfEnvQueue function
* Basic avatar setting working
* Avatar setting is working
* You can change the color of your icon
* Avatar improvements
* Bugfix for mising prop
* Removed some old env badges
* Fixed replaying
* Removed EnvironmentLabel
* Old env badge deleted, changed everywhere to the new one
* Fix for Slack integration paths
* Fix for waitpoint completion form moving
* Bulk replay/cancel env fix
* Fix for alert webhook path
* Redirect projects/v3/* to project/*
* Fixes for CLI redirect routes
* Remove welcome email (unused)
* Change how we count schedules towards your limits
* Use new schedules limits when checking a schedule
* Added projectId back in to task queries (indexes)
* WIP dev presence
* CLI modal
* Moved things around and use Context
* Fix for p inside p
* Dev connected status on run page
* Correct dev env (not a teammates)
* Show disconnected message at the end
* Minor tweak on project dropdown icon padding
* Fix for inconsistent date format for presence
* Added a message when pushing to the billing page
* Center the team page
* Project settings page centered
* Improvements to the dev presence
* Delete v2 Stripe routes
* Delete v2 billing/usage pages
* Delete v2 integration pages
* Delete v2 project pages
* Deleted a load of components and services
* Deleted a load more components, presenters and services
* Deleted another 100 files or so…
* Removed old v2 paths
* Removed named icons from form titles
* Removed more string icons
* Delete NamedIcon
* Fixed some type errors
* Delete endpointApi
* Removed v2 from core/sdk
* Post merge fixes
* added explicit return types
* using the new sdk export without v3
* Delete old v2 file
* Added explicit return types because TS was complaining…
* Don’t export RuntimeEnvironmentType from two core files. Was causing TS issue
* Fix for removal of NamedIcon in new route
* Removed strange eslintrc rule
* Use the new redis client
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* bump worker version
* Suggested glossary for the RunEngine, TBC
* Removed BatchTaskRun changes from this branch, they were done in main
* Set the BatchTaskRun status to completed when all runs are completed
* When dequeuing respect passed in maxResources
* Ported over the new run props: idempotencyKeyExpiresAt, versions, oneTimeUseToken, maxDurationInSeconds
* Didn’t hit save… the new props when triggering tasks passed through
* Idempotency expiration + waitpoint edge case
* WIP on creating checkpoint, parking for now
* fix worker routes
* upgrade webapp node types to support generic event emitter
* separate event bus handler singleton and run failure alerts
* duration waits
* fix execution snapshot debug spans
* task waits
* fix event bus types
* temporary fix for react hook run handle type
* disable run notifications for now
* convert any typecasts to expect errors to more easily fix later
* fix webapp types after node types upgrade
* updateEnvConcurrencyLimits across marqs and the runqueue
* Pass proper values into the run engine
* RunQueue settings and removed unused rebalancing workers
* Remove rebalancing prop
* Tidied more things up
* Update/remove queue limits for MARQS and RunQueue
* taskQueue/concurrencyLimit changes ported back into the RunEngine
* Reworked completing waitpoints to improve performance and reduce race conditions
* Improved test robustness
* Down to a single run lock only when a run is totally unblocked and ready to continue
* warm starts, worker notifications, wait fixes
* Fix for Run Engine poll interval env var
* Expect the waitpoint to be completed quickly
* If a run is locked then it’s too late to expire it
* Added VALKEY_ env vars and plugged them into the run engine
* Extracted and updated the guard queue function so it can be used when batching
* Added logging and universal concurrency changes to trigger task v1
* Added notes back in
* Bump @trigger.dev/worker to 3.3.7
* reportInvocationUsage for the runAttemptStarted event
* improve execution snapshot span debug span start times
* Unfriendly IDs
* update lockfile
* Created a shared determineEngineVersion function
* disable unfinished commands
* save new cli config to different location, misc fixes
* add basic engine version check via current deploy
* new run engine will default to node 22 runtime
* block some actions for projects on previous run engine
* fix worker group tests
* fix triggerAndWait test
* one typescript version to rule them all
* redlock type patch
* fix type issues caused by ts-reset
* improve cleanup scripts
* add missing socket.io dep
* fix run notification handler type
* fix worker group test again
* generate prisma client for e2e tests
* remove worker group tests for now
* prevent image pull rate limits during unit tests
* increase timeout for queue concurrency limit test
* generate prisma client for preview release
* same node types everywhere
* Updated engine readme, removed legacy system notes
* use default machine preset from platform package
* worker instances plural in schema
* disable pnpm update notifications
* return worker group details from connect call
* add workers admin route
* fix heartbeat route return type
* move deployment labels to core apps
* refactor run controller env schema
* Add firstAttemptStartedAt to TaskRun
* RunEngine 2.0 batch trigger support (#1581)
* Make it clear when BatchTriggerV2Service is used
* Copy of BatchTriggerV2Service
* WIP batch triggering
* Allow blocking a run with multiple waitpoints at once. Made it atomic
* Removed unused param
* New batch service
* Pass through the parentRunId and resumeParentOnCompletion
* Use the new batch service, and correct trigger task version
* Force V1 engine if using BatchTriggerV2Service, we’ve already done the check at this point
* Removed the $transaction and early exit if nothing changed
* Adedd a simple batch task to the hello world reference catalog
* Fix for batch waits not working
* Added parentRunId in a couple more places
* Removed waitForBatch log
* Added another parentRunId
* Expanded the example to include all the different triggers
* More changes to blocking to support continuing after idempotent completed runs
* Fix for the wrong type when blocking a run
* remove @map
* optimise worker auth query
* add engine version header to core api client requests
* remove unique constraint for default group id
* consolidate migrations
* the first managed worker becomes the global default
* Debug events off by default, added an admin toggle to show them
* worker group name can't be an empty string
* add exec helper to core
* move machine resources to core
* add pre-dequeue callback to determine max resources
* optionally skip dequeue
* bump worker package
* move worker to core
* fix ReadableStream type error
* fix another type issue
* update a few more tsconfigs
* add metadata changes introduced in #1563
* Run Engine 2.0 trigger idempotency (#1613)
* Return isCached from the trigger API endpoint
* Fix for the wrong type when blocking a run
* Render the idempotent run in the inspector
* Event repository for idempotency
* Debug events off by default, added an admin toggle to show them
* triggerAndWait idempotency span
* Some improvements to the reference idempotency task
* Removed the cached tracing from the SDK
* Server-side creating cached span
* Improved idempotency test task
* Create cached task spans in a better way
* Idempotency span support inc batch trigger
* Simplified how the spans are done, using more of the existing code
* Improved the idempotency test task
* Added Waitpoint Batch type, add to TaskRunWaitpoint with order
* Pass batch ids through to the run engine when triggering
* Added batchIndex
* Better batch support in the run engine
* Added settings to batch trigger service, before major overhaul
* Allow the longer run/batch ids in the filters
* Changed how batching works, includes breaking changes in CLI
* Removed batch idempotency because it gets put on the runs instead
* Added `runs` to the batch.retrieve call/API
* Set firstAttemptStartedAt when creating the first attempt
* Do nothing when receiving a BATCH waitpoint
* Some fixes in the new batch trigger service… mostly just passing missing optional params through
* Tweaked the idempotency test task for more situations
* Only block with a batch if it’s a batchTriggerAndWait… 🤦♂️
* Added another case to the idempotency test task: multiple of the same idempotencyKey in a single batch
* Support for the same run multiple times in the same batch
* Small tweaks
* Make sure to complete batches, even if they’re not andWait ones
* Export RunDuplicateIdempotencyKeyError from the run engine
* Latest lockfile
* Trigger with a machine (old run engine)
* RE2, allow setting machine when triggering
* Fix for new glob patterns
* add max run count to dequeue from version route
* add worker instance name env var and header
* queue consumer pre skip callback
* poll for more runs after final execution errors
* fix dequeue search param schema
* add shortcut to debug switch
* expose run engine timeouts as env vars
* make warm start durations configurable
* add optional status to json reply helper
* fix preSkip hook, add debug logs
* BLOCKED_BY_WAITPOINTS -> SUSPENDED
* exit controller when run suspended
* check if already replied before http reply
* run controller will wait for next run after the current one is suspended
* cancel run button shortcut
* minimal event repository environment type
* fix update metadata call
* run suspension and misc fixes wip
* change debug shortcut to shift + D
* Started work on the Dev supervisor
* Formatting
* Fix for bad imports
* Before rebuilding SSE
* Presence updating from the CLI working via SSE
* add worker notification debug logs
* send run:stop when exiting run phase
* skip current snapshot poll on worker notification
* add more logs and route to submit run debug logs
* add worker and runner ids to snapshots
* improve run notification debug logs
* add workload debug log route
* misc run controller fixes and refactor
* prevent parallel execution of critical functions
* update bun to 1.2.1
* WIP with dev dequeuing
* Method to convert friendlyIds to non-friendly, do nothing with actual ids
* Set the engine on BackgroundWorker, lazily upgrade projects to engine V2
* Runs with ttls were getting immediately expired… oops.
* Pass the Waiting for deploy reason through, so we have it on the execution snapshots
* Fixed the logic for getting the right background worker for a run
* Use the correct ID when dequeuing…
* determineEngineVersion is now fully functional
* Rate limiter ignores the dev endpoints
* Retrieving a batch gives you the runIds
* Set a unique version for the RE2 BatchTaskRun
* add provisional changeset
* The start of dev run execution is working
* First dev run working
* Moved the dev run controller closer to what Nick did with the managed one
* export exec output type
* Heartbeat fix: don’t heartbeat if _isHeartbeating == false
* Dev runs get notifications, some dev bug fixes
* Improved logging or dequeuing
* We need to dequeue runs from the latest version too, for triggerAndWait
* Ported Eric’s validateWorkerManifest with nicer errors
* When flattening an idempotency key if part is undefined, return undefined
* Dev logging fixes
* Remove sigterm listener
* Deprecating workers. Don’t specify a BackgroundWorker when dequeuing an environment
* Deleted some old files. Renamed “managed” to “deploy”
* When a build finishes, always copy the build dir (otherwise the first one gets trampled on by the 2nd)
* Dev master queues should work differently
* Deleting old workers
* Added debounce function to core
* Improvement to canceling
* WIP on debounce canceling on socket disconnection
* Added environment data to execution snapshots
* Dev runs that have stalled get “Canceled” with a reason explaining why
* Show CLI messaged when a connection to the platform is lost/restored
* Fix TriggerTask after merge
* Add trigger task v2 max attempts, replace some findUniques
* Port the new queue logic to the run engine
* More fixes post-merge
* We weren’t setting a `retryConfig` up for the tests… it’s now required
* Start the Redis worker inside the Run Engine… 🤦♂️
* Trying to make the testcontainers more reliable
* Added keyPrefix: "engine:”
* Badly placed bracket in trigger task
* Better Redis namespacing
* Fix for expired run not getting removed from the queue
* Don’t create a redis client in the testcontainers, return the redisOptions instead
* Cleanup redis client in the run lock tests
* Fix for the RunQueue not supporting keyPrefix
* Updated more of the RunQueue scripts rebalancing
* Trying to make Redis more robust in the tests…
* Improved test resiliciency more
* Fix for delays (checkpoint check)
* Increase the timeout slightly to fix ttl test
* Added priority support when triggering
* More wip trying to make test containers more reliable
* batchTriggerAndWait test is still failing… some wip to try fix it
* Fixed redis tests now we’re not providing a client
* Separate Redis clients for the run engine worker/queue/runlock
* Made the wait for duration test more resilient
* Added idempotencyKeyExpiresAt to Waitpoints
* Waitpoint timeouts and idempotency expiry
* Use finishWaitpoint, removed extra worker job
* Added waitpoint idempotency tests
* Creating resume tokens is working
* Some improvements to the resume tokens
* Moved resumeTokens to just be wait functions 🥳
* Delete old RuntimeManagers
* Wait for token is working
* Better test for the wait tokens
* Improved the test task some more
* Hide the accessories in the span inspector
* WIP on waitpoint inspector
* WIP on complete waitpoint form
* Span overview panel can be changed based on the entity type
* Improved the waitpoint display
* WIP on completing waitpoint form
* Use the existing CodeBlock for the tip
* Style improvements
* Complete waitpoint
* All waitpoint sidebar variants
* Waits now use a pause icon
* Durations waits use the API to create/block with a waitpoint, not the runtime
* Fix for engine.blockRunWithWaitpoint required org id
* Removed old wait code from the run controllers/task run process
* Form action for skipping a datetime waitpoint
* Move testDockerCheckpoint to a separate core package export (it can’t be bundled on the client)
* Fix for glitchy hourglass animation
* Completed waitpoints display better
* Increase Redis maxRetriesPerRequest to 20 (default)
* Completing and skipping waitpoints is working
* Remove the database prisma dev command, since we need to use create only now. Updated docs
* Added skip timeout, reworked the UI
* Tweaked spacing
* Added payload limit to waitpoint token completion from dashboard
* Test idempotency works on wait.for and wait.until
* Moved the worker-actions to /engine/ from /api/
* Moved dev engine endpoints to /engine/ from /api/
* Separate /engine/ rate limiter
* Added parallel wait prevention, it’s working for duration waits but not well for triggerAndWait yet
* WIP post-merge conflicts
* Set taskEventStore column in the new engine
* Remove duplicate keys
* Post-merge fixes
* Fix for span merge layout
* Use executedAt instead of firstAttemptStartedAt
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Record cold start and execution metrics on attempt executions. Add cold start metrics as span events on attempt spans and display them in the run dashboard
* Add deployed tasks run timeline metrics
* Add Dequeued event to run timeline and cleanup the run timeline code
* Adds variants to storybook
* WIP adding new span styles
* Added offset progress bar animation
* More storybook states
* Adds support for the full vertical span to show the same state
* Adds error state to timelineLine
* Added additional state
* Added more line styling
* Added progress state to dequeued
* Added another state to storybook
* Fixed classname error
* Updated styles for the span timeline points
* Fixes alignment of timeline follow cursor indicator
* Adds help text tooltip to timeline span type titles
* Fixes type error
* Tweaked wording of tooltips
* Fixed type error (check this)
* Moved isAdmin to a higher level
* removed unused admin props
* Removed unused Admin filter
* Fixed border styling
* made the opacity of the timeline states 30% less
* Undo type cast
* Added a diminished style that’s used for spans (grey progress bar)
* Adds new storybook state
* Fixed timeline state
* Removed state if span isn’t the first
* Changed the timestamp span icon
---------
Co-authored-by: James Ritchie <james@trigger.dev>
* Improve the MarQS priority system by moving future messages into the LRE worker and using a priority timestamp offset to define priority in messages
* Add a test to make sure priority offsets don't unfairly favor environments
* requeuing should clear concurrency sets
* Heartbeats should only reschedule existing heartbeat jobs
* Fix type error
* run engine v1: orgs are no longer considered for concurrency
* Add reserve concurrency concept to allow waiting to resume parent tasks to release concurrency at the env level for child tasks to use (or else there is a deadlock). WIP recursive tasks
* child tasks inherit the queue timestamp from their parent tasks to prioritize completing child tasks based on when their parent started
* handle reserve concurrency with recursive deadlocks
* Finish docs update for concurrency
* Some fixes from badge conflict resolution
* WIP priority queues
* Implement MarQS priority queues
* Fix the migrations
* WIP fix for ResumeAttemptService selecting the wrong attempt (which has no error or output)
* Don’t create an attempt if the run is already in a final status
* Don’t get all the columns for the query. Improved the logging.
* Added a log to the batch example
* Filter out the undefined values
* OOM retrying on larger machines
* Create forty-windows-shop.md
* Update forty-windows-shop.md
* Only retry again if the machine is different from the original
* Added isPrismaRetriableError()
* Retry completeBatchTaskRunItem if they fail because of a retriable Prisma error
* Retry using Redis worker
* Handle more retriable errors. Add special condition in for race condition error
* Added Postgres connection_timeout with default 20s
* Added a simple batchTriggerAndWait example
* Move the task run heartbeats to RedisWorker
* Move alerts to redis worker, improving redis worker
* Fix typecheck errors
* Use single threaded tests for redis worker
* Enable/disable the redis workers independently
* Remove preview release from PR checks
* Automatically retry TriggerTaskService when hitting a unique constraint error on idempotency key
* improve batch completion system for run engine v1
* Rename batch stuff to v3 so it's not confusing
* Handle unique constraint error on BatchTaskRunItem creation and allow different limits for batchTrigger and batchTriggerAndWait
* Remove unnecessary disabled org check
* Add support for separate redis servers for caching, rate limiter, marqs, and the pub/sub stuff (they all default to the previously used env vars for redis)
* engine v1 fairness improvements (env based queue selection)
* Add ability to bias env selection by concurrency limit and capacity
* Add some randomization to the list of queues within each evn
* Configure biases using env vars
* Typo
* revert v3 catalog project ref change
* Fix v2 marqs
* Efficiency improvements to SharedQueueConsumer and MarQS for run engine v1
* Reliabily resolve task queue for a run and ack runs where we can't find the queue
* engine v1: stop nacking after reaching the max nack count, ack to remove the message
* Handle division by 0 possibility in distributeQueues
* Show a good error if TRIGGER_ACCESS_TOKEN isn’t a PAT
* Require maxDuration in trigger.config from now on
* Changeset
* Update the trigger.config templates copy, and set the default to 1hr
* Set the maxDuration in the trigger.config files inside the repo
* Fixed the e2e tests by setting the package to 0.0.0-prerelease-20250116195421
* WIP preventing concurrent waits, throw an error
* Added ConcurrentWaitError (not retryable)
* Move preventMultipleWaits out of the RuntimeAPI
* Added preventMultipleWaits to the devRuntimeManager
* Added throwable InternalError. Plus new TASK_DID_CONCURRENT_WAIT code
* Docs link for troubleshooting concurrent waits
* Docs for troubleshooting concurrent waits
* preventMultipleWaits function
* Added TASK_DID_CONCURRENT_WAIT code
* Deal with InternalErrors that skipRetrying
* Added preventMultipleWaits to prod
* WIP langsmith & AI SDK integration
* Add changeset
* Add exporter support to deployed tasks
* Better support for external exporters and group exporters and instrumentations under the telemetry config property
* Missing changes
* Increase the number of active streams from 2 to 5
* WIP parent metadata updates
* Fix noop metadata manager
* Implement run metadata updates from ancestor tasks
* Add changeset
* Add ability to stream into parent and root task runs
* Don't duplicate exporting run metadata types
* Add ability to fetch streams through metadata
* Couple of fixes from CodeRabbit
* Fix metadata tests
* Make sure streams are subscribed in the "background"
* Move the stream subscription stuff to the API client, expose it through `runs.fetchStream`
* Fixed run stream tests
* Fix streaming splits in realtime streams v2
* Add changeset
* Skip all flaky tests 😡
* Improve the way we stream from tasks to the server
* Improve the v1 realtime streams (Redis)
* Turn on the relay realtime stream service
* Improved the relay realtime cleanup
* Fixed consuming realtime runs w/streams after the run is already finished
* Remove some logs
* Update changeset
* Fixed runStream tests
* Realtime streams now powered by electric, and fix the streaming update duplicate issues by converting the electric Shape materialized view into a ReadableStream of changes
* Ensure realtime subscription stops when runs are finished, and add an onComplete handle to use realtime hooks
* Fix tests
Eric Allam