* Refill release concurrency tokens when a run is cancelled
* Improved release concurrency accounting system + a sweeper to auto-refill tokens for snapshots that are no longer the latest snapshot on a run (e.g. the run has moved to a new snapshot state)
* Fix order of arguments to the releasings sweeper
* Add a heartbeat for SUSPENDED snapshots, where when stalled will attempt to continue the run if unblocked
* v4: fix race condition when continuing run when blocked at the same time
* Attempt to fix flaky e2e test
* Make waitpoint race condition test less flaky
* WIP clickhouse package with test containers setup
* More clickhouse client setup now with otel and real tests, and the v1 of raw run events
* Add some additional columns to raw_run_events_v1
* WIP runs dashboard service
* Create a new run engine event bus event for the runs dashboard to hook into
* Track run events in the run engine
* make sure engine v1 runs get synced to CH
* Update the attemptNumber of v3 task runs
* Restructure the run events to be more sparse
* emit more stuff
* Setup replication package
* scaffold the replication package
* replication wip
* resolve conflicts
* more replication stuff
* Add ability to drop the replication slot completely on teardown
* Use the new single replacingmergetree task events table for replication
* get it working
* insert payloads into their own table only on insert and then join
* prepare for using clickhouse cloud and now running ch migrations during boot in the entrypoint.sh
* Handover WIP and tests
* Testing the replication service
* Remove the runs dashboard stuff that we aren't using anymore
* Added a test for large payloads
* hacky typecheck fix
* Fix new internal package typecheck issues and start adding telemetry to the replication service
* tracing over spans, some other improvements
* Improvements to the runs replication service, now ready for testing
* Some fixes and cleanups
* Don't need this code anymore
* move transaction types into the runs replication service
* only send spans where there are transaction events
* A couple of suggested tweaks
* remove dead code
* rename managed to shared runtime manager
* rename to resolve waitpoint for clarity
* add resolver id helper
* store and correctly resolve waipoints that come in early
* fix ipc message type change
* branded type for resolver ids
* add fixme comments
* remove more unused ipc schemas
* fix entitlement validation when client doesn't exist
* restore hello world reference workspace imports
* runtime manager debug logs
* prefix engine run logs
* managed run logger accepts nested props
* runtime suspendable state and improved logs
* require suspendable state for checkpoints, fix snapshot processing queue
* add terminal link as cli module so we can more easily patch it
* apply cursor patch
* add license info
* remove terminal-link package and add deprecation notice
* remove old patch
* remove terminal-link from sdk
* rename snapshot module
* add cli test tsconfig
* add run logger base type
* add snapshot manager tests
* fix cli builds
* improve QUEUED_EXECUTING test
* changeset
* make testcontainers wait until container has stopped
* require unit tests for publishing again
* avoid mutation during iteration when resolving pending waitpoints
* improve debug logs and make them less noisy
* always update poller snapshot id for accurate logs
* detach task run process handlers
* check for env overrides in a few more places and add verbose logs
* log when poller is still executing when we stop it
* add supervisor to publish workflow
* always print full deploy logs in CI
* Revert "avoid mutation during iteration when resolving pending waitpoints"
This reverts commit 87b0ce1e5b.
* disable pre
* print prerelease script errors
* Revert "disable pre"
This reverts commit 9403409637.
* misc fixes
* better debug logs
* add snapshots since methods and route
* prep for snapshots since
* improve deprecated execution detection
* update supervisor and schema
* properly log http server errors
* detect restore after failed snapshot fetch
* run and snapshot id can be overridden
* fix restore detection
* fix deprecation checks, move into snapshot manager
* less logs
* rename snapshot manager stop
* restore detection was moved into snapshot manager
* fix notifier logs
* make runtime manager status a debug log
* no need to attach runtime status twice
* findUnique -> findFirst
* sort snapshots by created at everywhere
* update testcontainers packages
* increase cleanup timeout and add better logs
* small tweaks
* decrease docker network size so we can have more of them
* add a test flow to check this all works
* await all engine.quit calls
* add docker diagnostics
* reduce number of test runs
* improve network attachment output
* add setup logs
* log redis setup errors
* add cleanup log type
* stop redis container if setup fails
* disable ipv6
* tidy things up a bit
* fix unit tests workflow
* decrease container cleanup timeout
* fix types
* fix webapp typecheck
* make testcontainers wait until container has stopped
* require unit tests for publishing again
* add failing test case
* make it pass
* add retry threshold ms env var and use it
* Fix priority tests by dequeuing in sequence
* Expect this test to fail until we support parallel dequeuing of an env with streaming
* Fix one and expect fail from the other
* Fix for batch trigger test dequeuing
* Fixed pending version test
* Locked task runs will now require queues and tasks to be in the locked version
* Client errors caught in a run function now will skip retrying
* Extracted out the trigger queues logic
* extract validation, idempotency keys, payloads to concerns
* Extracted out a bunch of more stuff and getting trigger tests to work
* Add queue and locked version tests
* Deadlock detection WIP
* more deadlock detection
* Only detect deadlocks when the parent run is waiting on the child run
* Improve the error experience around deadlocks
* A couple tweaks to make CodeRabbit happy and fixing the tests in CI
* Fixed failing test
* Changeset
* wip
* Make sure to scope queries to the runtime env
* Added error throwing tests to runlock
* Added some more run lock tests
* Added customConditions so the test can use the Logger
* Fix for JSON error
* Added a span to the run lock with a name
* Increase v4 visibility timeouts
* Added some logging for #continueRunIfUnblocked: run has no checkpoint
* If a run is finished or pending cancel, don’t try and continue it
* Completed batch waitpoints when we completed the BatchTaskRun
* Try complete the batch faster now it’s being used operationally
* Fix for tests that were using the old engine.unblockRunForCreatedBatch() function
* Delete project
- Don’t schedule tasks if the project is deleted
- Delete queues from the master queues
- Add the old delete project UI back in
* Mark the project as deleted last
* Fix for overriding local variable
* Added a todo for deleting env queues
* Remove todo
* update nypm to support text-based bun lockfiles
* fix retry spans
* only download debug logs if admin
* add nypm changeset
* pull out env override logic
* use runner env gather helper
* handle dev flushing failures gracefully
* fix path normalization for init.ts
* add logger
* add execution heartbeat service
* add snapshot poller service
* fix poller
* add changesets
* create socket in constructor
* enable strictPropertyInitialization
* deprecate dequeue from version
* start is not async
* dependency injection in prep for tests
* add warm start count to all controller logs
* add restore count
* pull out run execution logic
* temp disable pre
* add a controller log when starting an execution
* refactor execution and squash some bugs
* cleanup completed docker containers by default
* execution fixes and logging improvements
* don't throw afet abort cleanup
* poller should use private interval
* rename heartbeat service file
* rename HeartbeatService to IntervalService
* restore old heartbeat service but deprecate it
* use the new interval service everywhere
* Revert "temp disable pre"
This reverts commit e03f4179de6a731c17253b68a6e00bcb7ac1736b.
* add changeset
* replace all run engine find uniques with find first
* run engine v1 will only lock to v1 deployments
* run engine v2 will only lock to managed v2 deployments
* test: create background worker and deployment with correct engine version
* don't hardcode max old space size in deployment images
* flags: treat underscores as hyphens
* append attempt number to runner name if >1
* improve retry spans for oom errors
* run engine: add ability to selectively disable release concurrency queue consumers via env var
Also added some additional logging
* actually pass the disableConsumers option through to the release concurrency system
* Fixed CLI e2e failing tests
* WIP
* Completing waitpoints using public access tokens
* fixed example
* adding agent-loops reference, adding useWaitToken react hook, adding a way to throw an error to output from the run, adding an icon option to logger.trace
* Fixed metadata system
* Check if the waitpoint is COMPLETED already in the complete endpoint and return true
* Added waitpoints/tokens to the sidebar
* Added indexes to the Waitpoint time for filtering
* Begun work on `WaitpointTokenListPresenter`, the pag is a copy of the Queues page for now
* MVP of waitpoint token page
* Added status
* Expiry of timeout/ttl
* Improvements to the waitpoint table
* Improved columns and icon
* Changes from the RunTag copy on hover branch
* Fix for nested button error
* Added waitpoint tags to the DB/table
* Applied Eric’s task run tag fix (it’s live on prod in the legacy run engine branch)
* Added tags to waitpoints
* Removed todos that have been done
* Added token support for releaseConcurrency. Also added a ton of JSDocs
* Added releaseConcurrency to the API token endpoint…
* WIP on waitpoint page filters
* Fix for tags filtering
* Waitpoint filters working
* Fix for badly named function
* WaitpointPresenter used from SpanPresenter
* Waitpoint detail panel WIP
* Fix for server client hydration issue with CodeBlock
* Selected waitpoint panel
* Added a blank state
* Added waitpoint docs link
* Fix for animated number going past the target
* Fix for the queue list pagination and upgrade status
* Engine version error for waitpoint token list
* RunTag component doesn’t get squished and hover behaviour is nicer
* Associating runs with waitpoints
* Added triggered icon
* Link directly to the waitpoint
* Fix for TS error on waitpoint retrieve
* Added CopyableText component, used for waitpoint id in the table
* Removed the confetti 🎊
* Deleted some old images
* Moved some schemas/types to core. Use `id` instead of `friendlyId`
* Added wait.listTokens() function. Made some changes to the types to make it nicer
* WIP wait.retrieveToken()
* wait.retrieveToken working
* Added data to retrieve token
* Separate ApiWaitpointPresenter completely
* Added completed time to the waitpoint detail panel
* Fix for the Avatar component having SSR issues. Specify the size in rems and removed the useLayoutEffect
* Fix for applied idempotency key filter dropdown showing the id field
* Use parentheses to make sure the token list query respects idempotency key correctly
* Use the proper logger, and have a decent message with info to track the bug down
* Pass the org title into the Avatar
* Better error when failing to creating a manual waitpoint after X attempts
* Improve the new run engine Trigger/Batch trigger service class names
* centralize queue timestamp logic in EnqueueSystem, adding queueTimestamp support and propagation
* Fixing the create tags in the new run engine trigger task service
Eric Allam