 nicktrn
|
14c2bdf89b
|
v3: checkpoint and reliability improvements (#1198)
* only checkpoint retries with delays greater than threshold
* rename checkpoint threshold env var
* log task monitor ignores
* crash runs with unbounded attempts
* fix retry check in shared queue consumer
* add missing stop for env var sync spinner
* prod entry point refactor
* missing awaits
* more verbose prod flush and exit logs
* reduce checkpoint support logs
* heartbeat while checkpointing between retries
* dynamic coordinator config
* measure lazy attempt creation time in prod
* simplify delay threshold
* heartbeat clarifications
* crash run if it doesn't reach checkpointable state
* require dynamic config threshold
* fix retry prep, await previous worker kill
* unify wait mechanics
* fix prod worker without tasks error
* ensure worker is ready to be checkpointed for dependency waits
* improve worker attempt creation logging
* prevent crashes caused by failed socket schema parsing
* fix dynamic imports in v3 catalog
* clarify attempt retry mechanics
* move backoff helper to core-apps
* remove core-apps barrel file
* add backoff execute with callback
* deprecate non-lazy attempt messages
* update socket.io-client to v4.7.5
* fix socket.io types for emits with timeout
* retry all the things
* remove todo
* fix retry restores
* improve index failure logs
* retry incomplete dependency waits
* fix checkpoint in-progress detection
* prevent losing messages during reconnect
* checkpoint when greater or equal to threshold
* improve handling of duration wait edge cases
* add ready for lazy attempt replay
* retry attempt completion
* allow failing runs with unfriendly run id
* fix min max jitter
* cancel checkpoints on run failure
* improve attempt creation errors
* prevent crashing run on failed cleanup
* handle at-least-once execute lazy attempt delivery
* log exit code on prepare for retry
* fix timeout promise
* mark some things
* chaos monkey superpowers
* refactor checkpointer
* set chaos monkey defaults
* less chaos
* fix backoff
* handle uncaught entry point exceptions
* only replay rpcs on true reconnects
* allow resume unless final run status
* add changeset
* small fixes
|
2024-07-03 16:54:09 +01:00 |
|