b017af4e6d
checkpoint readiness and cancel signals
2024-03-22 08:41:31 +00:00
cbc84e2011
measure basic checkpoint perf
2024-03-21 18:36:12 +00:00
e2b10f42a0
improve wait accuracy
2024-03-21 16:10:22 +00:00
9fc62f7f10
switch to run id for container names
2024-03-21 14:56:23 +00:00
5180aacd4a
atomic post-completion checkpoints
2024-03-21 14:14:46 +00:00
18bd7f4623
exit process after checkpointing
2024-03-20 14:38:21 +00:00
8fda3d7b36
structured logs for socket connections
2024-03-20 13:17:41 +00:00
c734db3951
log options on error
2024-03-20 10:05:03 +00:00
8ca21c1dd4
delete checkpoint archive after successful push
2024-03-20 10:00:15 +00:00
fdcb06a173
ensure valid registry login
2024-03-20 09:54:54 +00:00
6c0f8436ce
update coordinator image
2024-03-19 22:24:32 +00:00
a320f17bc4
Merge branch 'main' into v3/infra-updates
2024-03-19 19:22:27 +00:00
5ff738a18c
checkpointing fixes
2024-03-19 19:21:12 +00:00
dc5eb68a0b
Fix typecheck errors because of resolveJsonModule
2024-03-19 15:33:09 +00:00
9945980ab0
update kubernetes provider and fix builds again
2024-03-18 18:45:00 +00:00
1f190fe680
v3: fix dependency resume ( #946 )
...
* fix basic resume mechanics
* fix dependency resume
* fix deploy page sort
* remove task list console log
* checkpoint restore between attempts
* add checkpoint requirement checks to docker provider
* switch to raw execa for long statements
* catch and report pre-connection index errors
* graceful worker shutdown after completion
* fix final completion
* use same task process for each attempt
* update retry catalog entry
* add immediate return task
---------
Co-authored-by: Eric Allam <eric@trigger.dev >
2024-03-18 12:21:40 +00:00
478ce006cb
v3: task run status and canceling runs ( #941 )
...
* WIP task run status, revamped resuming task dependencies
* Don’t select a span when toggling collapsing
* Cancel runs and attempts, in prod and dev
2024-03-13 15:51:43 +00:00
20aa1cbe9b
v3: checkpoint and restore with sim for local dev ( #933 )
...
* zod ipc connection with acks
* pass sender through to ipc handlers
* bits and bobs
* add host networking back in
* disable verbose logs
* restore after tasks and batches.. almost
* restore and resume all the things
* fix for systems without checkpoint support
* Get deployment indexing errors to work with the new ZodIpc stuff
---------
Co-authored-by: Eric Allam <eallam@icloud.com >
2024-03-11 15:20:33 +00:00
83304f86f8
Capture uncaugh exceptions during indexing and fail deploys, with error output ( #934 )
2024-03-10 20:57:46 +00:00
c385db63be
v3: cli deploy command ( #931 )
...
* WIP proxy/deploy
* WIP, registry proxy in express working
* A couple of notes, preparing for indexing
* Move the changes to prod-worker into the new file
* Deploy command working with indexing and runs (docker provider only for now)
* Removed ts-expect-error directive
* Fixed build command
2024-03-08 10:49:42 +00:00
9b6f8f9238
v3: provider and coordinator updates ( #927 )
...
* add zod socket
* start using zod socket
* use zod socket for shared queue
* use shared provider shell
* update k8s provider task ops
* set otlp endpoint
* update docker actions and fix builds
* remove unused types
* update injected secret key env var name
* complete socket.io types to schemas migration
* update env example to new v3 key var
* fix coordinator build
* set task run label
* ensure attempt id is always set
* increase prod worker timeout
* set otlp endpoint for on runs
* pass otlp env var to prod worker
* auto-remove completed index jobs
2024-03-06 16:01:59 +00:00
52c9d485f8
v3: cli build command, prod runs, checkpoints ( #919 )
...
* fix trailing slash in api url
* add maybe platform down error
* shorten config path
* extend protobuf compiler install instructions
* build command and image model
* new image triggers task indexing
* index support for prod builds
* shorten example provider secret
* multi-stage task prod build
* lock prod tasks to node 18 image
* extract shared build and dev command libs
* pull out more shared deps
* add coordinator and providers
* fix core-apps build
* add dev builds for new apps
* fix cwd
* enable corepack
* some build fixes
* enable buildkit for old docker versions
* coordinator image fixes
* update provider containerfile
* build dev images in parallel
* upgrade pgadmin
* fix prod facade build
* prod runs
* fix merge
* don't knock out platform on invalid attempt id
* fix prod facade
* rename to build.ts
* fix prod builds
* set to executing after fetching payload
* make prod worker listen on random port if in use
* remove experimental warnings in dev
* prod resume
* prevent execution after completion
* exit prod worker after completion
* always restart otel collector
* docker checkpoints and prod runtime messaging
* don't retry indexing without chance of success
* make platform checkpoint aware
* deploy with existing hash sets latest worker
* log restore requests
* only try to checkpoint long waits
* tidying up
* lockfile
* fix build
* prod worker merge fixes
* fix prod complete and cancel
* fix lua nil checks
* socket namespace abstraction
* cleanup
* make all build args optional
* add build script
* don't require env vars for dev
* fix schema
* prod merge
* small fix
* bind correct logger
* fix v3 ref catalog entry
* resume prod batch
* pass socket to error and disconnect handlers
* fix non-batch resume
* fix batch resume
* send connection env vars when not in dev
* create worker via socket
* move api client back into v3 cli
* fix lockfile
* fix resume with failures
* marqs replace message
* typecheck prior to build
* don't define api url in prod builds
* support prod retries after resume
* skip typecheck option
2024-03-04 13:10:07 +00:00
f5ea64ef4f
Renamed the coordinator app to wss (web socket server)
2023-01-05 16:30:02 +00:00
87c781ca16
Clients can now reconnect if they lose access to the coordinator server
2023-01-05 16:19:03 +00:00
03f6c32051
Fixed some issues with disconnected runs and started the renaming process
2023-01-05 11:07:45 +00:00
f00e3c8633
The host now publishes the START_WORKFLOW_RUN message so there isn’t a race condition
2023-01-05 10:39:46 +00:00
edb1d9c95f
Added new INTERRUPTION step type to be able to display it in the UI
2023-01-04 17:23:09 +00:00
17683d2377
Workflow runs can now be interrupted and retried
2023-01-04 15:26:15 +00:00
22e02bcef4
Cleanup TriggerServer instances when the connection is lost
2023-01-04 14:25:35 +00:00
48bacb0e60
Make workflow run steps idempotent
2023-01-04 13:39:28 +00:00
d660c581f4
Add support for durable delays (either durations or scheduled times)
2023-01-02 16:17:28 +00:00
1fcc27b990
Resolve is the better name here again
2023-01-02 14:14:22 +00:00
ae63b49251
Initial work on the delay functionality
2023-01-02 13:41:11 +00:00
8967d5e216
“Resolve” is a better word for that message, and postMessage response schema is always success
2023-01-02 11:50:15 +00:00
d5e695dcfa
integration requests respond with just output
2022-12-30 19:03:14 +00:00
8b7dafcf2d
WIP sending/receiving requests
2022-12-29 15:29:49 +00:00
b9bf7033a7
Big overhaul to unify how events works across the different types
...
Workflow runs are still TODO but are pretty much ready to go
2022-12-23 16:34:42 +00:00
dd6937c84f
Deleted the coordinator specific readme
2022-12-22 12:39:36 +00:00
67612a8e0c
Registering webhooks with existing connections (and github only)
2022-12-21 16:56:32 +00:00
234f6a00bb
Add support for firing custom events in workflow runs
2022-12-20 14:47:37 +00:00
15e6af4f36
Added support for log messages
2022-12-20 13:44:22 +00:00
d457085f7b
Workflow runs, with a workflow step for output
2022-12-20 11:14:03 +00:00
6bf2ac423c
Got a trigger firing end-to-end
2022-12-18 21:58:40 +00:00
a0bb1a235c
Moves pulsar back into the docker compose file
...
Pulsar now starts/stops when running docker:db
2022-12-16 16:21:20 +00:00
36090aea80
Triggering workflows from custom events
2022-12-16 14:28:49 +00:00
d8c22b61ef
Workflows are now being registered in the webapp, along with their trigger
2022-12-15 16:41:08 +00:00
afc35caea6
Restructured internal-messages into internal-platform, to be used in the webapp
2022-12-14 15:58:42 +00:00
210f2e5ea3
Coordinator now publishing the Initialize Workflow message and it’s being consumed in the webapp
2022-12-14 14:40:20 +00:00
c1e7e80bf9
Reworked how the coordinator publishes and subscribes to messages in Pulsar
2022-12-14 14:21:16 +00:00
b8b558ef88
2 readme updates from when i ran the project for the first time
2022-12-14 13:30:41 +00:00
nicktrn