Commit Graph

27 Commits

Author SHA1 Message Date
nicktrn 671dff6d6f add secure flag to zod connection 2024-03-25 18:59:05 +00:00
nicktrn fe38d12661 fix hooks and improve reconnect 2024-03-23 11:17:38 +00:00
nicktrn 6a93aefa55 lifecycle hooks are back 2024-03-22 22:01:48 +00:00
nicktrn 30c74f3c4b remove attempt id env var 2024-03-22 15:38:32 +00:00
nicktrn 9fc62f7f10 switch to run id for container names 2024-03-21 14:56:23 +00:00
nicktrn 001bd97b63 remove init container from create 2024-03-21 11:21:37 +00:00
nicktrn 5017a8a808 don't restart restored containers 2024-03-21 11:20:34 +00:00
nicktrn d5b1bcac58 add missing restore label 2024-03-21 10:34:10 +00:00
nicktrn 0f6ea0b262 replace dapi with taskinfo 2024-03-21 10:07:12 +00:00
nicktrn e0507a9598 pass in coordinator host via volume 2024-03-20 21:56:44 +00:00
nicktrn 27ad6574c3 disable post start lifecycle hook 2024-03-20 19:18:46 +00:00
nicktrn b091347614 append shortcode to restore names 2024-03-20 15:15:51 +00:00
nicktrn f21c94c0ce update restore pull secret name 2024-03-20 15:13:03 +00:00
nicktrn 5ff738a18c checkpointing fixes 2024-03-19 19:21:12 +00:00
nicktrn d39aca8820 add missing run id on create 2024-03-19 15:45:42 +00:00
nicktrn 8f3bbc4c64 remove resource limits for now 2024-03-19 15:28:27 +00:00
nicktrn 561efe7bff rename type assertion 2024-03-19 15:14:41 +00:00
nicktrn b862724e6a fix error type assertion 2024-03-19 15:02:22 +00:00
nicktrn dc11de0714 shorten index container names 2024-03-19 13:54:44 +00:00
nicktrn ed7342bf49 rethrow k8s errors after logging 2024-03-19 13:49:42 +00:00
nicktrn 9945980ab0 update kubernetes provider and fix builds again 2024-03-18 18:45:00 +00:00
nicktrn ce9c5d22fe fix provider build and dev 2024-03-18 16:01:18 +00:00
nicktrn 0871efe103 fix tsconfig paths 2024-03-18 15:34:25 +00:00
nicktrn 20aa1cbe9b v3: checkpoint and restore with sim for local dev (#933)
* zod ipc connection with acks

* pass sender through to ipc handlers

* bits and bobs

* add host networking back in

* disable verbose logs

* restore after tasks and batches.. almost

* restore and resume all the things

* fix for systems without checkpoint support

* Get deployment indexing errors to work with the new ZodIpc stuff

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-03-11 15:20:33 +00:00
nicktrn 9b6f8f9238 v3: provider and coordinator updates (#927)
* add zod socket

* start using zod socket

* use zod socket for shared queue

* use shared provider shell

* update k8s provider task ops

* set otlp endpoint

* update docker actions and fix builds

* remove unused types

* update injected secret key env var name

* complete socket.io types to schemas migration

* update env example to new v3 key var

* fix coordinator build

* set task run label

* ensure attempt id is always set

* increase prod worker timeout

* set otlp endpoint for on runs

* pass otlp env var to prod worker

* auto-remove completed index jobs
2024-03-06 16:01:59 +00:00
Eric Allam b7845685eb v3: no bundle builds for the dev CLI (#923)
* WIP no bundling

* Convert dev CLI to use an unbundled build process to support otel instrumentation

* Fix pnpm lock file

* A couple of fixes to get typechecking to work
2024-03-04 15:36:09 +00:00
nicktrn 52c9d485f8 v3: cli build command, prod runs, checkpoints (#919)
* fix trailing slash in api url

* add maybe platform down error

* shorten config path

* extend protobuf compiler install instructions

* build command and image model

* new image triggers task indexing

* index support for prod builds

* shorten example provider secret

* multi-stage task prod build

* lock prod tasks to node 18 image

* extract shared build and dev command libs

* pull out more shared deps

* add coordinator and providers

* fix core-apps build

* add dev builds for new apps

* fix cwd

* enable corepack

* some build fixes

* enable buildkit for old docker versions

* coordinator image fixes

* update provider containerfile

* build dev images in parallel

* upgrade pgadmin

* fix prod facade build

* prod runs

* fix merge

* don't knock out platform on invalid attempt id

* fix prod facade

* rename to build.ts

* fix prod builds

* set to executing after fetching payload

* make prod worker listen on random port if in use

* remove experimental warnings in dev

* prod resume

* prevent execution after completion

* exit prod worker after completion

* always restart otel collector

* docker checkpoints and prod runtime messaging

* don't retry indexing without chance of success

* make platform checkpoint aware

* deploy with existing hash sets latest worker

* log restore requests

* only try to checkpoint long waits

* tidying up

* lockfile

* fix build

* prod worker merge fixes

* fix prod complete and cancel

* fix lua nil checks

* socket namespace abstraction

* cleanup

* make all build args optional

* add build script

* don't require env vars for dev

* fix schema

* prod merge

* small fix

* bind correct logger

* fix v3 ref catalog entry

* resume prod batch

* pass socket to error and disconnect handlers

* fix non-batch resume

* fix batch resume

* send connection env vars when not in dev

* create worker via socket

* move api client back into v3 cli

* fix lockfile

* fix resume with failures

* marqs replace message

* typecheck prior to build

* don't define api url in prod builds

* support prod retries after resume

* skip typecheck option
2024-03-04 13:10:07 +00:00