Commit Graph

159 Commits

Author SHA1 Message Date
github-actions[bot] 3aca603a33 chore: Update version for release (beta) (#1153)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-12 17:34:47 +01:00
nicktrn 01633c9c03 Print error logs on dev worker failure 2024-06-12 13:08:50 +01:00
nicktrn b2ba403dd3 Improve error logs cli message 2024-06-12 12:04:59 +01:00
nicktrn 68d32429b6 v3: checkpoint failover and misc fixes (#1157)
* configurable checkpoint registry namespace

* add missing task create await

* remove unused messages

* changeset

* update self-hosting docs

* capture and display stderr for failed deploys

* add missing lockfile changes

* stderr changeset

* fix cli stderr message

* update error logs label
2024-06-11 14:06:50 +01:00
Eric Allam ca94f0cac3 cli v3: fix deploys by using sub-path imports (fixes execa require ESM error) 2024-06-11 11:08:15 +01:00
nicktrn 35dbaedf69 v3: self-hosting (#1147)
* add amin email regex env var

* fix displayed init command for self-hosted setups

* shared env var to disable telemetry in cli and webapp

* pin sdk version during init

* if specified, add api url to dev command shown after init

* improve checkpoint support detection

* control forced checkpoint simulation via env var

* add public init to providers

* better checkpoint support check for coordinator

* add docker to coordinator image

* update docker provider containerfile

* bump remaining containers to node 20

* add infra image build to default publish workflow

* lockfile

* remove concurrency group from infra workflow

* add docker provider to build matrix

* fix var subst

* checkpoint test is docker specific

* enable v3 projects by default on self-hosted instances

* fix v3 setup command again

* add default posthog key

* self-hosting docs

* add latest tags to versioned infra and webapp builds

* some checkpoint errors should skip retrying

* add changeset

* shorten paragraph

* some docs updates

* update tunnelling section

* add registry setup section

* use correct cli push flag

* add checkout to v3 branch

* update the worker machine setup steps

* fix infra build

* small docs update

* remove unused feature function

* Revert "remove unused feature function"

This reverts commit cfe07887a12b6893dca8ce499964481a9b3dc9db.

* fix self-hosted v3 feature gate

* add note about missing arm support

* simplify helper script syntax
2024-06-10 14:13:04 +01:00
Eric Allam c11a77f50b cli v3: increase otel force flush timeout to 30s from 500ms 2024-06-07 20:46:03 +01:00
github-actions[bot] df7d1de16d chore: Update version for release (beta) (#1142)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-06-06 15:59:21 +01:00
nicktrn 8e5ef176a4 v3: registry maintenance (#1146)
* retry checkpoints with backoff and optional failover registry for deploys

* never abort checkpoint cleanup

* simulate checkpoint failure for 5 minutes

* add flag to simulate checkpoint push failure

* add flag to control push failure simulation duration

* backoff with helper

* handle all coordinator errors

* improve stop retrying

* increase cleanup ipc timeout

* improve webapp socket.io handler error logging

* remove unused backoff function
2024-06-05 11:06:18 +01:00
Eric Allam ba71f959e2 v3: Runs List Management SDK and SDK improvements (#1133)
* Improved the existing runs API

* WIP next runs API

* Improve the returned ApiPromise to add ability to return response

* More WIP

* WI{

* Added offset/limit pagination stuff like the cursor one, and converted all API methods to use ApiPromise

* More run API stuff

- Adding schedule  output from the retrieveRun endpoint
- Ability to filter by schedule and isTest

* Remove env from retrieve run in openAPI

* prefer duplication over merge

* WIP docs

* Use spread to DRY up some run API schemas

* Finish the overview docs

* Adding changeset

* Fixed typecheck errors

* Typo fix

* Re-export zodfetch from core so the v3 CLI can use it

* Fixed type errors

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-31 11:02:28 +01:00
github-actions[bot] ca9e827bd3 chore: Update version for release (beta) (#1130)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-30 12:15:10 +01:00
Matt Aitken 98ef170299 Set the CLI deploy timeout to 3mins from 1min 2024-05-30 12:08:14 +01:00
Eric Allam e69ffd314a v3: Refactor attempt creation to be worker requested (#1077)
* WIP worker TaskRunAttempt creation

* Handling failing task runs that cannot create an attempt for whatever reason

* Move the visibility queue stuff into a graphile job

* Fixed task runs with unsanitized queue names

* “Borrow” the code from alerts PR to get self hosted deployments working

* Add an admin API endpoint to get info about the shared marqs queue

* Allow admins to view any project metrics

* start adding lazy attempts to prod

* lazy attempt creation for prod workers

* resurrect prod stack traces

* add exception event to failed run spans

* simplify dependency resumes

* fix typecheck

* fix merge

* fresh process for all attempts

* always try sigterm first

* stop heartbeat timeout on non-inplace replace message

* add missing ack on checkpoint creation service failure

* bypass dequeue for retries with running worker

* respect retry delays

* crash runs with invalid run status for execution

* remove debug logs

* fix nack message

* fix version locking

* fresh attempt processes in dev and prod

* improve handling of ipc timeouts

* consider checkpoint failures on cancellation

* add basic chaos monkey to checkpointer

* changeset

* control forced checkpoint simulation via env var

* fix merge

* kill old attempt processes before checkpointing

* detailed perf logging for checkpointing

* add coordinator otlp endpoint example

* improve prod run cancellation

* rename supports lazy attempts migration

* fix graceful exit

* fix retry mechanics

* clear paused state before retry

* remove checkpoint image after push

* crash worker on unrecoverable errors

* refactor unrecoverable error emit

* switch to do hosted busybox image

* increase wait for duration ipc timeout

* add changeset for misc fixes

* fix merge

* fix retry delay span runId

* fix dev retries

* improve prod worker logging

* log checkpoint sizes

* add lazy attempts catalog entries

* Fixed merge issue: use zodFetch, not wrapZodFetch

* Revert "Fixed merge issue: use zodFetch, not wrapZodFetch"

This reverts commit d137e4e1fe.

* importEnvVars uses wrapZodFetch now

* add backwards compat for retries without checkpoints

* handle more cases of unrecoverable runs

* don't kill the child process if it shouldn't be killed

---------

Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-30 11:05:52 +01:00
github-actions[bot] ea4d15536c chore: Update version for release (beta) (#1115)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-24 14:55:09 +01:00
Eric Allam 3a1b0c486a v3: env var management API (#1116)
* WIP env var management API

* Add import env var API endpoint

* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints

* WIP envvar SDK

* Uploading env vars in a variety of formats now works

* Finish env var endpoints and add resolveEnvVars hook

* Add changeset
2024-05-23 16:11:25 +01:00
Eric Allam c092c0f9db v3: Prevent legacy-peer-deps=true from breaking deploys 2024-05-22 15:46:13 +01:00
Alexandre Costa a86f36cefa Fix TypeScript inclusion in tsconfig.json for cli-v3 init (#1105)
* Fix TypeScript inclusion in tsconfig.json for cli-v3 init

Fixed an issue where TypeScript files were included in the project directory when no include directive was present in tsconfig.json. Previously, the CLI added trigger.config.ts to the inclusion list by default, causing TypeScript compilation errors for other files. The fix ensures that trigger.config.ts is only added to the inclusion list if there's an existing include directive present in tsconfig.json

* Create hot-fishes-retire.md

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-05-21 14:40:49 +01:00
Eric Allam 5a6e79e0c0 Fix missing logs when client includes their own @opentelemetry/api package (#1106) 2024-05-21 14:38:06 +01:00
github-actions[bot] a7ca1222d7 chore: Update version for release (beta) (#1104)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-16 19:00:06 +01:00
Matt Aitken 598906fc4c Fix for typo in v3 CLI login command 2024-05-16 09:23:45 +01:00
github-actions[bot] 72748603ab chore: Update version for release (beta) (#1102)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-15 16:21:58 +01:00
Eric Allam 21c8b847e5 Better handle resolving dependencies when there are conflicts 2024-05-15 13:26:03 +01:00
github-actions[bot] 107f2e7bd5 chore: Update version for release (beta) (#1099)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-13 16:14:14 -04:00
Eric Allam b8477ea2b0 Fixes an issue with scoped packages in additionalPackages option 2024-05-13 15:31:19 -04:00
github-actions[bot] 2e21bc1563 chore: Update version for release (beta) (#1094)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-13 17:38:50 +01:00
Eric Allam 0e919f56f2 Alerts v1 (#1065)
* Alerts v1

* Encrypt alert webhook secrets and allow them to be generated by the server

* Alert v1 UI

* Remove unnecessary emails

* Move to using `@react-email/components`

* WIP slack alerts

* More slack alerts WIP

* Update pnpm lock after rebase

* Finish implementing Slack alerts

* Use a more error like emoji

* New secondary variant for the segmented control

* Added a simple checkbox style variant to storybook

* Style tweak to the segmented control

* UI improvements to the alert modal

* Use searchable Select for alerts. Changed default variant for SegmentedControl

* Secondary button now using secondary colour

* segmented control style tweak

* Improved the channel column in the alerts table

* Updated logo-mono.png

* Updated email styles

* Email templates updated to new styles

* Don’t log the decrypted secret

* await enqueing the deployment alert when an index fails

* await enqueing the timeout alert

---------

Co-authored-by: James Ritchie <james@jamesritchie.co.uk>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-05-09 12:12:00 +01:00
github-actions[bot] 223768a025 chore: Update version for release (beta) (#1091)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-08 17:25:22 +01:00
github-actions[bot] 56a063e586 chore: Update version for release (beta) (#1087)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-06 12:34:43 +01:00
Eric Allam 6406924b02 Ensure @trigger.dev/sdk and @trigger.dev/core are always in the list of deployed dependencies 2024-05-06 11:01:49 +01:00
Eric Allam 2156e15266 Adding some additional telemetry during deploy to help debug issues 2024-05-06 10:34:54 +01:00
github-actions[bot] 2e4f5d893c chore: Update version for release (beta) (#1086)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-05 08:19:13 +01:00
Eric Allam 1670c4c41d Remove "log" Log Level, unify log and info messages under the "info" log level 2024-05-03 14:55:24 +01:00
github-actions[bot] 0ee39e6101 chore: Update version for release (beta) (#1085)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-03 09:39:34 +01:00
Eric Allam e667028d4a Strip out server-only package from worker builds 2024-05-02 21:28:41 +01:00
github-actions[bot] c8538c236c chore: Update version for release (beta) (#1080)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-02 15:58:42 +01:00
Eric Allam c37c822314 Use locked package versions when resolving dependencies in deployed workers 2024-05-01 15:51:39 +01:00
Eric Allam e337b21650 Add a postInstall option to allow running scripts after dependencies have been installed in deployed images 2024-05-01 14:16:52 +01:00
github-actions[bot] 9feb0f70b0 chore: Update version for release (beta) (#1079)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-05-01 11:32:15 +01:00
nicktrn 25de9e59af run workflows 2024-05-01 10:57:09 +01:00
nicktrn 62700245a3 v3: fix consecutive waits (#1073)
* fix spacing for delete hints

* don't try to resume deps on wait resume

* sending duration wait resumes is not an error anymore

* set correct status with new wait resume flow

* cancel checkpoint schema v2

* don't mix messages and schemas

* prevent unintended case fallthrough in tree view

* completely switch to platform-led duration wait resumes

* prevent infinite restores

* some entries for the catalog

* add pg to additional packages

* add checkpoint safe timeout

* prevent duplicate spans after restore

* wait for post start

* add sdk version to deploy tab

* fail on impossible checkpoint scenarios

* remove debug logs
2024-05-01 10:33:30 +01:00
github-actions[bot] a42037da03 chore: Update version for release (beta) (#1070)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-29 14:48:01 +01:00
Eric Allam 43bc7ed94e Hoist uncaughtException handler to the top of workers to better report error messages 2024-04-29 14:22:55 +01:00
github-actions[bot] 469c8a2532 chore: Update version for release (beta) (#1067)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-26 17:13:20 +01:00
Eric Allam ebeb790522 Add typescript as a dependency so the esbuild-decorator will work even when running in npx 2024-04-26 17:00:14 +01:00
github-actions[bot] 38770a91c7 chore: Update version for release (beta) (#1061)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-26 14:35:30 +01:00
Eric Allam 9491a1649c v3: Support decorators (#1060)
* Adds support for `emitDecoratorMetadata: true` and `experimentalDecorators: true` in your tsconfig

* Implement task.onSuccess/onFailure and config.onSuccess/onFailure

* Added onStart and more docs for lifecycle functions

* Use onStart instead of init for TypeORM
2024-04-25 11:17:13 +01:00
github-actions[bot] 503882762c chore: Update version for release (beta) (#1059)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-24 15:06:25 +01:00
Eric Allam de1cc868e3 Fix dev CLI output when not printing update messages 2024-04-24 14:06:49 +01:00
github-actions[bot] d7911892ee chore: Update version for release (beta) (#1058)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-24 12:48:00 +01:00
Eric Allam f888f85a09 Make the update trigger packages not print anything in dev 2024-04-24 12:16:49 +01:00