Commit Graph

75 Commits

Author SHA1 Message Date
github-actions[bot] d462b7a51b chore: Update version for release (beta) (#1001)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-05 16:45:38 +01:00
nicktrn f2894c177a switch to busybox and only call post start hook for restore (#1005) 2024-04-05 15:47:58 +01:00
Matt Aitken e35f29764a We want retrying enabled in dev by default, it’s confusing otherwise (#1004) 2024-04-05 14:45:19 +01:00
nicktrn 1207efbbad v3: prod image upgrade and fixes (#1003)
🚢 Publish Infra Images / build (coordinator) (push) Has been cancelled
🚢 Publish Infra Images / build (kubernetes-provider) (push) Has been cancelled
* fix shutdown after final attempt

* fix workdir permissions

* correctly handle self-hosted deploy errors

* switch to node:20-bookworm-slim

* add binaries catalog file

* changeset

* skip recommended packages

* post start hook retries

* comment and improve lifecycle retry
2024-04-05 13:09:07 +01:00
nicktrn 7ea8532cce v3: crash observability (#1002)
* indexing resource limits

* refactor and storage limits

* handle and display worker crashes

* enable prod cancellation

* fix reconnect delay

* improve crash messages

* log and display crash events

* changeset
2024-04-05 13:00:20 +01:00
Eric Allam 6dcfeadaca v3: pnpm support (#1000)
* WIP fixing pnpm nested install strat

* Add changeset

* Pass (and log) cwd during background worker initialization

* Log out the env

* manually add the .pnpm/node_modules to NODE_PATH if in a pnpm repo

* Remove supports color dep
2024-04-04 16:13:04 +01:00
github-actions[bot] dc5ed0a0cd chore: Update version for release (beta) (#995)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-02 13:24:32 +01:00
Eric Allam e7e7397ed9 Force CLI change 2024-04-02 13:20:26 +01:00
github-actions[bot] 9a3eb289cd chore: Update version for release (beta) (#992)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-02 11:12:56 +01:00
github-actions[bot] ae27fd83af chore: Update version for release (beta) (#987)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-04-01 10:53:45 +01:00
Eric Allam c702d6a9ca v3: better task metadata errors (#991)
* v3: better handle task metadata parse errors, and display nicely formatted errors (dev, deploy, UI)

* Add changeset
2024-03-31 21:47:05 +01:00
Eric Allam 8c4df326cc v3: Better dev/deploy errors and handle image build errors (#988)
* Better error output when an ESM only package is required

* Add better errors for npm package errors, and bring deploy errors to the dev CLI

* Handle depot build errors and write out a temporary file to view the build logs

* Add changeset
2024-03-29 12:46:11 +00:00
Eric Allam 8694e573f5 Fix CLI logout and add list-profiles command 2024-03-28 22:18:15 +00:00
Eric Allam b271742dca v3: implement configurable log levels via config file and env var (#985)
* v3: implement configurable log levels via config file and TRIGGER_LOG_LEVEL

Also, test runs automatically set the TRIGGER_LOG_LEVEL to debug

* Fix type error and changeset

* Added a Node.js runtime check for the CLI dev command
2024-03-28 22:00:38 +00:00
github-actions[bot] 67aaffb6fc chore: Update version for release (beta) (#984)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-28 19:54:21 +00:00
Eric Allam e3cf456c69 v3: fix string and non-standard outputs (#983)
* Fixed string outputs missing and incorrectly formatted

* Handle non-serializable outputs like a function

* Add changeset
2024-03-28 19:51:29 +00:00
github-actions[bot] a3d809740d chore: Update version for release (beta) (#979)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-28 16:28:41 +00:00
Eric Allam f93eae300e v3: superjson dynamic import and deploy fixes (#982)
* v3: Dynamically import superjson and fix some bundling issues

* Added changeset

* Better error handling in the registry proxy and catch uncaught exceptions and unhandled promise rejections instead of crashing the server

* Await the prettyPrintPackage
2024-03-28 16:22:00 +00:00
nicktrn b66d5525ef v3: machine config (#978)
* add and use machine config

* assign tasks to worker nodes only

* add secure flag to zod connection

* changeset

* add pre stop hook

* don't use secure connection by default

* pass more identifiers to provider and apply labels
2024-03-28 14:21:49 +00:00
Eric Allam 719c0a0b94 v3: better clock management for spans/logs in CRIU envs (#980)
* durable clock WIP

* Fixed incorrect span timings around checkpoints by implementing a precise wall clock that resets after restores

* Add changeset
2024-03-28 14:20:44 +00:00
Matt Aitken d9c9e80bc4 Changed "Worker" to "Version" in the CLI dev command key 2024-03-28 12:13:58 +00:00
Matt Aitken 9bcb8cb42a Ignore the DEBUG env var when deploying 2024-03-28 11:28:34 +00:00
github-actions[bot] a22b5869e4 chore: Update version for release (beta) (#977)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-27 19:43:19 +00:00
github-actions[bot] 89c0e68fb5 chore: Update version for release (beta) (#975)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-27 18:54:28 +00:00
Eric Allam 5b745dc1a4 Improved the dev CLI output 2024-03-27 18:36:52 +00:00
Eric Allam a8263dec86 Include core-apps in trigger.dev typecheck tsconfig file 2024-03-27 15:01:02 +00:00
Eric Allam 055d468b38 Setup socket.io redis adapter and fix deployment builds for superjson error 2024-03-27 14:42:10 +00:00
Matt Aitken 3a35d60091 Change the npm tag from "latest" to "beta" in docs/cli/onboarding (#974)
* The v3 init command defaults to the “@v3” tag

* Change it to beta

* Updated the docs and in-app docs

* Updated GitHub action
2024-03-27 13:39:53 +00:00
Eric Allam 215b60c742 v3: superjson output support (#971)
* superjson output support

* Better support for superjson in the task events, limiting output attributes

* Offload large outputs to object store (r2)

* Finishing up the offloading of large outputs/payloads to an object store
2024-03-27 11:57:22 +00:00
Eric Allam 395abe1b92 v3: Various fixes to support papermark use-case (#970)
* Adding a changeset for v3

* Add a version field to @trigger.dev/core-apps package.json

* Build trigger.dev when doing a prerelease

* bundle @trigger.dev/core-apps with trigger.dev cli

* Fix the init command config template

* Don’t use * for the @trigger.dev/core dep version specifier

* strip workspace: from the package version before installing it

* Added dependenciesToBundle config option to bundle ESM only packages

* Added logging around resolving dependency paths

* Try again

* Resolve dependencies based on the project dir first

* flip the bundled default

* Adding some logs around dev task completion notifications

* Adding some additional logs

* Add more logs

* Write out the log using process.stdout

* Store pending completion notifications and resume them when awaited (fixes race condition)

* Cleanup some of the logs

* Copy over the postinstall step from the projects package.json

* Add support for including additional files when deploying (e.g. prisma schema)

* Don’t run scripts when resolving deps

* copy all the files just in case anything is needed in postinstall

* Remove duplicate option

* Use the tag when outputting the dev command

* Remove the postinstall script

* add the trigger dir to the config if the default is not chosen

* Remove the “hud” display in the dev command

* trigger file names with dashes now work

* Better file watching in dev

* Much better duplicate ID experience now

* Much better “Project not found” error

* Export the handleError function types from sdk

* Add support for configuring instrumentation

* Upgrade and unify @opentelemetry/* packages (and remove storybook from the webapp)

* Fix typescript error in react package

* Upgrade react types in webapp

* Allow span icons to be determined based on the span name (e.g. prisma:)

* Ignore built-in env vars when checking for env vars, and allow continuing the deployment even if missing env vars were detected

* Improve the retry.fetch default behavior and option structure

* Fixed typescript errors with packages/email react types

* Update the retry.fetch docs
2024-03-25 17:02:49 +00:00
nicktrn 9ecf07731a v3: small features and fixes for prod infra (#968)
* add checkpoint restore events

* fix retries.enabledInDev

* fix tsconfig paths

* fix provider build and dev

* update kubernetes provider and fix builds again

* upgrade prod base to node 20

* update infra publish workflow

* rethrow k8s errors after logging

* shorten index container names

* fix error type assertion

* rename type assertion

* remove resource limits for now

* add missing run id on create

* add push to deploy command for self-hosting

* checkpointing fixes

* update coordinator image

* ensure valid registry login

* delete checkpoint archive after successful push

* log options on error

* structured logs for socket connections

* fix structured log merge

* exit process after checkpointing

* update restore pull secret name

* append shortcode to restore names

* log handler payload

* disable post start lifecycle hook

* pass in coordinator host via volume

* replace dapi with taskinfo

* add missing restore label

* don't restart restored containers

* remove init container from create

* atomic post-completion checkpoints

* switch to run id for container names

* improve wait accuracy

* measure basic checkpoint perf

* always log disconnect reason

* use system clock to end wait spans

* checkpoint readiness and cancel signals

* restore from checkpoint events and fix statuses

* remove attempt id env var

* restore dependencies from events

* reconnect wip

* lifecycle hooks are back

* fix hooks and improve reconnect

* make docker send postStart hook

* only checkpoint for retry if large delay

* fix a few more resume issues

* lifecycle hook fixes

* skip connection handler when waiting for post start hook

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-25 16:28:56 +00:00
Eric Allam 7b3ffe040b v3: config file is now ts by default and uses named exports (#962)
* A bunch of changes to the config file:

- No more default export, the config is exported via the `config` named export
- `handleError` is moved to a separate named export
- Added the ability to include additional packages (e.g. wrangler) that aren’t included by default (think CLI usage)
- More reliably install packages in dev
- The config is now built first instead of directly imported, meaning we can now support typescript config files (`trigger.config.ts`)

* Update to trigger.config.ts in `trigger.dev init`
2024-03-21 10:40:44 +00:00
Eric Allam fc4b95d69e A bunch of deployment fixes (#960)
- Time out deployments to prevent never-ending building/deploying
- Use the CURRENT worker deployment when determining which version to execute
- Fix some of the deployment error handling logic
2024-03-20 13:15:48 +00:00
Eric Allam 0e347b001b Don’t override TRIGGER_SECRET_KEY and TRIGGER_API_URL with local env vars 2024-03-19 16:04:31 +00:00
Eric Allam 41e1ac9e0b v3: nanosecond precision task events and immediate mode for dev events (#956)
* Store startTime in nanoseconds since epoch, ensure logs using nanosecond precision for their timestamp (so they are not out of order)

* export task events immediately in dev
2024-03-19 15:06:47 +00:00
Eric Allam 86c24afb44 Support json traces and logs in the otel endpoints (#954) 2024-03-19 10:59:50 +00:00
Eric Allam e55ca0279b Adding profile support to the v3 CLI (#953) 2024-03-18 17:35:29 +00:00
Eric Allam 67cac97b26 Add DEPLOY_REGISTRY_HOST to allow configuring the deploy registry via the server 2024-03-18 15:56:28 +00:00
Eric Allam 0f32dee0eb Prevent depot command from displaying update message 2024-03-18 15:32:26 +00:00
Eric Allam 49e9dd8cb2 Disable fetch experimental warning and detect process resources (#950) 2024-03-18 13:46:24 +00:00
nicktrn 1f190fe680 v3: fix dependency resume (#946)
* fix basic resume mechanics

* fix dependency resume

* fix deploy page sort

* remove task list console log

* checkpoint restore between attempts

* add checkpoint requirement checks to docker provider

* switch to raw execa for long statements

* catch and report pre-connection index errors

* graceful worker shutdown after completion

* fix final completion

* use same task process for each attempt

* update retry catalog entry

* add immediate return task

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-03-18 12:21:40 +00:00
Eric Allam 52c364d691 v3: cli init command + CLI telemetry (#948)
* Adding telemetry to the deploy command

* Add ability to opt-out of telemetry in the CLI

* Remove log

* Unifying some telemetry stuff and adding it to the login command

* init command

* Remove extra trigger dirs

* Couple tweaks to the init CLI

* Don’t use npm init -y to create the package.json file
2024-03-18 10:12:11 +00:00
Eric Allam f424c81086 No staging by default, and project external refs now are prefixed with proj_ 2024-03-14 15:11:31 +00:00
Eric Allam 0a33bf7206 v3: handle errors and customizing retrying (#943)
* Add the ability to handle errors at the task and project level

* Cancel in progress runs when disconnecting the dev CLI (in between attempts)

* unify the task executor across dev & prod
2024-03-14 14:38:13 +00:00
Eric Allam 478ce006cb v3: task run status and canceling runs (#941)
* WIP task run status, revamped resuming task dependencies

* Don’t select a span when toggling collapsing

* Cancel runs and attempts, in prod and dev
2024-03-13 15:51:43 +00:00
Eric Allam fe9435f63f Allow alternate config file names and skipping using a config file by passing —project-ref (#940) 2024-03-11 21:31:15 +00:00
Eric Allam d7cc2c9d6c v3: task payload refactor and configurable default retries (#939)
* Make the payload parameter the first argument to the task functions

* Allow configuring default retry behavior, and disabling in dev
2024-03-11 20:30:29 +00:00
nicktrn 20aa1cbe9b v3: checkpoint and restore with sim for local dev (#933)
* zod ipc connection with acks

* pass sender through to ipc handlers

* bits and bobs

* add host networking back in

* disable verbose logs

* restore after tasks and batches.. almost

* restore and resume all the things

* fix for systems without checkpoint support

* Get deployment indexing errors to work with the new ZodIpc stuff

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-03-11 15:20:33 +00:00
Eric Allam 883feab257 v3: various deploy improvements (#936)
* Cache the dependency resolution step of deploys

* Better deploy output and fixed an issue with deleting env vars
2024-03-11 14:31:55 +00:00
Eric Allam 83304f86f8 Capture uncaugh exceptions during indexing and fail deploys, with error output (#934) 2024-03-10 20:57:46 +00:00