Commit Graph

266 Commits

Author SHA1 Message Date
github-actions[bot] 97533cba2e chore: Update version for release (#1442)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-11-01 21:32:04 +00:00
nicktrn fa18e6c012 Increase maximum retry count for deploy run controller (#1450)
* increase max retries for all deploy run controller operations

* add changeset
2024-11-01 11:23:11 +00:00
nicktrn 4194bce4a3 Forked worker processes now inherit PATH from parent (#1448)
* set PATH when forking

* add changeset
2024-11-01 11:23:00 +00:00
github-actions[bot] 53841d7bf0 chore: Update version for release (#1423)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-29 12:24:21 +00:00
nicktrn d2c779eb0f Fixes for internal error reattempts (#1436)
* magic links on span event errors

* prevent task monitor from processing errors handled elsewhere

* exclusively use internal error code enum for completion data

* add complete attempt service opts

* reattempts need to go via the queue for task controllers that may have exited

* only infer retry config if completed via crash or system failure

* enhance error before deciding if retriable

* retry on SIGTERM

* enable retry config helper for latest sdk

* don't retry heartbeat timeouts for now

* enable task monitor to update fatal errors

* add missing service

* update retry config since package version

* don't alter completion time when updating existing error
2024-10-28 19:15:47 +00:00
nicktrn 212f8539c3 Automatically reattempt after internal errors (#1424)
* refactor finalize run service

* refactor complete attempt service

* remove separate graceful exit handling

* refactor task status helpers

* clearly separate statuses in prisma schema

* all non-final statuses should be failable

* new import payload error code

* store default retry config if none set on task

* failed run service now respects retries

* fix merged task retry config indexing

* some errors should never be retried

* finalize run service takes care of acks now

* execution payload helper now with single object arg

* internal error code enum export

* unify failed and crashed run retries

* Prevent uncaught socket ack exceptions (#1415)

* catch all the remaining socket acks that could possibly throw

* wrap the remaining handlers in try catch

* New onboarding question (#1404)

* Updated “Twitter” to be “X (Twitter)”

* added Textarea to storybook

* Updated textarea styling to match input field

* WIP adding new text field to org creation page

* Added description to field

* Submit feedback to Plain when an org signs up

* Formatting improvement

* type improvement

* removed userId

* Moved submitting to Plain into its own file

* Change orgName with name

* use sendToPlain function for the help & feedback email form

* use name not orgName

* import cleanup

* Downgrading plan form uses sendToPlain

* Get the userId from requireUser only

* Added whitespace-pre-wrap to the message property on the run page

* use requireUserId

* Removed old Plain submit code

* Added a new Context page for the docs (#1416)

* Added a new context page with task context properties

* Removed code comments

* Added more crosslinks

* Fix updating many environment variables at once (#1413)

* Move code example to the side menu

* New docs example for creating a HN email summary

* doc: add instructions to create new reference project and run it locally (#1417)

* doc: add instructions to create new reference project and run it locally

* doc: Add instruction for running tunnel

* minor language improvement

* Fix several restore and resume bugs (#1418)

* try to correct resume messages with missing checkpoint

* prevent creating checkpoints for outdated task waits

* prevent creating checkpoints for outdated batch waits

* use heartbeats to check for and clean up any leftover containers

* lint

* improve exec logging

* improve resume attempt logs

* fix for resuming parents of canceled child runs

* separate SIGTERM from maybe OOM errors

* pretty errors can have magic dashboard links

* prevent uncancellable checkpoints

* simplify task run error code enum export

* grab the last, not the first child run

* Revert "prevent creating checkpoints for outdated batch waits"

This reverts commit f2b5c2ac42.

* Revert "grab the last, not the first child run"

This reverts commit 89ec5c8bfd.

* Revert "prevent creating checkpoints for outdated task waits"

This reverts commit 11066b4e74.

* more logs for resume message handling

* add magic error link comment

* add changeset

* chore: Update version for release (#1410)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Release 3.0.13

* capture ffmpeg oom errors

* respect maxAttempts=1 when failing before first attempt creation

* request worker exit on fatal errors

* fix error code merge

* add new error code to should retry

* pretty segfault errors

* pretty internal errors for attempt spans

* decrease oom false positives

* fix timeline event color for failed runs

* auto-retry packet import and export

* add sdk version check and complete event while completing attempt

* all internal errors become crashes by default

* use pretty error helpers exclusively

* error to debug log

* zodfetch fixes

* rename import payload to task input error

* fix true non-zero exit error display

* fix retry config parsing

* correctly mark crashes as crashed

* add changeset

* remove non-zero exit comment

* pretend we don't support default default retry configs yet

---------

Co-authored-by: James Ritchie <james@trigger.dev>
Co-authored-by: shubham yadav <126192924+yadavshubham01@users.noreply.github.com>
Co-authored-by: Tarun Pratap Singh <101409098+Wackyator@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-24 11:12:24 +01:00
Eric Allam c31700ae5f Fix issue with lost built trigger.dev files after prisma generate (#1429)
* Fix issue with lost built trigger.dev files after prisma generate on deploy

* Add changeset
2024-10-24 10:43:26 +01:00
Eric Allam 7f9091f205 fix metadata system not updating because of dual package hazard (#1428) 2024-10-23 14:02:13 +01:00
Eric Allam 2d8a41b18b feat: realtime (#1402)
* Denormalize run tags, increase character limit to 128

* WIP realtime subscribing to runs

* extracted the stream stuff into core, made it more reusable

* WIP tags

* Remove tags for now because it’s not support in electric

* Support async iterables, readable stream, and callback style subscription styles

* Remove tags streaming endpoint

* Add realtime rate limits and scope them to the /realtime path

* WIP rate limt per org

* Introduce per org rate limits

* WIP JWT auth

* Move migrations into new internal db package

* Resolve pnpm lock file

* Authenticating to the realtime API with JWTs are working

* realtime in the client

* Created react-hooks package and starting to move stuff in there

* Improve types for hooks

* schema tasks

* Added useBatch hook

* build uploadthing/fal demo and change how run metadata is synced to the server

* tweaks

* WIL realtime concurrency tracking

* Implement test for realtime client using testcontainers

also updated electric to latest version

* Allow customizing the expiration time of the automatic JWT created after triggering a task

* Add support for subscribing to run tags

* Improve auth types and API

* finalize the realtime API

* Fixed some example stuff

* Allow up to 10 run tags

* Remove core from docker-provider tsconfig paths to prevent it from being typechecked

* do the same for the kubernetes provider

* Fixing some typecheck errors

* Fix webapp type errors

* Update @trigger.dev/platform to 1.0.13

* Fix attw error

* Remove from/to in subscribeToRuns query params

* Add tests for the rate limit middleware and add custom JWT rate limits

* turn off webapp test parallelism

* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger

* Add changeset

* Attempt to fix unit tests in CI

* Skip running the auth rate limit middleware tests for now

* Try a beefier machine

* Try and run webapp tests separately

* Setup env vars

* Make sliding window test more reliabile
2024-10-21 15:07:08 +01:00
github-actions[bot] f1d89c413b chore: Update version for release (#1410)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-18 17:24:53 +01:00
nicktrn feb4fcdac6 Fix bun runtime by downgrading to 1.1.24 (#1409)
* pin bun to 1.1.24 and use slim image

* add changeset

* add warning about bun version to docs
2024-10-14 18:00:10 +01:00
Dan a6de3f0ff3 README updates (#1408)
* Main README edits

* SDK readme edits

* CLI readme edits

* Create cool-keys-sit.md

---------

Co-authored-by: Eric Allam <eric@trigger.dev>
2024-10-14 16:39:19 +01:00
github-actions[bot] 8a70ce4cff chore: Update version for release (#1394)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-09 16:15:58 +01:00
nicktrn 4f85119dfd Misc fixes related to error display (#1397)
* Fix execute error handling

* replace internal errors on attempts

* correctly display env var errors while indexing

* display correct errors on crashed attempts

* add changeset
2024-10-09 15:10:48 +01:00
nicktrn 94170dec3b Fix dev command args 2024-10-09 13:33:19 +01:00
nicktrn 215c8ed6b2 Remove unused path arg from the CLI dev command (#1395)
* remove path arg from dev cli command

* add changeset
2024-10-09 12:49:35 +01:00
nicktrn 4b0f677774 Prettier errors and related improvements (#1387)
* improve get attempt payload logging

* only log checkpoint readiness timeouts, don't crash the run

* improve coordinator errors

* add links export to core

* remove unused deploy error utilities

* update cli errors and use new links from core

* log remaining failure reasons

* also parse execute errors in deployed worker

* add signal to unexpected exit messages

* start displaying prettier internal errors

* improve unexpected exit error parsing

* better errors for crashes

* fix unexpected error parsing for dev runs

* changeset

* parse graceful exit timeouts
2024-10-09 12:12:47 +01:00
github-actions[bot] 13ebfcc5b9 chore: Update version for release (#1381)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-08 13:24:38 +01:00
nicktrn 002ae4bc0d Fix dotenv overrides for dev runs (#1388)
* override dashboard dev env vars with local .env

* add changeset

* add simple task for testing env vars
2024-10-08 09:22:29 +01:00
nicktrn a69e04f03a Include push output in logs for self-hosted deploys (#1382)
* include push output in logs

* changeset
2024-10-05 14:18:58 +01:00
nicktrn c5488df8ca Fix CLI downgrade check (#1380)
* fix downgrade detection

* remove unused semver package from webapp

* add changeset
2024-10-05 13:12:47 +01:00
Eric Allam 1caec27f72 docs: Max duration (#1379)
* maxDuration docs

* Update the init command to set the maxDuration and include a commented out maxDuration in the config file
2024-10-04 15:33:35 -07:00
github-actions[bot] 8e61f5d631 chore: Update version for release (#1378)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-04 14:59:07 -07:00
Eric Allam 6d0884254e feat: Add maxDuration to tasks (#1377)
* WIP

* Get max duration working on deployed runs

* Actually set the timed out runs to status = TIMED_OUT

* The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED

* New TimedOutIcon

* Added new timedout icon

* Add ability to opt-out of maxDuration with timeout.None

* MAX_DURATION_EXCEEDED -> TIMED_OUT

* changeset

* Improved styling for the status tooltip content

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2024-10-03 12:43:26 -07:00
github-actions[bot] a6ea8444c9 chore: Update version for release (#1366)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-29 19:17:26 -07:00
github-actions[bot] 926459893b chore: Update version for release (#1351)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-26 14:33:30 +01:00
Eric Allam d361e24bee Associate child runs with the parent span ID (#1352)
* Add safe rootTaskRunId index and a README to @trigger.dev/database

* Associate child runs with the span ID of the span in the parent run that triggered the child run

* Update deprecation notice doc links
2024-09-24 15:47:44 +01:00
github-actions[bot] af7194d078 chore: Update version for release (#1341)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-24 10:55:34 +01:00
Eric Allam d7a65f3e7a Fix resolving external packages that are ESM only (#1346)
* Fix resolving external packages that are ESM only by falling back to mlly resolvePathSync. This will fix mupdf

* when publishing a prerelease and aborting, clear the git stage
2024-09-23 16:21:04 +01:00
Harsh Shrikant Bhat 9ce3f790f9 fix: Added dev.vars into dotEnv file (#1340)
* Added dev.vars into dotEnv file

* Added changes

* Update yellow-knives-attack.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-09-21 09:12:48 +01:00
github-actions[bot] e47b6cf166 chore: Update version for release (#1328)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-20 16:24:45 +01:00
Eric Allam ba3c5bdf33 Adding missing task run hierarchy to TaskRun table (#1332)
* Add task run hierarchical relationships to the database

* Add depth and related runs to the retrieve run API response

* Remove prisma optimize

* restructure the migrations to create the index concurrently

* Delete these tsbuildinfo files

* Fix type error by adding depth to the run list presenter

* Cleanup the task hierarchy, share more code

* Remove some fields from the list run response
2024-09-20 11:09:00 +01:00
Niels 64862db846 Update dotEnv.ts to ignore OTEL_EXPORTER_OTLP_ENDPOINT as well (#1307)
* Update dotEnv.ts

* Create four-buttons-run.md

---------

Co-authored-by: Eric Allam <eallam@icloud.com>
2024-09-19 15:25:17 +01:00
Eric Allam b4be736555 prismaExtension fixes for #1325 and #1327 2024-09-19 15:21:03 +01:00
github-actions[bot] a538fa879f chore: Update version for release (#1322)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-18 22:07:50 +01:00
Eric Allam 3b15224453 Fix default machine preset in config not being used (#1321)
* Add additional error message and stack trace when a task file cannot be imported for run

* Apply default machine preset in config
2024-09-18 22:00:59 +01:00
github-actions[bot] 0ac7f4b24b chore: Update version for release (#1318)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-18 17:28:59 +01:00
nicktrn 8d1e41693a v3: self-hosting support for latest packages (#1319)
* display --profile flag after init if used

* add containerfile debug logs

* log all indexing errors in case of connection issues

* specify dockerfile syntax version

* add network flag for self-hosting

* move all self-hosting tags to latest

* add self-hosting update section

* manual setup subsection

* registry flag implies push

* add changeset
2024-09-18 17:07:47 +01:00
github-actions[bot] 33c108b739 chore: Update version for release (#1316)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-18 11:22:16 +01:00
Eric Allam 3d53d4c086 Improve the update CLI command and fix missing tsconfig.json error (#1315)
* Fixes for CLI update command, and make the hide the "whoami" command output when running in dev

* Fix an issue where a missing tsconfig.json file would throw an error on dev/deploy

* Don’t show latest CLI warning when using a prerelease

* Only print CLI update required message when update is embedded

* Strip out TRIGGER\_ keys when using syncEnvVars, to prevent deploy errors
2024-09-18 11:07:09 +01:00
github-actions[bot] b6b74046b9 chore: Update version for release (#1310)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-17 10:44:51 +01:00
Eric Allam 2b5771f38c Remove duplicate bin definition, fixes issue #1311 2024-09-16 21:40:01 +01:00
Eric Allam de135e4885 Fix for TASK_RUN_HEARTBEAT errors in deployed and dev works 2024-09-16 19:10:19 +01:00
github-actions[bot] 499fd86cf2 chore: Update version for release (#1309)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-16 18:11:20 +01:00
Eric Allam 3aa5811790 fix 3.0.0 update warning (#1308)
* Attempt to fix false package mismatch warnings

* Add changeset

* Add ability to test update checks in prerelease packages

* Resolve the trigger.dev package based on the package.json dir

* Try this

* Don’t use the version module, just resolve the packageJson

* One more dirname

* Comment

* Remove the version export because we aren’t using it anymore
2024-09-16 18:03:16 +01:00
github-actions[bot] 45287cd655 chore: Update version for release (#1269)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-16 15:37:20 +01:00
Eric Allam cf13fbdf32 Updating docs for the move from beta -> latest (with new build system) (#1299)
* Add triggerAndWait().unwrap() to more easily get at the output or throw the subtask error

* Add taskId and runId to SubtaskUnwrapError

* WIP docs update beta -> latest

* trigger.dev init now adds @trigger.dev/build to devDependencies

* How it works doc

* Restructure some docs and update the cli commands

* Config file docs, plus aptGet and ffmpeg extensions

* Update to latest from beta docs

* Add runtime to templates

* Add --runtime option to the init CLI command

* A bunch more doc updates after feedback

* Document triggerAndWait with unwrap and result types

* beta -> latest in the webapp

* CLI update check no longer references beta

* Add major release

* Leave changeset beta, back to normal package release

* Fixed default dirs option in init command

* exclude windows-yarn variation of cli e2e tests because it’s buggy

* Remove cache to try and fix yarn e2e test workflow errors
2024-09-16 15:30:32 +01:00
Eric Allam ca2cb7298c Another attempt at fixing the yarn CI failure 2024-09-16 15:13:19 +01:00
Eric Allam a50063ce04 Always insert the dirs option when initializing a new project in the trigger.config.ts 2024-09-12 14:12:16 +01:00
Eric Allam 8578c9b281 v3: new build system fixes round 2 (#1283)
* Fixed empty env vars overriding in dev runs

* Don’t import package.json anymore

* fix node10 moduleResolution in @trigger.dev/core

* Support self-hosters pushing to a custom registry when running deploy

* dev: Fixed stuck runs when a child run fails with a process exit

* Make some doc notes about known issues and docker hub private repos

* Fix --project-ref when running deploy

* Fix —config option when deploying

* Fixing the flushing/killing process with the new build system

* Add monorepo-react-email e2e test fixture

* Fix issue with emitDecoratorMetadata and tsconfigs with extends

* Got the emit decorator metadata fixture working

* Fixed typechecking yarn e2e CLI tests in monorepos

* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)

* Remote externals now powered by JSON Hero to be easier to update

* resolve config source files

* Add a —javascript option to init, defaults to typescript

* Add support for prisma typed sql

* Remove msw and retry.interceptFetch

* Add missing code to the openai retries example

* Don’t generate the v3 catalog prisma client during CI

* Fixed v3-catalog task imports

* Remove interceptor usage in task file

* Only import import-in-the-middle hook if there are instrumented packages

* Fix yarn.lock file
2024-09-11 15:48:20 +01:00