Commit Graph

12 Commits

Author SHA1 Message Date
Matt Aitken d214aa9093 Don't create an attempt if the run is final, batchTriggerAndWait bad continue fix (#1698)
* WIP fix for ResumeAttemptService selecting the wrong attempt (which has no error or output)

* Don’t create an attempt if the run is already in a final status

* Don’t get all the columns for the query. Improved the logging.

* Added a log to the batch example

* Filter out the undefined values
2025-02-12 14:02:29 +00:00
Matt Aitken efd2d21762 Detect ffmpeg OOM errors, added manual OutOfMemoryError (#1694)
* Detect ffmpeg OOM errors, added manual OutOfMemoryError

* Create eighty-spies-knock.md
2025-02-12 00:02:01 +00:00
Matt Aitken bb65b2614a OOM retrying on larger machines (#1691)
* OOM retrying on larger machines

* Create forty-windows-shop.md

* Update forty-windows-shop.md

* Only retry again if the machine is different from the original
2025-02-10 17:43:59 +00:00
Matt Aitken c49af774ba Retry batch item completion (#1675)
* Added isPrismaRetriableError()

* Retry completeBatchTaskRunItem if they fail because of a retriable Prisma error

* Retry using Redis worker

* Handle more retriable errors. Add special condition in for race condition error

* Added Postgres connection_timeout with default 20s

* Added a simple batchTriggerAndWait example
2025-02-06 19:16:30 +00:00
Matt Aitken dd321e3616 Throw an error for concurrent waits (with nice docs link) (#1618)
* WIP preventing concurrent waits, throw an error

* Added ConcurrentWaitError (not retryable)

* Move preventMultipleWaits out of the RuntimeAPI

* Added preventMultipleWaits to the devRuntimeManager

* Added throwable InternalError. Plus new TASK_DID_CONCURRENT_WAIT code

* Docs link for troubleshooting concurrent waits

* Docs for troubleshooting concurrent waits

* preventMultipleWaits function

* Added TASK_DID_CONCURRENT_WAIT code

* Deal with InternalErrors that skipRetrying

* Added preventMultipleWaits to prod
2025-01-16 14:33:14 +00:00
Matt Aitken f0996686e9 Added a db index to make the schedule list page faster
There was a slow query to get the latest run for a list of schedules
2024-11-06 11:56:28 +00:00
Eric Allam 2d8a41b18b feat: realtime (#1402)
* Denormalize run tags, increase character limit to 128

* WIP realtime subscribing to runs

* extracted the stream stuff into core, made it more reusable

* WIP tags

* Remove tags for now because it’s not support in electric

* Support async iterables, readable stream, and callback style subscription styles

* Remove tags streaming endpoint

* Add realtime rate limits and scope them to the /realtime path

* WIP rate limt per org

* Introduce per org rate limits

* WIP JWT auth

* Move migrations into new internal db package

* Resolve pnpm lock file

* Authenticating to the realtime API with JWTs are working

* realtime in the client

* Created react-hooks package and starting to move stuff in there

* Improve types for hooks

* schema tasks

* Added useBatch hook

* build uploadthing/fal demo and change how run metadata is synced to the server

* tweaks

* WIL realtime concurrency tracking

* Implement test for realtime client using testcontainers

also updated electric to latest version

* Allow customizing the expiration time of the automatic JWT created after triggering a task

* Add support for subscribing to run tags

* Improve auth types and API

* finalize the realtime API

* Fixed some example stuff

* Allow up to 10 run tags

* Remove core from docker-provider tsconfig paths to prevent it from being typechecked

* do the same for the kubernetes provider

* Fixing some typecheck errors

* Fix webapp type errors

* Update @trigger.dev/platform to 1.0.13

* Fix attw error

* Remove from/to in subscribeToRuns query params

* Add tests for the rate limit middleware and add custom JWT rate limits

* turn off webapp test parallelism

* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger

* Add changeset

* Attempt to fix unit tests in CI

* Skip running the auth rate limit middleware tests for now

* Try a beefier machine

* Try and run webapp tests separately

* Setup env vars

* Make sliding window test more reliabile
2024-10-21 15:07:08 +01:00
Eric Allam 6d0884254e feat: Add maxDuration to tasks (#1377)
* WIP

* Get max duration working on deployed runs

* Actually set the timed out runs to status = TIMED_OUT

* The client status for TIMED_OUT is now MAX_DURATION_EXCEEDED

* New TimedOutIcon

* Added new timedout icon

* Add ability to opt-out of maxDuration with timeout.None

* MAX_DURATION_EXCEEDED -> TIMED_OUT

* changeset

* Improved styling for the status tooltip content

---------

Co-authored-by: James Ritchie <james@trigger.dev>
2024-10-03 12:43:26 -07:00
Matt Aitken 0bf500fa95 Prioritize finishing waited runs (#1375)
* If a tree node is missing, estimate the size as zero

* Task to test prioritizing finishing existing runs after triggerAndWaits

* When requeuing a run with a checkpoint, put it in the queue with the parent run time so it’s correctly prioritized

* The same change but if there’s no checkpoint
2024-10-02 15:30:16 -07:00
Eric Allam e81380314c Add retrieve background worker API endpoint 2024-09-23 14:23:40 +01:00
Eric Allam 8578c9b281 v3: new build system fixes round 2 (#1283)
* Fixed empty env vars overriding in dev runs

* Don’t import package.json anymore

* fix node10 moduleResolution in @trigger.dev/core

* Support self-hosters pushing to a custom registry when running deploy

* dev: Fixed stuck runs when a child run fails with a process exit

* Make some doc notes about known issues and docker hub private repos

* Fix --project-ref when running deploy

* Fix —config option when deploying

* Fixing the flushing/killing process with the new build system

* Add monorepo-react-email e2e test fixture

* Fix issue with emitDecoratorMetadata and tsconfigs with extends

* Got the emit decorator metadata fixture working

* Fixed typechecking yarn e2e CLI tests in monorepos

* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)

* Remote externals now powered by JSON Hero to be easier to update

* resolve config source files

* Add a —javascript option to init, defaults to typescript

* Add support for prisma typed sql

* Remove msw and retry.interceptFetch

* Add missing code to the openai retries example

* Don’t generate the v3 catalog prisma client during CI

* Fixed v3-catalog task imports

* Remove interceptor usage in task file

* Only import import-in-the-middle hook if there are instrumented packages

* Fix yarn.lock file
2024-09-11 15:48:20 +01:00
Eric Allam f040417440 v3: new build system fixes (#1278)
* Support custom config file names & paths

* Fix entry point paths on windows

* Support custom conditions

Add support for custom conditions (for bundling and running), to support being able to import `ai/rsc` with the “react-server” condition.

- Fixed an issue where symlinking unresolvable externals after rebuilding caused the build to hang
- Fixed an issue with external not working with subpath exports (e.g. “ai” would not match “ai/rsc”)
- Protect better against build extensions breaking builds

* Add changeset

* Fix passing CLI process.env down to the task processes

* Remove unused import

* reviving the e2e CLI tests

* Another attempt at fixing windows

* yet another windows attempt (yawa)

* Output index child stdout and stderr (yawa)

* normalize import paths for windows

* Added some logging (yawa)

* normalize the loader path as well

* Added some logging to figure out why the entry points aren’t being found on windows

* Fix for entry point detection on windows

* Normalize runner import loader path

* Normalize import paths in dev and make sure rewritten build manifest paths are correct on windows as well

* Various cleanup after windows fixes

* Remove the webapp e2e for now

* Add node10 type resolution support for subpath exports (still does not actually import in Node 10 or Browserify/Parcel)

https://github.com/andrewbranch/example-subpath-exports-ts-compat

* init using templates again but downloaded from the repo this time

* Adding init schedule example

* Support for js init

* init now working with js

* Fix issues with links in terminals that don’t support them. Also skip update check of version starts with 0.0.0
2024-09-05 12:56:23 +01:00