Commit Graph

24 Commits

Author SHA1 Message Date
nicktrn 2b586c87a0 Fix controller waitpoint resolution, suspendable state, and snapshot race conditions (#2006)
* remove dead code

* rename managed to shared runtime manager

* rename to resolve waitpoint for clarity

* add resolver id helper

* store and correctly resolve waipoints that come in early

* fix ipc message type change

* branded type for resolver ids

* add fixme comments

* remove more unused ipc schemas

* fix entitlement validation when client doesn't exist

* restore hello world reference workspace imports

* runtime manager debug logs

* prefix engine run logs

* managed run logger accepts nested props

* runtime suspendable state and improved logs

* require suspendable state for checkpoints, fix snapshot processing queue

* add terminal link as cli module so we can more easily patch it

* apply cursor patch

* add license info

* remove terminal-link package and add deprecation notice

* remove old patch

* remove terminal-link from sdk

* rename snapshot module

* add cli test tsconfig

* add run logger base type

* add snapshot manager tests

* fix cli builds

* improve QUEUED_EXECUTING test

* changeset

* make testcontainers wait until container has stopped

* require unit tests for publishing again

* avoid mutation during iteration when resolving pending waitpoints

* improve debug logs and make them less noisy

* always update poller snapshot id for accurate logs

* detach task run process handlers

* check for env overrides in a few more places and add verbose logs

* log when poller is still executing when we stop it

* add supervisor to publish workflow

* always print full deploy logs in CI

* Revert "avoid mutation during iteration when resolving pending waitpoints"

This reverts commit 87b0ce1e5b.

* disable pre

* print prerelease script errors

* Revert "disable pre"

This reverts commit 9403409637.

* misc fixes

* better debug logs

* add snapshots since methods and route

* prep for snapshots since

* improve deprecated execution detection

* update supervisor and schema

* properly log http server errors

* detect restore after failed snapshot fetch

* run and snapshot id can be overridden

* fix restore detection

* fix deprecation checks, move into snapshot manager

* less logs

* rename snapshot manager stop

* restore detection was moved into snapshot manager

* fix notifier logs

* make runtime manager status a debug log

* no need to attach runtime status twice

* findUnique -> findFirst

* sort snapshots by created at everywhere
2025-05-07 13:02:32 +01:00
nicktrn 49a3f72e13 Publish redis-worker and add graceful shutdown manager (#1810)
* add shutdown manager

* update ai test instructions

* add shutdown timeout to redis-worker

* move redis worker to packages

* add unregister method

* prep for publishing package

* fix types

* update ai files

* fix cursor terminal links

* prevent overly friendly ids

* use structured logger

* use unique shutdown handler names

* rework suspend completion

* add trycatch util

* rework suspend restore

* add http server metrics

* add missing prom-client to core

* add prom metrics to redis worker

* bundle redis-worker

* fix esm/cjs interop

* remove proxy from changeset ignore and add supervisor

* add pause to prerelease script for any manual edits

* unregister the correct handler and add early detection

* small change to http handler return

* fix worker tests

* fix shutdown manager tests
2025-03-21 14:53:27 +00:00
nicktrn 4fda8a5ee1 Fix prerelease script (#1794)
* clean before building

* add main branch commit protection
2025-03-15 12:20:54 +00:00
Eric Allam 0a18e1d919 Additional dev queue consumer logging (#1606) 2025-01-13 16:37:19 +00:00
Eric Allam 67592ec2b4 Multiple streams can now be consumed at the same time (#1522)
* Mutliple streams can be now consumed simultaneously

* Update prerelease script

* Add changeset

* Make it core

* Handle API error responses when streaming
2024-12-02 17:58:10 +00:00
Eric Allam 40152af775 Load GITHUB_TOKEN env var in publish-prerelease, as it’s not required 2024-09-23 21:30:35 +01:00
Eric Allam d7a65f3e7a Fix resolving external packages that are ESM only (#1346)
* Fix resolving external packages that are ESM only by falling back to mlly resolvePathSync. This will fix mupdf

* when publishing a prerelease and aborting, clear the git stage
2024-09-23 16:21:04 +01:00
Eric Allam 3aa5811790 fix 3.0.0 update warning (#1308)
* Attempt to fix false package mismatch warnings

* Add changeset

* Add ability to test update checks in prerelease packages

* Resolve the trigger.dev package based on the package.json dir

* Try this

* Don’t use the version module, just resolve the packageJson

* One more dirname

* Comment

* Remove the version export because we aren’t using it anymore
2024-09-16 18:03:16 +01:00
Eric Allam 8578c9b281 v3: new build system fixes round 2 (#1283)
* Fixed empty env vars overriding in dev runs

* Don’t import package.json anymore

* fix node10 moduleResolution in @trigger.dev/core

* Support self-hosters pushing to a custom registry when running deploy

* dev: Fixed stuck runs when a child run fails with a process exit

* Make some doc notes about known issues and docker hub private repos

* Fix --project-ref when running deploy

* Fix —config option when deploying

* Fixing the flushing/killing process with the new build system

* Add monorepo-react-email e2e test fixture

* Fix issue with emitDecoratorMetadata and tsconfigs with extends

* Got the emit decorator metadata fixture working

* Fixed typechecking yarn e2e CLI tests in monorepos

* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)

* Remote externals now powered by JSON Hero to be easier to update

* resolve config source files

* Add a —javascript option to init, defaults to typescript

* Add support for prisma typed sql

* Remove msw and retry.interceptFetch

* Add missing code to the openai retries example

* Don’t generate the v3 catalog prisma client during CI

* Fixed v3-catalog task imports

* Remove interceptor usage in task file

* Only import import-in-the-middle hook if there are instrumented packages

* Fix yarn.lock file
2024-09-11 15:48:20 +01:00
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
nicktrn e23047f9ad update pre-release script 2024-06-12 09:49:47 +01:00
Eric Allam 8fc8f57b39 v2: MarQS powered job executions (#1149)
* WIP

* Allow marqsv2 and v2 graphile to run in parallel

* Fix missing GraphileLogger import

* Fixed heartbeat after rebase

* Replace postgres based run counters with redis ones with a backfill

* Add back in the graphile logger

* Remove duplicate visibility timeout calls

* Clamp simple weighted strategy to max of 5
2024-06-06 13:56:27 +01:00
Eric Allam c3f6557eb6 Create publish preprelease script 2024-04-25 11:26:14 +01:00
Eric Allam 17f6f29d05 Feature: Support multiple runtimes other than Node.js (#774)
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 0s
🚀 Publish Trigger.dev Docker / units (push) Failing after 5s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 5s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
2023-12-06 10:23:37 +00:00
Eric Allam fd94a32b95 Removing a bunch of old stuff 2023-06-21 17:07:17 +01:00
Eric Allam 683f24d4e4 Getting ready to update all templates to the newest SDK after it’s released 2023-03-15 15:05:43 +00:00
Eric Allam 4e83ef642d Update prerelease template deps script 2023-03-14 09:15:12 +00:00
Eric Allam 5eeb275b32 Release template deps script 2023-03-13 15:33:23 +00:00
Eric Allam f39bc44eec Projects
- Deploy a new VM when a push event comes through
- Live updating project overview page
2023-03-07 15:12:42 +00:00
Eric Allam ab512157a5 Added ability to patch all template repos and push the changes 2023-03-03 08:34:15 +00:00
Eric Allam 6deabb21ba Bump notion 2023-03-02 22:30:43 +00:00
Eric Allam 408bd6ddc8 Added a script to automatically update template repo @trigger.dev packages instead of having to do it manually 2023-03-02 15:34:53 +00:00
Eric Allam 5ad386e306 Make it easier to start ngrok in different regions 2023-01-29 22:23:38 +00:00
Eric Allam 1f1195a0d6 Change how ngrok is being used again, and update DEVELOPMENT instructions 2022-12-30 11:44:53 +00:00