Commit Graph

35 Commits

Author SHA1 Message Date
nicktrn cad8791859 chore: make changeset:version atomic (#3505)
Follow-up to the v4.4.5 release incident where the release PR (#3406)
was merged with a stale lockfile and stale Chart.yaml, breaking npm +
helm releases. The two automation jobs (`update-lockfile`,
`bump-chart-version`) got cancelled mid-flight by `cancel-in-progress`
when the merge fired the workflow again on `main`.

This restructures `changeset:version` so all the post-version-bump
fixups happen in the same script and end up in a single atomic commit on
`changeset-release/main`, via `changesets/action`'s normal commit step.

Pattern borrowed from Cloudflare workers-sdk, Astro, shadcn/ui.

## Before

```
push: main
└── release-pr (changeset version → bumps package.jsons, opens PR)
    └── update-lockfile (separate job, separate commit)
        └── bump-chart-version (separate job, separate commit)
```

Three jobs, three commits to the release branch.

## After

```
push: main
└── release-pr
    └── changesets/action runs:
          changeset version
          pnpm install --lockfile-only
          node scripts/bump-helm-chart.mjs
          node scripts/cleanup-server-changes.mjs
        ...all staged and committed as ONE commit by the action
```

One job, one commit.
2026-05-02 09:45:44 +01:00
Eric Allam 6f6523ff78 chore(repo): remove unnecessary "trigger.dev v4.4.2" header from the release PR description (#3183) 2026-03-06 14:07:32 +00:00
Eric Allam 5612383684 chore(repo): Improve formatting of server entries in release notes (#3134) 2026-02-26 11:59:47 +00:00
Eric Allam fe193418d0 chore(repo) auto-link server change entries to their PRs via GitHub API (#3129) 2026-02-25 16:02:32 +00:00
Eric Allam c05b30adfe chore(repo): fix enhanced release pr description to filter out dependency only updates (#3128) 2026-02-25 15:50:06 +00:00
Eric Allam 3c0644a3b8 feat: unified GitHub release, server change tracking, and enhanced release PR (#3085)
- Add .server-changes/ convention for tracking server-only changes
- Create scripts/enhance-release-pr.mjs to deduplicate and categorize
changeset PR body
- Create scripts/generate-github-release.mjs to format unified GitHub
release body
- Change release.yml to create one unified GitHub release instead of
per-package releases
- Add update-release job to patch Docker image link after images are
pushed to GHCR
- Update changesets-pr.yml to trigger on .server-changes, enhance PR
body, and clean up consumed files
- Document server changes in CLAUDE.md, CONTRIBUTING.md, CHANGESETS.md,
and RELEASE.md
2026-02-25 13:54:19 +00:00
Matt Aitken 6055c7d050 Recover runs that failed to dequeue (#2931)
There’s an edge case that means runs can end up in the
currentConcurrency set when they’re not in the correct state for
execution. This means they will be permanently stuck in queued.

Given an environmentId this will fix those runs.

This is a temporary fix while we permanently fix the issue.

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Eric Allam <ericallam@users.noreply.github.com>
2026-01-23 16:25:58 +00:00
Eric Allam bd449f75dc fix(migrations): Add IF NOT EXISTS to 20260116154810_add_idempotency_key_options_to_task_run (#2923)
## Summary
- Adds `IF NOT EXISTS` to the migration that adds
`idempotencyKeyOptions` column to prevent errors if the column already
exists

## Migration Checksum Fix

If you've already applied the previous version of this migration, you'll
need to update the checksum in your `_prisma_migrations` table to match
the new migration file.

**Previous checksum:**
`f8876e274e3f7735312275eb24a9c4b40f512ac12a286b2de3add47f66df5b27`
**New checksum:**
`0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397`

### Fix instructions

Run this SQL command against your database:

```sql
UPDATE "_prisma_migrations"
SET checksum = '0620a914ddbaf01279576274432e51c41f41502cd4c8de38621625380750e397'
WHERE migration_name = '20260116154810_add_idempotency_key_options_to_task_run';
```

This updates the stored checksum to match the modified migration file,
allowing future migrations to proceed without checksum mismatch errors.

## Test plan
- [x] Verified migration applies cleanly on fresh database
- [ ] Verified checksum update works on database with previous migration
applied

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-01-21 14:43:37 +00:00
Eric Allam fc351cb6c4 chore(repo): remove GITHUB_TOKEN requirement for publishing prerelease packages (#2679) 2025-11-24 14:33:40 +00:00
nicktrn 885ae5e06f fix(runner): immediate poll to decrease restore time (#2516)
* fix(runner): immediate poll to decrease restore time

* chore: bump default prerelease tag

* fix(cli): s is not a function

* add changeset
2025-09-16 16:17:04 +01:00
Eric Allam 65da20c225 feat: replicate task runs to clickhouse to power dashboard improvements (#2035)
* WIP clickhouse package with test containers setup

* More clickhouse client setup now with otel and real tests, and the v1 of raw run events

* Add some additional columns to raw_run_events_v1

* WIP runs dashboard service

* Create a new run engine event bus event for the runs dashboard to hook into

* Track run events in the run engine

* make sure engine v1 runs get synced to CH

* Update the attemptNumber of v3 task runs

* Restructure the run events to be more sparse

* emit more stuff

* Setup replication package

* scaffold the replication package

* replication wip

* resolve conflicts

* more replication stuff

* Add ability to drop the replication slot completely on teardown

* Use the new single replacingmergetree task events table for replication

* get it working

* insert payloads into their own table only on insert and then join

* prepare for using clickhouse cloud and now running ch migrations during boot in the entrypoint.sh

* Handover WIP and tests

* Testing the replication service

* Remove the runs dashboard stuff that we aren't using anymore

* Added a test for large payloads

* hacky typecheck fix

* Fix new internal package typecheck issues and start adding telemetry to the replication service

* tracing over spans, some other improvements

* Improvements to the runs replication service, now ready for testing

* Some fixes and cleanups

* Don't need this code anymore

* move transaction types into the runs replication service

* only send spans where there are transaction events

* A couple of suggested tweaks
2025-05-12 22:12:36 +01:00
nicktrn 2b586c87a0 Fix controller waitpoint resolution, suspendable state, and snapshot race conditions (#2006)
* remove dead code

* rename managed to shared runtime manager

* rename to resolve waitpoint for clarity

* add resolver id helper

* store and correctly resolve waipoints that come in early

* fix ipc message type change

* branded type for resolver ids

* add fixme comments

* remove more unused ipc schemas

* fix entitlement validation when client doesn't exist

* restore hello world reference workspace imports

* runtime manager debug logs

* prefix engine run logs

* managed run logger accepts nested props

* runtime suspendable state and improved logs

* require suspendable state for checkpoints, fix snapshot processing queue

* add terminal link as cli module so we can more easily patch it

* apply cursor patch

* add license info

* remove terminal-link package and add deprecation notice

* remove old patch

* remove terminal-link from sdk

* rename snapshot module

* add cli test tsconfig

* add run logger base type

* add snapshot manager tests

* fix cli builds

* improve QUEUED_EXECUTING test

* changeset

* make testcontainers wait until container has stopped

* require unit tests for publishing again

* avoid mutation during iteration when resolving pending waitpoints

* improve debug logs and make them less noisy

* always update poller snapshot id for accurate logs

* detach task run process handlers

* check for env overrides in a few more places and add verbose logs

* log when poller is still executing when we stop it

* add supervisor to publish workflow

* always print full deploy logs in CI

* Revert "avoid mutation during iteration when resolving pending waitpoints"

This reverts commit 87b0ce1e5b.

* disable pre

* print prerelease script errors

* Revert "disable pre"

This reverts commit 9403409637.

* misc fixes

* better debug logs

* add snapshots since methods and route

* prep for snapshots since

* improve deprecated execution detection

* update supervisor and schema

* properly log http server errors

* detect restore after failed snapshot fetch

* run and snapshot id can be overridden

* fix restore detection

* fix deprecation checks, move into snapshot manager

* less logs

* rename snapshot manager stop

* restore detection was moved into snapshot manager

* fix notifier logs

* make runtime manager status a debug log

* no need to attach runtime status twice

* findUnique -> findFirst

* sort snapshots by created at everywhere
2025-05-07 13:02:32 +01:00
nicktrn 49a3f72e13 Publish redis-worker and add graceful shutdown manager (#1810)
* add shutdown manager

* update ai test instructions

* add shutdown timeout to redis-worker

* move redis worker to packages

* add unregister method

* prep for publishing package

* fix types

* update ai files

* fix cursor terminal links

* prevent overly friendly ids

* use structured logger

* use unique shutdown handler names

* rework suspend completion

* add trycatch util

* rework suspend restore

* add http server metrics

* add missing prom-client to core

* add prom metrics to redis worker

* bundle redis-worker

* fix esm/cjs interop

* remove proxy from changeset ignore and add supervisor

* add pause to prerelease script for any manual edits

* unregister the correct handler and add early detection

* small change to http handler return

* fix worker tests

* fix shutdown manager tests
2025-03-21 14:53:27 +00:00
nicktrn 4fda8a5ee1 Fix prerelease script (#1794)
* clean before building

* add main branch commit protection
2025-03-15 12:20:54 +00:00
Eric Allam 0a18e1d919 Additional dev queue consumer logging (#1606) 2025-01-13 16:37:19 +00:00
Eric Allam 67592ec2b4 Multiple streams can now be consumed at the same time (#1522)
* Mutliple streams can be now consumed simultaneously

* Update prerelease script

* Add changeset

* Make it core

* Handle API error responses when streaming
2024-12-02 17:58:10 +00:00
Eric Allam 40152af775 Load GITHUB_TOKEN env var in publish-prerelease, as it’s not required 2024-09-23 21:30:35 +01:00
Eric Allam d7a65f3e7a Fix resolving external packages that are ESM only (#1346)
* Fix resolving external packages that are ESM only by falling back to mlly resolvePathSync. This will fix mupdf

* when publishing a prerelease and aborting, clear the git stage
2024-09-23 16:21:04 +01:00
Eric Allam 3aa5811790 fix 3.0.0 update warning (#1308)
* Attempt to fix false package mismatch warnings

* Add changeset

* Add ability to test update checks in prerelease packages

* Resolve the trigger.dev package based on the package.json dir

* Try this

* Don’t use the version module, just resolve the packageJson

* One more dirname

* Comment

* Remove the version export because we aren’t using it anymore
2024-09-16 18:03:16 +01:00
Eric Allam 8578c9b281 v3: new build system fixes round 2 (#1283)
* Fixed empty env vars overriding in dev runs

* Don’t import package.json anymore

* fix node10 moduleResolution in @trigger.dev/core

* Support self-hosters pushing to a custom registry when running deploy

* dev: Fixed stuck runs when a child run fails with a process exit

* Make some doc notes about known issues and docker hub private repos

* Fix --project-ref when running deploy

* Fix —config option when deploying

* Fixing the flushing/killing process with the new build system

* Add monorepo-react-email e2e test fixture

* Fix issue with emitDecoratorMetadata and tsconfigs with extends

* Got the emit decorator metadata fixture working

* Fixed typechecking yarn e2e CLI tests in monorepos

* Add remote forced externals system, in case we come across another package that cannot be bundled (spurred on by header-generator)

* Remote externals now powered by JSON Hero to be easier to update

* resolve config source files

* Add a —javascript option to init, defaults to typescript

* Add support for prisma typed sql

* Remove msw and retry.interceptFetch

* Add missing code to the openai retries example

* Don’t generate the v3 catalog prisma client during CI

* Fixed v3-catalog task imports

* Remove interceptor usage in task file

* Only import import-in-the-middle hook if there are instrumented packages

* Fix yarn.lock file
2024-09-11 15:48:20 +01:00
Eric Allam f9ec66c562 v3: new build system (#1265)
* upgrade @opentelemetry packages to the latest versions

* remove v2 only packages, will be moved to a dedicated repo

* remove more v2 code and run pnpm install

* use the npm yalt package in the webapp

* convert @trigger.dev/core to tshy

* Switch from jest to vitest in @trigger.dev/core

* Fixed core test

* move core-backend code into core subpath export

* convert @trigger.dev/sdk to tshy

* Removed hono

* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3

* Fix up some of the commands

* cli now building and loadable

* using package-json-from-dist to get package version now in core and cli

* dev command WIP

* cleaned up some repetition and structure of the entry point stuff

* bringing back the background worker stuff

* Indexing of the v3 catalog

* getting closer to executing dev runs...

* centralize dev logging using event emitter

* Move indexing to it’s own entry point, simplify code

* dev runs working

* Get instrumentation to work with openai

* debugging achieved internally

* provide worker files as part of the worker creation on the server

* support for cjs and esm javascript

* Fixed timeout

* worker manifest now has the config path

* auto-upgrade config to non-deprecated alternatives

* Adding package preview release

* deployment WIP

* improve the syncEnvVars output and adapt resolveEnvVars

* WIP bun runtime

* WIP bun support

* seed tasks with the machine preset if listed in the config

* deploy run executions WIP, extracted TaskRunProcess into 1 place

* deployed tasks running and executing 🎉

* support for waits and better flushing & process cleanup

* Fixed the heartbeating

* Better warning messages

* Improve and unify the indexing between dev and deploy

* Support for external deps that need node-gyp to build

* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files

* Add back in the main/types/module to sdk

* dev no longer is Ink/React, grace period for disconnections in dev

* Fix the changeset config

* More changeset fixes

* Remove config packages

* More changeset fixes

* Fixed typescript issues (needed to revert back to zod 3.22.3

* Fix pr_checks workflow

* Remove the prepare script

* Fixed tests and package versions

* Remove cli test script

* Remove packages from tailwind watch paths

* Add repo to public packages

* Just commit the generated files and do the building at dev time

* Try and get pkg.pr.new working

* Try again

* Fix emitDecoratorMetadata importing named export from typescript

* config file backwards compat with export const config

* Fixed issue where import errors weren’t coming through

* p-retry is a prod dep

* typescript needs to be a prod dependency for emitDecoratorMetadata

* Add better debug logging to help track down import-in-the-middle bug

* An external is only considered resolvable if it resolves to the same path as the collected external

* Fix runtime checks to allow >=18.20

* Move extensions to a new build package

* Fixed building packages in dockerfile

* Remove the e2e test from publish workflow for now

* Don’t treat pkg.pr.new versions has needing upgrading

* making sure config handleError works, and discovered path aliases don’t work in config files

* Strip empty string env vars so they accidentally override real values

* Couple of things

* Update version to use preview instead of beta

* Hopefully fix re-attempts with >30s delay

* Match socket emit messages to current latest in main

* Initial guide

* Go back to beta

* Go back to the preview, and update guide to use pr preview tags

* Go back to beta

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-08-23 13:10:15 +01:00
nicktrn e23047f9ad update pre-release script 2024-06-12 09:49:47 +01:00
Eric Allam 8fc8f57b39 v2: MarQS powered job executions (#1149)
* WIP

* Allow marqsv2 and v2 graphile to run in parallel

* Fix missing GraphileLogger import

* Fixed heartbeat after rebase

* Replace postgres based run counters with redis ones with a backfill

* Add back in the graphile logger

* Remove duplicate visibility timeout calls

* Clamp simple weighted strategy to max of 5
2024-06-06 13:56:27 +01:00
Eric Allam c3f6557eb6 Create publish preprelease script 2024-04-25 11:26:14 +01:00
Eric Allam 17f6f29d05 Feature: Support multiple runtimes other than Node.js (#774)
🚀 Publish Trigger.dev Docker / e2e (push) Failing after 0s
🚀 Publish Trigger.dev Docker / units (push) Failing after 5s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 5s
🚀 Publish Trigger.dev Docker / publish (push) Has been skipped
2023-12-06 10:23:37 +00:00
Eric Allam fd94a32b95 Removing a bunch of old stuff 2023-06-21 17:07:17 +01:00
Eric Allam 683f24d4e4 Getting ready to update all templates to the newest SDK after it’s released 2023-03-15 15:05:43 +00:00
Eric Allam 4e83ef642d Update prerelease template deps script 2023-03-14 09:15:12 +00:00
Eric Allam 5eeb275b32 Release template deps script 2023-03-13 15:33:23 +00:00
Eric Allam f39bc44eec Projects
- Deploy a new VM when a push event comes through
- Live updating project overview page
2023-03-07 15:12:42 +00:00
Eric Allam ab512157a5 Added ability to patch all template repos and push the changes 2023-03-03 08:34:15 +00:00
Eric Allam 6deabb21ba Bump notion 2023-03-02 22:30:43 +00:00
Eric Allam 408bd6ddc8 Added a script to automatically update template repo @trigger.dev packages instead of having to do it manually 2023-03-02 15:34:53 +00:00
Eric Allam 5ad386e306 Make it easier to start ngrok in different regions 2023-01-29 22:23:38 +00:00
Eric Allam 1f1195a0d6 Change how ngrok is being used again, and update DEVELOPMENT instructions 2022-12-30 11:44:53 +00:00