github-featured/triggerdotdev--trigger.dev

Fork 0

Files

T

History

Saadi Myftija 6d6ed471d1

⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled

Details

feat(cli): deterministic image builds for deployments (#2778 )

This PR makes our image builds deterministic and reproducible by
ensuring that identical source code always produces the same image
layers and image digest. This means that deployments where nothing has
changed will no longer invalidate the image cache in our worker cluster
nodes, thus avoid making the cold starts for runs worse.

**Context**
New deployments currently increase the cold start times for runs, as
they generate a new image which needs to be pulled in the worker cluster
where runs are executed. It happens also when the source code for the
deployment has not changed due to non-deterministic steps in our build
system. This addresses the latter issue by making builds reproducible.

**Main changes**
- Avoided baking `TRIGGER_DEPLOYMENT_ID` and
`TRIGGER_DEPLOYMENT_VERSION` in the image, we now pass these via the
supervisor instead.
- Used `json-stable-stringify` for consistent key ordering in the files
we generate for the build, e.g., `package.json`, `build.json`,
`index.json`.
- Removed `metafile.json` from the image contents as it is not actually
used in the container. This is only relevant for the `analyze` command.
- Added `SOURCE_DATE_EPOCH=0` and `rewrite-timestamp=true` to Docker
builds to normalize file timestamps.
- Removed some `timings` and `outputHashes` from build outputs and
manifests.

The builds are now reproducible for both native build server and Depot
paths. This should also lead to better image layer cache reuse in
general.

2025-12-12 09:48:04 +01:00

src

feat(cli): deterministic image builds for deployments (#2778 )

2025-12-12 09:48:04 +01:00

.env.example

Fix: fixes and prerequisites for v4 self-hosting (#2150 )

2025-06-04 18:02:38 +01:00

.nvmrc

Add RE2 supervisor (#1769 )

2025-03-06 13:20:45 +00:00

Containerfile

fix(supervisor): image builds with pnpm v10 (#2718 )

2025-12-01 11:28:05 +00:00

package.json

feat(supervisor): add ecr support to docker client (#2424 )

2025-08-20 10:49:19 +01:00

README.md

Allow self-hosted deploys locally by pointing to docker.host.internal on macOS (#2064 )

2025-05-16 15:20:48 +01:00

tsconfig.json

re2: env based queue selection algo (#1775 )

2025-03-07 14:30:19 +00:00

README.md

Supervisor

Dev setup

Create a worker group

api_url=http://localhost:3030
wg_name=my-worker

# edit this
admin_pat=tr_pat_...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\"}"

If the worker group is newly created, the response will include a token field. If the group already exists, no token is returned.

Create .env and set the worker token

cp .env.example .env

# Then edit your .env and set this to the token.plaintext value
TRIGGER_WORKER_TOKEN=tr_wgt_...

Start the supervisor

pnpm dev

Build CLI, then deploy a test project

pnpm exec trigger deploy --self-hosted

# The additional network flag is required on linux
pnpm exec trigger deploy --self-hosted --network host

Worker group management

Shared variables

api_url=http://localhost:3030
admin_pat=tr_pat_... # edit this

These are used by all commands

Create a worker group

wg_name=my-worker

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\"}"

If the worker group already exists, no token will be returned

Set a worker group as default for a project

wg_name=my-worker
project_id=clsw6q8wz...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\", \"projectId\": \"$project_id\", \"makeDefaultForProject\": true}"

If the worker group doesn't exist, yet it will be created
If the worker group already exists, it will be attached to the project as default. No token will be returned.

Remove the default worker group from a project

project_id=clsw6q8wz...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"projectId\": \"$project_id\", \"removeDefaultFromProject\": true}"

The project will then use the global default again
When removeDefaultFromProject: true no other actions will be performed