github-featured/triggerdotdev--trigger.dev

Files

T

Saadi Myftija 45ed802e54 feat(supervisor): add node affinity rules for large machine worker pool scheduling

**Background**
Runs with `large-1x` or `large-2x` machine presets are disproportionally
affected by scheduling delays during peak times. This is in part caused
by the fact that the worker pool is shared for all runs, meaning large
runs compete with smaller runs for available capacity. Because large runs require
significantly more CPU and memory, they are harder for the scheduler to bin-pack onto existing
nodes, often requiring a node with a significant amount of free resources or
waiting for a new node to spin up entirely. This effect is amplified during peak times
when nodes are already densely packed with smaller workloads, leaving insufficient contiguous resources for large runs.

**Changes**

This PR adds Kubernetes node affinity settings to separate large and standard machine workloads across node pools.

  - Controlled via KUBERNETES_LARGE_MACHINE_POOL_LABEL env var (disabled when not set)
  - Large machine presets (large-*) get a soft preference to schedule on the large pool, with fallback to standard nodes
  - Non-large machines are excluded from the large pool via required anti-affinity
  - This ensures the large machine pool is reserved for large workloads while allowing large workloads to spill over to standard nodes if needed

2026-01-12 13:35:49 +01:00

src

feat(supervisor): add node affinity rules for large machine worker pool scheduling

2026-01-12 13:35:49 +01:00

.env.example

Fix: fixes and prerequisites for v4 self-hosting (#2150 )

2025-06-04 18:02:38 +01:00

.nvmrc

Add RE2 supervisor (#1769 )

2025-03-06 13:20:45 +00:00

Containerfile

fix(supervisor): image builds with pnpm v10 (#2718 )

2025-12-01 11:28:05 +00:00

package.json

feat(supervisor): add ecr support to docker client (#2424 )

2025-08-20 10:49:19 +01:00

README.md

Allow self-hosted deploys locally by pointing to docker.host.internal on macOS (#2064 )

2025-05-16 15:20:48 +01:00

tsconfig.json

re2: env based queue selection algo (#1775 )

2025-03-07 14:30:19 +00:00

README.md

Supervisor

Dev setup

Create a worker group

api_url=http://localhost:3030
wg_name=my-worker

# edit this
admin_pat=tr_pat_...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\"}"

If the worker group is newly created, the response will include a token field. If the group already exists, no token is returned.

Create .env and set the worker token

cp .env.example .env

# Then edit your .env and set this to the token.plaintext value
TRIGGER_WORKER_TOKEN=tr_wgt_...

Start the supervisor

pnpm dev

Build CLI, then deploy a test project

pnpm exec trigger deploy --self-hosted

# The additional network flag is required on linux
pnpm exec trigger deploy --self-hosted --network host

Worker group management

Shared variables

api_url=http://localhost:3030
admin_pat=tr_pat_... # edit this

These are used by all commands

Create a worker group

wg_name=my-worker

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\"}"

If the worker group already exists, no token will be returned

Set a worker group as default for a project

wg_name=my-worker
project_id=clsw6q8wz...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"name\": \"$wg_name\", \"projectId\": \"$project_id\", \"makeDefaultForProject\": true}"

If the worker group doesn't exist, yet it will be created
If the worker group already exists, it will be attached to the project as default. No token will be returned.

Remove the default worker group from a project

project_id=clsw6q8wz...

curl -sS \
    -X POST \
    "$api_url/admin/api/v1/workers" \
    -H "Authorization: Bearer $admin_pat" \
    -H "Content-Type: application/json" \
    -d "{\"projectId\": \"$project_id\", \"removeDefaultFromProject\": true}"

The project will then use the global default again
When removeDefaultFromProject: true no other actions will be performed