## Summary
Speeds up webapp test jobs by balancing measured work across runners,
reducing repeated container setup, and ensuring test workers release
shutdown resources promptly. Unit tests run across 24 duration-aware
shards, while E2E tests run across two balanced shards.
## Design
`RunEngine` shutdown now closes processing resources before support
resources, continues cleanup if one close fails, and reuses one shutdown
promise for concurrent callers. Redis workers clear completed shutdown
deadlines so finished tests no longer wait on idle timers.
Container-heavy suites are split only where it improves parallelism, and
repeated replication and engine fixtures are consolidated where one
end-to-end case provides coverage. Timing weights are refreshed for all
affected files.
Dependency installation overlaps container pulls, and both workflows use
WarpBuild's Node setup action.
<!-- ccr-slack-attribution -->
_Requested by **Iss** · [Slack
thread](https://triggerdotdev.slack.com/archives/C045W9WM3E1/p1787161814493949)_
**Before:** archiving a branch dropped the query string on the way back
to the branches list, so the list reset to page 1. Working down a long
list meant re-navigating to the page you were on after every archive.
**After:** you land back on the exact page you archived from, with
`page`, `search` and `showArchived` intact.
The archive action now redirects to the page the request came from
instead of rebuilding a bare branches path.
## How
The archive dialog already submits the page it was opened from as a
hidden `redirectPath` field (`${location.pathname}${location.search}`),
and the failure path already redirected to it — only the success path
ignored it and rebuilt the path with `branchesPath`/`branchesDevPath`,
which have no query string. Both paths now redirect to the submitted
path, run through the existing `sanitizeRedirectPath` helper to keep the
redirect same-origin (the same idiom used by
`resources.batches.$batchId.check-completion`).
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Three files change:
- `apps/webapp/app/routes/resources.branches.archive.tsx` — the fix.
- `apps/webapp/test/archiveBranchRedirect.test.ts` — new test that
drives the archive action and asserts the redirect `Location`: the query
string survives on both success and failure, and an off-origin
`redirectPath` falls back to `/`. Reverting the fix makes two of the
three cases fail, so the test covers the regression.
- `.server-changes/archive-branch-keeps-list-page.md` — release-note
entry, since this is a user-facing server-only change.
Also ran `pnpm run typecheck` and `oxlint` for `apps/webapp` — both
clean.
---
## Changelog
Archiving a branch now returns you to the same page of the branches list
instead of resetting it to page 1.
---
## Screenshots
_None — no visual change._
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Problem
Every merge to main touching the agent queued a gated `staging`+`prod`
deploy that sat `pending` on a reviewer approval nobody grants
routinely. Because the gated runs never completed, they never drained
the concurrency queue and cancelled each other, so the Actions tab
filled with never-completing runs and the agent only ever actually
deployed via a manual dispatch + approval.
The reviewer gate bought nothing here: the agent deploys with
`--skip-promotion`, so a deploy lands **dormant** and nothing goes live
until the consuming webapp flips `DASHBOARD_AGENT_VERSION`. Promotion is
already a deliberate act (the env-var flip); gating the dormant deploy
on top of that just created the pile-up.
## Change
- **Remove the reviewer gate** by dropping the required-reviewers rule
on the `dashboard-agent-*` environments (repo-settings change, done).
The `environment:` key **stays** so the per-environment scoped deploy
token still resolves — no secret migration.
- **`workflow_dispatch` `ref` input** — deploy a specific commit SHA,
branch, or tag; defaults to the ref the run launches from. Checkout uses
`github.event.inputs.ref || github.sha`.
- **Require the ref to be an ancestor of `main`.** Constrains which
commit gets deployed to merged code only. A push is always main's tip
(passes trivially); a dispatched unmerged ref is rejected before the
deploy step. Because an explicit `ref:` checkout doesn't create
remote-tracking branches, `origin/main` is fetched explicitly before
`git merge-base --is-ancestor`.
- **`cancel-in-progress: false`** (kept). Cancelling the runner wouldn't
stop the remote build (it finishes server-side), and a superseding
concurrent deploy would race the same project's indexer. With the gate
gone, deploys are short, so a brief queue can't pile up.
- `max-parallel: 1` stays (parallel deploys of the same project race at
the indexer).
## Owner actions (repo settings — not in the diff)
1. **Remove required-reviewers** on `dashboard-agent-staging` and
`dashboard-agent-prod` — done.
2. **Add a deployment branch policy** on both environments restricting
deployments to `main`. This is the authoritative token guard:
`workflow_dispatch` runs the workflow file from the selected ref, so the
in-file ancestor check alone can't protect `TRIGGER_ACCESS_TOKEN` (a
branch could edit the check out). GitHub enforces the branch policy
server-side against `GITHUB_REF` regardless of file contents. With it in
place, the workflow only runs (and the token is only exposed) when
dispatched from `main`, and the in-file check then constrains the
independent `ref` input to merged commits.
## Pile-up root cause
The stacking was caused by the **reviewer gate** (runs waited forever,
so the queue never drained), not by `cancel-in-progress`. Removing the
gate is what fixes it; `cancel-in-progress` stays `false`.
Two defects that surface when a run parked on an external deployment id
gets pushed by a debounce key. Both were reproduced against a local
instance before being fixed.
## 1. The run is expired before it is due
```
now | status | statusReason | delayUntil | expiredAt
13:57:06 | EXPIRED | EXTERNAL_DEPLOYMENT_NOT_FOUND | 14:01:37 | 13:57:02
```
Killed 4m35s before its own scheduled start, blaming a missing
deployment.
**Why.** The park deadline is armed **once**, when the run is first
parked, from `max(now, delayUntil) + deadline`. Debounce pushes
`delayUntil` out afterwards and nothing re-arms it:
- `rescheduleDelayedRun` reschedules `enqueueDelayedRun:<id>`, not
`expireParkedExternalDeploymentRun:<id>`
- the redis-worker reschedule is an update-only `ZADD … XX`, and a
parked run has no `enqueueDelayedRun` job, so that call is a silent
no-op
Repeat triggers on one key walk `delayUntil` away from a deadline that
no longer moves. Once it crosses, the run dies while parked and not yet
due.
**Fix.** The expiry job already loads `delayUntil`, so it re-arms from
the current value and returns instead of expiring a run that is not due.
The guard lives in the expiry job rather than the debounce path
deliberately: it covers **every** caller that moves `delayUntil`, so a
future call site can't reintroduce this by forgetting to re-arm. It
stays bounded by the debounce max-duration contract, so a hot key can't
postpone expiry indefinitely.
## 2. The run reports itself as delayed while it is parked
```
RUN_CREATED | PENDING_VERSION | Run is waiting for a deployment of 'debounce-test-2'
DELAYED | DELAYED | Delayed run was rescheduled to a future date ← after one debounce push
```
The row stays `PENDING_VERSION`; the latest snapshot claims `DELAYED`,
so the run page describes a parked run as delayed. Happens on the
*first* push.
**Fix.** `rescheduleRun` hardcoded `DELAYED`/`DELAYED`. The snapshot
statuses are now supplied by the caller and **default to `DELAYED`**, so
the ordinary delayed path is byte-identical, and `rescheduleDelayedRun`
passes the parked statuses through when the run is parked.
## Reproducing
Repeated triggers on one debounce key against an id that hasn't landed:
```bash
curl … -d '{"options":{"externalDeploymentId":"x","debounce":{"key":"k","delay":"5m"}}}'
```
Three triggers correctly fold into one parked run; the defects show up
on the pushes.
## Testing
Two tests, each verified red before green and failing alone:
- a run whose delay was pushed past the deadline stays `PENDING_VERSION`
instead of expiring
- a debounce push on a parked run leaves a
`RUN_CREATED`/`PENDING_VERSION` snapshot, not `DELAYED`
`56 passed` across parking, pendingVersion, delayedRunSystem and
debounce; `43 passed` in `PostgresRunStore`. Typecheck, lint, format
clean.
## Notes
- Stacks on #4665, so it lands after the whole external-deployment-id
series.
- No changeset: this fixes unreleased behaviour introduced by the stack
below it, so no user has seen it.
- Both found by Devin's review on #4664, and both confirmed end to end
on a local instance before fixing.
Deployments page: an always-visible External ID column after Deployed
by, and an External ID row in the deployment inspector under Worker
type, both showing an en dash when a deploy carried no id. The Vercel
Linked column now renders before Git, still only when a Vercel
integration is connected. Also corrects the blank-row colSpan, which was
already off by one before this column existed.
Run inspector: an External deployment ID row between Version and SDK
version, read from the run annotations, so an operator can see which id
a run was pinned to - including a run that expired before its deployment
ever arrived, where the locked version is empty but the id is the whole
story. Buffered runs read the id from the same annotations rather than
reporting none.
Long ids are head-truncated with the full value behind the copy button:
a commit SHA is meaningful in its prefix, and the inspector panel can be
narrowed to 250px, where an unbroken 40-character SHA would otherwise
scroll the properties list sideways and push the copy button off-panel
(TRI-12923, TRI-13000).
The SDK discovers an external deployment id at runtime (explicit
TRIGGER_EXTERNAL_DEPLOYMENT_ID always; platform commit-SHA variables and
generic fallbacks when TRIGGER_AUTOMATIC_SKEW_VERSION_PROTECTION=1) and
sends it alongside lockToVersion; the server resolves precedence
(version > external id > current). An id held by a deployed deployment
pins the run to that worker; an in-flight or unknown id parks the run in
PENDING_VERSION with the id in TaskRun.annotations, wakes it pinned when
a deployment carrying the id finalizes (ClickHouse candidates, Postgres
authoritative), and expires it after a deadline that re-checks Postgres
before acting. Parking outranks delaying and preserves delayUntil. The
id is projected to ClickHouse task_runs_v2.external_deployment_id during
replication. Redis cache for id-to-worker resolution, guarded
version-aware writes.
Ids are not unique. Several deployments can hold one id - a --force
rebuild is the ordinary way to get there - so resolution always picks
the highest version among the candidates, never the newest by timestamp.
The rule is applied identically on both paths that can bind a run to a
worker: resolveExternalDeployment at trigger time, and
PendingVersionSystem when a landing deployment wakes a parked run.
Version comparison is numeric on the counter half, so 20260807.10
outranks 20260807.9.
A run whose id never lands expires at the deadline with
EXTERNAL_DEPLOYMENT_NOT_FOUND and an error naming the id it waited for,
which is what a failed build or a typo looks like from the caller.
Default deadline is one hour (EXTERNAL_DEPLOYMENT_PARK_DEADLINE_MS).
Debounce registration happens in both the parked and the delayed branch
through one helper, so a debounced run that parks still binds its
debounce key; without it every later trigger for the same key created
another parked run, and all of them executed when the deployment landed.
The two DELAYED-only status checks in DebounceSystem also accept
PENDING_VERSION, without which the lock-contention fallback would
rethrow a 5xx the SDK retries and amplifies, and the fast path would
push every trigger on a parked key through the redlock.
Resolution is skipped in development. A dev environment cannot hold a
WorkerDeployment - trigger dev registers a BackgroundWorker with nothing
behind it, and deploy --env refuses dev - so an external deployment id
there could only ever park, and the parked run then expired against the
dev TTL while a connected dev worker sat idle. The id is still annotated
so the dashboard shows what the app sent (TRI-13000).
A deploy can carry an opaque external id (commit SHA, CI run id, release
tag). Repeating an id that already deployed returns the existing version
as a no-op instead of rebuilding; an id with a build in flight is
rejected with 409 naming that version; a failed id rebuilds freely.
--force is non-destructive to deployments that already succeeded - both
persist and the higher version wins - but cancels a build still in
flight, so one id never has two live builds racing to define it.
Cancelling writes a terminal status and appends a finalized event, which
aborts a build the platform drives; a build it does not drive keeps
running but can never land, and the CLI says so. Ids are deliberately
not unique - reuse is resolved in application code by highest version,
never timestamps. The no-op path mints no build credentials and no event
stream (TRI-12923).
What that means for callers: a --force rebuild leaves two deployments
holding one id, and runs triggered with it go to the higher version once
the rebuild lands, so the takeover needs no separate promotion. Until a
successful build exists for an id, runs triggered with it park and then
expire rather than falling back to current - a failed build is therefore
visible to the caller as expired runs, not as runs on the wrong release.
An external deployment id is an opaque, caller-chosen name for a release
- a commit SHA, a CI run id, a release tag. This adds the shared
contract that both halves of the feature read, and nothing else: no
deploy writes one yet and no trigger sends one.
ExternalDeploymentId is defined once and reused by
InitializeDeploymentRequestBody.externalId and
TriggerTaskRequestBody.options.externalDeploymentId, so a value accepted
by one half can never be rejected by the other. A value that is blank
once trimmed is treated as absent rather than rejected, so an unset CI
variable expanding to an empty string is not a 400. The 128 character
limit fits a SHA-256 commit hash with room for composite ids, and
EXTERNAL_DEPLOYMENT_ID_MAX_LENGTH is the single source of truth that the
request schemas and the CLI both read.
RunAnnotations.externalDeploymentId records the request, not the
outcome: lockedToVersionId and taskVersion are overwritten when a run
locks, whereas this stays true forever, and it can carry the pin for a
run parked before its deployment exists.
Also lands the runtime discovery helpers as pure functions over an
environment reader: the explicit TRIGGER_EXTERNAL_DEPLOYMENT_ID
variable, the platform and CI commit-SHA table, and the
TRIGGER_AUTOMATIC_SKEW_VERSION_PROTECTION gate. Nothing calls them yet.
refs TRI-13000
Migrations only, no code reads them yet. Postgres: nullable non-unique
externalId on WorkerDeployment plus a CONCURRENTLY-built (environmentId,
externalId) index in its own migration file. ClickHouse:
external_deployment_id String DEFAULT '' on task_runs_v2 (plain String,
not LowCardinality - commit SHAs are high-cardinality). Part of task run
version skew protection (TRI-12998).
## Summary
Move tree selection onto semantic tree items and use native expansion
buttons.
Dashboard and story tree rows now share mouse and keyboard selection
through `getNodeProps`. Expand and collapse affordances are named
buttons instead of clickable layout elements.
Base: [#4700](https://github.com/triggerdotdev/trigger.dev/pull/4700)
## Summary
Use native controls for sortable columns and selectable prompt versions.
Table headers keep filter actions separate from sort buttons, prompt
version rows expose pressed state, and a redundant deployment click
interceptor is removed.
Base: [#4699](https://github.com/triggerdotdev/trigger.dev/pull/4699)
## Summary
Make time-filter mode selection keyboard accessible.
Duration and exact-range modes now use native pressed buttons. Nested
date, duration, and quick-select controls no longer depend on click
propagation blockers.
Base: [#4698](https://github.com/triggerdotdev/trigger.dev/pull/4698)
## Summary
Replace mouse-only dashboard actions with native buttons.
Copy, remove, and stop-generation controls now expose keyboard focus and
accessible names. Hover-revealed actions remain mounted so keyboard
users can discover them, and a decorative clipboard icon no longer
captures clicks.
Base: [#4697](https://github.com/triggerdotdev/trigger.dev/pull/4697)
## Summary
Use native label and checkbox behavior for `CheckboxWithLabel` and
enforce `jsx-a11y/no-noninteractive-element-interactions`.
The component no longer simulates checkbox activation with click
handlers on non-interactive wrappers. Native change events now drive the
controlled checked state.
Base: [#4696](https://github.com/triggerdotdev/trigger.dev/pull/4696)
## Summary
Require accessible names for dashboard controls.
Filter menu action items and chart color controls now expose explicit
names. The chart legend action uses a native button, while lint depth
and spacer-cell configuration match the rendered control structure.
Base: [#4695](https://github.com/triggerdotdev/trigger.dev/pull/4695)
## Summary
Finish associating dashboard form labels with their controls and enforce
`jsx-a11y/label-has-associated-control`.
Repeated data store dialogs use unique generated IDs, story controls and
notification filters have explicit associations, and display-only status
text no longer uses label elements.
Base: [#4694](https://github.com/triggerdotdev/trigger.dev/pull/4694)
## Summary
Associate internal model administration labels with their form controls.
The model editor, creator, and tester now use explicit `htmlFor` and
`id` pairs. Section titles that do not label controls now use headings
instead of label elements.
Base: [#4693](https://github.com/triggerdotdev/trigger.dev/pull/4693)
## Summary
Enable foundational JSX accessibility checks for image text alternatives
and valid ARIA roles.
The avatar color picker now has an explicit accessible name and
decorative image alternative. Dashboard chat styling props no longer
reuse the reserved DOM `role` name.
Base: [#4692](https://github.com/triggerdotdev/trigger.dev/pull/4692)
## Summary
Add explicit types to native dashboard buttons and enforce
`react/button-has-type`.
This prevents action buttons from accidentally submitting a surrounding
form. Shared button primitives retain their caller-selected submit and
reset semantics with documented lint exceptions.
Base: [#4691](https://github.com/triggerdotdev/trigger.dev/pull/4691)
## Summary
Remove redundant React fragments from the dashboard and enforce
`react/jsx-no-useless-fragment`.
The cleanup returns existing nodes, arrays, and empty states directly
without adding wrapper elements.
Base: [#4689](https://github.com/triggerdotdev/trigger.dev/pull/4689)
## Summary
Keep component and renderer identities stable across dashboard renders.
Inline icon components, chart renderers, table cells, and select render
callbacks now use module-level implementations. Oxlint enforces the
pattern across the dashboard.
Base: [#4688](https://github.com/triggerdotdev/trigger.dev/pull/4688)
## Summary
Enforce stable React hook ordering in the dashboard and React hooks
package.
Conditional hook calls now keep a consistent order, and overloaded
realtime stream arguments are resolved before entering the shared hook
implementation.
Base: `main`
`@grpc/grpc-js` sat at 1.12.6 in the lockfile. `dockerode` is the only
consumer and already declares `^1.11.1`, so a scoped override is enough:
```json
"@grpc/grpc-js@>=1.12.0 <1.12.7": "1.12.7"
```
Pinned exactly to stay on the 1.12 line; a caret would pull 1.14.x.
## Summary
Listing schedules could block the event loop for seconds. A page of 100
timezone-aware schedules spent over two seconds on cron arithmetic
alone, after the database work was already done, which stalls every
other request on that process. The same page now resolves in tens of
milliseconds.
## Root cause and fix
`cron-parser` walks the calendar unit by unit, and under a named
timezone every step goes through luxon. Parsing an expression is cheap
(single-digit microseconds); *stepping* it is not, ranging from a couple
of hundred microseconds for a common expression to several milliseconds
for a sparse one like `0 0 29 2 *`. The presenter did three independent
walks per row, one backwards for "last run" and two forwards (re-parsing
each time) for the next run and the occurrence after it. At 100 rows
that is 300 calendar walks in one uninterrupted tick.
Run times now resolve for the whole page in one pass, in a new
`resolveScheduleTimings` that takes plain values rather than Prisma rows
so it can be tested and benchmarked on its own.
- **Nominal times are cached per `(cron, timezone)`** against a single
`now` pinned for the batch, so cost scales with the number of distinct
expressions instead of the number of rows. Rows in one response also
stop disagreeing about the current time.
- **The backwards walk is opt-in.** It is the most expensive of the
three and only the dashboard renders the column; the public API never
returned it at all.
- **Windowless schedules take one step instead of two.** The second step
only measures the interval to the following occurrence, and that
interval reaches the result solely through `min(intervalMs,
max(MINIMUM_SCHEDULE_RANGE_MS, windowMs))`. With no window `windowMs` is
0, and `CronPattern` rejects expressions with a seconds field, so
occurrences are always at least `MINIMUM_SCHEDULE_RANGE_MS` apart and
that `min` can never bind. It is also the costlier step, since it walks
a whole period rather than the remainder of the current one.
- **`nextScheduledTimestamps` steps one parsed expression** instead of
re-parsing per step, which also helps the single-schedule callers.
Behaviour is unchanged, error semantics included: a malformed expression
still throws for the next run and still degrades to an undefined last
run.
## Verification
Measured inside a real request against a live environment, 100
schedules: sparse expressions went from 2250-2652 ms to 23-30 ms, and
five distinct timezone expressions from 463-500 ms to 9.7-10.6 ms.
The new suite checks the optimized code against an inline copy of the
previous implementation across eleven cron and timezone combinations
plus five DST transitions, so the rewrite is verified as
behaviour-preserving rather than just faster. Separate tests pin the
invariant the single-step path depends on, so if sub-minute crons are
ever allowed they fail loudly instead of the timings quietly going
wrong.
Worth knowing for later: `cron-parser` v5 is a much faster rewrite on
exactly this workload (`prev()` under a timezone drops from roughly 2700
to 60 microseconds), but it is a breaking API change across several call
sites including the schedule engine, so it belongs on its own. The
differential test added here is the tool to de-risk it.
## Summary
Allow the logs search schema migration to run on ClickHouse versions
that require text index options to be literals.
## Root cause
The text index declared `lowerUTF8(search_text)` as a preprocessor
option. Some ClickHouse versions reject that column expression while
parsing index settings. The projected `search_text` is already
normalized to lowercase before insertion, so removing the redundant
preprocessor preserves search behavior.
Verified with the task events search integration tests.
## What
The one-off worker container boot is billed to whichever test resolves
the fixture first. This moves it into a `beforeAll` with its own
timeout.
## Why
vitest runs the fixture chain *inside* the test timer:
```js
// @vitest/runner 4.1.7
setFn(task, withTimeout(...withFixtures(handler)..., timeout, ...))
```
There is no `fixtureTimeout`. So booting Postgres (plus `CREATE
DATABASE`, schema push, ClickHouse and Redis) lands on the first test
and consumes a budget sized for test work.
That is why losing the image pre-pull on fork PRs was fatal rather than
merely slower: the extra ~10s crossed the 60s cap. Since fork time is
roughly internal + 10s and forks exceed 60s, internal runs were already
clearing that cap by under 10s — a latent flake regardless of forks.
## How
`withWarmup` wraps each fixture family and lazily registers a
`beforeAll` on first touch, with its own generous timeout. Registration
is lazy so only files that actually use a family pay for it —
`@internal/testcontainers` is imported by hundreds of test files, many
of which only need Redis. It registers once per file, since `isolate`
gives each file a fresh module registry.
Eight families are wrapped. `isolatedRedisTest`,
`replicationContainerTest` and `postgresAndRedisTest` are deliberately
untouched: they use per-test containers by design, so there is no
one-off boot to hoist.
No test file or CI changes, and it applies to every package using these
fixtures.
## Verification
Proven by mutation. `src/warmup.test.ts` runs container tests under a
deliberately tight cap:
| | Result |
| --- | --- |
| with the warm-up | passes |
| warm-up neutered | fails, `Test timed out` |
It is kept as a regression test — without it, unwrapping a fixture would
break nothing visibly.
`triggerFailedTask.call.test.ts`, one of the five shard casualties,
passes locally in 20.4s.
## Also here
`@internal/testcontainers` had no `test` script, so `turbo run test
--filter "@internal/*"` skipped the package and its existing
`heteroDedicated.test.ts` never ran in CI. Adding the script (matching
the sibling packages') runs both files; verified green through turbo
exactly as CI invokes it.
## What
Three corrections to the pre-pull lists, each verified against what the
suites actually use.
## Changes
**`ryuk:0.11.0` -> `0.14.0`** in `e2e-webapp.yml` and
`e2e-webapp-auth-full.yml`. The installed testcontainers hardcodes the
image it starts:
```js
// testcontainers@11.14.0 build/reaper/reaper.js
: ImageName.fromString("testcontainers/ryuk:0.14.0").string;
```
So those two lines were pre-pulling an image nothing starts, and the one
actually used was never pre-pulled. The other three workflows already
say 0.14.0.
**`postgres:17` added** to `unit-tests-webapp.yml`. The webapp suite
references `docker.io/postgres:17` across 10 files but only
`postgres:14` was pre-pulled. `unit-tests-internal.yml` already pulls
both.
**Electric pinned to its digest** in `unit-tests-webapp.yml`. The tests
run `electricsql/electric:1.2.4@sha256:20da...` while the pre-pull asked
for the bare tag, so the pre-pull did not necessarily populate the
manifest the tests then request.
## Not changed
The otel collector and s2 images are pulled by other workflows but are
not used by the webapp suite, so they are deliberately not added here.
`postgresAndRedisTest` uses per-test containers by design and needs
nothing pre-pulled.
## What
The `Pre-pull testcontainer images` step is gated on
`env.DOCKERHUB_USERNAME`. Fork PRs receive no repository secrets, so
that variable is empty and the step is skipped along with the DockerHub
login it was grouped with.
## Why
With the pre-pull skipped, testcontainers pulls images lazily — inside
the first test that resolves the fixture, against that test's
`testTimeout`. On PR #4534 that pushed five webapp shards past their 60s
cap across three runs, each failing as `Test timed out in 60000ms` while
42 of 43 files in the shard passed.
Measured cost of the missing pre-pull, comparing the delta from vitest
start to the first container fixture on the same runner class:
| Run | Delta |
| --- | --- |
| internal x2 | +139.9s, +139.4s |
| fork x2 | +149.7s, +149.4s |
A 10.0s penalty, bimodal to within 0.3s.
Note the pulls themselves succeed anonymously — there are no rate-limit
errors in any of the failing logs. Only the login needs credentials, so
the pre-pull can run unconditionally.
## Scope
Removes the `if:` from the pre-pull step in all five workflows that have
one. The DockerHub login stays gated, since it genuinely needs secrets.
## Summary
Memoize shared context values so provider renders do not unnecessarily
rerender every consumer. Oxlint now enforces this pattern for the rest
of the dashboard.
Base: [#4677](https://github.com/triggerdotdev/trigger.dev/pull/4677)
## Summary
Enable lint rules that prefer direct iteration and concise function
callback types.
The existing code now uses direct iteration where no index is needed,
and callback contracts use function types consistently.
Base: [#4675](https://github.com/triggerdotdev/trigger.dev/pull/4675)
## Summary
Enable JSX cleanup rules for shorthand fragments and self-closing
components.
The existing JSX is automatically simplified, and future components will
follow the same concise form.
Base: [#4673](https://github.com/triggerdotdev/trigger.dev/pull/4673)
## Summary
Enable small cleanup rules for redundant boolean expressions, object
ownership checks, assignments, and object construction.
The existing call sites now use the simpler equivalent forms, keeping
future code consistent without changing behavior.
Base: [#4672](https://github.com/triggerdotdev/trigger.dev/pull/4672)
## Summary
Enable additional lint rules that catch unsafe optional-chain
assertions, inherited-property iteration, anonymous symbols, and unsafe
external links.
The existing violations now use explicit values and own-property checks,
so the rules can prevent those patterns from returning.
Adds an optional priority class for run pods.
```
KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME
```
When set, the value is applied as `priorityClassName` on the run pod
spec. When unset, pods are created exactly as before.
Off by default, and inert unless set. It sits beside the existing
`KUBERNETES_SCHEDULER_NAME` option and follows the same conditional
shape:
```ts
...(env.KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME
? { priorityClassName: env.KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME }
: {}),
```
## Verification
`typecheck --filter supervisor`, `format` and `lint` clean. No changeset
or `.server-changes/` note: off by default, no user-visible behaviour
change.
## What
Runs that carry no external trace context (schedules, task-to-task
triggers) fall back to a trace id generated once in the [`TracingSDK`
constructor](https://github.com/triggerdotdev/trigger.dev/blob/main/packages/core/src/v3/otel/tracingSDK.ts#L165).
With `experimental_processKeepAlive` the SDK outlives the run, so every
run on a warm process is exported to the external OTLP endpoint under
that one id.
Across our production traces, 80.3% contained spans from more than one
run, worst case 25. Per-run cost and latency attribution is unusable as
a result. This is the same warm-start hazard c043c4a6a fixed for the
external-context path, which left the fallback captured at construction.
## How
`FallbackExternalTraceIds` hands out one id per internal trace, shared
by the span and log wrappers so a run's spans and logs agree.
The id is keyed off the record's own internal trace id rather than
ambient state at export time, because batch processors drain
asynchronously and a run's records routinely export after the next run
has started. The map is bounded and evicts least-recently-used, so a run
that is still exporting can't lose its id.
Granularity follows the internal trace, so a run and the runs it
triggers stay on one trace.
**Risk:** the wrappers only exist when `exporters` / `logExporters` are
configured, so deployments that don't export externally are untouched.
Nothing outside `tracingSDK.ts` changes.
**Known gap (pre-existing):** sampling and id selection still branch on
ambient `getExternalTraceContext()`, so records draining across a run
boundary in mixed mode are misplaced in both directions. It can't use
the approach here — the external id comes from the run's incoming
`traceparent`, which isn't carried on the record — so closing it means
capturing `internalTraceId -> external context` in a span processor.
Happy to follow up separately.
---
## Testing
`packages/core` suite passes. `pnpm run format` and `pnpm run lint:fix`
produce no diff.
Six cases in `externalSpanExporterWrapper.test.ts`, each
mutation-checked rather than just observed passing: one id per run,
stability within a run, correct id when records drain after the next run
started (spans and logs together), external export stays off when
unconfigured, retention of a run still exporting while the map churns,
and the bound itself.
**CI:** the five failing `webapp` shards are the ones containing
`containerTest` suites. Fork PRs receive no repository secrets, so
`unit-tests-webapp.yml` skips the DockerHub login and the image pre-pull
(both gated on `env.DOCKERHUB_USERNAME`) and the container tests time
out at 60s. Same five shards across five runs, every failure a 60s
timeout, and those shards pass on internal PRs. Happy to be corrected if
you can run them with secrets available.
---
## Changelog
Unrelated runs are no longer merged into a single trace in your external
observability tool when they happen to execute on the same warm worker
process. A run and the runs it triggers still share one trace, so a run
tree stays together.
---
## Screenshots
_n/a_
---
_Supersedes #4526 (auto-closed before I was vouched) and #4533 (opened
ready rather than as a draft). GitHub won't reopen either._
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Iss <74388823+isshaddad@users.noreply.github.com>
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
<!-- ccr-slack-attribution -->
_Requested by **Matt Aitken** · [Slack
thread](https://triggerdotdev.slack.com/archives/C0BKB98B84W/p1787045331358929)_
Copy-only reword of the banner shown to org admins who have not set a
billing limit yet.
**Before** — the banner read "Protect your organization from unexpected
usage spikes." with a button labelled "Configure billing limit".
**After** — it reads "Add a billing limit to your account to prevent
overspending" with a button labelled "Billing limit settings".
The new wording names the action up front and matches the destination it
sends you to, so the banner reads as a settings link rather than a
one-off setup step.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Formatting and linting pass (`oxfmt --check`, `oxlint`). No tests or
snapshots assert this copy. The change is two string literals in one
component, with no behaviour attached.
---
## Changelog
Reworded the billing-limit banner for organizations without a limit
configured, and relabelled its button to "Billing limit settings".
---
## How
Both strings live in `NoLimitConfiguredBanner` in
`apps/webapp/app/components/billing/OrgBanner.tsx`: the heading is the
`canManageBillingLimits` branch of the banner's children, and the label
is the `<span>` inside the `LinkButton`. Only those two literals
changed. The button still points at `v3BillingLimitsPath(organization)`
(`/orgs/{slug}/settings/billing-limits`), so routing, permissions and
the non-admin variant of the message are untouched.
Co-authored-by: Claude <noreply@anthropic.com>
Adds KUBERNETES_RUNNER_SECURITY_CONTEXT (off | baseline | restricted), selecting how constrained the run container is.
baseline drops the capability bounding set and blocks privilege escalation. restricted additionally pins the container to a non-root uid, chosen by runtime so bun images get their own.
Default is off, so this is inert on merge.
## Summary
A run triggered with a `ttl` could get permanently stuck showing as
queued. If the run started executing and was then requeued after a
failure (a stalled heartbeat, a worker dying mid-run) once its TTL had
already elapsed, the next dequeue pass silently dropped it from every
queue structure. The run stayed QUEUED in the database forever, and
nothing (dequeue, the TTL consumer, queue repair) could ever see it
again.
## Root cause
Enqueue registers a TTL entry for the TTL consumer, and the first
dequeue removes it ("the run is executing, not expired"). A nack rewrote
the message preserving the original `ttlExpiresAt` without
re-registering that entry. The next dequeue pass then took the
expired-TTL branch: remove the run from the queue sorted sets and leave
the message for the TTL consumer to finalize. But the consumer's entry
was gone, so nothing ever finalized the run.
The fix has two halves:
- `nackMessage` drops `ttlExpiresAt` from the rewritten message. TTL
only applies to runs that have never been dequeued (the same contract as
`includeTtl` on re-enqueues), so a requeued run stays dequeuable and is
never expired by its original deadline.
- The dequeue expired-TTL branches now (re-)register the TTL entry
instead of assuming it exists, so any message still carrying a lapsed
`ttlExpiresAt` with no TTL entry (including ones written before this
fix) finalizes as EXPIRED instead of orphaning.
## Verification
New engine test suite `ttlNackRequeue.test.ts` (testcontainers, real
Redis and Postgres). All four tests fail before the fix and pass after:
- a heartbeat-stalled EXECUTING run with a lapsed TTL is requeued and
dequeued again instead of orphaned (the full production failure chain)
- requeue-after-failure strips `ttlExpiresAt` so later dequeues do not
treat the run as expired
- a lapsed-TTL message whose TTL entry is missing is re-registered by
dequeue and finalized as EXPIRED, for both plain and concurrency-key
queues
Also ran the existing ttl, heartbeats, dequeuing and attemptFailures
engine suites plus the full run-queue suite (149 tests) against the
change.
Adds two optional env vars that rewrite the registry host of run pod images at pod creation, so a supervisor can pull from a registry in its own region. Off by default and inert unless both are set. Exact host-prefix matching, so look-alike hosts pass through untouched.
dependabot[bot]