## Summary
Keep component and renderer identities stable across dashboard renders.
Inline icon components, chart renderers, table cells, and select render
callbacks now use module-level implementations. Oxlint enforces the
pattern across the dashboard.
Base: [#4688](https://github.com/triggerdotdev/trigger.dev/pull/4688)
## Summary
Enforce stable React hook ordering in the dashboard and React hooks
package.
Conditional hook calls now keep a consistent order, and overloaded
realtime stream arguments are resolved before entering the shared hook
implementation.
Base: `main`
## Summary
Listing schedules could block the event loop for seconds. A page of 100
timezone-aware schedules spent over two seconds on cron arithmetic
alone, after the database work was already done, which stalls every
other request on that process. The same page now resolves in tens of
milliseconds.
## Root cause and fix
`cron-parser` walks the calendar unit by unit, and under a named
timezone every step goes through luxon. Parsing an expression is cheap
(single-digit microseconds); *stepping* it is not, ranging from a couple
of hundred microseconds for a common expression to several milliseconds
for a sparse one like `0 0 29 2 *`. The presenter did three independent
walks per row, one backwards for "last run" and two forwards (re-parsing
each time) for the next run and the occurrence after it. At 100 rows
that is 300 calendar walks in one uninterrupted tick.
Run times now resolve for the whole page in one pass, in a new
`resolveScheduleTimings` that takes plain values rather than Prisma rows
so it can be tested and benchmarked on its own.
- **Nominal times are cached per `(cron, timezone)`** against a single
`now` pinned for the batch, so cost scales with the number of distinct
expressions instead of the number of rows. Rows in one response also
stop disagreeing about the current time.
- **The backwards walk is opt-in.** It is the most expensive of the
three and only the dashboard renders the column; the public API never
returned it at all.
- **Windowless schedules take one step instead of two.** The second step
only measures the interval to the following occurrence, and that
interval reaches the result solely through `min(intervalMs,
max(MINIMUM_SCHEDULE_RANGE_MS, windowMs))`. With no window `windowMs` is
0, and `CronPattern` rejects expressions with a seconds field, so
occurrences are always at least `MINIMUM_SCHEDULE_RANGE_MS` apart and
that `min` can never bind. It is also the costlier step, since it walks
a whole period rather than the remainder of the current one.
- **`nextScheduledTimestamps` steps one parsed expression** instead of
re-parsing per step, which also helps the single-schedule callers.
Behaviour is unchanged, error semantics included: a malformed expression
still throws for the next run and still degrades to an undefined last
run.
## Verification
Measured inside a real request against a live environment, 100
schedules: sparse expressions went from 2250-2652 ms to 23-30 ms, and
five distinct timezone expressions from 463-500 ms to 9.7-10.6 ms.
The new suite checks the optimized code against an inline copy of the
previous implementation across eleven cron and timezone combinations
plus five DST transitions, so the rewrite is verified as
behaviour-preserving rather than just faster. Separate tests pin the
invariant the single-step path depends on, so if sub-minute crons are
ever allowed they fail loudly instead of the timings quietly going
wrong.
Worth knowing for later: `cron-parser` v5 is a much faster rewrite on
exactly this workload (`prev()` under a timezone drops from roughly 2700
to 60 microseconds), but it is a breaking API change across several call
sites including the schedule engine, so it belongs on its own. The
differential test added here is the tool to de-risk it.
## Summary
Memoize shared context values so provider renders do not unnecessarily
rerender every consumer. Oxlint now enforces this pattern for the rest
of the dashboard.
Base: [#4677](https://github.com/triggerdotdev/trigger.dev/pull/4677)
## Summary
Enable lint rules that prefer direct iteration and concise function
callback types.
The existing code now uses direct iteration where no index is needed,
and callback contracts use function types consistently.
Base: [#4675](https://github.com/triggerdotdev/trigger.dev/pull/4675)
## Summary
Enable JSX cleanup rules for shorthand fragments and self-closing
components.
The existing JSX is automatically simplified, and future components will
follow the same concise form.
Base: [#4673](https://github.com/triggerdotdev/trigger.dev/pull/4673)
## Summary
Enable small cleanup rules for redundant boolean expressions, object
ownership checks, assignments, and object construction.
The existing call sites now use the simpler equivalent forms, keeping
future code consistent without changing behavior.
Base: [#4672](https://github.com/triggerdotdev/trigger.dev/pull/4672)
## Summary
Enable additional lint rules that catch unsafe optional-chain
assertions, inherited-property iteration, anonymous symbols, and unsafe
external links.
The existing violations now use explicit values and own-property checks,
so the rules can prevent those patterns from returning.
Adds an optional priority class for run pods.
```
KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME
```
When set, the value is applied as `priorityClassName` on the run pod
spec. When unset, pods are created exactly as before.
Off by default, and inert unless set. It sits beside the existing
`KUBERNETES_SCHEDULER_NAME` option and follows the same conditional
shape:
```ts
...(env.KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME
? { priorityClassName: env.KUBERNETES_RUN_POD_PRIORITY_CLASS_NAME }
: {}),
```
## Verification
`typecheck --filter supervisor`, `format` and `lint` clean. No changeset
or `.server-changes/` note: off by default, no user-visible behaviour
change.
<!-- ccr-slack-attribution -->
_Requested by **Matt Aitken** · [Slack
thread](https://triggerdotdev.slack.com/archives/C0BKB98B84W/p1787045331358929)_
Copy-only reword of the banner shown to org admins who have not set a
billing limit yet.
**Before** — the banner read "Protect your organization from unexpected
usage spikes." with a button labelled "Configure billing limit".
**After** — it reads "Add a billing limit to your account to prevent
overspending" with a button labelled "Billing limit settings".
The new wording names the action up front and matches the destination it
sends you to, so the banner reads as a settings link rather than a
one-off setup step.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Formatting and linting pass (`oxfmt --check`, `oxlint`). No tests or
snapshots assert this copy. The change is two string literals in one
component, with no behaviour attached.
---
## Changelog
Reworded the billing-limit banner for organizations without a limit
configured, and relabelled its button to "Billing limit settings".
---
## How
Both strings live in `NoLimitConfiguredBanner` in
`apps/webapp/app/components/billing/OrgBanner.tsx`: the heading is the
`canManageBillingLimits` branch of the banner's children, and the label
is the `<span>` inside the `LinkButton`. Only those two literals
changed. The button still points at `v3BillingLimitsPath(organization)`
(`/orgs/{slug}/settings/billing-limits`), so routing, permissions and
the non-admin variant of the message are untouched.
Co-authored-by: Claude <noreply@anthropic.com>
Adds KUBERNETES_RUNNER_SECURITY_CONTEXT (off | baseline | restricted), selecting how constrained the run container is.
baseline drops the capability bounding set and blocks privilege escalation. restricted additionally pins the container to a non-root uid, chosen by runtime so bun images get their own.
Default is off, so this is inert on merge.
Adds two optional env vars that rewrite the registry host of run pod images at pod creation, so a supervisor can pull from a registry in its own region. Off by default and inert unless both are set. Exact host-prefix matching, so look-alike hosts pass through untouched.
Replaces the hardcoded runner seccomp profile path with KUBERNETES_RUNNER_SECCOMP_PROFILE_PATH, and the node-24-only condition with KUBERNETES_RUNNER_SECCOMP_PROFILE_RUNTIMES (none | node-24-plus | all).
Both defaults reproduce current behaviour, so this is inert on merge. Widening the scope or turning attachment off becomes a config change rather than a deploy.
The supervisor now supports routing an organization's runs to specific
nodes. `KUBERNETES_ORG_PLACEMENT_OVERRIDES` takes JSON keyed by the
internal org ID, adding node selector entries and tolerations to that
org's run pods, e.g. to route an org onto a dedicated, tainted node
pool:
```json
{"<orgId>": {"nodeSelector": {"pool": "dedicated"}, "tolerations": "dedicated=runs:NoSchedule"}}
```
The node selector merges over the defaults (the override wins on key
collision, with a warning logged). Tolerations append to the existing
runner and scheduled-run sets. Overrides are validated at startup
similar to `KUBERNETES_RUNNER_TOLERATIONS`.
Exposed in the Helm chart as
`supervisor.config.kubernetes.orgPlacementOverrides`, where tolerations
can also be given as a list.
## What & why
The dashboard agent can now run its model calls through AWS Bedrock
instead of the direct Anthropic API, chosen by a single env switch. It's
**off by default** (`DASHBOARD_AGENT_MODEL_PROVIDER` unset ⇒
`anthropic`), so merging changes nothing at runtime — the Bedrock path
is a dormant branch until an operator sets the switch and AWS config.
The default Anthropic path is byte-for-byte unchanged.
This also carries a related tenant-isolation hardening for the agent's
delegated token (kept together deliberately — both land the agent on
Bedrock for HIPAA readiness). Refs: TRI-13251, TRI-11032.
## What's inside
**Provider seam** —
`internal-packages/dashboard-agent/src/model-provider.ts`: the registry
now holds both `anthropic` and `bedrock`; `resolveDashboardAgentModel()`
maps the canonical `"anthropic:<id>"` strings the managed prompts carry
to the active provider, and the cache-breakpoint helpers emit the active
provider's shape — Anthropic `cacheControl` vs Bedrock `cachePoint`.
Managed prompt strings stay canonical, so stored prompts don't change
meaning. Unmapped model ids throw rather than shipping a guaranteed-404
profile. All agent, watch, compaction and title callsites route through
the resolver; the `dashboardAgentModelKey` locals override (test mock
injection) is preserved.
**Cache telemetry** — `step-cache.ts`: cache token usage is read from
the active provider (Anthropic reports it on provider metadata; Bedrock
reports the write on metadata and the read via standard usage), so
`gen_ai.usage.cache_*` is populated on both. This also fixes a latent
ordering bug where step attributes could null-overwrite the prompt-cache
read count.
**Webapp callsites** — `dashboardAgentHeadStart.server.ts` and the
head-start route resolve the model and the cache breakpoint through the
same seam, so the warm-up prefix and the following turn share one
provider. The head-start firing gate is provider-aware: on Bedrock it
gates on `AWS_REGION` and lets the SDK resolve credentials (IAM role /
static keys / session token / bearer), so a role-based deploy still
warms; on Anthropic it stays `Boolean(ANTHROPIC_API_KEY)`.
`app/env.server.ts` gains the optional AWS vars and validates
`DASHBOARD_AGENT_MODEL_PROVIDER`. `ANTHROPIC_API_KEY` is untouched and
not required on a Bedrock deploy.
**Tenant-isolation hardening** —
`internal-packages/rbac/src/fallback.ts`: for a **scoped** context, the
OSS `authenticateUserActor` now applies the same membership floor as the
session path — a delegated user-actor token whose user is not a member
of the scoped org/project is denied (403). Unscoped tokens keep their
prior behavior (no tenant claim, no lookup). The user lookup falls back
replica→primary so replication lag can't spuriously 401 a just-joined
member. Members and admins are unaffected. Previously this invariant
held only through per-route discipline; this makes it structural.
## Enabling Bedrock (later, ops)
- Set `DASHBOARD_AGENT_MODEL_PROVIDER=bedrock` **identically** in both
the webapp and the agent task container — the webapp warms the cache
prefix and the task reads it, so a split would silently miss the cache.
- Set `AWS_REGION` and provide credentials the Bedrock SDK can resolve
(IAM role preferred). For v1 this runs **without** an Anthropic API key.
Note: with no Anthropic key set, rollback is "turn the agent off", not
"unset the switch" (unsetting falls back to the Anthropic provider,
which then has no key).
- Two things to confirm before rollout: the Sonnet inference-profile id
is validated against the SDK's own model-id union but still warrants a
live smoke test; and Bedrock prompt caching for Sonnet is a 5-minute
window (not Anthropic's 1h), so input-token cost rises when flipped.
## Testing
Unit tests cover both provider paths: the provider switch and
per-provider cache shapes, a structural regex asserting Bedrock ids are
real inference profiles (not an echo of the table), the split-metadata
cache telemetry, and real-Postgres RBAC tests — member allowed, scoped
non-member denied (org-only and project-only), missing user → 401, admin
non-member exempt, unscoped success. `typecheck --filter webapp` and the
dashboard-agent + rbac suites pass.
## Summary
When an AI SDK tool call failed inside a run, the span showed up under
the "Errors only" filter but the span inspector gave no hint of what
went wrong. The exception was recorded on the span all along; the
`ai.toolCall` and `ai.embed` inspector views just never rendered span
events. Failed tool call and embedding spans now show the standard error
block (message plus stack trace) below the Input section.
## Root cause
Generic spans render exception span events via the `SpanEvents`
component, but the AI-specific span entities replace the whole panel
with their own layout and dropped the events entirely. The span's events
are now passed into `AIToolCallSpanDetails` and `AIEmbedSpanDetails` and
rendered with the same `SpanEvents` component the generic view uses.
Errored generation spans (`ai.generateText` and friends) use a tabbed
view and still don't surface errors; that needs its own design pass and
is left for a follow-up.
## Summary
`nanoid` was pinned at exactly `3.3.8` in five manifests. Two of those
five never imported it: in `internal-packages/schedule-engine` and
`internal-packages/webhook-engine` the only occurrence of the string
`nanoid` in the entire package was the `package.json` line itself. Both
are removed rather than bumped.
The three that genuinely use it move to `3.3.18`, a version already
present in the tree via `postcss`, so this pulls in nothing new.
| Package | Uses it | Change |
| --- | --- | --- |
| `internal-packages/schedule-engine` | no | removed |
| `internal-packages/webhook-engine` | no | removed |
| `apps/webapp` | yes | `3.3.8` to `3.3.18` |
| `packages/core` | yes | `3.3.8` to `3.3.18` |
| `internal-packages/run-engine` | yes | `3.3.8` to `3.3.18` |
| `packages/redis-worker` | yes | `^5.0.7` to `^5.1.16` |
`redis-worker` is on the 5.x line and is included because its declared
range already permitted a newer release; the lockfile had simply not
re-resolved, leaving it on `5.1.2`.
The unused declarations were found with `pnpm run knip:deps`, which the
repo already ships.
`pnpm run typecheck` passes across all 57 workspaces.
## Summary
`socket.io` was pinned at exactly `4.7.4` in three manifests
(`apps/webapp`, `apps/supervisor`, `packages/core`). That pin capped
`engine.io` at 6.5.4, because 4.7.4 declares `engine.io: ~6.5.2`.
Moving all three pins to `4.8.3` lifts that cap: 4.8.3 declares
`engine.io: ~6.6.0`. The webapp's direct `engine.io` devDependency moves
from `^6.5.4` to `^6.6.7` to match.
These are direct dependencies, so they are bumped in place rather than
forced with an override.
## Result
The tree previously carried two `engine.io` copies. It now carries one:
```
engine.io@6.6.8
└─┬ socket.io@4.8.3
├── @trigger.dev/core (dependencies)
├─┬ react-email
│ └── emails (devDependencies)
├── supervisor (dependencies)
└── webapp (dependencies)
```
`react-email` was already resolving `socket.io@4.8.3` in this same tree,
so that combination was already running here before this change.
## Servers move, clients do not
This bumps `socket.io` (the server) only. `socket.io-client` stays at
`4.7.5` in `packages/core` and `packages/cli-v3`, deliberately: the fix
is server-side, and clients ship inside user deployments, so leaving
them alone keeps the blast radius small. That means a 4.8.3 server will
be talking to 4.7.5 clients indefinitely, which is worth being explicit
about.
That pairing is safe because neither wire protocol changed. Both
versions report the same protocol numbers:
| | 4.7.4 | 4.8.3 |
| --- | --- | --- |
| Socket.IO protocol (`socket.io-parser`) | 5 | 5 |
| Engine.IO protocol (`engine.io-parser`) | 4 | 4 |
The version bump moves `socket.io-parser` 4.2.6 to 4.2.7 and `engine.io`
6.5.4 to 6.6.8, but the protocol constants each exports are unchanged.
The 4.8.0 changes are additive on the client (custom transport
implementations, a `tryAllTransports` option) and bug fixes on the
server.
Verified rather than assumed, with a cross-version matrix covering both
transports and both directions:
```
PASS server 4.8.3 <- client 4.7.5 websocket / polling
PASS server 4.8.3 <- client 4.8.3 websocket / polling
PASS server 4.7.4 <- client 4.7.5 websocket / polling
PASS server 4.7.4 <- client 4.8.3 websocket / polling
```
Each case exercised connect, a server-initiated emit, `emitWithAck`,
room join, room broadcast, and a binary payload. Compatibility holds in
both directions, so there is no upgrade-ordering requirement between
server and client.
`pnpm run typecheck` passes across all 57 workspaces.
Stacked on #4634.
## Summary
`npm-run-all` has had no release since 4.1.5 in 2018, and pnpm now
covers the one thing we used it for. The webapp's `build` script was its
only consumer anywhere in the repo, so the dependency goes away
entirely.
`run-s build:**` becomes an explicit chain:
```
pnpm run build:remix && pnpm run build:server && pnpm run build:otlpworker && pnpm run build:sentry && pnpm run upload:sourcemaps
```
## Why this shape
I compared both forms side by side against the real `run-s` before
swapping:
| Behaviour | `run-s build:**` | explicit chain |
| --- | --- | --- |
| Scripts selected | remix, server, otlpworker, sentry | identical |
| Order | declaration order | identical |
| `upload:sourcemaps` matched by the glob | no | no |
| Second script fails | aborts, third never runs | identical |
| Exit code on failure | `1` | `1` |
`pnpm run --sequential "/^build:/"` was the closer-looking option, but
it keeps running scripts after one fails, so it is not a faithful
replacement.
The one thing given up is that `build:**` automatically picked up any
new `build:*` script, where the chain has to be edited. With four
entries that felt like the better trade.
`pnpm run build --filter webapp` passes end to end locally, all five
steps in order.
## Summary
Following a link straight into an organization's settings (for example
the usage limit link in a billing email) and then clicking "Back to app"
took you to `/`, which resolves to whichever organization you last had
selected, not the one whose settings you were looking at. The button now
links to the organization in the URL, so you land back in the org you
came from.
The org index route already redirects to the best project in that org,
so the destination is unchanged apart from being the right org.
Account settings still links to `/`, since that page is not org scoped
and has no org to return to.
## Summary
The server half of hosted webhooks: the public ingress endpoint,
signature verification, the delivery pipeline (Postgres partitioned
storage + ClickHouse for ordering), the in-app partition manager, the
HTTP API, and the dashboard (Deliveries, Endpoints, and the in-app test
console).
The public SDK and docs half is #4537. That PR carries the user-facing
API (`webhook()`, `chat.event` / `chat.channels`, the
`@trigger.dev/slack` connector) and builds on the shared
`@trigger.dev/core` schemas that ship here.
## Shipping behind a flag
A `WEBHOOK_ENABLED` env var (default off) gates the public ingress route
and the engine worker plus partition cron, so merging and deploying this
changes nothing in production until it is flipped on per environment.
The dashboard is separately gated per org by the `hasWebhooksAccess`
feature flag.
## Note on packages
This PR includes the `@trigger.dev/core` schema additions the server
compiles against, but carries no changeset. Core is not consumed
independently of the SDK, so it is released together with the SDK via
#4537. Keeping its changeset off `main` means no release cut from `main`
publishes it early.
## What
Makes two transaction-resilience behaviors real and env-var
configurable, defaults set to the good values, so we can tune during and
after the Aug 15 database patch window without a redeploy:
- **maxWait 2s → 10s** (TRI-12982): how long Prisma waits to borrow a
connection before it can `BEGIN`. A restart freeze holds the pool full,
and the only thing that errored was transaction starts giving up at 2s.
- **Retry transaction-start P2028-at-acquisition** (TRI-12984): when
Prisma can't borrow a connection within `maxWait` it raises P2028
(`Unable to start a transaction in the given time`) and **no SQL ran**,
so retrying is safe. Scoped narrowly: only that error (never P2024
pool-exhaustion), 2 attempts, jittered backoff, and a token-bucket
budget so a mass freeze can't amplify into a retry storm.
## Env vars (`DATABASE_*` convention)
Generic defaults:
| var | default |
|---|---|
| `DATABASE_TRANSACTION_MAX_WAIT_MS` | `10000` |
| `DATABASE_TRANSACTION_START_RETRY_ENABLED` | `true` (kill switch) |
| `DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS` | `2` |
| `DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS` | `50` |
| `DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS` | `250` |
| `DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC` | `50` |
| `DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST` | `100` |
Per-writer-pool overrides, each falling back to the generic when unset
(same pattern as the per-client pool/connect-timeout work):
`RUN_OPS_DATABASE_TRANSACTION_*` and
`RUN_OPS_LEGACY_DATABASE_TRANSACTION_*` (all 7 knobs each). Transactions
only open on writer pools, so those are the only pools with their own
knobs. Each pool gets its **own** token bucket, so a storm on one pool
can't drain another's retry budget.
## Design
- The retry primitives live in `internal-packages/database` and never
read `process.env` (IoC): a P2028-at-acquisition classifier, a
`TokenBucketRetryBudget`, and `withTransactionStartRetry`, folded into
the `$transaction` helper via a new `startRetry` option. Config is
resolved at the app boundary and threaded in.
- The `$transaction` helper is the chokepoint (wraps the whole
transaction), not the per-statement `$allOperations` extension.
- The run engine's writes go through `PostgresRunStore`'s own
`.$transaction(...)`, not the webapp helper, so both the helper and the
two `PostgresRunStore` sites apply maxWait + retry (sharing the per-pool
config). Builds on the `options?: { timeout, maxWait }` seam added in
#4514.
- Webapp `$transaction` call sites get the default `maxWait` + retry
injected at one merge point, so no call site needed editing.
## Evidence
- Unit red/green in `internal-packages/database`: reverting the helper
wiring turned the acquisition-retry test red (`Unable to start a
transaction in the given time`), re-applying it green. Full package
suite 25/25. Covers: classifier (P2028-acq yes, P2024 no, in-tx P2028
no), retry (retry-then-succeed, no-retry P2024, stop at maxAttempts,
disabled, budget-exhausted, jitter bounds), token bucket, and
`$transaction` wiring.
- Typecheck clean: webapp, run-store, run-engine.
- Full-stack run: bounded queue-ay pass (15 projects, real dev runs
through the run-engine `PostgresRunStore` transaction path). 13 pass;
the 2 failures are one documented known-failure and one
stale-worker-state flake that passes 2/2 with this change active on a
fresh app.
- Boots cleanly with per-pool overrides set.
## Configuration & rollout
Ship **inert** first (zero behavior change), then flip to the good
values **live via env** — no redeploy needed for either.
### Inert — behaves exactly as today
```
DATABASE_TRANSACTION_MAX_WAIT_MS=2000 # Prisma's built-in default (change defaults to 10000)
DATABASE_TRANSACTION_START_RETRY_ENABLED=false # disable the new retry entirely
```
`maxWait=2000` is what every path used before (Prisma's default; the
run-store sites and the helper passed no maxWait). `retry=false`
short-circuits `withTransactionStartRetry` to a single run and makes the
serialization-retry exclusion a no-op. Verified on the pooler-freeze
rig: identical fail-fast P2028 at ~2003ms with zero retries —
byte-for-byte current behavior, across all pools.
### Production ("good") — the baked defaults
Rely on defaults (nothing to set) or set explicitly:
```
DATABASE_TRANSACTION_MAX_WAIT_MS=10000
DATABASE_TRANSACTION_START_RETRY_ENABLED=true
DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS=3 # 3 attempts (2 retries); ~30s acquisition tolerance covers a ~20-25s freeze
DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS=50
DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS=250
DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC=50
DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST=100
```
Per-pool overrides `RUN_OPS_DATABASE_TRANSACTION_*` and
`RUN_OPS_LEGACY_DATABASE_TRANSACTION_*` (all seven knobs each) are
optional and fall back to the generic set — not needed for v1; the
generic set covers the control-plane, run-ops, and run-ops-legacy writer
pools. Readers open no transactions and take nothing.
**Guardrail:** the retry only engages when a pool's `pool_timeout` >
`maxWait`. Prod is fine (`DATABASE_POOL_TIMEOUT=60` >> 10). Do not set
any writer pool's `pool_timeout` at or under `maxWait`, or saturation
failures flip from retryable P2028 to non-retryable P2024 and the retry
silently stops helping.
### Rollback
Env flip (set inert) or revert. Retry only fires where no SQL ran, and
the per-pool token bucket caps a storm. No migration.
refs TRI-13295, TRI-12982, TRI-12984
<!-- ccr-slack-attribution -->
_Requested by **Matt Aitken** · [Slack
thread](https://triggerdotdev.slack.com/archives/C045W9WM3E1/p1786732623292829?thread_ts=1786732623.292829&cid=C045W9WM3E1)_
**Before:** you pause an environment, then a deploy lands (or a
background worker is created, or an admin changes the
concurrency/burst-factor). The environment starts picking up runs again
even though the dashboard still shows it as paused.
**After:** a paused environment stays paused until it is resumed, no
matter what else pushes its concurrency limit.
Pausing an environment sets `paused` in the database and writes a `0`
env concurrency limit into the run queue — the `0` is the only thing
that actually stops dequeueing. Any caller that pushed the limit without
an explicit value (`finalizeDeployment`, `createBackgroundWorker`, the
two admin environment routes) rewrote the real limit and silently
un-paused the environment.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
`apps/webapp/test/pauseEnvironment.server.test.ts` gains two
`containerTest` cases that wire a real `RunEngine` (real Redis) in place
of the stubbed app singleton and assert the actual run-queue env limit:
- pause a PRODUCTION env → limit is `0` → run the real
`FinalizeDeploymentService` → limit is still `0`, plus a control on a
running env in the same test proving that deploy path really does push
the limit (so the `0` can't just mean "nothing happened").
- pause → resume → the real limit is restored, so the clamp can't
regress resuming.
Both cases fail on `main` (`expected 17 to be +0` and `expected +0 to be
17`) and pass with this change. `pnpm run typecheck --filter webapp` is
clean.
---
## Changelog
Fix paused environments starting to run work again after a deploy.
---
## How
The clamp lives in the shared `updateEnvConcurrencyLimits` helper in
`apps/webapp/app/v3/runQueue.server.ts`, so every present and future
caller is covered: when no explicit limit is passed and the environment
is paused, `0` is written instead of the stored maximum. An
explicitly-passed limit still wins, which is what pausing itself relies
on. The resume path now passes the post-update environment state (its
in-memory copy was read before the un-pause and would otherwise be
clamped back to `0`), and the helper no longer mutates the caller's
environment object — that aliasing made a pause followed by a resume on
the same object write `0` twice. The existing `!paused` guards in
`allocateConcurrency` and the queue-level guard in
`createBackgroundWorker` are left in place as defence in depth, and
queue-level `TaskQueue.paused` behaviour is untouched.
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
After a deployment promotion or rollback, newly triggered runs could
keep dispatching onto the previously deployed version for up to 30
seconds. Runs now resolve the current version fresh on every dequeue, so
a promotion or rollback takes effect immediately.
## Fix
The dequeue path resolved the worker version through a 30s in-process
cache that nothing invalidated on promotion, and it loaded the worker's
entire task and queue set only to keep the single row matching the run.
Both go away: the resolve now fetches just the matched task and queue by
unique index and reads them fresh, so there is no cache left to serve a
stale version.
```
- cache.get(env:current) # 30s TTL, never invalidated -> stale
- worker + ALL tasks + ALL queues
+ worker + one task WHERE slug=... + one queue WHERE id/name=... # fresh
```
A kill-switch env var (`RUN_OPS_WORKER_VERSION_FRESH_READ_ENABLED`,
default on) falls back to the old cached path without a code deploy.
Verified end-to-end on an isolated stack: a run triggered after a
mid-stream promotion now dequeues onto the new version, with the
previous stale behavior reproduced first.
## Summary
When resolving the current worker for a development environment,
`findCurrentWorkerFromEnvironment` loaded the entire `BackgroundWorker`
row,
including the large `metadata` JSON, even though it only ever returns a
handful
of small fields. It is a frequently-run query, so the wasted payload
adds up:
every call pulled data it immediately threw away.
## Fix
Add a `select` to the development-environment lookup listing exactly the
fields
the function returns (`id`, `friendlyId`, `version`, `sdkVersion`,
`cliVersion`,
`supportsLazyAttempts`, `engine`). The query plan is unchanged, still a
single-row indexed lookup; only the row width shrinks. No behavior
change: the
dropped columns were never read.
## Summary
The worker-version resolve path fetched every column of every
`BackgroundWorkerTask` for a worker (`include: { tasks: true }`), plus
full `WorkerDeployment` and `TaskQueue` rows, just to match one task at
dequeue. That pulls large JSON columns none of this path reads (task
`payloadSchema`/`config`/`queueConfig`/`description`, deployment
`externalBuildData`/`buildServerMetadata`/`errorData`/`git`, queue
`rateLimit`), so each resolve transfers and deserializes far more than
it uses.
## Fix
Replace the includes with explicit `select`s of only the columns dequeue
reads, in both the passthrough resolver and the app resolver:
- task: `id`, `slug`, `machineConfig`, `retryConfig`,
`maxDurationInSeconds`
- deployment: `id`, `friendlyId`, `imageReference`, `imagePlatform`
- queue: `id`, `name` (the queue matcher keys on both)
The shared `ResolvedWorkerVersion` element types narrow to match
(mirrored in the cache), which also shrinks each cached worker-version
entry.
## Impact
The `tasks` read fetches every task of a worker to match one, so its
cost scales with task count and payload-schema size. For a worker with
~70 registered tasks, dropping the unread columns cuts the per-query
transfer roughly:
| Task shape | Before | After | Reduction |
|---|---|---|---|
| Light (no payload schema, small config) | ~28 KB | ~14 KB | ~54% |
| Typical (mixed schemas / config) | ~62 KB | ~14 KB | ~77% |
| Schema-heavy (large `payloadSchema`) | ~200 KB | ~14 KB | ~93% |
The `after` size is roughly fixed because the kept columns are small;
the win grows with how heavy the dropped JSON is. Narrowing `deployment`
(four JSON columns off a single row) and `queues` saves further on top.
No behavior change: pure read-shape narrowing, no flag and no schema
change, so rollback is a plain revert. Verified with a red/green
run-engine test that asserts the resolved task, deployment, and queue
carry only the used columns, plus the queue feature-matrix runs (batch,
retry-policy, machine-preset, plain trigger) that exercise the kept
columns.
## What
A relation with `onDelete: Cascade | SetNull` whose child FK column has
no index makes every parent delete fire a cascade that sequentially
scans the whole child table. That has shipped three times recently and
had to be fixed after the fact (#4554 `ProjectAlert.channelId`, #4555
`EnvironmentVariableValue.valueReferenceId`, #4588
`PersonalAccessToken.userId`).
This adds a schema-aware CI guard that catches the next one before it
merges.
## How
`apps/webapp/scripts/fkCascadeIndexGuard.ts` parses both Prisma schemas
(`@trigger.dev/database`, `@internal/run-ops-database`) and flags any
`onDelete: Cascade | SetNull` relation whose leading FK scalar is not
the leading column of some index (`@@index` / `@@unique` / `@@id` /
field-level `@id`/`@unique`) on the child model. A leading FK column
lets the cascade's `WHERE fk = $1` use the index instead of a seq scan.
It is modeled on the existing `runOpsLegacyGuard` (same `--check` gate,
same baseline-regenerate pattern), and it is lighter: it only reads
`schema.prisma` as text, so its CI job needs no Prisma client generation
and no raised heap.
## Why a baseline, not a hard rule
Not every unindexed cascade FK is a live bug. When the parent is only
ever soft-deleted, the cascade never fires, so the missing index is
harmless. Hard vs soft delete lives in application code
(`parent.delete()` vs `parent.update({ deletedAt })`), not in the
schema, and a `deletedAt` column proves neither direction. So the guard
makes no such judgment: it flags every unindexed cascade FK uniformly
and carries a baseline of the 72 currently-accepted cases. Only
violations **not** in the baseline fail `--check`.
The value is the forcing function: a newly added cascade FK stops CI and
makes the author answer "is the parent ever hard-deleted?" Add the index
if yes; regenerate the baseline with a reason if no.
## Wiring
- `apps/webapp/package.json`: `guard:fk-cascade-index` script
(regenerate with no args, gate with `-- --check`).
- `.github/workflows/fk-cascade-guard.yml`: the reusable workflow.
- `.github/workflows/pr_checks.yml`: runs on webapp-affecting changes,
aggregated into `all-checks`.
## Verification
- The three already-fixed columns are correctly seen as indexed (absent
from the baseline).
- `--check` passes on the current schemas (72 baselined, 0 new).
- A synthetic new unindexed cascade FK fails with exit 1 and an
actionable message.
- Adding `@@index([fk])`, or a composite leading with the FK, clears it.
No false positives.
- `oxfmt` and `oxlint` clean on the new script.
## Rollback
Pure tooling addition, no runtime code, no schema or data change. Revert
to remove.
## Summary
The notifications admin list loaded every interaction row for the
notifications on the current page just to show three per-notification
counters (seen, clicked, dismissed), then counted them in memory. On
notifications with many interactions this made the page slow to load and
heavy on memory, even though only 20 notifications are shown.
## Fix
Compute the counters in a single grouped aggregate in the database
instead, returning one row per notification rather than one row per
interaction:
```sql
SELECT "notificationId",
COUNT(*) AS seen,
COUNT(*) FILTER (WHERE "webappClickedAt" IS NOT NULL) AS clicked,
COUNT(*) FILTER (WHERE "webappDismissedAt" IS NOT NULL OR "cliDismissedAt" IS NOT NULL) AS dismissed
FROM "PlatformNotificationInteraction"
WHERE "notificationId" IN (...)
GROUP BY "notificationId"
```
Behavior is unchanged; notifications with no interactions report zero.
## Summary
Since the move from the Remix compiler to Vite
([#4188](https://github.com/triggerdotdev/trigger.dev/pull/4188)), the
"App version" on the organization settings page shows `v0.0.0` unless
the image was built from a semver release tag (which bakes in
`BUILD_APP_VERSION`). Self-hosted builds and any image built from `main`
are affected. This restores the real version.
## Root cause
The Vite SSR bundle resolves workspace packages to TS source via the
`@triggerdotdev/source` condition, so `@trigger.dev/core`'s `VERSION`
constant is bundled as its raw `"0.0.0"` placeholder.
`scripts/updateVersion.ts` still stamps the real version at build time,
but only into the packages' dist output, which the bundle no longer
reads. The old Remix compiler bundled the stamped dist, which is why
this used to work.
The fix is a small Vite plugin that applies the same substitution to the
source version modules of `@trigger.dev/core` and `@trigger.dev/sdk`
during bundling. Beyond the settings page, this also restores real
values in the `trigger-version` request header and the version
attributes the bundled packages emit.
Verified by building the server bundle and confirming the VERSION
constants carry the package versions, with no `"0.0.0"` occurrences left
in the build output.
## Summary
Adds execution-window product surfaces for both declarative and
imperative schedules.
- Declarative schedules can set `window` through `schedules.task()`,
with support for whole-minute, hour, and percentage values.
- Imperative schedules can create, update, clear, and inspect windows
through the API and dashboard.
- Schedule API responses preserve `nextRun` as the nominal CRON time and
expose `nextRunEffectiveAt` as the stable assigned time.
- The dashboard displays configured windows alongside assigned
upcoming-run times.
- Deploy output summarizes declarative schedules and suggests adding a
wider window when the default 60-second placement range is used.
## Design
Window validation remains authoritative on the server and ensures each
window is compatible with the schedule cadence. Omitting a window uses
the default 60-second range, while explicit zero-duration windows remain
supported.
Deployment summaries are derived from the deployment's stored task
metadata, so they reflect the declarations associated with that
deployment.
## Summary
Follow-up to #4595. Dashboard pages under an environment loaded every
environment in the project on each page just to resolve the one named in
the URL. On projects with many preview branches that meant reading
hundreds of (mostly archived) rows on every page load.
## Fix
The environment-scoped layout loader now scopes its lookup to the slug
in the URL (`where: { slug: envParam }`), resolving the current
environment through the `projectId, slug` composite index instead of
loading the whole project. Archived branches stay viewable by slug.
`BatchListPresenter` is bounded to the current environment, since every
batch in that list already belongs to it.
Verified on a project seeded with 2,000 archived branch environments:
the layout lookup drops from all environments to one, and both a normal
environment page and an archived branch page render correctly.
## Summary
On the Concurrency page, the dev environment row's Total always showed
the plan's included dev concurrency, even when the environment's limit
had been raised. The row's own "Extra concurrency" value was already
derived from the real limit, so the two columns could disagree with each
other.
## Root cause
The Total cell renders `planConcurrencyLimit + allocation`, where
`allocation` is the state behind the editable prod/staging inputs. Dev
environments are deliberately excluded from that allocation map (dev
concurrency is not purchasable), so the dev row's allocation always
resolved to 0 and the Total fell back to the plan value. The dev row now
renders the environment's actual `maximumConcurrencyLimit` instead.
## What & why
Org avatars disappeared from the sidebar, replaced by alt text spilling
across it. Two bugs stacked: the document img-src CSP pins the Google
favicon endpoint org avatars are stored as, but Google 302-redirects it
to `tN.gstatic.com` and CSP re-checks the redirect target, so the avatar
is refused. Changelog images served from `trigger.dev` in the agent chat
were also missing from the allowlist. And `Avatar.tsx` had no clipping
and no error fallback, so a refused image degraded into overflowing alt
text.
## What's inside
**CSP allowlist** — `app/utils/cspImageOrigins.ts`: the base sources
gain the four gstatic shards `t0`–`t3.gstatic.com`, path-pinned to
`/faviconV2`, plus `https://trigger.dev/changelog/` as a path prefix. No
wildcards — the no-wildcard beacon policy stands. The shard hosts are
Google-operated with no public write path, so the enumeration is as
narrow as the existing `s2/favicons` entry; if Google ever adds a `t4`,
the failure mode is one broken avatar, not a broken page.
**Avatar fallback** — `app/components/primitives/Avatar.tsx`: the image
box clips, and a failed load falls back to the globe icon. That covers
failures before hydration too — `onError` never replays for a node that
already failed, so a ref checks `complete && naturalWidth === 0` at
attach time. The error state resets when the URL changes
(`key={avatar.url}`).
**Radio card theming** — `app/components/primitives/RadioButton.tsx`: in
the dark themes the checked radio card rendered darker than the
unchecked ones. Unchecked cards now sit on `background-bright`
(near-black in dark, unchanged white in light) and the checked card uses
the `surface-control` tokens, so selection reads black → grey in dark
themes; light theme keeps its current look. The API keys route keeps its
indigo checked-hover via an explicit override.
## Testing
The CSP test helper now implements CSP's real path-matching rule
(trailing slash = prefix, otherwise exact, query ignored) and asserts
the pins hold: the gstatic redirect target passes, `beacon.png` on
gstatic, a `t9` shard, and non-changelog `trigger.dev` paths stay
blocked. 39 tests green plus webapp typecheck. Verified against a
running webapp that the served directive contains the new sources.
## What & why
The dashboard agent's upkeep — retention deletes and the investigation
sweep — ran as cron jobs on the webapp's common worker, even though it
only touches the agent's own datastore. This moves that upkeep into the
agent's Trigger project as scheduled tasks (TRI-13182).
## What's inside
**Retention** — `internal-packages/dashboard-agent/src/maintenance.ts`,
a daily task (03:00 UTC). Deletes turn evals older than 30 days,
hard-deletes chats soft-deleted more than 30 days ago, and purges
terminal watches and submission rows older than 7 days. It used to run
every 5 minutes; nothing needs a hard delete that fast, so it is daily
now, draining in bounded batches and warning if it hits the cap. It
retries (3 attempts) because the next run is a day away. It connects
with `DASHBOARD_AGENT_DATABASE_URL`, falling back to `DATABASE_URL` like
every other task in the package (the deletes are confined to the agent's
own Postgres schema), and skips when neither is set.
**Investigation sweep** — `src/investigation-sweep.ts`, every 5 minutes,
same as before: settles investigation cards stuck `in_progress`
(30-minute window, attempt cap, force-abandon note). It keeps the fast
cadence because it fixes live state the UI is showing.
**What stays in the webapp.** The watch finalize/deliver sweep and batch
rearm: they cover a dead agent-side tick chain — a backstop can't live
inside the thing it backstops — and they need the main database and the
alerts worker. The org-deletion chat purge also stays: deletion must not
depend on the agent project being deployed. The removed cron job keeps a
cron-less tombstone entry so already-queued items drain cleanly; remove
it in a follow-up.
**Test plumbing** — the drizzle migration replayer that webapp tests
hand-rolled is now exported once from
`@internal/dashboard-agent-db/testing`; the moved tests live in the
agent package as `src/*.test.ts` against real Postgres.
## Testing
Agent package: retention passes (backlog drain, batch cap, no-op guard,
chat-delete cascade) and the sweep, on testcontainers Postgres. Webapp:
the watch/chat suites, plus a test that a settlement card stops the
dashboard spinner. Full typecheck on both.
## What
`CheckScheduleService.call` loaded **every** environment of a project
(`{ id, type, archivedAt }`, no filter) and then immediately narrowed to
just the requested `environmentIds` via
`resolveProjectScopedEnvironments`. It only ever uses the requested envs
(to reject foreign env ids and reject archived branches). On a
preview-heavy project that meant loading hundreds of archived branch
rows to validate one, on a path called in a per-scheduled-task loop on
the deploy path (`createBackgroundWorker` -> `syncDeclarativeSchedules`)
and from `upsertTaskSchedule`.
The query is index-backed and individually fast (rows_read/returned = 1
per predicate), so this is about result-set width / egress and wasted
work at scale (~580k calls/24h observed via Insights), not a slow plan.
## Change
Bound the `environments` relation load to `boundedIn(environmentIds)`:
```ts
environments: {
where: { id: { in: boundedIn(environmentIds) } },
select: { id: true, type: true, archivedAt: true },
}
```
Returns `<=` the number of requested envs (usually 1) instead of the
whole project. Both existing behaviors are preserved:
- **Foreign-id rejection**: the relation is still scoped to the project,
so a requested id belonging to another project never comes back and
`resolveProjectScopedEnvironments` reports it as `foreign` (a missing
requested id is already treated as foreign).
- **Archived-branch rejection**: a requested id that is an archived
branch still comes back with `archivedAt` set, so the downstream `Can't
add or edit a schedule for an archived branch` check still fires.
`archivedAt` is kept in the select deliberately, so this bounds by id
rather than filtering archived rows out.
## Evidence (isolated stack, seeded 1 prod env + 40 archived branch
envs)
Local `EXPLAIN (ANALYZE)` of the exact environments sub-select:
| | rows returned | buffers |
|---|---|---|
| before (unbounded) | **41** | shared hit=12 |
| after (`id IN (requested)`) | **1** (`Rows Removed by Filter: 40`) |
shared hit=4 |
Same `RuntimeEnvironment_projectId_idx`, no plan change. Rows to the
client drop to `len(environmentIds)`, which is the point.
**Unit (vitest, testcontainers, real Postgres):**
`apps/webapp/test/checkSchedule.test.ts` extended to prove, on real
rows, that the bounded load returns only the requested env (1 of 10),
still reports a foreign id as foreign, and still surfaces an archived
branch when it is the requested one. 5/5 pass.
**Full e2e (both execution modes, real stack):** a purpose-built project
with two declarative `schedules.task`s.
- `trigger dev`: dev worker created, both schedules synced through the
edited `checkSchedule` loop, no errors.
- `trigger deploy` (managed deployment): PRODUCTION worker registered,
both schedules synced against the **prod** environment through the same
loop, prod + dev schedule instances active, no errors.
`typecheck --filter webapp` clean.
## Rollout / rollback
Straight deploy, no flag, no migration. Rollback is revert-only
(read-path narrowing, no data change). Old and in-flight rows read
correctly under both the old and new code.
## Out of scope
The two lower-priority sibling reads in the ticket (the Query/metrics
env id->slug map and the env-var repository fan-out) are left for
follow-ups; they need caching / per-method scoping rather than this
single bound.
## What
The environment variables settings page loaded **every** variable in the
project in one shot, with a nested `values` read plus a `valueReference`
(SecretReference) sub-load that was selected but never read. For a
project with many variables this pulled `variables × environments` value
rows (~18k for large projects) on every page load, plus a matching
~18k-row `SecretReference IN` query.
This paginates the presenter by variable key and removes the dead
include.
- Remove the never-read `valueReference: { select: { key } }` include →
the `SecretReference` query is gone entirely.
- Paginate the parent variable query: `count` + `orderBy key` +
`skip/take`, page size 50 → the value read is bounded to `pageSize ×
environments` per page.
- Scope the count and the page to variables that have a value in a
displayed environment (`values: { some: { environmentId: { in } } }`),
so `totalCount`/`totalPages` and the `skip/take` window match what
actually renders (no phantom empty pages from variables that live only
in archived branches or another member's dev env).
- Display order comes from the DB `orderBy: { key: "asc" }` — the
presenter no longer re-sorts each page with `localeCompare`, which under
pagination could disagree with the DB collation at page boundaries.
- The secret-value lookup (`SecretStore` keys) and the updater lookup
(`user` by id) are now scoped to the current page instead of the whole
project.
- Search moves server-side (variable key, case-insensitive) and drives
both the count and the page; the UI gains standard pagination controls.
## Why
The two correlated ~18k-row control-plane queries flagged in the ticket
come from this settings-page presenter, not from any hot path. Both are
index-covered (`rows_read == rows_returned`); the issue is the sheer
volume fetched in one burst. Bounding it per page removes the burst.
## Evidence
Measured on an isolated stack with a seeded project of 1000 variables ×
3 environments (3000 value rows), using Prisma's emitted-SQL log:
| | SecretReference query | value rows fetched |
| --- | --- | --- |
| before | 1 | 3000 |
| after | **0** | **150** (page 1) + one `count` |
`EXPLAIN` on Prisma's verbatim statements (index confirmed via
`enable_seqscan=off`; the local table is too small for the planner to
choose them by default):
- `count` (`WHERE projectId AND EXISTS(values in displayed envs)`) →
Hash Join: Index Scan on `EnvironmentVariable_pkey` + Bitmap Index Scan
on `EnvironmentVariableValue_environmentId_idx`
- paginated parent (`WHERE projectId AND EXISTS(...) ORDER BY key
LIMIT/OFFSET`) → Nested Loop Semi Join: Index Scan on
`EnvironmentVariable_projectId_key_key` (**no Sort node**) driving an
Index-Only Scan on
`EnvironmentVariableValue_variableId_environmentId_key`
- nested values (`variableId = ANY … AND environmentId = ANY …`) → index
scan on `EnvironmentVariableValue_environmentId_idx`
- `SecretStore` keys (`key = ANY …`) → index scan on
`SecretStore_key_idx`
No new index required. Verified in the browser on the seeded project: 20
pages, page navigation, server-side search (matches across all pages),
last page renders, no app console errors. `typecheck`, `oxlint`, `oxfmt`
all clean.
## Behavior change
The previous client-side search matched variable **name and value** (and
environment type / branch name). Values are encrypted at rest and
resolved separately, so they cannot be searched server-side under
pagination. Search is now **variable-name only**, server-side,
case-insensitive. Projects with fewer than one page of variables see no
pagination bar and no visible change.
## Rollout / rollback
Pure read-path change on a dashboard loader, no schema or data
migration. Rollback is a straight revert.
## Screenshots
<img width="2400" height="1794" alt="01-page1"
src="https://github.com/user-attachments/assets/d4a7effd-d167-4dd6-92f4-6e9174818acd"
/>
<img width="2400" height="1794" alt="02-search-single"
src="https://github.com/user-attachments/assets/cd113ca9-ff87-431f-b2f6-7f7d36f2b32a"
/>
Chris Arderne
