Adds an opt-in way for operators to route deployed runs' API traffic
through a different origin than the public one, per organization. Set
`INTERNAL_API_ORIGIN` on the webapp and enable the
`internalApiOriginEnabled` feature flag (globally or per org, with the
org override winning in both directions): deployed runs for enabled orgs
then get `TRIGGER_API_URL` set to the internal origin instead of
`API_ORIGIN`. Useful for gradually moving run traffic onto a private
network path.
## Design
The origin is resolved when an attempt starts, so flag changes take
effect on the next attempt and roll back the same way, with no task
redeploys. The org override is read fresh per attempt; the global
default comes from the cached flags registry (a cold read fails safe to
the public origin). When `INTERNAL_API_ORIGIN` is unset the flag is a
no-op and no extra queries run, so existing deployments are unaffected.
Dev runs always use the public origin, and `TRIGGER_STREAM_URL` remains
unchanged.
<!-- ccr-slack-attribution -->
_Requested by **Eric Allam** · [Slack
thread](https://triggerdotdev.slack.com/archives/C0AU83M3136/p1785222101937829?thread_ts=1785207509.304669&cid=C0AU83M3136)_
Removes two dead Remix routes and the helpers only they used.
`app/routes/sync.traces.runs.$traceId.ts` (`/sync/traces/runs/:traceId`)
and `app/routes/sync.traces.$traceId.ts` (`/sync/traces/:traceId`) were
added with the original ElectricSQL run page and lost their only
consumers when the dashboard hooks that called them were deleted.
Nothing in the repo references either route today.
Also removed, because the deleted routes were their only callers:
- `OtelTraceIdSchema`, `RESERVED_ELECTRIC_SHAPE_PARAMS`, `TraceScope`,
`buildElectricTraceWhereClause` from `app/v3/electricShape.server.ts`
(the file stays — `UNSAFE_REALTIME_TAG_CHARS` /
`sanitizeRealtimeTagForSql` / `sanitizeRealtimeTagsForSql` are still
used by `realtime.v1.runs.ts` and `realtimeClient.server.ts`)
- the loader-specific cases in
`apps/webapp/test/spanTraceRoutes.replicaLag.test.ts` and
`internal-packages/run-store/src/runOpsStore.routesSpanTraceReadView.replicaLag.test.ts`
`app/utils/longPollingFetch.ts` is untouched —
`realtimeClient.server.ts` still uses it. `runOpsStore.ts` /
`PostgresRunStore.ts` are untouched too; the unrouted-lookup mechanism
there is generic and stays.
As a plain code fact: the run lookup these loaders performed keyed on
`TaskRun.traceId` alone, which is not an index-backed query shape. That
is noted only as context for why the code is not worth keeping around
unused.
### Judgement call worth a maintainer's opinion
The request was specifically about `/sync/traces/runs/:traceId`, the
route that looks up a run by `traceId`. This PR **also** deletes its
sibling `/sync/traces/:traceId`. The reasoning:
- both routes came in with the same ElectricSQL run-page work
- both lost their only consumers in the same later commit
- neither has any caller anywhere in the repo
- they share the same helper module, so keeping one means keeping the
helpers half-used
If you would rather keep the sibling, reverting just that one file
deletion is easy and does not affect the rest of this PR — say the word
and I will restore it along with the helpers it needs.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Verification run locally from the repo root:
| Command | Result |
| --- | --- |
| `pnpm run format` | clean, no changes produced |
| `pnpm run lint:fix` | clean |
| `pnpm run lint` | pass (exit 0, no findings) |
| `pnpm run typecheck --filter webapp` | pass |
| `pnpm run typecheck --filter @internal/run-store` | pass |
A ripgrep sweep for `sync.traces`, `sync/traces`, `syncTraceRunsLoader`,
`buildElectricTraceWhereClause`, `OtelTraceIdSchema` and
`RESERVED_ELECTRIC_SHAPE_PARAMS` (excluding `node_modules`) returns zero
hits.
**Not fully verified:** both edited test files are testcontainers suites
and need a Docker runtime, which was not available in my environment. I
confirmed each file *collects* correctly with exactly the three intended
remaining tests and no import errors — notably, dropping the
`session.server` / `controlPlaneResolver.server` / `longPollingFetch` /
`env.server` mocks does not break module loading for the surviving
loaders. The assertions themselves then failed only on `Could not find a
working container runtime strategy`. CI should be the real signal here.
Per `apps/webapp/CLAUDE.md`, `pnpm run build --filter webapp` was
deliberately not run.
---
## Changelog
Removed two unused sync routes left over from the original ElectricSQL
run page, along with the helpers and tests that existed only to serve
them. No behaviour change — neither route had any caller.
---
## Screenshots
_n/a — no user-visible surface changes._
💯
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
The project Integrations page now uses the same settings layout as the
org SSO page: a centered column of titled rows with dividers, instead of
headings over bordered boxes. GitHub, Vercel and build settings read as
one consistent list, and the page titles itself "Integrations".
Confirmations persist rather than vanishing once you move past them
(`GitHub app: Installed`, `Vercel project: Connected`), plan-gated rows
offer an Upgrade button instead of a dead toggle, a disabled toggle
explains why in place and highlights the control that unlocks it, and
warnings are rows with a hazard icon and their recovery action on the
right. Copy throughout leads with the outcome instead of restating the
field label.
Two fixes along the way: a nested `<form>` in the Vercel panel that
failed hydration and silently truncated the page, and every settings row
carrying a few pixels more space above its title than below its
description.
### Before
<img width="1160" height="1972" alt="CleanShot 2026-07-26 at 21 56
42@2x"
src="https://github.com/user-attachments/assets/ed0fd676-36d8-4eb7-a16e-827a24f007d9"
/>
### After
<img width="1358" height="4455" alt="CleanShot 2026-07-26 at 19 14
28@2x"
src="https://github.com/user-attachments/assets/6a635e6a-c0eb-4a4c-a68f-fcde4d25e8a6"
/>
## Summary
Favorite any dashboard page and it appears in a new "Favorites" section
at the top of the side menu. The star next to the page title (or
Option+F) saves the exact view, filters and tabs included, with a name
derived from the URL ("Runs: Completed successfully, last 7d", "Run:
05hrqq9n") that you can rename inline from each item's hover menu.
The sidebar is customizable too: "Customize sidebar" (on section header
menus and in each "More" menu) opens a modal where you can reorder
sections, drag items into a new order, hide items behind a per-section
"More" popover, and rename or remove favorites. Changes apply on
Confirm, Reset restores the default layout without touching favorites,
and everything is stored per user in dashboard preferences.
## Screenshots
| Favorites in the side menu | Customize sidebar modal |
| --- | --- |
| 
| 
|
## Design notes
- Favorite links carry a small marker search param so the favorite, not
its identical main menu item, highlights as active. Markers from shared
or stale links are cleaned on load, and changing any filter hands the
highlight back to the regular menu item.
- Preference writes are serialized with a row lock: several writers
(debounced collapse and width saves, favorite toggles, the customize
modal) can land concurrently and would otherwise clobber each other's
read-modify-write of the JSON column.
- Option+F is matched on `event.code` with a raw listener because macOS
reports Option-modified letters as symbols, which the `event.key` based
shortcut hook can't capture.
Verified end-to-end in the browser: star toggle and shortcut, instant
section appearance, inline rename and staged modal removal, filter-aware
labels and unique active states, shared-link normalization, drag
reordering, and persistence across reloads.
## Summary
The runs list on a task's page now updates live, matching the main Runs
page. Run rows update their status, duration, and cost in place as runs
progress, and a "N new runs" button appears in the header when newer
runs come in so you can pull them into the list without a manual
refresh. This applies to both standard and scheduled task pages.
## Design
It reuses the Runs page's polling hook. A task page scopes its runs by
the task in the URL path rather than a `tasks` query filter, so the hook
now takes an optional task slug and scopes new-run detection to it. The
"new runs" button sits in the header, outside the deferred runs table,
so the count is lifted to the page and the click action is passed
through a ref. That keeps the table streaming on first load instead of
blocking the header on the runs query.
When newer runs come in, a `1 new run` button appears in the task page
header, to the left of the time filter. Clicking it pulls the new runs
into the list.
The SSO & Directory Sync settings page decided access by comparing the
organization's plan code against the literal string `"enterprise"`. The
webapp now reads a `hasSso` entitlement from plan limits.
## Changes
- **`settings.sso` route** — `planAllowsSso` reads `limits.hasSso`
rather than the plan code; the loader and the action gate on a shared
`getSsoEntitlement` helper.
- **`platform.v3.server`** — new `getSsoEntitlement(orgId)` returning
`entitled | not_entitled | unknown`, behind a new SWR cache namespace
(60s fresh / 120s stale, memory + Redis). This replaces an uncached
billing round-trip that previously ran on every settings load, so the
page gets cheaper than it was.
- **`directorySyncEffects`** — the entitlement is now checked before
applying membership effects, per organization and memoised across a
batch.
- **`@trigger.dev/platform` 1.2.0 → 1.3.0** — required, see below.
## Behaviour worth reviewing
**Revocation now stops SCIM.** Previously the plan check existed only on
the settings page, so an org that lost access kept receiving
directory-sync pushes indefinitely; only the config UI froze. Provision
*and* deprovision are gated, so a revoked entitlement can't remove
members either.
**An unreadable entitlement throws instead of skipping.** Effects are
idempotent and the worker retries, so retrying is lossless where
dropping would silently lose a directory change. It's raised at `warn`
level so a transient billing blip doesn't page anyone.
**The login path is deliberately untouched.** A hard entitlement check
there turns a billing outage into a login outage. Consequence: an org
that loses the entitlement keeps its existing SSO logins working until
the connection is removed. Gating sign-in is a separate decision.
**Self-hosted is unaffected.** With no billing service configured the
helper returns `entitled`, leaving plugin presence and the kill switch
as the only gates — a self-hoster who installed the plugin isn't locked
out of it.
## The dependency bump is load-bearing
The `Limits` schema is a plain `z.object`, so it *strips* unknown keys.
On 1.2.0 the `hasSso` field was silently discarded during parsing and
read as `undefined` no matter what billing sent — a structural accessor
would not have helped. Verified against both builds:
```
1.2.0 → parsed: true | hasSso survives: false
1.3.0 → parsed: true | hasSso survives: true
```
This PR therefore cannot merge before 1.3.0 is published, which it now
is.
## Testing
`apps/webapp/test/directorySyncEffects.server.test.ts` — 7 tests over
the gate: applies when entitled, skips provision and deprovision when
not, throws a warn-level retryable error when unreadable, resolves once
per org across a batch, and gates per org so one unentitled org doesn't
block another.
`pnpm run typecheck --filter webapp` passes (18/18), oxfmt and oxlint
clean.
<!-- ccr-slack-attribution -->
_Requested via [Slack
thread](https://triggerdotdev.slack.com/archives/C097ZHVKZFA/p1785082528841609)_
`TaskListPresenter` created promises that nothing ever consumed. Two of
the three deferred metrics promises it returned had no reader, no
`await` and no `.catch()`, so when the query behind one of them failed
the rejection had nowhere to go.
## Before / After
**Before**
- `TaskListPresenter.call()` returned four things: `tasks`, `activity`,
`runningStats` and `durations`. Its only caller reads `tasks` and
`runningStats`.
- Every load of the tasks page therefore fired two ClickHouse queries
whose results were thrown away.
- If either of those two queries failed, the resulting promise rejection
was unhandled — nothing was awaiting it and nothing had attached an
error handler, so it surfaced as an unhandled rejection at the process
level rather than as an error anyone could attribute to a request.
**After**
- `TaskListPresenter.call()` returns `tasks` and `runningStats` only.
- Two fewer queries run per tasks-page load.
- There is no longer an unconsumed promise that can reject without a
handler. `runningStats` is awaited by its caller, so its failures
continue to be handled the way they always were.
Nothing changes on screen: the tasks page renders `hourlyActivity` and
`runningStates`, and neither of the removed values fed either of those.
## How
The removed values were verified unreferenced before deleting anything:
- `TaskListPresenter` has exactly one caller,
`UnifiedTaskListPresenter`, which reads `taskResult.tasks` and
`taskResult.runningStats` and nothing else.
- No file anywhere in the repo — app code, tests, or type re-exports —
reads an `activity` or `durations` field off the presenter's result.
- `UnifiedTaskListPresenter` builds its own
`unifiedTaskListHourlyActivity` query for the 24h chart the page
actually renders, which is what made the presenter's separate 7-day
daily activity data redundant.
- `getDailyTaskActivity` and `getAverageDurations` on
`ClickHouseEnvironmentMetricsRepository` had no callers other than the
two lines being deleted, so they and their now-orphaned helpers and
types were removed too.
Changes:
- `apps/webapp/app/presenters/v3/TaskListPresenter.server.ts` — drop the
`activity` and `durations` fields (both from the main return and from
the no-current-worker early return) and the two repository calls behind
them. Drop the unreferenced `TaskActivity` type alias. The "don't await
this" comment on the remaining `runningStats` promise now spells out
that the caller has to consume it.
- `apps/webapp/app/services/environmentMetricsRepository.server.ts` —
remove `getDailyTaskActivity` and `getAverageDurations` from the
`EnvironmentMetricsRepository` interface and its ClickHouse
implementation, along with `fillInDailyTaskActivity` and the
`DailyTaskActivity` / `AverageDurations` types.
`getCurrentRunningStats` is the control that shows the diagnosis is
right. It throws on query failure in exactly the same way as the two
removed methods — `if (queryError) throw queryError` — but it never
produced an unhandled rejection, because `UnifiedTaskListPresenter`
passes its promise into a `Promise.all(...).then(...)` chain that the
route then awaits. Same failure mode, opposite outcome, and the only
difference is whether anything consumes the promise.
Follow-ups, not in this PR:
- `AgentListPresenter` returns three sparkline promises in the same
shape and they look similarly unconsumed. Left alone here to keep this
change reviewable.
- With these two callers gone, the `getTaskActivity` and
`getAverageDurations` query builders in `@internal/clickhouse` have no
remaining callers in this repo. Whether to remove them is a separate
call for someone who owns that package.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
- `pnpm run typecheck --filter webapp` — passes. This is the meaningful
check here: it proves nothing still references the removed fields,
methods or types.
- `pnpm run format` and `pnpm run lint:fix` — clean, no changes
produced.
- No test file referenced the removed symbols, so no test needed
updating.
---
## Changelog
Server-only change, so this carries a `.server-changes/` note rather
than a changeset:
`.server-changes/task-list-remove-unused-metrics-queries.md`.
> The tasks page no longer runs two queries whose results were never
displayed, cutting wasted work on every page load and removing a source
of hidden server errors
---
## Screenshots
_No visual change — the removed data was never rendered._
Co-authored-by: Claude <noreply@anthropic.com>
🦋 Changesets PR / Create Release PR (push) Has been cancelled
🚀 Publish Trigger.dev Docker / 📣 Dispatch main image (push) Has been cancelled
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 2s
🚀 Publish Trigger.dev Docker / units (push) Failing after 2s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
🚀 Publish Trigger.dev Docker / scan-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / scan-supervisor (push) Has been skipped
## Summary
The realtime runs feed hydrates run rows from read replicas, which means
it needs a replica-lag gate to avoid serving a run's previous state
right after a write. Setting
`REALTIME_BACKEND_NATIVE_RUN_READS_FROM_PRIMARY=1` reads those rows from
each run store's primary instead, so there is no lag to gate against: no
probe, no wake delay, no stale-read retries. Off by default, so nothing
changes unless you set it.
## Design
The run stores already decide replica-vs-primary from the *brand* on the
read client they are handed: a branded replica keeps the read on the
owning store's replica, an unbranded writer escalates it to that store's
own primary. So this is a one-line choice at the hydrator, and it stays
correct across topologies. With the run-ops split on, each leg lands on
its own writer and the caller's client is never forwarded across
databases; with the split off, it is the single database's primary.
```ts
const runReader = new RunHydrator({
readClient: runReadsFromPrimary ? prisma : $replica,
runStore,
});
```
The same flag skips constructing the lag estimator, since probing a
replica the feed no longer reads would be measuring the wrong thing.
Independently, `AuroraReplicaLagSource` detected Aurora by letting
`aurora_replica_status()` fail, on the assumption that the app-level
catch made that free. It isn't: an unresolvable function is a query
error the driver reports to the error log on every sample, so a
non-Aurora replica produced a continuous stream of error events while
the estimator quietly fell through to its next candidate. It now
resolves the function with `to_regproc` and memoizes the answer, so the
unparseable call never reaches the wire.
## Summary
Several list endpoints accepted an unbounded page size (`perPage` /
`per_page` / `pageSize`). An unbounded page lets one request pull an
arbitrarily large result set and do a proportional amount of work, which
is a poor default for a shared API.
This clamps the page size to 100 on every list endpoint that was
uncapped, matching the existing cap on `api.v1.runs` and
`api.v1.sessions`. Clamping rather than rejecting keeps existing clients
working: a request for a larger page returns up to 100 items and offset
pagination continues from there.
## Endpoints capped
- `api.v1.schedules` (`perPage`)
- `api.v1.queues` (`perPage`)
- `resources.…versions` (`per_page`)
- `resources.…queues` (`per_page`)
- `admin.api.v1.…engine.report` (`per_page`)
- `admin.api.v1.llm-models` (`pageSize`)
Already capped, left as-is: `api.v1.runs`, `api.v1.sessions`,
`api.v1.deployments`.
## Summary
The split run-store's id-set read path (`#findRunsByIdSet`, used by the
runs-list hydrate, the realtime hydrator, and engine sweeps) queried the
new store for the entire id set and then probed the legacy store for the
misses. A run's residency is a total function of its id (run-ops ids
live in the new store, every other id in legacy), so each id belongs to
exactly one store. Route each id to its owner and query each store only
for its own ids, in parallel. Same result set, and while a split is
active with most runs still on legacy it removes a wasted new-store
query from every id-set read.
## Change
`#findRunsByIdSet` now partitions the ids by `classifyResidency` and
runs one bounded query per store (skipping an empty side), in parallel,
mirroring `expireRunsBatch` and the single-run `#route`. `finalizeRows`
still applies orderBy/take/skip globally over the merged set.
This drops the id-set path's cross-store fallback, which existed to
prefer the new-store copy when the same id was present in both stores.
That collision cannot arise when each id maps to exactly one store
(nothing writes a legacy-shaped id into the new store), so the fallback
is dead code. The two id-set tests that asserted "new copy wins on
collision" now assert the routing invariant: a legacy-shaped id resolves
to the legacy store and the path never consults the new store.
The open-predicate path (`#findRunsOpen`) is unchanged: an open `where`
has no id to route on, so it still unions both stores and dedupes.
## Default billing alerts + billing limit page UX
- New orgs get default billing alerts: $5, $100, $500, $1000, $2500.
Existing orgs are backfilled by a billing-side data migration (companion
[PR](https://github.com/triggerdotdev/cloud/pull/1657)).
- The billing limit form starts with nothing selected for orgs that
never set a limit — the save button appears once an option is picked.
- The yellow banner now also shows on the billing limits page itself,
asking to configure a limit. Hidden everywhere for members who can't
manage billing.
- Also fixes billing limit alert preview.
Tests
- `apps/webapp/test/billingLimitsRoute.test.ts` — dirty logic for
empty/selected mode
- `apps/webapp/test/billingAlertsDefaults.test.ts` — default values
- `apps/webapp/test/billingAlertsFormat.test.ts` — preview after a limit
change
## Summary
Adds two counters to the native realtime backend so we can see how much
duplicate row serialization the change router does per batch. When a run
changes it can match several held feeds at once (a run subscription plus
one or more tag/list feeds), and today each matching feed serializes
that run's wire value independently. These counters quantify that
fan-out so we can decide whether a shared serialization step is worth
it.
## What they measure
- `realtime_native.emission_run_serializations`: total wire-value
serializations performed across feeds per batch (what the current path
does).
- `realtime_native.emission_distinct_serializations`: distinct (columns,
run) rows those serializations cover (what a serialize-once-per-batch
step would do).
Average feeds-per-run is `run_serializations / distinct_serializations`,
and `1 - distinct / run_serializations` is the serialization work a
shared step could save. Wired through a new optional `onEmissionFanout`
callback on the router. No behavior change.
Wraps the workload_auth_gate_total Counter in the singleton helper (same
pattern as reloadingRegistry.server.ts) so a dev hot reload doesn't
crash with "A metric with the name workload_auth_gate_total has already
been registered". No production behavior change.
Restores the debug panel on the **Tasks** and **Runs** pages, and makes
the data it shows copyable.
Admin/impersonation only — no change for regular users, so there's no
`.server-changes`
<img width="909" height="1420" alt="CleanShot 2026-07-22 at 12 05 27@2x"
src="https://github.com/user-attachments/assets/ce2da167-dc23-422f-83f3-4f4aee9ed32c"
/>
## Summary
The empty-state panel on the Prompts page linked to a docs path that no
longer exists, so the "Prompts docs" button returned a 404. It now
points to the current prompts documentation at /docs/ai/prompts,
matching the link already used in the page header.
## What
The organization side menu previously showed an "Enterprise" badge next
to the SSO & Directory Sync item for any org not on the enterprise plan.
That badge is now removed so the item renders without it.
## Screenshot (before)
<img width="1428" height="649" alt="CleanShot 2026-07-10 at 08 24 41"
src="https://github.com/user-attachments/assets/b9787363-972f-4dd5-bf61-486680f49f4c"
/>
## Summary
Limits user account email addresses to 254 characters in profile
settings and onboarding. Oversized values are rejected before the
uniqueness lookup, and the form fields enforce the same limit in the
browser.
## Fix
Both email update flows use a shared bounded email schema. Basic
validation completes before the uniqueness lookup runs.
## Summary
Allow each organization member to use the same development branch name
without colliding with another member's environment. Fixes#4320.
## Fix
Development branches now use the existing member-scoped project, slug,
and organization-member key for upserts. Preview branches retain their
project-wide shortcode behavior.
New development branches receive distinct shortcodes while keeping their
readable, member-scoped slugs. Existing branches continue to resolve
through the member-scoped key, so this requires no migration or
backfill.
Two small corrections to the organization **Usage** page credits
display.
### 1. Label the credits panel "Credits" (was "Promo credits")
The panel surfaces any credit balance, not only promo-code redemptions,
so "Promo credits" is misleading when the credits come from another
source. Renamed the heading to "Credits".
### 2. Don't show "Included usage" for Enterprise orgs
Enterprise inherits the Pro plan's `includedUsage` value, so the Usage
bar rendered an "Included usage: $50" tier marker for Enterprise
organizations. Enterprise bills against prepaid credits rather than a
per-month included-usage tier, so the marker was misleading. The
`tierLimit` marker is now suppressed for Enterprise (`plan.type ===
"enterprise"`).
Verified with `pnpm run typecheck --filter webapp`.
Redesign of the main side menu: separates Projects and Accounts from the
Organization menu and makes the menu resizable.
**Main changes**
- **Organization & Account menus**: the top-left is now a dedicated
organization menu (Settings, Usage, Billing, Team, SSO, integrations),
with a separate account menu beside it (Profile, PATs, Security,
Logout).
- **Project switcher**: a new Project section above the Environment
selector.
- **Resizable side menu**: drag the right edge to set a custom width
(saved per user), or click the edge to collapse/expand.
- **Environment selector**: reworked to match the Project menu,
including dev-branch handling.
- **Account Profile page**: redesigned into the Security page's
row-and-divider layout.
Preview URL: https://samejr-org-menu-update.triggerlabs.dev/https://github.com/user-attachments/assets/9b199576-6037-4ea6-9bdb-3ee15265b8c2
Fixes TRI-12078
## Summary
Prevents concurrent environment setup requests from creating duplicate
Staging and Preview environments.
## Fix
Adds database-enforced uniqueness for root Staging and Preview
environments.
If two requests race, the losing request loads the environment created
by the winner and continues successfully instead of creating a duplicate
or returning an error.
## Summary
Upgrades the workspace to TypeScript 6.0.3 and applies the compiler,
type, and build configuration changes required to preserve package
layouts and existing runtime behavior, apart from correcting the HTTP
status field used for deployment connection errors.
## Compatibility
- Centralizes TypeScript 6.0.3 through the pnpm workspace catalog.
- Replaces compiler options and module resolution modes that TypeScript
6 no longer accepts.
- Restores explicit Node types where TypeScript 6 no longer includes
them transitively.
- Adds explicit declaration build roots that preserve each package's
existing output layout.
- Patches tsup to stop injecting the removed `baseUrl` option during
declaration builds.
- Uses type-only assertions for stricter typed-array and stream
definitions without changing runtime behavior.
- Reads the EventSource v3 HTTP status from `code`, so deployment
connection errors include it correctly.
- Keeps standalone CLI compatibility fixtures pinned to their existing
TypeScript version and lockfiles.
`turbo run typecheck` and the complete PR test suite are green.
## What
The `/promo` page heading rendered with overlapping lines — the two
lines of "Promo codes are for new accounts" collided.
## Why
The page used `Header2` stretched to display sizes (`sm:text-2xl
md:text-3xl lg:text-4xl`), but `Header2` bakes in a fixed `leading-6`
(24px). A 36px font in a 24px line box makes wrapped lines overlap. It
only showed at `sm`+ widths and only on headings that wrap to 2+ lines,
which is why it slipped through — the short single-line headings on the
same page looked fine.
## Fix
Switch both headings to `Header1` — the page-title primitive the sibling
login pages (`login._index`, `login.magic`) already use for exactly this
size. Add `leading-tight` (relative line-height, scales with font size,
and this heading uniquely wraps to two lines) and `pb-4` to match the
login pages' spacing convention.
## Testing
Manually verified the signed-in view (`/promo` while logged in) renders
as two clean, non-overlapping lines across breakpoints. Pure CSS/layout
change — no automated test.
## Stacked on #4284 — tests only
This PR contains **only the tests** that guard the production fixes in
#4284 (its base). Review #4284 first; this branch adds no production
code.
## What
Caller-driven replica-lag and idempotency guards for every fixed site:
- Each guard **drives the real exported caller** (route loader/action,
presenter `.call()`, service, or engine method) against a **real
Postgres** with the owning replica frozen via the shared
`laggingReplica` testcontainer primitive — never a store-seam
reimplementation.
- For a **fixed** site the guard goes **RED when the production change
is reverted**; for a **tolerated read-view** site it's a caller-driven
**GREEN** proof the miss self-heals (returns null/empty, no mutation,
row live on primary).
- The **global-scope idempotency** guard drives the real dedup + claim
path through a **real `MollifierBuffer` over a Redis testcontainer**
(real SETNX/poll/publish), and covers the cross-DB **andWait** waitpoint
wiring and the **expired/failed clear-and-recreate** reacquire cases.
Run with `vitest --no-file-parallelism` (testcontainers). Verified
GREEN, and revert→RED verified per fixed site.
## What & why
Two related correctness fixes for the run-ops DB split. Under the split,
run-store reads can route to a **lagging read replica**; a just-written
run/waitpoint/batch can then be missed, causing a wrong decision.
**1. Read-your-writes → owning primary.** Surfaced first as an
intermittent `wait.until({ idempotencyKey })` re-wait on retry. Auditing
the run-store read surface found the same class at sibling sites (some
gating mutations or returning spurious 404s, others
tolerable/self-healing). Reads that must observe their own writes now
route to the owning **primary**
(`findRun`/`findWaitpoint`/`findBatchTaskRunByFriendlyId` →
`*OnPrimary`, a primary re-read on a miss, or a retryable 404 where the
SDK polls). Read-view reads stay on the replica. All additive — the
happy path is unchanged.
**2. Global-scope idempotency across the split.** A `global`-scope key
carries no per-run salt, so the same `(env, task, key)` triggered
concurrently from parents resident on **different** run-ops DBs could
dedup-miss on each DB and create a duplicate (the per-DB unique index
can't enforce cross-DB uniqueness). Such triggers (global scope, or
scope-absent, while split is active) are serialized through the existing
Redis idempotency claim, the loser resolves the winner by id across both
DBs, and the claim is reacquired on the expired/failed
clear-and-recreate path. `run`/`attempt` scope embed the run id and
never contend.
## Stacked for review
This is the **base** of a 2-PR stack, split so review is easier:
- **This PR** — production code only (34 files).
- **Stacked tests PR →
https://github.com/triggerdotdev/trigger.dev/pull/4285** — the
caller-driven guards (55 test files) on top of this branch.
## Validation
Local run-ops split, **both 2-DB and 3-DB**, fresh boot on this branch:
SDK canary 64/71 (only the known concurrency/input-streams/s3 failures),
quarantine sweep **0 unexpected** (340 pass / 16 known / 4 local) in
each topology, dashboard e2e 0 failed. No product regressions.
## What
- Runs `apps/webapp/scripts/runOpsLegacyGuard.ts --check` as its own PR
job (`runops-guard`), so code that reaches a run-graph table through the
control-plane Prisma client instead of the RunStore fails the build.
- Adds a `trigger-runops` oxlint plugin with two fast, in-editor rules
scoped to `apps/webapp/app`: one for direct `prisma.taskRun`-style
access, one for a control-plane client wired into a read-through slot.
These are the cheap fence; the guard is the type-aware gate.
- Fixes `CancelTaskRunService.callV1`: historical V1 runs are
legacy-resident, so its two finalize writes now go through
`runOpsLegacyPrisma` instead of the control-plane client (they'd miss
the row once legacy is a separate database).
- Regenerates the guard baseline, which had drifted stale (it referenced
files deleted in an earlier PR).
## Why
The guard existed but ran nowhere, so its baseline rotted and a real
residency gap (the V1 cancel writes) sat undetected. Wiring it into CI
turns it into a ratchet against new control-plane run-graph access.
## Verification
Local, against a clean regen: `oxfmt --check`, `oxlint .`, `guard
--check`, and `typecheck --filter webapp` all pass. Remaining baseline
entries are 4 batch-results router reads through type-opaque `as
PrismaReplicaClient` casts (correct at runtime, accepted) + 2 sanctioned
legacy annotations.
## Summary
Correctness and performance fixes for deployments that split run data
across more than one database. Single-database / self-hosted deployments
are unaffected (they collapse to a single read/write path).
- **Batches list (dashboard):** for some organizations the Batches list
could hide older batches or show them out of order. It now orders and
paginates by creation time (with the id as a stable tiebreak), so every
batch appears exactly once, newest first. The pagination cursor format
changes; older in-flight cursors simply restart from the first page.
- **Reads:** waitpoint and snapshot lookups that are keyed by a single
run now read only the database that holds that run instead of querying
both, removing redundant queries on hot paths (unblock, snapshot reads).
- **Writes:** environment-scoped writes with no owning run (standalone
wait tokens, waitpoint tags, idempotency-key resets) now land in the
same database as that environment's runs, rather than defaulting to the
other one. An idempotency-key reset also falls back to the other
database when it matches nothing, so a reset still clears the key
wherever the run actually lives.
## Notes
Verified end-to-end against multi-database setups: run-keyed reads and
env-scoped writes land on the correct database with no cross-database
writes, and the batches list surfaces every batch in creation order. New
tests cover the batches ordering/reachability and the write-residency
routing.
## Problem
The webapp's HTML references content-hashed `/build` assets, and each
running
instance contains exactly one build and returns 404 for asset hashes it
doesn't
have. During a rolling deploy a client can hold HTML from one build
while a
request for one of its assets is served by an instance on a different
build →
missing styles or a failed chunk load.
## What this does
On a `/build` stylesheet/script/chunk load failure, the client does a
**bounded
full-document reload** (at most 2 per 5 minutes, tracked in
`sessionStorage`) so
the page reloads onto a single consistent build. That's the whole
mechanism — no
polling, no `fetch` interception, no blocking overlay, no form
snapshotting.
- `apps/webapp/app/components/StaleAssetRecovery.tsx` — authored as a
typed,
lint-checked function and serialized to an inline script via
`.toString()` (so
the logic is real, reviewable code, not an opaque string), injected
before
`<Links />`, production only.
- Detection: capture-phase `error` listener for
`<link>`/`<script>`/modulepreload
failures under `/build/`, plus an `unhandledrejection` guard for
dynamic-import
failures.
- Guards: once-per-page re-entrancy guard, the bounded reload budget,
and a
`navigator.onLine` check so it never reloads into an offline error page.
- Unit tests in `StaleAssetRecovery.test.ts`.
## Relationship to #4260
Replaces the recovery introduced in #4260 (reverted in #4280) with a
much
smaller, reload-only approach — the previous version intercepted `fetch`
and
could turn a data request into a navigation, and showed a full-screen
overlay on
any asset error; this drops both.
## `/build-version` compatibility shim
`apps/webapp/server.ts` adds a tiny `GET /build-version` endpoint (build
id only,
`no-store`). A previously-deployed client build polls it after an asset
failure
and reloads once it sees a newer build, so those older tabs recover in
one reload
instead of getting stuck. Temporary — safe to remove once older clients
have
cycled out. It deliberately does **not** re-add an `X-Build-Id` response
header.
## Also
Restores the `.server-changes` writing guidance in
`.claude/rules/server-apps.md`
(reverted alongside #4260).
## Self-hosting note
Recovery is most reliable when your load balancer keeps a client on one
instance
for the duration of a deploy (short session stickiness) — the reload
then lands
on a consistent build in one hop.
Removes the deprecated realtime stream write action kept for retired v3
task clients. Supported clients use the targeted stream write routes,
while the existing stream read loader remains unchanged.
Standard `git revert` of #4260.
Its client-side stale-asset recovery is net-negative during normal
deploys:
- The `fetch` interception treats any `?_data=` request (Remix loader
**and** action traffic) as a navigation and, on a build-id mismatch,
`location.assign`es the tab to the fetched URL — an open dashboard tab
can be hard-navigated to a raw data URL during a rolling deploy, losing
unsaved input.
- Any transient `/build` asset error (a network blip, an extension, an
unrelated failed dynamic import) blanks the page behind a full-screen
overlay for ~60s before offering a manual reload.
- It serialized form field values to `sessionStorage` to restore them
across the reload.
This returns the webapp to the pre-#4260 baseline as a fast, low-risk
step.
Follow-ups (separate PRs):
- a minimal reload-only recovery to replace this,
- restore the unrelated `.claude/rules/server-apps.md` docs tidy-up from
#4260 (via cherry-pick),
- a load-balancer stickiness change addressing the root cause.
The signup → Attio sync (`attio.server.ts` `#assert`) logged every
non-2xx response at `error` level and threw the same way regardless of
status. Transient upstream failures (5xx/429) are retried by the common
worker and self-heal, so treating them as errors created false alerts
for something that isn't actually a bug.
Now `#assert` splits the two cases:
- **5xx / 429** — Logged at `warn` and thrown with `logLevel: "warn"`,
so they continue to be retried but don't raise error-level alerts. This
reuses the same pattern the worker already honors
(`directorySyncEffects`).
- **4xx** — Unchanged: logged at `error` and thrown, so genuine
integration bugs (schema, permissions, auth, etc.) remain visible.
There is no behavior change to retries or the signup flow. This is a
server-only change.
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
## Summary
Adds optional configuration to send ClickHouse read traffic to a
separate instance (for example a read replica) while writes stay on the
primary `CLICKHOUSE_URL`. This lets operators offload read load (runs
list, traces, logs, queries) from the cluster that handles inserts.
Fully backwards compatible: with nothing new set, every client resolves
to `CLICKHOUSE_URL` exactly as before.
## What it adds
- `CLICKHOUSE_READER_URL` (optional): a single reader endpoint that the
read-only clients fall back to. Read clients resolve `<own URL> ??
CLICKHOUSE_READER_URL ?? CLICKHOUSE_URL`. The task-events client (which
both inserts events and reads traces, spans, and logs) is built as a
reader/writer pair so queries use the reader while inserts stay on
`CLICKHOUSE_URL`.
- `RUNS_LIST_CLICKHOUSE_URL` (optional): a dedicated client for the runs
list (dashboard list, runs list API, live reload, child-status counts),
so the highest-traffic read path can target its own instance.
## Safety
Only read-only clients fall back to the reader: logs, query, admin, runs
list, the pending-version lookup, and the realtime run-id resolver. The
query page is constrained to read-only (the TSQL parser rejects anything
that is not a `SELECT`, and a `readonly` setting is applied). The
task-events client routes inserts to the writer and queries to the
reader per method, so a write can never reach the reader. Pure-write
clients (event inserts, replication) always use `CLICKHOUSE_URL`.
Note: this PR targets a baseline branch rather than `main` so the diff
stays scoped to the read-replica changes. It will be retargeted to
`main` before merge.
---------
Co-authored-by: Eric Allam <eallam@icloud.com>
## Summary
Adds a set of PAT-authenticated management API endpoints so orgs,
projects, members/invites, environment variables, and a few
project/environment settings can be managed programmatically (scripting,
automation) rather than only through the dashboard. Each route is a thin
wrapper over the **existing** service the dashboard already uses, with
the same authorization applied at the route layer - no new business
logic.
## Endpoints
**Organizations**
- `POST /api/v1/orgs` - create an org (`createOrganization`)
- `PATCH /api/v1/orgs/:orgParam` - rename (title)
- `DELETE /api/v1/orgs/:orgParam` - soft-delete
(`DeleteOrganizationService`; keeps the active-subscription guard)
**Members & invites**
- `GET /api/v1/orgs/:orgParam/members` - list members + pending invites
- `DELETE /api/v1/orgs/:orgParam/members/:memberId` - remove a member
(last-member guarded)
- `POST /api/v1/orgs/:orgParam/invites` - invite by email
(`inviteMembers`, sends the invite email)
- `DELETE /api/v1/orgs/:orgParam/invites/:inviteId` - revoke an invite
**Projects**
- `PATCH /api/v1/projects/:projectRef` - rename
(`ProjectSettingsService`)
- `DELETE /api/v1/projects/:projectRef` - soft-delete
(`DeleteProjectService`)
- `PUT /api/v1/projects/:projectRef/default-region` - set the default
region by worker-group name (`SetDefaultRegionService`)
- project GET/list now return `defaultRegion` (worker-group name, or
null when unset)
**Environments**
- `POST /api/v1/projects/:projectRef/:env/pause` and `/resume`
(`PauseEnvironmentService`)
- `POST /api/v1/projects/:projectRef/:env/regenerate-api-key` - rotate
the env secret key (`regenerateApiKey`, RBAC `write:apiKeys`)
- env var create now accepts an optional `isSecret` flag
## Auth & authorization
- All routes authenticate with a **Personal Access Token**
(`Authorization: Bearer tr_pat_...`).
- Org/project routes are built on the PAT route builders in
`apiBuilder.server.ts`: `createLoaderPATApiRoute` (already existed) and
**`createActionPATApiRoute`** (added here - the loader builder had no
mutation counterpart). The builder runs auth, resolves the org/project
role-floor via `context`, and enforces a declarative `authorization`
block using the same RBAC actions the dashboard applies
(`manage:organization` / `read:members` / `manage:members` /
`manage:project`). Handlers keep a membership-scoped query as the floor,
so a non-member gets a 404. This also gives these routes `tenantContext`
user attribution (Sentry) and `ServiceValidationError`-to-status mapping
for free.
- **Membership floor (important).** The OSS RBAC fallback grants a
permissive ability, so `ability.can(...)` can't reject a non-member on
self-hosted. Every handler therefore resolves the target scoped to the
caller's membership (`members: { some: { userId } }`) → 404 for
non-members. `authorization` is the *role* gate; this is the *tenant*
gate. `resolveOrganizationForApiUser`
(`organizationApiAccess.server.ts`) is the org-tier version of the
existing `findProjectByRef` - org-addressed PAT routes are new, so no
such helper existed before.
- Env-tier routes reuse the existing `authorizePatEnvironmentAccess`
(`write:apiKeys`).
### What `createActionPATApiRoute` gives you
A route is pure declaration - the builder handles auth, RBAC,
validation, tracing, and error mapping:
```ts
export const action = createActionPATApiRoute(
{
method: "PUT", // one verb, or ["PATCH", "DELETE"] for multi-verb routes
params: ParamsSchema,
body: SetDefaultRegionRequestBody, // zod-validated
context: async ({ projectRef }) => { // resolve the org for the RBAC role-floor
const project = await prisma.project.findFirst({
where: { externalRef: projectRef, deletedAt: null },
select: { organizationId: true },
});
return project ? { organizationId: project.organizationId } : {};
},
authorization: { action: "manage", resource: () => ({ type: "project" }) },
},
async ({ params, body, authentication, ability }) => {
// auth + authz already enforced. Just do the work.
// `throw new ServiceValidationError("Region not found", 400)` → mapped to that status.
return json({ ok: true });
}
);
```
Handled for you, so handlers stay thin:
- **Method allowlist** - `method` accepts a verb or an array; any other
verb → `405` with an `Allow` header, *before* auth runs:
```ts
const allowedMethods = method ? (Array.isArray(method) ? method :
[method]) : undefined;
if (allowedMethods && !(allowedMethods as
string[]).includes(request.method.toUpperCase())) {
return json({ error: "Method not allowed" }, { status: 405, headers: {
Allow: allowedMethods.join(", ") } });
}
```
- **PAT / user-actor auth** in a single roundtrip → `401` on
missing/invalid/revoked token.
- **RBAC** - `context` computes the caller's role-floor for the target
org/project; `authorization` gates it → `403` with a structured error
body.
- **Sentry attribution** - `tenantContext.enrich({ userId })` so events
from the handler carry the acting user.
- **Typed errors** - a thrown `ServiceValidationError` is mapped to its
`.status` (default 400); anything else → `500`, and expected boundary
errors are logged as `warn` (kept out of Sentry).
- **Validation** - params / query / headers / body are all zod-checked →
`400` with details.
## Notes for reviewers
- Everything wraps an existing service; the intent is API parity for
things that are currently dashboard-only, not new behaviour.
- `createActionPATApiRoute` is new shared infra (the PAT + RBAC mutation
builder that didn't exist). It's self-contained - the loader builder and
existing routes are untouched.
- `@trigger.dev/core` gets one additive field (`defaultRegion` on the
project response, optional/nullable for client-server version skew) -
changeset included, patch.
- `removeTeamMember`'s last-member guard is now atomic (Serializable
transaction via the `$transaction` helper, with retry), so the dashboard
and API both get it server-side. Added a `## Transactions` rule to
`apps/webapp/CLAUDE.md` (always use the `$transaction` helper);
migrating the remaining direct usages is tracked in TRI-11698.
## Open questions
- ~~Is PAT the right auth (vs OAT for automation)?~~ **Resolved: PAT.**
Organization Access Tokens are currently internal-only (used by the
image builder) and not user-accessible, so they can't back this yet.
- Should any of these be gated behind a flag or scope?
- Naming/shape of the routes.
## Summary
Batch triggers that use per-item idempotency keys could take seconds
instead of milliseconds when the target task had a large run history.
This keeps the idempotency lookup fast regardless of how many runs a
task has accumulated.
## Root cause
The batch path checks which items already have runs by looking up their
idempotency keys with a single `WHERE runtimeEnvironmentId = ? AND
taskIdentifier = ? AND idempotencyKey IN (...)` query. On a very large
`TaskRun` table Postgres underestimates the row count of a specific
`(environment, task)` pair, so once the `IN` list grows past a handful
of keys it stops doing per-key index probes and instead scans every run
for that `(environment, task)` and filters the keys in memory. The cost
is then flat and large regardless of how many keys are being checked,
and a routine `ANALYZE` does not correct the estimate at that table
size.
## Fix
Look each idempotency key up on its own, batched into a `UNION ALL` of
point lookups (chunked, run with bounded concurrency). Each branch is an
equality on all three columns of the unique index, so the planner can
only do a per-key index probe and can never fall back to the range scan.
Same results, same columns, confined to the batch trigger path.
### Problem
Webapp HTML references content-hashed /build assets, and each Docker
image contains exactly one build with a hard 404 for unknown hashes.
During a rolling deploy, a client holding HTML from the old build may
request old asset hashes from a replica running the new image, causing
missing styles or failed chunk loads.
The page should recover automatically once a compatible build becomes
available, without reload loops or unnecessary interruptions during
normal deployments.
### What changed
- Build changes alone do nothing — no polling, no automatic reloads.
- If a CSS or JavaScript asset fails to load, a recovery overlay is
shown immediately.
- While the server still reports the same build, the client polls for a
newer build using exponential backoff (up to ~60s). As soon as a newer
build is detected, the page reloads automatically.
- If no newer build appears within the timeout, recovery falls back to a
manual Reload action.
- If recovery still fails after the automatic reload, the client stops
retrying and displays a final recovery screen instead of entering a
reload loop.
- Recovery preserves form values and scroll position across the
automatic reload.
## Summary
When the run-ops split is enabled, the legacy run-ops database client
was always constructed as its own connection pool, even when it points
at the same database as the primary (control-plane) client. On setups
where those two DSNs resolve to the same physical database, this opened
a second, redundant pool and doubled the number of connections used
against that database. This change makes the legacy client reuse the
primary client's pool whenever their DSNs point at the same database,
and only open a separate pool when they genuinely differ.
## Fix
A small `sameDatabaseTarget` comparison (host, port, database name,
user) decides whether the legacy DSN points at the same database as the
primary. When it does, the legacy handle reuses the primary client by
reference, so no second pool is opened. When the DSNs diverge, the
legacy client is built independently as before, so the split still works
once the databases are actually separate.
Two smaller changes ride along:
- An optional per-pool limit for the run-ops read replica, which
connects unpooled and so draws raw backend connections; unset, it falls
back to the existing default and behaviour is unchanged.
- A startup warning about a missing legacy replica URL is now suppressed
when the legacy client shares the primary pool, where it would be
misleading.
## Verification
Booted the webapp end-to-end in three modes and confirmed the pools
opened as expected via the client's own startup logs and live backend
connection counts: split off (single pool), split on with a shared
database (legacy reuses the primary pool, no doubling), and split on
with separate databases (legacy opens its own pool).
The `ResourceMonitor` server-side logging helper is no longer used. It
periodically logged the webapp process own memory, disk, and CPU usage
behind the `RESOURCE_MONITOR_ENABLED` flag (off by default), and was
also exported from `@trigger.dev/core/v3/serverOnly` with no other
consumers.
This removes the helper, its `@trigger.dev/core` export, the webapp
wiring, and the `RESOURCE_MONITOR_ENABLED` env var. The supervisor has
its own unrelated `ResourceMonitor` class, which is left untouched.
## Summary
Adds `EVENT_REPOSITORY_POSTGRES_WRITES_DISABLED` (default off), which
makes the task-event store skip all PostgreSQL `TaskEvent` writes. It's
for deployments that store task events in ClickHouse
(`EVENT_REPOSITORY_DEFAULT_STORE=clickhouse_v2`) and no longer want the
PostgreSQL copy.
## How it works
The guard sits at the single postgres write boundary,
`TaskEventStore.create` / `createMany`, so it covers every write path
(OTLP ingestion and run-lifecycle events) with one check. Reads are
untouched (`findMany` / trace queries / streaming), so existing
PostgreSQL events remain readable.
Leave it off unless the default store is `clickhouse_v2`, otherwise task
events for any run still routed to PostgreSQL would be dropped.
## What
The `GET
/api/v1/projects/:projectRef/background-workers/:envSlug/:version`
endpoint loaded each file's tasks through the nested `files.tasks`
relation. Prisma resolves that as a separate query:
```sql
SELECT id, slug, "fileId" FROM "BackgroundWorkerTask" WHERE "fileId" IN (...)
```
`BackgroundWorkerTask.fileId` is not indexed — the FK constraint exists,
but Postgres does not auto-create an index for foreign keys — so on a
large table this can only run as a sequential scan, which gets
progressively slower as the table grows and was observed taking minutes
per call in production.
The loader already loads every task for the worker via `tasks: true`,
which uses the indexed `workerId` relation, and those rows already
include `fileId`. This PR groups task slugs by `fileId` in memory from
that already-loaded data and drops the `files.tasks` include entirely.
## Behavior change (latent bug fix)
The response shape is unchanged, but there is a semantic correction for
**source files reused across worker versions** (files are de-duplicated
by `@@unique([projectId, contentHash])`, so one file row can be linked
to many workers).
- **Before:** `file.tasks` came from the `BackgroundWorkerFile.tasks`
relation, i.e. *every* `BackgroundWorkerTask` with that `fileId` —
across all workers sharing the file. So a worker's manifest could list
tasks it doesn't actually have.
- **After:** `file.tasks` is grouped from the queried worker's own
tasks, so it reflects only that worker version's tasks.
Verified on a local DB: 460 files are referenced by tasks from more than
one worker; of 6819 (worker, file) pairs, 6 differ — all one file where
the old union leaked a task slug (`cancellation-test`) into worker
versions that never had it. The new per-worker behavior is the correct
one for a worker-version manifest. (Thanks to the automated review for
flagging this.)
## Analysis
Captured the exact SQL before/after by instrumenting Prisma against real
data (a worker with 62 files):
- **Before:** 5 statements, including the `WHERE "fileId" IN (...)`
scan.
- **After:** 4 statements; the `fileId` query is gone and the other four
are identical.
EXPLAIN of the two access paths:
```
Before WHERE "fileId" IN (...)
Seq Scan on "BackgroundWorkerTask"
Filter: ("fileId" = ANY (...)) -- reads the whole table, scales with table size
After WHERE "workerId" IN (...)
Index Scan using "BackgroundWorkerTask_workerId_slug_key"
Index Cond: ("workerId" = ...) -- bounded by matching rows, scale-independent
```
No new index is required: the `workerId` access path is already covered
by the existing `BackgroundWorkerTask_workerId_slug_key` unique index.
## Testing
- `pnpm run typecheck --filter webapp` passes.
- Query capture + EXPLAIN performed against a local database seeded with
real worker/file/task data.
## Summary
Run-graph data (runs, batches, waitpoints, and their related tables) can
now live in a database separate from the control plane, with every read
and write routed to the correct database by each run's residency. This
makes reading and writing run data more reliable once the two are split,
and is a no-op for single-database installs.
## Design
- Run-graph table access goes through the run-store router, which
selects the legacy or the new run-ops store per run instead of assuming
one shared client.
- The legacy run-ops client is now independently pointable, so legacy
run data can be served from its own database (and replica) rather than
the control-plane connection.
- Run-graph writes go straight to the run-graph database instead of
being forwarded through the control plane, and replication targets are
split so runs in the new database still replicate to analytics without
under-counting.
- Read-through slots refuse the control-plane client, so a missing
residency fails loudly instead of silently reading the wrong database.
- Migration `20260710120000_drop_remaining_run_graph_seam_foreign_keys`
drops the foreign keys that still crossed the run-graph / control-plane
seam, which is what lets the two live in separate databases.
The split stays off unless explicitly enabled and the two databases are
confirmed physically distinct; startup fails closed otherwise.
Verified by running the full dashboard end-to-end suite against both a
single-database configuration and a three-database configuration
(control plane, the new database, and a physically separate legacy
database), with runs on both residencies. No misrouted reads in either
configuration.
## Summary
v3 (the engine that ran the SDK v3 era, internally
`RunEngineVersion.V1`) is end-of-life. Following the removal of the v3
execution apps
([#4194](https://github.com/triggerdotdev/trigger.dev/pull/4194)) and
the legacy dev websocket
([#4198](https://github.com/triggerdotdev/trigger.dev/pull/4198)), this
removes the remaining v3 execution stack from the server.
Clients still on v3 (an old SDK or CLI that has not upgraded) keep
getting a clear "upgrade to v4" response. Triggers, batch triggers,
reschedules, and deploys that resolve to v3 are rejected with a graceful
4xx pointing at the migration guide, never a 5xx, so a stale client
cannot affect server health. Self-hosted instances still running v3
should stay on the 4.5.x release line until they migrate.
## What is removed
- The MarQS queue and its shared/dev queue consumers.
- The v3 socket.io namespaces (coordinator, provider, shared-queue) and
the v3 run lifecycle services (attempt, checkpoint, and batch-resume).
- The graphile-worker background job system; all live jobs already run
on `@trigger.dev/redis-worker`.
- The `DEPRECATE_V3_ENABLED` flag: v3 is now rejected unconditionally,
so the flag is gone.
- Unused v3 exports from `@trigger.dev/core` (the `v3/zodNamespace`
subpath and the legacy socket message catalogs) and the now-dead MarQS
environment variables.
## What stays
The v4 engine is untouched. The graceful v3 rejection boundary stays,
`determineEngineVersion` still detects a v3 project so it can reject it,
and the batch service plus batch-completion worker stay for current
clients. Live queue concurrency limits and metrics now read from the v4
run engine instead of MarQS, and a brand-new dev environment now
defaults to v4.
## Dependency cleanup
Removes webapp dependencies left unused by this change: `seedrandom` and
`semver` (only the removed v3 code used them) plus a set that was
already dead, their orphaned `@types` packages, and two dead files. Adds
a `knip:deps` script and a `knip.json` config so unused dependencies can
be found the same way going forward.
Chris Arderne
