The pre-gate idempotency claim was eligible only when the org was on the mollifier. Concurrent same-key triggers that straddle a runTableV2 flip can mint into different physical tables, whose per-table unique constraints can't see each other, so two runs could share one key. The claim is now also eligible when the org is cut over to the v2 run table, serialising those triggers through Redis.
A single-format id-list narrows findRuns to one physical table, but the ordered+limited path still built the cross-table comparator and threw the time-key guard; it now delegates natively to the one table (Postgres orders within a single table fine). Separately, the in-memory merge comparator ordered strings by code unit while the Postgres keyset continuation orders by the database collation (en_US); switching the comparator to localeCompare makes them agree, so a tied-createdAt boundary spanning both tables no longer skips or duplicates a row.
findRuns now throws when given skip: offset pagination cannot span the two run tables, where each would independently skip N rows from its own result rather than N from the merged result. For an id-list predicate (id in [...]), it now queries only the table whose id format can contain those ids, avoiding a wasted query against an empty task_run_v2 while it is unpopulated during rollout.
## Summary
Adds read methods to `RunStore` (`findRun`, `findRunOrThrow`,
`findRuns`) and routes every Postgres read of `TaskRun` through them,
mirroring how writes already go through the store. Behavior-preserving:
each relocated read keeps its exact query, field selection, and database
client (writer, replica, or transaction). This lets `TaskRun` reads be
retargeted to a different backing store later without touching call
sites.
Stacked on #3981 (the write adapter); that PR is the base of this one.
## Scope
In scope: the run engine, webapp services, presenters, and route
loaders. Three reads that pulled `TaskRun` in through a parent model's
relation `include` (alert delivery, batch results, attempt-dependency
cancellation) are decomposed to fetch the run(s) through the store and
stitch them back, since a relation include would not follow `TaskRun` to
a new table.
Left reading the existing table (out of scope): the legacy MarQS paths,
the legacy trigger idempotency read, and one raw-SQL recovery script
(commented for revisiting at cutover).
## Notes
Reads default to the read replica; callers pass the writer or a
transaction client wherever the original read did, so writer-vs-replica
behavior is unchanged.
Poll for the ClickHouse row with a bounded deadline instead of a fixed sleep, which is flaky under replication lag variance, and stop the replication service in a finally block so a failing assertion cannot leak it into later tests.
Follow-up to #3992, which gated the send runner-side - but only for new
runner images. Existing runners still POST a debug log per line.
When `SEND_RUN_DEBUG_LOGS` is off (default), the route now drops the
request immediately: `skipBodyParsing` skips the body read/parse, a bare
handler returns 204, no wide event. The route stays registered so it
avoids the `No route match` error log; the only per-request log left is
the framework's `logger.debug` trace, suppressed at the default `info`
level. Still counted by request metrics, and 204 is non-retryable so no
retry storm.
Adds a `skipBodyParsing` flag to the internal HTTP server.
Runners were POSTing a debug log to the supervisor for every log line -
one request per line, unbatched and unconditional. The supervisor
already has a `SEND_RUN_DEBUG_LOGS` toggle (off by default) that
discards them on receipt, but the runner fired the request regardless,
so the traffic hit the supervisor either way.
This gates the send at the source. The runner now reads
`TRIGGER_SEND_RUN_DEBUG_LOGS` (off by default, injected by the
supervisor from its existing `SEND_RUN_DEBUG_LOGS` setting) and skips
the POST entirely when disabled. Local log output is unchanged. Dev runs
use a separate path and are unaffected.
## Summary
Adds an environment-scoped HTTP API over the Errors feature, mirroring
the runs API. Task-run failures are grouped by a fingerprint into "error
groups," and this exposes everything you can do with them in the
dashboard:
- `GET /api/v1/errors` lists error groups, with
`filter[taskIdentifier]`, `filter[version]`, `filter[status]`
(`unresolved`/`resolved`/`ignored`), `filter[search]`, a time range, and
cursor pagination.
- `GET /api/v1/errors/{errorId}` retrieves a single group (summary,
lifecycle state, affected versions).
- `POST /api/v1/errors/{errorId}/{resolve,ignore,unresolve}` changes its
state.
- `GET /api/v1/runs?filter[error]={errorId}` lists the runs behind a
group.
Request and response schemas are exported from `@trigger.dev/core/v3` so
the SDK can reuse them, and all endpoints are documented in the API
reference (OpenAPI). `errorId` is the `error_<fingerprint>` friendly id.
## Attribution
State changes record who made them. A plain environment API key has no
user, so `resolvedBy`/`ignoredByUserId` stay null. When the caller uses
an environment JWT obtained by exchanging a personal access token or a
delegated user token at `POST /api/v1/projects/:ref/:env/jwt`, that
exchange now stamps an `act` delegation claim, and the write endpoints
read `act.sub` to attribute the change to the acting user. This is the
first endpoint to consume the `act` claim, so two small pieces of
plumbing ride along: the exchange stamps `act` for personal-access-token
subjects too (it was delegated-token-only), and the public-JWT
bearer-auth path surfaces `act.sub` to the handler.
Built on the delegated-token work in #3997.
## Summary
The CLAUDE.md audit job (`.github/workflows/claude-md-audit.yml`)
frequently hits its 15-turn cap before it finishes reviewing a PR, so
the job fails without posting a verdict. For example, the audit job
failed on [this
run](https://github.com/triggerdotdev/trigger.dev/actions/runs/27837408945/job/82390460772?pr=3990).
This raises `--max-turns` from 15 to 25 to give the review room to
complete, and pins `--model claude-opus-4-8` (the job previously
inherited the action default model).
runsReplicationService co-publishes task_run_v2 alongside TaskRun. It is a column-identical clone, so its WAL rows flow through the same transform into the same ClickHouse table, keeping the mirror complete once orgs cut over to v2 run ids. task_run_v2 needs REPLICA IDENTITY FULL, applied the same out-of-band way as TaskRun, so update and delete events carry the old row.
LogicalReplicationClient gains an optional additionalTables option. These are published alongside the primary table in the same publication, and their WAL events stream through the same data handler. When the publication already exists, missing tables are added via ALTER PUBLICATION ADD TABLE (online, slot-preserving) instead of erroring, so a publication can gain a table without a drop and recreate.
Adds a per-org runTableV2 feature flag, read in memory at the single run-id mint site in the trigger path. When on, the org mints a KSUID id for new runs (routing them to task_run_v2); off, the default, keeps minting legacy ids. The read is a pure lookup on the org featureFlags already loaded at auth, so the trigger path adds no query. RunStore routes purely by id format and never sees this flag.
findRun and findRunOrThrow route by the id/friendlyId in the predicate. A lookup that carries neither (the idempotency-key dedup, or an "are there any runs in this environment" check) previously defaulted to the legacy table and would miss a match that lives in task_run_v2. Such predicates now query both tables in parallel and return the first match, so a reused idempotency key is found wherever its run lives and no duplicate is created.
runsBackfiller paginates on a (createdAt, id) keyset instead of id alone. The ClickHouse runs list restores ClickHouse ranking in memory after hydrating rows by id, since a single SQL order cannot span the two tables.
findRuns now queries both TaskRun and task_run_v2 and merges the two ordered streams into one result. Ordered, limited reads require a time-based key (createdAt) because cuid and ksuid ids do not sort into a shared range, so id alone cannot order the union.
## Summary
Adds a short-lived, delegated token (`tr_uat_...`) that authenticates
against the API as a user without handing out a long-lived personal
access token. You mint one from a PAT, optionally narrow it to a set of
scopes, and give it a lifetime; the API then treats requests as that
user, subject to their role.
`trigger.dev mint-token` is the entry point (it uses your stored PAT):
```bash
UAT=$(trigger.dev mint-token --ttl 3600 --cap read:runs)
```
The token works anywhere a PAT does for user-level endpoints, and can be
exchanged for an environment JWT at `POST
/api/v1/projects/:ref/:env/jwt` to reach environment-scoped data (the
same exchange a PAT supports).
## How it works
A user-actor token is a short-lived JWT verified by a new first-class
`authenticateUserActor` method on the RBAC plugin. Self-hosters get a
built-in fallback; role-aware enforcement comes from the plugin.
Effective permissions are the intersection of the user's role and the
token's optional scope cap, so a token is only ever narrower than the
user, never broader.
Minting is restricted to personal access tokens (a token can't mint
another one, and an environment key can't mint one). Tokens default to a
1 hour lifetime (max 365 days). When exchanged for an environment JWT,
the user is stamped on it for attribution and the scope cap is carried
through.
Select the TaskRun or task_run_v2 table per operation from the run id's
format (KSUID routes to v2, anything else to legacy) via a runModel helper,
so a run is read and written in its own table. Batch and predicate-keyed
operations span both tables. Behavior-preserving for legacy runs.
Production drops the foreign keys on and referencing the run tables, but the
test harness builds via prisma db push, which recreates them from the schema
relations. Drop them after the push so test databases match production and a
run can live in either run table.
Give TaskRunV2 the same relation surface as TaskRun (belongs-to plus child
collections, with child relations sharing the existing scalar fields) so run
reads through the store can include relations regardless of table. No DB
foreign keys: stripped in production migrations and in the test harness.
Drop the 14 child-table foreign keys that referenced TaskRun.id so a child
row can reference a run in either the legacy or the new run table by plain
scalar. Run integrity moves to app code, symmetric with TaskRun's already
dropped outgoing foreign keys. Relations stay in the Prisma schema.
Add task_run_v2 as a scalar clone of TaskRun with no foreign-key
constraints, plus a (createdAt, id) index for keyset pagination. Unused for
now; new runs are routed to it by id format in a later change.
Add an isomorphic generateKsuid() and an isKsuidId() format check to the
run-id scheme, so a run's id can encode which table it belongs to. Additive
groundwork: nothing mints KSUIDs yet, and generate() (cuid) is unchanged.
## Summary
Replaces the multi-select popover task type filter on the Tasks page
with a single-select segmented control: **All** plus icon-only
**Agent**, **Standard**, and **Scheduled** segments. Each segment has a
tooltip showing its label and a number-key shortcut (0-3), and the
search field no longer autofocuses so the shortcuts work on page load.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
## Summary
The Personal Access Tokens page now shows each token's maximum role in a
new column, so you can see at a glance what a token is capped to. The
column only appears when an RBAC plugin is installed, and shows "-" for
tokens with no cap. Its header tooltip reuses the same explanation shown
in the create-token panel.
Importing the service pulls the cancel chain, which eagerly initializes the
concurrency tracker singleton and requires REDIS_HOST/REDIS_PORT at import
time, so the suite cannot load in the unit-test shards without stacking
mocks. The decompose it covered is exercised by the analogous batch-results
container test and confirmed by review, so drop this one rather than mock
the tracker and cancel chain.
## Summary
Adds the `4.5.0-rc.7` entry to the AI chat changelog, covering the
agent-facing changes in
[v4.5.0-rc.7](https://github.com/triggerdotdev/trigger.dev/releases/tag/v4.5.0-rc.7):
- `chat.headStart` now works with the `chat.customAgent` and
`chat.createSession` backends, not just `chat.agent`
- Opt-in Anthropic system-prompt caching via
`chat.toStreamTextOptions()`
- Three custom-agent-loop fixes: continuation replay, mid-stream
steering, and task-backed tools
- `trigger skills` follow-ups: `trigger-` namespacing, SDK-bundled docs,
and a new cost-savings skill
Generic, non-agent rc.7 items (the CLI uninitialized-project error
message, run-span cost fields) are intentionally left out to keep this
changelog scoped to AI chat agents.
The new container tests import the service and presenter, which pull the
db.server singleton in through their base classes. Mock it so the tests do
not try to connect to the env database when none is reachable (the CI unit
shards), matching the existing webapp container-test pattern. The tests use
the injected testcontainer prisma for all reads.
The recovery script joins TaskRunExecutionSnapshot to TaskRun in raw SQL, so
it is the one TaskRun read not routed through the run store. Add a note to
revisit it at table cutover.
Decompose the three reads that pulled TaskRun in through a parent model's
relation include (alert, batch results, attempt dependencies): query the
parent without the include, hydrate the run(s) via RunStore in a single
batched read, and stitch them back. Preserves field selection, ordering,
null handling and the query client. Adds container-backed tests for the
batch-results and cancel-dependencies paths.
Relocate the route and loader TaskRun reads to the RunStore read methods,
preserving the exact client per site, including the replica-resolve then
writer-recheck realtime paths. Behavior-preserving.
Relocate the direct TaskRun reads in webapp services, run-engine concerns,
realtime, mollifier and metadata to the RunStore read methods, preserving
the exact client (writer, replica, or transaction) at each site. The run
hydrator now receives the store by injection. Behavior-preserving.
Relocate the direct TaskRun reads in the engine and its systems to the
RunStore read methods, preserving the exact client (writer, replica, or
transaction) at each site. Behavior-preserving; the engine test suite is
unchanged.
Add a no-args overload to findRun, findRunOrThrow and findRuns that
returns the whole TaskRun row, for callers that read a run without a
select or include.
Add findRun, findRunOrThrow and findRuns to RunStore, mirroring the
existing write methods. They pass where/select/include through the same
Prisma generics and default to the read replica, while letting the caller
pass the writer or a transaction client when needed. This lets Postgres
reads of TaskRun be routed through the store the same way writes already
are. Additive only; no call sites change yet.
Pushes new organizations and users into the Attio CRM at signup time,
for Customer Success (TRI-10431).
- Orgs → Attio `workspaces`, users → Attio `users`, keyed on Attio's
built-in unique `workspace_id` / `user_id` so writes are idempotent
upserts.
- Runs on the common Redis worker (not inline), so a slow or unavailable
Attio never blocks the signup path; failures retry (3 attempts).
- Hooks: user-created (alongside the existing Loops call) and
org-created (`createOrganization`).
- Gated behind `ATTIO_API_KEY`, no key means the sync is skipped
entirely, so OSS / self-hosted installs are unaffected.
Only creation is covered here (the record "shell"); spend, runs, plan
changes, churn, and role/relationship linking are populated by the
scheduled full sync, tracked separately.
**Deploy note:** requires an Attio API key set as `ATTIO_API_KEY` in the
webapp env, with scopes **Records (read-write)** + **Object
Configuration (read)**, the assert/upsert endpoint reads object config
to resolve the matching attribute. Without the key the sync no-ops.
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Testing
Ran the webapp locally with the change applied; it compiles and serves.
The edit only swaps the chart card title string from "LLM spend" to "LLM
spend ($)" on the agent landing page.
---
## Changelog
The agent dashboard "LLM spend" chart label now includes the currency
unit, reading "LLM spend ($)".
---
## Screenshots
_[Screenshots]_
💯
A batch of technical-SEO fixes across the docs, all reader-facing
(titles, links, redirects):
- Canonicalize the duplicate CLI command pages: the bare `/cli-dev` and
`/cli-deploy` paths now permanently redirect to their `-commands`
equivalents, and a duplicate navigation entry is removed.
- Give the three pages that all rendered as "Overview" distinct titles
(Building with AI, self-hosting overview, Management API overview), with
sidebar labels unchanged.
- Replace the generic "Learn more" links in the introduction's
build-extension list with descriptive anchor text.
- Switch two http links to https in the Supabase guides, point a
troubleshooting page's help link to Discord, and add missing meta
descriptions to three help and troubleshooting pages.
## Summary
Several dashboard routes performed actions a restricted role should not
be able to do (cancel or replay runs, manage prompt versions, invite and
manage members, manage billing) without any permission check. This adds
role-based permission enforcement to those routes, and disables the
matching UI controls (with a tooltip) when the current role lacks
permission.
Covered actions:
- Runs: cancel and replay (single, bulk create, bulk abort)
- Prompts: create or edit override versions, and promote a version to
current
- Members: invite, resend invite, revoke invite
- Billing: change plan, billing alerts, and the customer portal
## How
Each affected route now goes through the `dashboardLoader` /
`dashboardAction` route builders with an `authorization` block declaring
the required permission (or a per-intent check where one route handles
several intents). Existing tenancy and data-scoping queries are
untouched; this only layers permission checks on top. The UI follows
disable-don't-hide: controls stay visible but disabled with a "You don't
have permission to ..." tooltip.
Two reusable pieces support this: `checkPermissions(ability, checks)`
turns a set of checks into a boolean map a loader returns to the client,
and `PermissionButton` / `PermissionLink` disable the underlying control
and show a tooltip when a permission flag is false.
## Behaviour
No change in the default configuration: permissions are permissive, so
every control stays enabled and every route behaves as before. The
checks only take effect when an RBAC plugin is installed. This also
makes role assignment on invite-accept non-fatal, so a failure there
cannot block joining an org.
Verified with `pnpm run typecheck --filter webapp`; `checkPermissions`
has unit tests.
## Summary
Adds a "Stopping generation" section to the Custom agents page. It
documents how stop works when you drop down from `chat.agent` to
`chat.createSession`: pass `turn.signal` (a combined stop-and-cancel
`AbortSignal`) to `streamText`, and `turn.complete()` cleans up the
aborted partial, accumulates it as its own assistant message, and keeps
the run alive for the next turn. `turn.stopped` distinguishes a user
stop from a full run cancel.
Until now the createSession stop story only existed as scattered fields
in the reference table; the client side (`transport.stopGeneration`) and
the `chat.agent` run-callback signals were documented, but not the
custom-agent turn loop. Steering for these backends is already covered
on the pending messages page, which this page links to.
## Summary
Adds a "Duration and cost while paused" section to the human-in-the-loop
page. It explains that a HITL pause (a no-execute tool waiting on
`addToolOutput`) suspends the run and frees compute, so the human's
thinking time does not count against `maxDuration` (which measures
active CPU time and excludes suspended waitpoint time, the same as
`wait.for`). Customers don't need to raise `maxDuration` or end the run
to support long human waits.
This was a recurring point of confusion: readers assumed the pause holds
the run open and burns the budget. Also updates the how-it-works
pseudocode ("Agent suspends (compute freed)") and links `wait.for` and
`maxDuration` on first mention.
Dan Sutton