## Summary
Adds read methods to `RunStore` (`findRun`, `findRunOrThrow`,
`findRuns`) and routes every Postgres read of `TaskRun` through them,
mirroring how writes already go through the store. Behavior-preserving:
each relocated read keeps its exact query, field selection, and database
client (writer, replica, or transaction). This lets `TaskRun` reads be
retargeted to a different backing store later without touching call
sites.
Stacked on #3981 (the write adapter); that PR is the base of this one.
## Scope
In scope: the run engine, webapp services, presenters, and route
loaders. Three reads that pulled `TaskRun` in through a parent model's
relation `include` (alert delivery, batch results, attempt-dependency
cancellation) are decomposed to fetch the run(s) through the store and
stitch them back, since a relation include would not follow `TaskRun` to
a new table.
Left reading the existing table (out of scope): the legacy MarQS paths,
the legacy trigger idempotency read, and one raw-SQL recovery script
(commented for revisiting at cutover).
## Notes
Reads default to the read replica; callers pass the writer or a
transaction client wherever the original read did, so writer-vs-replica
behavior is unchanged.
Two defensive fixes to the native realtime backend's run-change
publishing (behind a feature flag, off by default), so turning it on can
never destabilize the run lifecycle.
**Never throws at the caller.** Publish sites run synchronously on the
run-engine event bus and the metadata flush loop. The internal publish
was already wrapped in try/catch, but lazy construction (singleton +
metrics) and record encoding ran before that guard, so a throw could
propagate into a run lifecycle operation. The public
`publishChangeRecord` / `publishManyChangeRecords` helpers now wrap the
whole call and log-and-drop on failure.
**Bounds outage buffering.** The publisher connection caps
`maxRetriesPerRequest` at 1 (vs ioredis's default of 20), so during a
pub/sub Redis outage a publish rejects after ~1 reconnect cycle instead
of holding commands in memory for ~20s. A dropped publish is
latency-only, since the consumer has a periodic backstop full-resolve.
The offline queue stays on, so the first publish after a process boots
still flushes once the connection is ready.
## Summary
Two read-replica races on the session APIs could break chats whose first
activity lands inside the replication window (or any time the replica
lags):
1. A session's first `.in` append or `.out` subscribe could fail with a
404 for a session that exists on the writer, because the route resolved
the Session row on the replica only.
2. `ensureRunForSession` probed run liveness on the replica, so a probe
miss on a run triggered moments earlier was judged "run is dead" and a
second live run was spawned for the same session. Both runs then
consumed the same input stream, producing duplicated turns and doubled
responses (and doubled LLM cost).
## Fix
Liveness now re-probes the writer before declaring the current run dead
(the old code already fell back to the writer, but only to recover the
friendlyId, after the wrong verdict was made). Session resolution on the
append and subscribe/init routes goes through a new
`resolveSessionWithWriterFallback`, which stays replica-first on the hot
path and only touches the writer on a miss.
Reproduced and verified against a local streaming replica with an
artificial apply delay: pre-fix, a send immediately after session
creation reliably produced either the 404 or two executing runs with a
doubled response; post-fix, the same flow produces exactly one run and
one response.
Also rides along: the local docker replica's default apply delay drops
from 150ms to a realistic 20ms (override via `REPLICA_APPLY_DELAY` when
you want to deliberately widen the race window).
## Summary
When the realtime runs feed (the backend behind the `realtimeBackend`
feature flag) hydrates a change from a Postgres read replica, the read
can race the replica's apply of the very write that triggered it. The
delivered row then carries the previous change's content, and an
isolated final change (for example a last `metadata.set` before a run
goes quiet) is not corrected until the roughly 20 second backstop poll.
Measured against a replica with deliberate apply delay, every delivery
trailed exactly one change behind and a final change stranded for the
full backstop interval.
## Fix
Publishers stamp each change record with the committed row's
`updatedAt`, taken from writes they already perform, so the stamp costs
no extra queries. The router delays its wake hydrate until the replica's
measured lag has passed, anchored to that timestamp: a record that has
already spent longer than the lag in transit is hydrated immediately, so
only the racing leading edge ever waits. After hydrating, a tripwire
compares each row against its record's watermark. Still-stale rows are
withheld and retried briefly, and each detection feeds the lag estimate.
If retries run out, the rows are delivered anyway (liveness over
freshness) and follow-up re-hydrates emit the fresh version through the
normal working-set diff once the replica catches up, with the backstop
as the terminal net.
Replica lag is sampled reader-side only, and only while feeds are
active. Aurora reports live lag via `aurora_replica_status()`; vanilla
Postgres can only report "caught up or not" (mid-apply lag is not
honestly measurable from a replica), so tripwire observations floor the
estimate there. Deployments without a replica resolve to zero lag and
skip the gate entirely. Tunables live under
`REALTIME_BACKEND_NATIVE_REPLICA_LAG_*`, and
`realtime_native.stale_hydrates` plus
`realtime_native.replica_lag_estimate_ms` make replica health
observable.
Two adjacent fixes: a metadata update that writes nothing no longer
publishes a change record, and buffered parent and root metadata
operations now publish when the flusher writes them, so those changes
wake live feeds instead of waiting for the backstop.
For local testing, `docker-compose` gains an opt-in `database-replica`
service (compose profile `replica`) with a configurable
`recovery_min_apply_delay`, which reproduces replica-lag behavior
deterministically. With the gate disabled this rig reproduces the
one-change-behind delivery exactly; with it enabled, deliveries arrive
with current content at roughly the true replica lag, across write rates
faster and slower than the lag itself.
## Summary
Reliability and authorization fixes for realtime chat sessions:
- Session-stream waitpoint delivery is scoped to the environment, so two
environments using the same session `externalId` can no longer complete
each other's waitpoints.
- The session snapshot-url routes now enforce per-session authorization,
and appending to a session's `out` channel requires secret-key auth, so
a session-scoped token can't read another session's snapshot or forge
assistant output.
- Appends that carry an `X-Part-Id` header are deduplicated on retry, so
a retried send can't duplicate a message.
- Session creation rejects expired sessions (instead of triggering a run
that can never receive input), `externalId` is immutable after creation,
and the sessions list endpoint returns friendly `run_*` ids to match the
single-session routes.
## Rollout
The waitpoint cache key gains an environment prefix. To keep waitpoints
registered by the previous deploy working across the boundary, the drain
reads both the new and the previous key for this release; the legacy
read can be removed a release later once no pre-deploy waitpoints
remain.
## Summary
Adds a second backend for the realtime runs feed (`useRealtimeRun`,
`subscribeToRunsWithTag`, `subscribeToBatch`), built to stay healthy
when a single busy environment has many subscribers watching many runs
at once. It is gated behind a feature flag with the existing backend as
the default, so nothing changes for users until it is enabled per
environment.
## Design
A run change is published once, as a small self-describing record, to a
single per-environment channel. Every feed is then a predicate over that
one stream rather than owning a channel:
- A per-instance router indexes the currently-held feeds by run, tag,
and batch. When a run changes it hydrates the affected rows once and
serializes them once, then fans the result to every matching feed. One
hot shared tag watched by many subscribers costs a single database query
and serialize, not one per subscriber.
- Feeds that don't match a change are never woken, wake delivery per
environment is coalesced on a leading edge (250ms default) so a burst of
changes costs one wake, and cold reads coalesce onto a single
short-TTL-cached resolve.
- An admission gate bounds how many cold ClickHouse resolves run
concurrently, so a mass reconnect across many distinct filters queues
instead of stampeding the database.
- Changes that land while a client is between long-polls are delivered
on its next poll instead of waiting for the periodic backstop: each
environment buffers its recent change records, subscriptions linger
briefly after the last feed closes, and a newly-armed poll replays
exactly the connection's gap.
- The per-connection replay cursors behind that are shared across
instances via Redis (a single timestamp each), so a poll landing on a
different instance behind the load balancer still reads the connection's
true gap instead of falling back to a cold resolve. Cursor reads have a
bounded deadline and degrade to the cold-read path on any Redis trouble.
- Tag subscriptions with multiple tags match runs carrying all of the
tags, mirroring the existing backend's filter semantics, and live
long-polls hold for about 20 seconds to match its cadence.
- The per-environment channel supports Redis Cluster sharded pub/sub, so
the wake path scales horizontally across shards by environment.
- The backend reports its health through OpenTelemetry metrics (delivery
lag, poll resolution paths, backstop outcomes, replay and cursor-store
activity), with a provisioned Grafana dashboard for local development.
Everything is behind the feature flag and tunable via env vars; the
existing backend remains the default.
## Summary
Three fixes to the AI agent surface in the dashboard, all surfaced while
verifying AI SDK 7 support (the SDK side is #3833):
- **AI SDK 7 telemetry rendering.** The generation-span inspector and
run metrics now read both the v6 (`ai.*`) and v7 (`gen_ai.*`) telemetry
attribute shapes. On v7 the Messages, Provider, and Tools views showed
empty/unknown before; now they render correctly.
- **HITL approvals in the conversation view.** The agent conversation
view renders human-in-the-loop tool approvals and denials (awaiting
approval, approved, denied with reason) instead of leaving the tool part
blank. This gap predated v7.
- **Chat snapshot store mismatch.** Chat session snapshots now resolve
through a single storage-key helper shared by the SDK write and the
dashboard read. Previously the write applied the default object-store
protocol to the key while the read fell back to a different store, so
the dashboard 404'd on the snapshot and showed only a partial
conversation.
## Summary
Two improvements to session `.in/append`:
- Oversize-body 413 responses now carry CORS headers, so browser fetches
see a readable status instead of an opaque `TypeError: Failed to fetch`.
App-side retry-on-disconnect loops no longer spin forever on a
permanently-rejected payload.
- The per-record cap is now computed precisely against S2's actual
ceiling instead of the conservative 512 KiB floor. Legitimate ~600-900
KiB tool outputs (search results, file content) now succeed;
pathological all-quote content that would double under JSON escape still
rejects cleanly.
## Design
S2 enforces a per-record metered size of `8 + 2*H + Σ(header name +
value) + body ≤ 1048576` bytes. With no record headers (our case), the
budget reduces to `body ≤ 1048568`. Verified empirically against cloud
S2 — append succeeds at metered=1048576 and 422s at 1048577 with `record
must have metered size less than 1 MiB`.
The old `MAX_APPEND_BODY_BYTES = 512 KiB` was derived by assuming
worst-case JSON escape doubling (every byte becomes `\"` or `\\`),
giving `(1 MiB - overhead) / 2`. Safe, but rejects ~half the legitimate
input space.
The new flow:
1. Pre-cap the HTTP body at 1 MiB (DoS guard against reading arbitrary
garbage before we can compute the wrap).
2. After reading, `S2RealtimeStreams.#appendPartByName` computes
`Buffer.byteLength(JSON.stringify({data: part, id: partId}), "utf8") +
8` and throws `S2RecordTooLargeError` (a `ServiceValidationError` with
status 413) if it would exceed S2's ceiling. The route's existing error
branch maps the throw to a 413 with a descriptive message.
The 413 CORS fix is a single-line change in `apiBuilder.server.ts` —
`wrapResponse` was being skipped on the body-too-large branch; every
other error branch wraps; the 413 was the exception.
## Test plan
- Empirically verified against cloud S2 with a boundary scan across
`[1048568, 1048569, ..., 1048576]` and across H ∈ {0, 1×5 hdr bytes,
1×14 hdr bytes} — the formula matches exactly
- Browser-side fetch on a 700 KiB POST now resolves with a readable
`status: 413` (no `TypeError: Failed to fetch`)
- A 900 KiB ASCII tool output now passes (would have 413'd at 512 KiB
pre-fix)
(`OBJECT_STORE_BASE_URL`) and a named protocol provider
(`OBJECT_STORE_DEFAULT_PROTOCOL=s3`), chat.agent session snapshot writes
landed in the named provider but reads fell through to the default — so
the recovery boot couldn't find the snapshot it had just written.
After a mid-stream cancel, the missing snapshot triggered a fallback
replay path that dropped the user's follow-up message, leaving the chat
stuck in `submitted` indefinitely.
Fix:
- New `/api/v1/sessions/:id/snapshot-url` route handles PUT + GET
symmetrically — both prefix unprefixed keys with
`OBJECT_STORE_DEFAULT_PROTOCOL` so they always round-trip through the
same store.
- `Session.chatSnapshotStoragePath` persists the resolved URI on first
write so future protocol changes don't strand existing snapshots.
Reads prefer the stored URI and fall back to the computed default for
pre-column sessions.
- SDK calls `createChatSnapshotUploadUrl` / `getChatSnapshotUrl`; the
generic v1/v2 packets endpoints are unchanged.
## Test plan
- [x] Configure local with two providers (R2 default + MinIO `s3` named)
and `OBJECT_STORE_DEFAULT_PROTOCOL=s3`.
- [x] Reproduce hang: send a message, cancel mid-stream, send another —
without the fix it hangs in `submitted`; with the fix it streams.
- [x] Snapshot lands in the `s3`-protocol bucket and
`Session.chatSnapshotStoragePath` is set after first write.
- [x] SDK unit tests pass; webapp typecheck passes.
## Summary
The S2 access-token cache key was `${basin}:${streamPrefix}` — purely
server-derived but blind to the **scope/ops list** hardcoded one method
away. When the ops list changes in code (e.g. #3644 added `trim` so
`chat.agent`'s per-turn trim chain can issue `AppendRecord.trim()`),
pre-deploy tokens still in cache get returned to SDK callers for up to
the token's TTL (24h default), surfacing as `Operation not permitted`
403s on any op outside the old scope.
## Fix
Lift the ops list to a module constant and fold its sorted-join
fingerprint into the cache key:
```ts
const S2_TOKEN_OPS = ["append", "create-stream", "trim"] as const;
const S2_TOKEN_OPS_FINGERPRINT = [...S2_TOKEN_OPS].sort().join(",");
// in getS2AccessToken
const cacheKey = `${this.basin}:${this.streamPrefix}:${S2_TOKEN_OPS_FINGERPRINT}`;
// in s2IssueAccessToken
scope: { /* ... */ ops: [...S2_TOKEN_OPS], /* ... */ }
```
The fingerprint is derived from the single source of truth, so any
future scope change auto-invalidates without anyone remembering to bump
a literal version. The Unkey L1 (in-memory LRU) and L2 (Redis) layers
share the same key derivation, so both reset together on the next deploy
with no manual cache busting.
## Test plan
- [ ] `pnpm run typecheck --filter webapp`
- [ ] Run a multi-turn `chat.agent` chat via `references/ai-chat` and
confirm no `chat.agent: trim failed; will retry next turn` warn span
fires across turn-completes.
## Summary
Long-running chat agents were filling `session.out` forever — every
`chat.agent` turn appended to the same S2 stream with no trim, and the
Sessions dashboard re-streamed the entire history from `seq_num=0` on
every page load. After this change the agent appends an S2 `trim`
command record after each `trigger:turn-complete`, pointing back at the
previous turn-complete's seq_num. `session.out` stays roughly one turn
long at steady state, regardless of session age.
`trigger:turn-complete` and `trigger:upgrade-required` move from
`chunk.type`-shaped data records into header-form control records under
a uniform `trigger-control` namespace. Built-in transports
(`TriggerChatTransport`, `AgentChat`, the dashboard's `AgentView`)
handle the new shape transparently. Custom transports need a one-line
filter on the `trigger-control` header — see the rewritten "Records on
session.out" section in the client-protocol docs.
The Sessions detail page in the dashboard fetches the agent's per-turn
S3 snapshot via a presigned URL and seeds the transcript view, then
SSE-tails from the snapshot's `lastOutEventId`. Bandwidth and
time-to-first-render scale with unread turns instead of session
lifetime.
Resume contract is now explicit: single-turn-boundary resume always
works (the prior turn-complete is still on the stream), the S2 trim is
eventually consistent over 10-60s, and multi-turn-away resume falls back
to a snapshot reload.
Adds Sessions, a durable, run-aware stream primitive that scopes
session.in / session.out records to a session (not a single run).
Records survive run boundaries; reconnect-from-last-event-id is built in.
Server foundation:
- New /realtime/v1/sessions/:session/:io/append + /records routes
- sessionRunManager + sessionsRepository + clickhouseSessionsRepository
- mintRunToken for short-lived per-session tokens
- s2Append retry-with-backoff + undici cause diagnostics
- /api/v[12]/packets/* exempt from customer rate limits
- BackgroundWorker schema gains taskKind enum (TASK, AGENT, SCHEDULED)
- TaskRun.taskKind column + clickhouse 029_add_task_kind_to_task_runs_v2
Core types:
- new sessionStreams, inputStreams, realtimeStreams packages in @trigger.dev/core
- session-streams-api / realtime-streams-api surface
Sessions dashboard UI (the primitive's own viewer):
- /sessions index + detail routes
- SessionsTable, SessionFilters, SessionStatus, CloseSessionDialog
- AGENT/SCHEDULED filter in RunFilters + TaskTriggerSource
Includes the sessions-primitive changeset.
## Summary
Consolidates the webapp's authentication and authorization into a small
set of route helpers, replacing the ad-hoc `requireUser` /
`requireUserId` / `authenticatedEnvironmentForAuthentication` calls
scattered across routes. Same security model, but the per-request flow
(authenticate → authorize → load) now lives in one place per route
family.
Introduces a plugin seam (`@trigger.dev/plugins`) that lets the cloud
build install a richer RBAC implementation without touching webapp code.
The OSS fallback keeps the pre-RBAC permissive behaviour intact, so
self-hosted deployments work unchanged.
Adds a comprehensive end-to-end auth test suite that didn't exist before
— 193 `it()` blocks (vitest reports ~199 after `it.each` expansion)
covering API key, PAT and JWT auth across the public API surface, plus
dashboard session auth for admin pages.
## Changes
### Plugin contract — `@trigger.dev/plugins`
`RoleBaseAccessController` interface authoritative for both OSS
(fallback) and cloud (enterprise plugin):
- `authenticateBearer(request, { allowJWT? })` — API-key / public-JWT
auth, returns env + ability
- `authenticateSession(request, { userId, organizationId?, projectId?
})` — dashboard auth, caller resolves `userId` from the session cookie
and passes it in (no `helpers.getSessionUserId` callback — decouples the
plugin host from session-cookie code)
- `authenticatePat(request, { organizationId?, projectId? })` — PAT
auth, returns identity + `lastAccessedAt` so the host can throttle the
per-request update
- `authenticateAuthorize*` variants for the auth-and-check-in-one-call
cases
- `isUsingPlugin(): Promise<boolean>` — capability flag for UI /
branching where plugin-present-ness matters; replaces the
sentinel-string coupling that had `personalAccessToken.server` matching
`"RBAC plugin not installed"` literally
### Dashboard auth (started, partial rollout)
Admin and settings pages migrated to a unified `dashboardLoader` /
`dashboardAction` helper that authenticates the session, runs an
authorization check, and exposes the result to the route. Other
dashboard routes still on the old pattern; remaining migration tracked
in TRI-8730.
Migrated routes:
- `admin.*` (14 admin / back-office / feature-flags / LLM-models /
notifications / orgs / concurrency pages)
- `_app.orgs.$organizationSlug.settings.team`
- `_app.orgs.$organizationSlug.settings.roles`
### API / realtime / engine auth (complete for the migrated families)
71 routes migrated to a unified `apiBuilder` that centralizes Bearer /
PAT / Public-JWT authentication and applies the per-route authorization
check before the handler runs. Includes:
- `api.v1.*` and `api.v2.*` and `api.v3.*` — tasks, runs, batches,
queues, prompts, deployments, query, sessions, waitpoints, packets,
workers, idempotency keys
- `realtime.v1.*` — runs, batches, sessions, streams
- `engine.v1.*` — dev / worker-action protocols
29 routes still on the legacy `authenticateApiRequest*` helpers —
tracked as a post-deploy follow-up in TRI-9228.
Multi-resource auth direction is now explicit at the call site via
`anyResource(...)` (OR) and `everyResource(...)` (AND). Bare arrays no
longer typecheck — fixes a class of bug where a JWT scoped to one
resource could implicitly access others under OR semantics.
PAT auth path consolidated: was three DB queries per request (legacy
`authenticateApiRequestWithPersonalAccessToken` findFirst +
`rbac.authenticatePat` join + `lastAccessedAt` update). Now one query in
the steady state — plugin returns `lastAccessedAt`, host smart-skips the
update via JS-side throttle when fresh.
Side effect: action aliases preserved historic JWT scope semantics where
the new model is stricter (e.g. a `write:tasks` JWT now also satisfies
`trigger` / `batchTrigger` / `update` actions on the same resource —
matched at the auth boundary, not in the route handler).
### Backwards-compat fixes
The strict-match model regressed several real-world JWT shapes. Each
preserved via explicit `anyResource(...)` entries in the route's authz
block:
- **Batch retrieve routes** (`api.v1.batches.$batchId`, `api.v2.*`,
`realtime.v1.batches.*`) accept `read:runs` JWTs again (pre-RBAC
literal-match superScope behaviour)
- **Runs list routes** (`api.v1.runs`, `realtime.v1.runs`) accept
type-level `read:tasks` / `read:tags` on unfiltered queries (matched the
legacy `Object.keys` iteration semantic)
- **PAT/OAT auth shape** normalized through `toAuthenticated` so all
auth methods return the same slim `AuthenticatedEnvironment` (was:
API-key returned the slim shape but PAT/OAT returned raw Prisma
`Decimal` / no `orgMember`)
- **Scope `:` preservation** in resource ids — `read:tags:env:staging`
now correctly identifies the tag id as `env:staging`, not `env`
### Slim `AuthenticatedEnvironment`
Extracted to `@trigger.dev/core/v3/auth/environment` — a structural
shape independent of `@trigger.dev/database`. The plugin contract
returns this; webapp consumers import from there; the cloud plugin
(Drizzle) returns the same shape without Prisma's `Decimal` class
leaking into the public surface. Lets internal-packages (run-engine,
etc.) refer to `AuthenticatedEnvironment` without pulling Prisma in.
### Auth test suite (new — `*.e2e.full.test.ts`)
193 e2e tests run against a real spawned webapp + Postgres (no mocks).
Coverage matrix:
- **API key auth** — read / write / trigger / batchTrigger / deploy
actions across runs, batches, deployments, prompts, queues, query,
sessions, input-streams, waitpoints, tasks, idempotency keys; multi-key
resources (a run carries batch / tag / task identifiers — auth must
accept any matching scope)
- **Personal Access Token auth** — comprehensive matrix: scope match,
scope mismatch, missing scope, expired token, malformed token
- **Public JWT auth** — sub-vs-URL environment resolution, expired JWTs,
signature verification, scope checking, otu (one-time-use) token
semantics, branch-environment signing-key fallback
- **Dashboard session auth** — admin-only pages reject non-admins;
per-action gating
- **Cross-cutting edge cases** — revoked API key grace window, JWT
cross-environment isolation, MissingResource branch behaviour
### Hygiene cleanups
- Deleted dead `app/services/authorization.server.ts` (legacy
`checkAuthorization` + types — no live consumers post-migration) and its
orphaned test
- Dropped the never-populated `scopes` field from
`ApiAuthenticationResultSuccess`
- `scheduleEmail` moved out of `email.server.ts` into its own module —
breaks a `commonWorker → marqs/V1` import chain that was poisoning the
auth test graph
- OSS Roles page shows a deployment-aware empty state ("Roles aren't
available in this self-hosted deployment" vs the plan-upsell copy) via
`rbac.isUsingPlugin()`
- Team action handler: explicit per-intent ability gates
(`manage:billing` for purchase-seats, `manage:members` for set-role +
remove-member with self-leave carve-out)
### Cross-repo coordination
All public-package contract changes paired in `triggerdotdev/cloud#763`
(rbac-packages branch) — the enterprise plugin implements the same
`RoleBaseAccessController` interface against Drizzle.
## Test plan
- [x] `pnpm run typecheck --filter webapp` clean
- [x] `pnpm --filter webapp exec vitest run --config
vitest.e2e.full.config.ts` — 193/193 pass (requires Docker for
testcontainers)
- [x] Spot-check an authed API endpoint with a valid + invalid API key
against a local stack
- [x] Spot-check the migrated admin pages render and gate non-admins
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
## Summary
During an ElastiCache role swap (failover) or node-type change (vertical
scale), the ioredis TCP/TLS connection stays open but the server starts
answering with `READONLY` (the client is talking to a node that became a
replica) or `LOADING` (node still loading data from disk). Without an
explicit hook, those errors surface to caller code as `ReplyError`
instances — every write op on the affected connection fails until the
cluster fully cuts over.
This PR adds `reconnectOnError` to every prod ioredis client so the
disconnect + reconnect + retry cycle absorbs these errors and caller
code never sees them.
## Fix
```ts
export function defaultReconnectOnError(err: Error): boolean | 1 | 2 {
const msg = err.message ?? "";
if (msg.startsWith("READONLY") || msg.startsWith("LOADING")) return 2;
return false;
}
```
Returning `2` tells ioredis to disconnect, reconnect, and re-issue the
failed command. After reconnect, DNS / SG state routes the new socket to
a writable node.
The helper lives in `@internal/redis` and is wired into both the shared
`createRedisClient` (which covers RunQueue, schedule-engine,
redis-worker, and every other internal-package consumer) and the direct
`new Redis(...)` call sites in the webapp.
V1-only marqs files are intentionally not migrated.
## Test plan
- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm run typecheck --filter @internal/run-engine`
- [x] Verified end-to-end against a live ElastiCache vertical-scale
event — caller-surfaced errors went from tens of thousands during the
cutover window down to a handful per ioredis client
- [ ] Confirm steady-state behavior unchanged after deploy
## Summary
Move from a single shared S2 basin to **per-org basins** with retention
tied to the org's billing plan. Stops S2 from deleting streams out from
under live chat sessions when basin retention fires before the chat
ends, and unlocks per-org cost attribution.
OSS / s2-lite installs are unaffected: provisioning is gated by
`REALTIME_STREAMS_PER_ORG_BASINS_ENABLED` (default `false`), and the
read precedence falls back to the global basin env var when an entity
has no stamped basin.
```
basin = run.streamBasinName ?? session.streamBasinName ?? env.REALTIME_STREAMS_S2_BASIN
```
## Design
Three nullable `streamBasinName` columns (`Organization`, `TaskRun`,
`Session`) plus a provisioner that idempotently creates the basin and
reconfigures retention on plan changes. The trigger and session-create
paths stamp the org's basin onto new rows; the realtime read path picks
the basin from the entity context.
Admin routes back-fill existing orgs and force-reconfigure a single org.
## Test plan
- [x] `pnpm run typecheck --filter webapp --filter @internal/run-engine`
- [x] Backfill admin route end-to-end (provision + DB stamp + S2 basin
config).
- [x] Reconfigure on plan change (all retention tiers).
- [x] chat.agent multi-turn drives streams into the per-org basin.
- [x] Legacy fallback when entity has no stamped basin.
- [x] Provisioner is a no-op when the flag is off.
## Summary
Follow-up to #3420. PATs (public access tokens) minted before an API key
rotation 401'd immediately on the realtime stream endpoints, even though
the rotation flow advertises a 24h overlap. This fixes the gap.
## Root cause
PATs are JWTs signed with the env's `apiKey` at mint time. When that
secret is rotated, `validatePublicJwtKey`
(`apps/webapp/app/services/realtime/jwtAuth.server.ts`) only verifies
the signature against `environment.parentEnvironment?.apiKey ??
environment.apiKey` — i.e. the env's *current* canonical key. Any PAT in
the wild signed with the previous key fails signature verification →
401, even within the grace window.
#3420 wired up the grace-window fallback in two places —
`findEnvironmentByApiKey` (raw secret-key auth) and `api.v1.auth.jwt.ts`
(signs new JWTs with the canonical key when minting from an old one) —
but the *verify* path for already-issued PATs was never updated.
In a typical app, `POST /api/v1/tasks/.../trigger` (Bearer secret) keeps
working through rotation because that path has the fallback, but `GET
/realtime/v1/streams/run_*/...` and `POST
/realtime/v1/streams/run_*/input/...` 401 for runs that were already in
flight when the rotation happened.
## Fix
After the primary `validateJWT` against the env's current `apiKey`, fall
back to non-expired `RevokedApiKey` rows for the signing env (parent env
when the request is against a child) — but **only on the failure path**,
so the hot success path is unchanged. Uses `$replica` to match the rest
of the auth path.
Symmetrical to the `findEnvironmentByApiKey` two-step from #3420.
## Changes
- `apps/webapp/app/services/realtime/jwtAuth.server.ts` —
`validateAgainstRevokedApiKeys` helper invoked only on `!result.ok`
- `apps/webapp/app/models/runtimeEnvironment.server.ts` —
`findEnvironmentById` also selects `parentEnvironment.id` so we can
scope the revoked-keys lookup to the correct env
## Test plan
E2E verified locally via curl against `GET /realtime/v1/runs/{runId}`
(PAT-authenticated):
- [x] Pre-rotation, PAT signed with K1 → **200** with run body
- [x] Simulate rotation (insert `RevokedApiKey` row + flip env `apiKey`
to K2 in a single transaction, mirroring `regenerateApiKey`)
- [x] Same PAT (K1) within grace window → **200** with run body —
fallback hits
- [x] Fresh PAT signed with K2 → **200** — current key still works
- [x] Set `RevokedApiKey.expiresAt` to past → **401** — fallback finds
no live row
- [x] Bogus signature (no rotation) → **401**
- [x] Cleanup verified: env `apiKey` restored, `RevokedApiKey` row
deleted
- [x] `pnpm run typecheck --filter webapp` passes
> ⚠️ **Not released yet.** This PR is the server-side foundation only.
The SDK changes that customers will actually use (`chat.agent`
migration, `chat.createStartSessionAction`, `useTriggerChatTransport`
updates) live on a separate branch and ship together in an upcoming
`@trigger.dev/sdk` prerelease. Until that prerelease is published, this
surface is reachable only via direct HTTP.
## What this gives Trigger.dev users
A new first-class primitive, **Session**, for durable, task-bound,
bidirectional I/O that outlives any single run. Sessions are the run
manager for `chat.agent` going forward, and they unblock anything else
that needs "one identifier, many runs over time" with a stable channel
pair the client can write to and subscribe to.
### Use cases unblocked
- **Chat agents that persist across many runs.** One session per chat
(keyed on your own `chatId` via `externalId`), turns 1..N attach to the
same Session, the UI subscribes once and keeps receiving output as new
runs take over.
- **Approval loops and long-running tasks with user feedback.** The task
waits on `.in`, the client writes to `.in`, the server enforces
no-writes-after-close.
- **Workflow progress streams that live past the run.** Subscribe to
`.out` after the task finishes to replay history.
- **Resume-next-day flows.** A session is a durable row, not a transient
stream. Send a message a day later and the server triggers a fresh run
on the same session.
### How it works (Session-as-run-manager)
A Session row is task-bound (`taskIdentifier` + `triggerConfig` are
required) and owns its current run via `currentRunId` +
`currentRunVersion` for optimistic claim. Three trigger paths:
1. **Session create** — `POST /api/v1/sessions` creates the row and
triggers the first run synchronously.
2. **Append-time probe** — `POST
/realtime/v1/sessions/:session/in/append` checks if the current run is
alive; if it has terminated (idle exit, crash, etc.), the server
triggers a new run before processing the append.
3. **End-and-continue handoff** — `POST
/api/v1/sessions/:session/end-and-continue`, called by the running
agent, triggers a fresh run and atomically swaps `currentRunId`. Used by
`chat.requestUpgrade()` for version handoffs.
Every triggered run is recorded in the `SessionRun` audit table with a
reason (`initial`, `continuation`, `upgrade`, `manual`).
## Public API surface
### Control plane
- `POST /api/v1/sessions` — create. Idempotent on `(env, externalId)`.
Triggers the first run, returns the session and a session-scoped public
access token. Returns 409 if the upserted row is already closed.
- `GET /api/v1/sessions/:session` — retrieve by friendlyId
(`session_abc...`) or by your own externalId (server disambiguates by
prefix).
- `GET /api/v1/sessions` — list with filters (`type`, `tag`,
`taskIdentifier`, `externalId`, derived `status` ACTIVE/CLOSED/EXPIRED,
created-at range) and cursor pagination. Backed by ClickHouse.
- `PATCH /api/v1/sessions/:session` — update tags / metadata /
externalId.
- `POST /api/v1/sessions/:session/close` — terminate. Idempotent,
hard-blocks new server-brokered writes.
- `POST /api/v1/sessions/:session/end-and-continue` — agent-only handoff
to a fresh run.
### Realtime
- `PUT /realtime/v1/sessions/:session/:io` — initialize a channel.
Returns S2 credentials in headers so high-throughput clients can write
direct to S2.
- `GET /realtime/v1/sessions/:session/:io` — SSE subscribe. Supports
Last-Event-ID resume and an opt-in `X-Peek-Settled: 1` header that
fast-closes the stream when the upstream is already settled
(`trigger:turn-complete`), eliminating long-poll wait on
reconnect-on-reload paths.
- `POST /realtime/v1/sessions/:session/:io/append` — server-side
appends.
- `POST /api/v1/runs/:runFriendlyId/session-streams/wait` — runs wait on
a session stream as a waitpoint, with a race-check to avoid suspending
if data already landed.
### Auth scopes
`sessions` is a new resource type. `read:sessions:{id}`,
`write:sessions:{id}`, `admin:sessions:{id}` flow through the existing
JWT validator. Session-scoped public access tokens minted by the server
replace browser-held trigger-task tokens for chat-style flows — the
browser never sees a run identifier or a run-scoped token in steady
state.
## What's coming after this PR
- **SDK + chat.agent migration**: separate branch, separate PR, ships in
the next `@trigger.dev/sdk` prerelease alongside this server deploy.
Customers using the prerelease `chat.agent` will follow the [upgrade
guide](https://github.com/triggerdotdev/trigger.dev/blob/docs/tri-7532-ai-sdk-chat-transport-and-chat-task-system/docs/ai-chat/upgrade-guide.mdx).
- **Dashboard surfaces**: dedicated agent list, agent playground, agent
view on the run dashboard. Tracking separately.
## Implementation notes
- **Postgres `Session` table**: scalar scoping columns (`projectId`,
`runtimeEnvironmentId`, `environmentType`, `organizationId`) without
FKs, matching the January TaskRun FK-removal decision. Point-lookup
indexes only — list queries go to ClickHouse. Terminal markers
(`closedAt`, `expiresAt`) are write-once.
- **ClickHouse `sessions_v1`**: ReplacingMergeTree, partitioned by
month, ordered by `(org_id, project_id, environment_id, created_at,
session_id)`. Tags indexed via `tokenbf_v1` skip index.
- **`SessionsReplicationService`**: mirrors `RunsReplicationService`
exactly — leader-locked logical replication consumer,
`ConcurrentFlushScheduler`, retry with exponential backoff + jitter,
identical metric shape. Dedicated slot + publication so the two consume
independently.
- **S2 keys**: `sessions/{addressingKey}/{out|in}`. The existing
`runs/{runId}/{streamId}` key format for run-scoped streams is
untouched.
- **Optimistic claim**: `ensureRunForSession` triggers a run upfront
(cheap to cancel if it loses the race), then attempts an `updateMany`
keyed on `currentRunVersion`. Loser cancels its triggered run and reuses
the winner's. No DB lock held across the trigger.
### What did NOT change
Run-scoped `streams.pipe` / `streams.input` and the existing
`/realtime/v1/streams/{runId}/...` routes are unchanged. Sessions are
net-new — not a reshaping of the current streams API.
## Deploy notes
- Set `SESSION_REPLICATION_CLICKHOUSE_URL` and
`SESSION_REPLICATION_ENABLED=1` to enable the replication consumer.
- The `Session` table needs `REPLICA IDENTITY FULL` set on the prod
source DB before the publication is created (same one-time DDL we did
for `TaskRun`). Required for delete events to carry full column values.
- Cross-form authorization on the `GET /api/v1/sessions/:session` loader
(a JWT minted for either form authorizes both URL forms). Action routes
are URL-form-specific, matching how the SDK mints PATs.
## Verification
- Webapp typecheck clean (10/10).
- `apps/webapp/test/sessionsReplicationService.test.ts` — round-trip
tests for insert/update/delete through Postgres logical replication into
ClickHouse via testcontainers.
- Live end-to-end against local dev: create + retrieve (both forms) +
update + close, `.out.initialize` + `.out.append` x2 + `.in.send` +
`.out.subscribe` over SSE, list with all filter combinations +
pagination, `end-and-continue` swap, `X-Peek-Settled` fast-close
(verified in browser via reconnect-on-reload and via curl). Replicated
row lands in ClickHouse within ~1s.
- Multi-round Devin + CodeRabbit review feedback addressed
(read-after-write paths use `prisma` writer, info-leak on auth-routes
masked as 403, peek-settled discriminator parsing fix, etc.).
## Test plan
- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run test --filter webapp
./test/sessionsReplicationService.test.ts --run`
- [ ] Start the webapp with `SESSION_REPLICATION_CLICKHOUSE_URL` and
`SESSION_REPLICATION_ENABLED=1`. Confirm the slot and publication
auto-create on boot.
- [ ] `POST /api/v1/sessions` and verify the row replicates to
`trigger_dev.sessions_v1` within a couple of seconds.
- [ ] `POST /api/v1/sessions/:id/close`, then confirm `POST
/realtime/v1/sessions/:id/out/append` returns 400.
- [ ] Reuse a closed session's `externalId` on `POST /api/v1/sessions`
and confirm 409.
- [ ] `GET /realtime/v1/sessions/:id/out` with `X-Peek-Settled: 1` after
a turn completes and confirm `X-Session-Settled: true` response header +
immediate close.
Pool Redis connections for non-blocking ops (ingestData, appendPart,
getLastChunkIndex)
using a shared singleton instead of new Redis() per request. Use
redis.disconnect()
for immediate teardown in streamResponse cleanup. Add 15s inactivity
timeout fallback.
Fix broken request.signal in Remix/Express by wiring Express
res.on('close') to an
AbortController via httpAsyncStorage. All SSE/streaming routes now use
getRequestAbortSignal() which fires reliably on client disconnect,
bypassing the
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain.
Input streams enable sending typed data to executing tasks from external
callers — backends, frontends, or other tasks. This unlocks interactive
use cases like approval UIs, cancel buttons, chat interfaces, and
human-in-the-loop AI workflows where the task needs to receive data
while running.
Three consumption patterns inside a task:
* `.wait()` — Suspend the task until data arrives (process freed, most
efficient)
* `.once()` — Wait for the next message (process stays alive)
* `.on()` — Subscribe to a continuous stream of messages
One send pattern from outside:
* `.send(runId, data)` — Send typed data to a specific run's input
stream
## User-facing API
### Define a typed input stream
```ts
import { streams, task } from "@trigger.dev/sdk";
const approval = streams.input<{ approved: boolean; reviewer: string }>({ id: "approval" });
```
### Consume inside a task
```ts
export const myTask = task({
id: "my-task",
run: async () => {
// Pattern 1: Suspend until data arrives (most efficient — frees the process)
const result = await approval.wait({ timeout: "5m" });
// Pattern 2: Wait for next message (process stays alive)
const data = await approval.once().unwrap();
// Pattern 3: Subscribe to multiple messages
approval.on((data) => { /* handle each message */ });
},
});
```
### Send from outside
```ts
// From a backend (using secret API key)
await approval.send(runId, { approved: true, reviewer: "alice" });
// From a frontend (using public JWT token from trigger response)
const { send } = useInputStreamSend("approval", runId, { accessToken });
send({ approved: true, reviewer: "alice" });
```
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(webapp): add support for running web services (api, engine, webapp) in cluster mode for better perf
* cleaned up signal handling and resolved some valid 🐇 issues
* Remove variables from dequeue log message
* Continue snapshot throw json
* Waiting for deploy error removed
* Realtime ECONNRESET is expected
* Redis worker logErrors changes, removed ids
* Preview branch without a branch shouldn't log an error, it's a user provided issue
* "Task run is not in a cancellable state" isn't an error, it's expected
* "CreateCheckpointService: Child run already resumed" is expected
* "CreateCheckpointService: Batch already resumed" is expected
* "Failed to insert events, will attempt bisection" changed to info, we have errors for complete failures
* Ignore "PrismaClient error"
* Don't log Redis worker DLQ errors if we're ignoring
* "Failed to parse machine config" is fine, sometimes a config is null or undefined
* "Failed to parse machine config" for v3
* MetadataTooLargeError shouldn't log an error
* WIP
* Make release concurrency system extremely simple, everything just releases all the time
* update the deadlock detection to use the new lockedQueueReleaseConcurrencyOnWaitpoint column
* WIP new release concurrency system
* Remove releaseConcurrency and releaseConcurrencyOnWaitpoint
Also removed deadlock detection, and added environment burst concurrency
* Added new DEQUEUED status
Cleaned up the API run statuses, including now detecting new clients and not breaking older clients by adding an API version header to all requests
* Introduce the new "current dequeued concurrency set"
* Remove QUEUED_EXECUTING because we no longer "eagerly" release before checkpointing
* Remove waitpoint test for QUEUED_EXECUTING
* Add isWaiting
* Add changeset
* Use createdAt for ordering realtime runs instead of number
* Clarify the envCurrentDequeuedKey usage
* mock the db.server file to fix the tests
* Updated changset "EXECUTED" -> "EXECUTING"
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Initial preview migrations
* Modified the staging endpoint to create preview environments
* Added isBranchableEnvironment to RuntimeEnvironment
* Staging = yellow Preview = orange
* Changed the env sort order
* Set isBranchableEnvironment correctly. Create preview for new projects
* Very basic branch menu
* Creating branches from the dashboard
* Fix for string icons on project delete page
* Don’t show branch API keys
* WIP on the manage branches page
* RuntimeEnvironment added projectId index
* Only create the parentEnvironmentId column if it doesn’t exist already
* Improved the limit wording
* Add search to the branch list
* contains in both places
* Many style improvements
* Branch dropdown and v4 badge
* Arching/unarchive branches working in the dashboard
* Tidied imports
* Change preview slug from `prev` to `preview`
* Use correct color for side menu preview branch icon
* Upsert the branch and use the shortcode as a unique constraint
* Upserting working with nice messages in the dashboard
* Better errors when upserting branches
* Button shortcut, don’t allow event to propagate
* Better duplicate error message
* Filter out archived branches from the env selector
* Archiving/creating tweaked some more
* Add an archived banner to the app, fixes for archived branches and upsells
* Fixed pagination
* Disable editing schedules, pausing queues, testing tasks
* Don’t allow replaying if the env is archived
* When deploying detect the correct environment
* Get the projectClient when there’s a branch
* createGitMeta function, most code from the vercel CLI repo
* Deploy, getting the correct environment client
* Added git column to WorkerDeployment
* Add GitMeta to core schemas
* Create branch when deploying
* WIP on branch support in the API
* Delete old createTaskRunAttempt fn
* apiAuth remove export from internal functions
* Rename env var to “TRIGGER_PREVIEW_BRANCH”
* Add TRIGGER_PREVIEW_BRANCH to resolved env vars for runs
* First preview deploy and run working
* Set the preview branch in the main SDK
* Added git links to the preview branches table
* Better errors when replaying/testing archived branches
* Don’t dequeue archived environments
* Env var resolution with parent environment
* Hello world default machine small-2x to save my memory
* Fix for more env var functions
* Only return non-archived envs
* Switch to controlled state for the checkboxes
* Uncheck everything when PREVIEW is checked
* WIP on branch UI
* Show the preview branch label on the env vars list
* Fix for overriding env vars
* Adding preview branch env vars working
* Progress on new env vars
* Only allow selecting a single branch
* Layout fix when there are errors
* Set the defaultValue so there are some fields
* Conform fix for team invite page
* Archived environments don’t run scheduled tasks
* Added Git data to deployments
* Added git data to the deployment inspector
* Don’t allow upserting schedules when archived
* Deduplicate and blacklist some env vars
* Fix for wrong conform function being used
* Show a better error if all vars were blacklisted
* Added environment variable search (by key and value)
* Improved preview branch icon
* Replay now supports branches
* Schedule page render branches properly
* Show the env icon in bottom-left of the test page
* When editing older schedules (that have multi-env) show preview branches correctly
* Fix for incorrect disallowed branch name character
* Extract and improve the directory verification code
* WIP for CLI preview archive command
* Improved the preview branch action buttons
* Redirect to the project if we don’t find a matching env
* Archiving branch via the CLI working
* Fix for archiving branches
* Public access token test task
* JWTs working are with preview branches
* Add branch and git data to the Run ctx
* Updated GitMeta functions to work in CI
* Added pullRequestState
* Archive when deploying if the PR is closed/merged
* Fix for the changesets guide
* Fix for CLI dev bug introduced
* CLI promote now supports preview branches
* Add PR title. Reordered them and added tooltips
* syncEnvVars working with branches
* Added preview branch support to syncVercelEnvVars()
* Detect the branch from Vercel env var (set during build)
* Allow passing a branch in
* Use process.env.VERCEL_TOKEN as well… this used in Vercel CI
* Temp delete
* Improved regenerate api key modal
* Added Accordion component (with styles)
* Redesigned the API keys page
* Revert "Temp delete"
This reverts commit 177b92cd935a6161456bde65d01294e23ecfd47f.
* Changeset
* Fixed docs link
* The new branch panel closes when a branch is created
* Update apps/webapp/app/services/upsertBranch.server.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Removed findUniques from WorkerGroupTokenService
* Made the parentEnvironmentId migrations safe
* Latest lockfile
* Update packages/cli-v3/src/commands/workers/build.ts
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Move isValidGitBranchName to a separate file
* Move the sanitize fn too
* removeBlacklistedVariables moved to a separate file
* Moved deduplicateVariableArray to a separate file…
* Fix broken sanitizeBranchName import
* Another import fix…
* Improved blacklisted error message
* SImplified migration to use `ADD COLUMN IF NOT EXISTS "parentEnvironmentId" TEXT`
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* realtime: fix streams missing chunks when streams are longer than 5 minutes and receive 408 Request Timeout errors. Also now support multiple client streams being sent to a single stream key
* Safely release the reader lock
* Fix streaming splits in realtime streams v2
* Add changeset
* Skip all flaky tests 😡
* Improve the way we stream from tasks to the server
* Improve the v1 realtime streams (Redis)
* Turn on the relay realtime stream service
* Improved the relay realtime cleanup
* Fixed consuming realtime runs w/streams after the run is already finished
* Remove some logs
* Update changeset
* Fixed runStream tests
* Realtime streams now powered by electric, and fix the streaming update duplicate issues by converting the electric Shape materialized view into a ReadableStream of changes
* Ensure realtime subscription stops when runs are finished, and add an onComplete handle to use realtime hooks
* Fix tests
* Stop using Response errors when validating API Keys, instead introduce a new "Result" type that has success and failure conditions. Adding in a way to progressively adopt because this touches everything.
* Make sure authenticateApiRequestWithFailure always returns a result
* WIP realtime streams
* Handle realtime with large payloads or outputs #1451
* feat: optimize Redis stream handling with batching
Add STREAM_ORIGIN to environment schema. Improve performance in
RealtimeStreams by using TextDecoderStream for simpler text
decoding and implementing batching of XADD commands for Redis
streams. Limit stream size using MAXLEN option. Update
environment variable repository with new variable type. Adjust
import statements for Redis key and value types.
* 🔧 chore: add dev dependencies for bundle analysis
* add metadata tests and a few more utilties
* Add stream tests and improve streaming
* Added AI tool tasks, descriptions to tasks
* Use the config file path to determine the workingDir, then the package.json path
* Remove stream test files
* useTaskTrigger react hook that allows triggering a task from the client
* Add streaming support for the realtime react hooks
* Add ability to stream results after useTaskTrigger
* Improve the stream throttling
* Use the runId as the ID key to bust the cache after triggering
* Upgrade to to the latest electric sql client and server
* Make realtime server backwards compat with 3.1.2 release
* Pass the runId into useRealtimeRun
* Fix scopes when specifiying reading all runs
* WIP @trigger.dev/rsc package
* Various fixes and accepted recommendations by CodeRabbit
* Regenerate pnpm lock file
* A couple tweaks to rsc and give up on rendering react in tasks for now
* Add changeset
* Remove triggerRequest from the useEffect deps
* Improve realtime & frontend authentication errors
* Fixed authorization tests
* Remove unnecessary log
* Add metadata.stream limits and improve the metadata streams structure
* Streams can now have up to 2500 entries
* Various coderabbit fixes
* additional react-hooks jsdocs
* Denormalize run tags, increase character limit to 128
* WIP realtime subscribing to runs
* extracted the stream stuff into core, made it more reusable
* WIP tags
* Remove tags for now because it’s not support in electric
* Support async iterables, readable stream, and callback style subscription styles
* Remove tags streaming endpoint
* Add realtime rate limits and scope them to the /realtime path
* WIP rate limt per org
* Introduce per org rate limits
* WIP JWT auth
* Move migrations into new internal db package
* Resolve pnpm lock file
* Authenticating to the realtime API with JWTs are working
* realtime in the client
* Created react-hooks package and starting to move stuff in there
* Improve types for hooks
* schema tasks
* Added useBatch hook
* build uploadthing/fal demo and change how run metadata is synced to the server
* tweaks
* WIL realtime concurrency tracking
* Implement test for realtime client using testcontainers
also updated electric to latest version
* Allow customizing the expiration time of the automatic JWT created after triggering a task
* Add support for subscribing to run tags
* Improve auth types and API
* finalize the realtime API
* Fixed some example stuff
* Allow up to 10 run tags
* Remove core from docker-provider tsconfig paths to prevent it from being typechecked
* do the same for the kubernetes provider
* Fixing some typecheck errors
* Fix webapp type errors
* Update @trigger.dev/platform to 1.0.13
* Fix attw error
* Remove from/to in subscribeToRuns query params
* Add tests for the rate limit middleware and add custom JWT rate limits
* turn off webapp test parallelism
* Finish renaming jwt -> publicAccessToken and automatically give the JWT read access to the tags when using trigger
* Add changeset
* Attempt to fix unit tests in CI
* Skip running the auth rate limit middleware tests for now
* Try a beefier machine
* Try and run webapp tests separately
* Setup env vars
* Make sliding window test more reliabile
Chris Arderne
