max-compute-seconds
4339 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Eric Allam
|
d343727021 |
fix(webapp,sdk): keep chat.agent snapshots on one object store (#3679)
(`OBJECT_STORE_BASE_URL`) and a named protocol provider
(`OBJECT_STORE_DEFAULT_PROTOCOL=s3`), chat.agent session snapshot writes
landed in the named provider but reads fell through to the default — so
the recovery boot couldn't find the snapshot it had just written.
After a mid-stream cancel, the missing snapshot triggered a fallback
replay path that dropped the user's follow-up message, leaving the chat
stuck in `submitted` indefinitely.
Fix:
- New `/api/v1/sessions/:id/snapshot-url` route handles PUT + GET
symmetrically — both prefix unprefixed keys with
`OBJECT_STORE_DEFAULT_PROTOCOL` so they always round-trip through the
same store.
- `Session.chatSnapshotStoragePath` persists the resolved URI on first
write so future protocol changes don't strand existing snapshots.
Reads prefer the stored URI and fall back to the computed default for
pre-column sessions.
- SDK calls `createChatSnapshotUploadUrl` / `getChatSnapshotUrl`; the
generic v1/v2 packets endpoints are unchanged.
## Test plan
- [x] Configure local with two providers (R2 default + MinIO `s3` named)
and `OBJECT_STORE_DEFAULT_PROTOCOL=s3`.
- [x] Reproduce hang: send a message, cancel mid-stream, send another —
without the fix it hangs in `submitted`; with the fix it streams.
- [x] Snapshot lands in the `s3`-protocol bucket and
`Session.chatSnapshotStoragePath` is set after first write.
- [x] SDK unit tests pass; webapp typecheck passes.
|
||
|
Eric Allam
|
aec7e0a93d |
perf(webapp): index EnvironmentVariableValue.environmentId (#3675)
Env-var lookups via `GET /api/v1/projects/:projectRef/envvars/:slug/:name` run a Prisma `findMany` on `EnvironmentVariableValue` filtered by `environmentId` + `isSecret`. The only existing indexes are the primary key and a unique on `(variableId, environmentId)`, so `environmentId` is never the leading column — the planner falls back to a Parallel Seq Scan over the whole table to find what is, in practice, a handful of rows per environment. Two changes: - Add a btree index on `EnvironmentVariableValue(environmentId)` so the planner switches to an index scan. The composite `(variableId, environmentId)` unique stays in place; the new index is purely additive. - Route the `findMany` inside `getEnvironmentWithRedactedSecrets` through the read replica via a new `replicaClient` constructor param on the repository (defaulting to `$replica`, mirroring how `prismaClient` defaults to `prisma`). Writes and read-after-write methods stay on the primary. ## Test plan - [ ] `pnpm run typecheck --filter webapp` - [ ] Confirm `EXPLAIN` plan flips from Parallel Seq Scan to an index scan - [ ] Existing env-var route tests still pass |
||
|
Daniel Sutton
|
6b46a34c46 |
fix(webapp): return 404 instead of 500 for missing env/project/schedule loaders (#3663)
## Summary
- Dashboard loaders for runs / sessions / batches / schedule-detail
threw bare `Error("X not found")` when a slug didn't resolve. Remix
surfaces this as a 500 and Sentry captures it via auto-instrumentation,
producing ongoing noise from real users following stale preview-branch
or deleted-resource links (the URLs in those Sentry events all carry
`?_data=routes/...`, i.e. client-side revalidation, not full-page
navigation).
- Added a `throwNotFound(statusText)` helper in
`app/utils/httpErrors.ts` that throws a Response with status 404,
matching the established pattern in sibling routes (agents, alerts,
bulk-actions, etc.).
- Migrated 5 loader sites to `throwNotFound` (4× "Environment not
found", 1× "Schedule not found").
- Migrated 1 loader site (`runs._index` project branch) to
`redirectWithErrorMessage("/", request, "Project not found")` to match
the pre-existing convention used by every other dashboard route's
project-not-found branch.
- Intentionally **not** touched: bare `throw new Error("X not found")`
inside `resources.*` action routes (sit inside try/catch blocks that
already redirect with a flash message), the invariant assertion in
`vercel.connect.tsx`, and the admin config check in
`admin.api.v1.runs-replication.backfill.ts`.
## Where the fix is visible
Normal browser navigation to these URLs doesn't reach the buggy loaders
— the parent env-layout
(`_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam/route.tsx`)
already filters missing envs/projects and redirects/404s before the
child loader runs. The bug fires exclusively when Remix calls a single
child loader via `?_data=routes/...`, which happens during client-side
navigation or `useRevalidator`. That matches every Sentry event URL.
## Test plan
- [x] Unit test for the new helper —
`apps/webapp/test/httpErrors.test.ts`
- [x] `pnpm run typecheck --filter webapp` clean
- [x] Manual verification via Playwright on `main` vs this branch (6
cases): main returns 500 for each defective `_data` URL; branch returns
404 or 204 + `X-Remix-Redirect` as designed
- [x] Verified user-visible 404 catch boundary on `schedules/<missing>`
(the one case reachable via normal nav)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Eric Allam
|
f91b96efb7 | feat(sdk,core): preserve chat.agent context after cancel / OOM / crash (#3671) | ||
|
Iss
|
204a766bb1 |
feat(webapp): expose is_warm_start in TRQL runs schema (#3667)
Add is_warm_start to TRQL runs schema so warm vs cold start data is queryable |
||
|
Eric Allam
|
436b7a9ea1 |
fix(webapp): fold S2 token scope into access-token cache key (#3668)
## Summary
The S2 access-token cache key was `${basin}:${streamPrefix}` — purely
server-derived but blind to the **scope/ops list** hardcoded one method
away. When the ops list changes in code (e.g. #3644 added `trim` so
`chat.agent`'s per-turn trim chain can issue `AppendRecord.trim()`),
pre-deploy tokens still in cache get returned to SDK callers for up to
the token's TTL (24h default), surfacing as `Operation not permitted`
403s on any op outside the old scope.
## Fix
Lift the ops list to a module constant and fold its sorted-join
fingerprint into the cache key:
```ts
const S2_TOKEN_OPS = ["append", "create-stream", "trim"] as const;
const S2_TOKEN_OPS_FINGERPRINT = [...S2_TOKEN_OPS].sort().join(",");
// in getS2AccessToken
const cacheKey = `${this.basin}:${this.streamPrefix}:${S2_TOKEN_OPS_FINGERPRINT}`;
// in s2IssueAccessToken
scope: { /* ... */ ops: [...S2_TOKEN_OPS], /* ... */ }
```
The fingerprint is derived from the single source of truth, so any
future scope change auto-invalidates without anyone remembering to bump
a literal version. The Unkey L1 (in-memory LRU) and L2 (Redis) layers
share the same key derivation, so both reset together on the next deploy
with no manual cache busting.
## Test plan
- [ ] `pnpm run typecheck --filter webapp`
- [ ] Run a multi-turn `chat.agent` chat via `references/ai-chat` and
confirm no `chat.agent: trim failed; will retry next turn` warn span
fires across turn-completes.
|
||
|
Oskar Otwinowski
|
2fbac48e0d |
feat(webapp): prompt to clear TRIGGER_VERSION on disabling Vercel atomic deployments (#3666)
- Ask user if they want to remove TRIGGER_VERSION when they disable atomic deployments, and explain what is the situation if they leave it as it is - Install TRIGGER_SECRET keys as sensitive values in Vercel <img width="1136" height="714" alt="image" src="https://github.com/user-attachments/assets/a7351da1-5b2a-44e5-acdd-d30c9359f3ed" /> <img width="1136" height="714" alt="image" src="https://github.com/user-attachments/assets/e773ede2-74cb-438e-811c-338f678d2f7d" /> <img width="1136" height="714" alt="image" src="https://github.com/user-attachments/assets/c7b235a8-e06d-48d3-ac28-c5c9aacc6069" /> |
||
|
Daniel Sutton
|
2f261e5e69 |
fix(webapp): catch loader/action throws before Remix serializes them (#3664)
## Summary Companion to #3536, which patched routes that already had a leaking `catch (e) { return json({error: e.message}, 500) }`. That pattern can't reach routes which have no catch in the first place — when those throw, Remix's default error path serializes `error.message` into the response body, and the SDK then wraps the leaked string as `TriggerApiError`. Across 28 raw api.v1 loaders/actions plus one dashboard polling endpoint, each handler now: - Wraps its body in `try { ... } catch (error) { ... }`. - Re-throws `Response` instances so auth helpers' `throw json(...)` / `throw redirect(...)` pass through unchanged. - Logs non-Response errors via `logger.error` so server-side visibility is preserved. - Returns a generic body — `{"error": "Internal Server Error"}` 500 for raw API routes, or `{ changelogs: [] }` 200 for the polling widget (degrade silently across transient blips; the consumer hook already coped with empty payloads). For six routes where #3536 left an inner try/catch covering only a service call (`alertChannels`, `batches.results`, `deployments.finalize`, `deployments.background-workers`, `deployments.promote`, `projects.background-workers`): an outer try/catch is added so auth/parsing failures are also sanitized. Inner typed-error handling (`ServiceValidationError` → 422 with message, etc.) is preserved exactly. For two routes whose existing catch returned 400 + `error.message` (`api.v1.authorization-code`, `api.v1.orgs.\$orgParam.projects` action): the body is sanitized to a generic per-route string. **Status code stays 400** — clients that key on the 4xx/5xx distinction (and the SDK's no-retry-on-4xx behavior) are unaffected. ## Test plan - [x] \`pnpm run typecheck --filter webapp\` - [x] Per-route synthetic-throw probe: inject \`throw new Error("SYNTHETIC ...")\` at the top of each catch'd try, curl the route with a dummy bearer, confirm the response body is the generic shape and that the synthetic message lands server-side via \`logger.error\`. 29 routes verified. - [x] Real-P1001 probe on the envvars loader: \`docker stop database\` mid-flight, confirm response is generic 500 (not the leaked Prisma message). - [x] Sampled legitimate 4xx/2xx paths across each pattern variant (naked-wrap, partial-expanded, 400-preserved) to confirm the wraps don't interfere with normal control flow. |
||
|
nicktrn
|
5dacab0c72 |
fix: validate email format on magic link login (#3660)
Reject non-email strings at the magic link form instead of accepting any string and proceeding through rate-limit / authenticator steps. |
||
|
Oskar Otwinowski
|
02d61afc1c |
fix(webapp): sanitize OTel attributes on ClickHouse JSON parse rejection (#3659)
Before fix: <img width="1264" height="987" alt="image" src="https://github.com/user-attachments/assets/24b8b85c-b89f-4109-9004-8d6af61d2849" /> After fix: <img width="1264" height="987" alt="image" src="https://github.com/user-attachments/assets/89bbc587-c50a-45ab-b203-dbe91028e918" /> |
||
|
Daniel Sutton
|
906d5fafb6 |
feat(mollifier): trigger burst smoothing — Phase 1 (monitoring) (#3614)
## Summary - Introduce the Mollifier: a Redis-backed buffer for `trigger()` API calls during traffic spikes, with a per-env trip evaluator and a drainer ack-loop. - Phase 1 is dual-write monitoring — every mollified trigger is buffered to Redis AND continues to `engine.trigger`. No customer-facing behaviour change. - Telemetry events: `mollifier.would_mollify`, `mollifier.buffered`, `mollifier.drained`, plus the `mollifier.decisions` counter. - Gated behind a feature flag (default off). ## Test plan - [x] `pnpm run test --filter @trigger.dev/redis-worker` - [x] `pnpm run test --filter webapp -- mollifier` - [x] Manual: with flag off, no behaviour change vs main - [x] Manual: with flag on + threshold lowered, observe `mollifier.buffered` + `mollifier.drained` log pairs with matching `runId` --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Eric Allam
|
82853debea |
feat(webapp,core,sdk,cli): bound session.out via per-turn trim (#3644)
## Summary Long-running chat agents were filling `session.out` forever — every `chat.agent` turn appended to the same S2 stream with no trim, and the Sessions dashboard re-streamed the entire history from `seq_num=0` on every page load. After this change the agent appends an S2 `trim` command record after each `trigger:turn-complete`, pointing back at the previous turn-complete's seq_num. `session.out` stays roughly one turn long at steady state, regardless of session age. `trigger:turn-complete` and `trigger:upgrade-required` move from `chunk.type`-shaped data records into header-form control records under a uniform `trigger-control` namespace. Built-in transports (`TriggerChatTransport`, `AgentChat`, the dashboard's `AgentView`) handle the new shape transparently. Custom transports need a one-line filter on the `trigger-control` header — see the rewritten "Records on session.out" section in the client-protocol docs. The Sessions detail page in the dashboard fetches the agent's per-turn S3 snapshot via a presigned URL and seeds the transcript view, then SSE-tails from the snapshot's `lastOutEventId`. Bandwidth and time-to-first-render scale with unread turns instead of session lifetime. Resume contract is now explicit: single-turn-boundary resume always works (the prior turn-complete is still on the stream), the S2 trim is eventually consistent over 10-60s, and multi-turn-away resume falls back to a snapshot reload. |
||
|
Eric Allam
|
f88d4018cc |
fix(webapp): dedupe realtimeStreams array push on stream create (#3653)
## Summary
The PUT handler at `/realtime/v1/streams/:runId/:target/:streamId` ran
`taskRun.update({ realtimeStreams: { push: streamId } })` on every call,
even when the `streamId` was already present. SDK call patterns that
re-initialize the same stream key on every chunk produce a per-write row
UPDATE, duplicate entries pile up in the array, and the row-lock + TOAST
rewrite cost grows unbounded on long-running stream sessions.
## Fix
Mirror the sibling append handler: read the array first and only push
when the `streamId` isn't already present. Identical behavior for
first-time stream creation; repeat creates short-circuit to a single
indexed read. The dashboard's per-run stream listing keeps working
because the first create still records the entry.
## Test plan
- [ ] A fresh PUT for a new `(run, streamId)` adds the entry to the
array
- [ ] A repeat PUT for the same pair leaves the array unchanged
- [ ] 404 is returned when the run doesn't exist; 400 when the run is
completed
|
||
|
Daniel Sutton
|
9623e88b05 |
fix(webapp): collapse Prisma P1001 errors into a single Sentry issue (#3632)
## Summary
- Adds a `beforeSend` rule in `apps/webapp/sentry.server.ts` that
collapses Prisma `P1001` ("Can't reach database server") errors into a
single Sentry issue regardless of which call site threw, by setting
`event.fingerprint = ["prisma-p1001-db-unreachable"]` and tagging
`db_unreachable:true`.
- Matches both `err.code === "P1001"` (Prisma's `KnownRequestError` when
a connection drops mid-query) and `err.errorCode === "P1001"`
(`InitializationError` when the client fails to connect at startup).
- Implemented as a small extensible `FINGERPRINT_RULES` table so further
fan-out errors can be added with one entry.
## Verification
End-to-end verified locally with `debug: true` on the SDK:
- Real Prisma `P1001` thrown from a loader (DB stopped mid-request) is
captured by Sentry's Remix auto-instrumentation
- `beforeSend` fires with `originalException.code === "P1001"`, rule
matches
- `event.fingerprint = ["prisma-p1001-db-unreachable"]` and
`tags.db_unreachable = "true"` applied
- Event lands in Sentry under the new fingerprint
## Test plan
- [ ] Deploy to staging; confirm P1001 events appear under a single
`prisma-p1001-db-unreachable` issue rather than fanning out
- [ ] Confirm `db_unreachable:true` tag is filterable in Sentry
- [ ] Verify non-P1001 errors are unaffected (event passes through
`beforeSend` untouched)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Eric Allam
|
4c42f6cc0b |
feat(webapp,core,cli): filter runs by region in dashboard, API, and MCP (#3612)
## Summary
Adds a Region column and Region filter (under More filters) to the runs
list dashboard, the same filter on the public runs list API
(`filter[region]`), and a matching `region` input on the MCP `list_runs`
tool. Each run's executing region is also surfaced as a new optional
`region` field on the runs list and run retrieve responses, populated
from the worker instance group's `masterQueue` identifier.
Useful when you run tasks across multiple regions and want to slice the
runs list — or your existing run-querying scripts — by where the run
actually executed.
## Design
The filter value in the URL / API is the `masterQueue` identifier (the
same string already persisted on `TaskRun` and replicated to ClickHouse
as `worker_queue`), so the query just becomes `worker_queue IN (...)`
with no server-side translation. The Region dropdown options come from a
new resource loader backed by `RegionsPresenter`, which now also exposes
`masterQueue` alongside the existing region metadata.
```ts
// public API
const runs = await runs.list({ region: ["us-east-1", "eu-west-1"] });
// each item: { id, status, ..., region?: "us-east-1" }
```
```ts
// MCP
list_runs({ environment: "prod", region: "us-east-1" })
```
|
||
|
Eric Allam
|
454f0c949a |
perf(webapp): cache task metadata in Redis for the trigger hotpath (#3625)
## Summary The trigger-task hotpath used to early-return without a DB query when a caller passed both a queue override and a per-trigger TTL — the hottest configuration on the trigger API. Adding `triggerSource` to the resolver so the runs-list "Source" filter could distinguish STANDARD / SCHEDULED / AGENT runs removed those early-returns, costing +2 DB queries per trigger on non-locked calls and +1 on locked calls. This change caches `BackgroundWorkerTask` metadata (`ttl`, `triggerSource`, `queueId`, `queueName`) in Redis so the resolver can satisfy every caller configuration with a single `HGET` on the warm path. PG fallback on miss back-fills the cache. Follow-up to #3542. ## Design Two key spaces: - `task-meta:env:{envId}` — the "current worker" view, refreshed at every deploy promotion. 24h safety TTL. - `task-meta:by-worker:{workerId}` — used for `lockToVersion` triggers. Immutable post-create. 30d sliding TTL so historical workers age out. Cache writes use Lua scripts via `defineCommand` so `DEL` + `HSET` + `EXPIRE` land atomically — concurrent readers never see the empty intermediate state of a naive pipeline. Read-path back-fill uses single-field upserts so concurrent back-fills don't wipe each other's siblings. The cache lives behind its own `TASK_META_CACHE_REDIS_*` env-var prefix that falls back to the default `REDIS_*` set, so operators can route the cache to a dedicated Redis instance if they want. The service/instance file split (`taskMetadataCache.server.ts` for the pure class, `taskMetadataCacheInstance.server.ts` for the env-wired singleton) mirrors the existing `runsReplicationService` / `runsReplicationInstance` pattern. ## Test plan - [ ] `pnpm run typecheck --filter webapp` - [ ] `pnpm run test ./test/engine/triggerTask.test.ts --run` — 8 existing tests untouched + 5 new tests covering warm cache, cold miss with back-fill, queue + ttl path, by-worker vs env keyspace, and the promotion cache write - [ ] End-to-end against a dev worker: registering writes both keyspaces with the expected TTLs, and `redis-cli HGETALL "tr:task-meta:env:<envId>"` returns the cached entries ## Benchmark Measured `DefaultQueueManager.resolveQueueProperties` against a real Postgres + Redis (vitest `containerTest`, single-host docker). 500 sequential calls and 2,000 parallel calls (concurrency=50) per scenario, request shaped as `{ taskId, queue: "bench-queue", ttl: "5m" }` — the hot path this PR restores. ``` sequential (one in flight at a time): [noop cache (baseline)] n=500 mean=1.423ms p50=1.394ms p95=1.735ms p99=2.629ms max=11.100ms [redis cache, cold ] n=500 mean=1.346ms p50=1.283ms p95=1.688ms p99=2.463ms max=5.058ms [redis cache, warm ] n=500 mean=0.084ms p50=0.078ms p95=0.105ms p99=0.156ms max=1.129ms speedup (warm vs baseline, sequential): 16.95x parallel (concurrency=50): [noop cache (baseline)] n=2000 mean=10.069ms p50=8.850ms p95=14.718ms p99=31.887ms total=405ms ops/s=4,940 [redis cache, warm ] n=2000 mean=0.614ms p50=0.568ms p95=1.189ms p99=1.432ms total=25ms ops/s=80,389 throughput speedup (warm vs baseline, parallel): 16.27x ``` Read: - **Warm cache cuts resolver latency 17×** at p50 — from ~1.4 ms to ~78 µs per call. - **Cold cache is on par with baseline** — the extra `HGET` miss adds <50 µs against the two Postgres queries that follow, so the worst case is not worse than today. - **Under burst load (50 concurrent triggers)**, the baseline's p99 jumps to ~32 ms as Postgres connections queue up; warm stays at ~1.4 ms. The cache moves the saturation point from ~5k ops/s (PG pool) to ~80k ops/s (single-client Redis pipelining). Caveats: single-host docker, local Postgres + Redis, resolver-only measurement (excludes the rest of the trigger transaction). Prod adds region-local Redis RTT (~0.3–0.8 ms) which shifts warm absolute numbers up but keeps the ratio intact. |
||
|
Daniel Sutton
|
bff4b46b22 |
fix(webapp): log Google auth conflict as warn instead of error (#3627)
## Summary A "Google auth conflict" Sentry alert fires whenever a user signs in via Google whose Google account is linked to one user row but whose Google-provided email is now on a *different* user row. The handler in `apps/webapp/app/models/user.server.ts:236` already does the right thing — it returns the existing auth-linked user and skips the update path so neither row gets mutated — but it logs the situation with `logger.error`, which routes to Sentry as an exception and pages the on-call channel. There's no exception to chase here: the branch is the intended outcome for a known data shape (user changed their email on one account after originally signing up via Google on another). Downgrading the call to `logger.warn` keeps the diagnostic record in our logs (with all the same context fields — email, both user IDs, authIdentifier) but stops it firing the production error alert. ## Change - `logger.error` → `logger.warn` for the conflict branch in `findOrCreateGoogleUser`. Context payload is unchanged. ## Test plan - [x] Typecheck only — there's no behavioural change to test, the log level is the entire diff. |
||
|
Eric Allam
|
16538f6fdb |
feat(webapp): agent-view dashboard for chat.agent runs (3/4) (#3545)
## Summary A chat-aware run inspector and a `/playground` UI for testing `chat.agent` tasks interactively. Builds on #3543's runtime. ## Design The run inspector grows a new tab that renders the conversation chain for any `chat.agent`-kind run. It subscribes to the run's session streams, threads chat parts through a per-message renderer, and uses a shared markdown + Shiki component for code highlighting (also used by the test-payload panel). The playground is a standalone `/playground` route that lets you drive a deployed chat agent from the dashboard — pick a task, send messages, watch tool calls render, and see span detail on every turn. The matching `/agents` list view shows all deployed agents in the project. |
||
|
Eric Allam
|
23ff763359 |
feat(webapp): agent-view dashboard for chat.agent runs
Dashboard surfaces for inspecting and debugging chat.agent runs. Depends on the Sessions primitive (L1) and chat.agent runtime (L2+L3). Run inspector — chat-aware: - AgentView + AgentMessageView (run inspector tab for chat.agent runs) - AIChatMessages + AISpanDetails + types.ts (per-span chat message rendering, tool-call/tool-output handling) - PromptSpanDetails (gen_ai.* span detail panel) - StreamdownRenderer + shikiTheme (markdown renderer with shiki highlighting and v2 patch) - useAutoScrollToBottom hook Playground UI (interactive chat.agent debugger): - /playground index + /playground/$agentParam routes - /agents route + AgentListPresenter - PlaygroundPresenter (per-org basin variants, clientData wiring) - realtime session routes for playground + run inspector chat - AI-generate-payload + AIPayloadTabContent for the test panel Navigation + theming: - SideMenu links for Agents and Playground - BlankStatePanels copy updates - tailwind config + tailwind.css storybook hooks - streamdown@2 dep in apps/webapp/package.json Includes agent-view-sessions, playground-trigger-config-fields, run-agent-view, and streamdown-v2-upgrade .server-changes. |
||
|
Daniel Sutton
|
5022769f70 |
fix(webapp): retry on version collision when initializing a deployment (#3610)
Concurrent `POST /api/v1/deployments` requests for the same environment race on the `WorkerDeployment(environmentId, version)` unique constraint. Both requests read the same latest deployment via `findFirst`, compute the same next version via `calculateNextBuildVersion`, and both attempt `prisma.workerDeployment.create()` — one wins, the other crashes with Prisma `P2002`. The bug is a classic TOCTOU between the version read and the version write; it's been latent since the version-assignment logic was first added but only fires when two deploys land within milliseconds of each other (CI matrices, retried CLI calls, webhook-triggered redeploys). ## Approach Extracts the version assignment + create into a small helper `createDeploymentWithNextVersion` (`apps/webapp/app/v3/services/initializeDeployment/createDeploymentWithNextVersion.server.ts`). The helper retries on `P2002 (environmentId, version)` up to 5 times with randomised 5–50ms jitter so N concurrent racers don't loop in lockstep. Each attempt re-reads the latest version, recomputes via `calculateNextBuildVersion`, and re-runs the caller's `buildData` callback so version-dependent fields (image ref tag, friendlyId) are always consistent with the version actually persisted. A `logger.warn` fires per collision so the retry rate is observable in production logs. When retries are exhausted, the helper throws a dedicated `DeploymentVersionCollisionError` carrying `environmentId`, `attempts`, and `lastAttemptedVersion`, with the original `PrismaClientKnownRequestError` attached as `cause`. Sentry walks the `cause` chain natively, so contention exhaustion shows up as a distinguishable wrapper exception linked to the underlying `P2002` rather than a generic unique-constraint violation that looks identical to every other duplicate-key bug. The behavioural change is limited to "catch P2002 and retry instead of crashing." The image ref computation stays inside the builder callback (same call site as before the refactor), so ECR / non-ECR behaviour, S2 stream creation order, and all downstream side effects are unchanged. ## Non-goals - No new database migrations, no schema changes, no isolation-level / locking changes. A serialisable transaction or advisory lock would also fix this; retry-on-conflict is the smaller change that keeps the existing version-allocation logic intact. - Does not touch the analogous `calculateNextBuildVersion` call in `createBackgroundWorker.server.ts`, which likely has the same race shape against `BackgroundWorker`'s unique constraint — flagged as a follow-up. ## Test plan - [x] `pnpm run typecheck --filter webapp` passes (no new errors in the modified files). - [x] Three real-Postgres tests in `apps/webapp/test/createDeploymentWithNextVersion.test.ts` via `containerTest`: - 5 concurrent calls all produce distinct, persistable versions (`Set(versions).size === concurrency`). The naive read-then-create version of the helper fails this test with the exact same `P2002` seen in production; the retry version passes. - Non-`P2002` errors raised from the `buildData` callback propagate immediately without retry, builder invoked exactly once. - With `maxRetries: 0`, concurrent racers surface the wrapped `DeploymentVersionCollisionError` (not a raw `P2002`); `environmentId`, `attempts`, `lastAttemptedVersion` are populated and `error.cause.code === "P2002"`. - [x] Existing `apps/webapp/test/getDeploymentImageRef.test.ts` still green (the file was untouched in the final diff). ## Follow-ups (not in this PR) - `createBackgroundWorker.server.ts` likely has the same TOCTOU shape against its background-worker version unique constraint — should use the same helper. - Sentry visibility check: confirm `error.cause` chain renders as a linked exception in the Sentry UI when the wrapped error fires (requires a sandboxed triggering of the exhaustion path). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Eric Allam
|
16720a5e62 |
feat(sdk): chat.agent — runtime + browser transport
Adds the chat.agent({...}) task definition (server runtime) and the
browser-side TriggerChatTransport + AgentChat that drives it from a
React or Next.js app. The runtime sits on top of the Sessions primitive
and handles the durable conversational task lifecycle.
Server runtime:
- chat.agent({...}) — session-aware task definition
- Lifecycle hooks: onChatStart, onTurnStart, onTurnComplete, onAction,
onValidateMessages, hydrateMessages
- chat.history read primitives for HITL flows
- chat.local, chat.headStart, chat.handover, oomMachine
- Delta-only wire + S3 snapshot reconstruction at run boot
- Actions are no longer turns
Browser transport:
- TriggerChatTransport (ai-sdk Transport): delta-only wire sends,
SSE reconnection with lastEventId resume, stop/abort cleanup,
dynamic accessToken refresh
- AgentChat: direct programmatic API
- useTriggerChatTransport (React hook)
- chat-tab-coordinator: cross-tab leader election
Includes the chat-agent, chat-agent-delta-wire-snapshots,
chat-history-read-primitives, chat-head-start, chat-actions-no-turn,
chat-session-attributes, agent-skills, and mock-chat-agent-test-harness
changesets.
|
||
|
Eric Allam
|
979655c281 |
feat: Sessions dashboard, task_kind, and chat-ready hardening (1/4) (#3542)
## Summary A `/sessions` dashboard for inspecting durable Sessions, an `AGENT` / `SCHEDULED` task-kind filter for the runs list, and the server-side hardening (rate-limit exemption for packets, retry-with-backoff on stream appends, typed too-large-chunk error) that the `chat.agent` runtime in #3543 needs. Builds on the Sessions primitive shipped in #3417. ## Design The Sessions list + detail routes mirror the run inspector pattern. `TaskTriggerSource` gains `AGENT` and `SCHEDULED` values, persisted on `BackgroundWorker.taskKind` and `TaskRun.taskKind` (plus a matching Clickhouse column), so the runs list can filter by kind. New `@trigger.dev/core` modules — `sessionStreams`, `inputStreams`, a `sessionStreamInstance` for realtime streams, and the `realtime-streams-api` / `session-streams-api` surfaces — expose the typed shapes that chat.agent will use to drive `session.out`. `ChatChunkTooLargeError` lets the runtime drop oversized chunks with a typed surface instead of failing the run. `s2Append` retries transient failures with exponential backoff. `/api/v[12]/packets/*` is exempt from customer rate limits so chat snapshot reads and writes don't get throttled under load. ## Stack Part of a 4-PR stack. Merge bottom-up. 1. **This PR** (#3542) → `main` 2. #3543 → #3542 — `chat.agent` runtime + browser transport 3. #3545 → #3543 — agent-view dashboard 4. #3546 → #3545 — ai-chat reference + MCP tooling Replaces #3173 (closed). <!-- GitButler Footer Boundary Top --> --- This is **part 5 of 5 in a stack** made with GitButler: - <kbd> 5 </kbd> #3612 - <kbd> 4 </kbd> #3546 - <kbd> 3 </kbd> #3545 - <kbd> 2 </kbd> #3543 - <kbd> 1 </kbd> #3542 👈 <!-- GitButler Footer Boundary Bottom --> |
||
|
Daniel Sutton
|
09f5354a03 |
fix(core): cap idempotencyKey length at the API boundary (#3560)
`tasks.trigger`, `tasks.batchTrigger`, `batch.create`, `wait.createToken`, `wait.forDuration`, and the input/session stream waitpoint endpoints all accept a caller-supplied `idempotencyKey` and store it verbatim against a composite-unique index on `TaskRun`, `BatchTaskRun`, or `Waitpoint`. The schemas had no length cap, so a sufficiently long high-entropy key produced an index row larger than the underlying storage layer can hold. The insert failed at the database, and the caller saw a generic 500 from `RunEngineTriggerTaskService.call()` / `CreateBatchService` / waitpoint creation, depending on the endpoint. Keys produced by `idempotencyKeys.create()` are 64-character SHA-256 hashes and never trip this — it only manifests for direct REST callers (or SDK callers passing a raw string they generated themselves). Low-entropy keys also sail through, because the storage layer compresses repeated bytes before they reach the index, which is why the failure mode is intermittent and tied to caller-side key shape. ## Fix Add `.max(2048, "<field> must be 2048 characters or less")` to the seven schemas that feed an indexed `idempotencyKey` column: - `TriggerTaskRequestBody.options.idempotencyKey` - `BatchTriggerTaskItem.options.idempotencyKey` - `CreateBatchRequestBody.idempotencyKey` - `CreateWaitpointTokenRequestBody.idempotencyKey` - `CreateInputStreamWaitpointRequestBody.idempotencyKey` - `CreateSessionStreamWaitpointRequestBody.idempotencyKey` - `WaitForDurationRequestBody.idempotencyKey` Plus the `idempotency-key` HTTP header on the trigger route (and the three batch routes that re-export `HeadersSchema`). The header schema is lifted out of `api.v1.tasks.$taskId.trigger.ts` into `apps/webapp/app/v3/triggerHeaders.server.ts` so it can be exercised in tests without dragging the route's import-time side effects. The 2048 character ceiling is chosen to sit safely under the per-row index limit while staying generous against existing callers — keys that fit before still fit. Oversized keys now return a structured Zod 400 instead of a generic 500. Limit is documented under `Idempotency key` in `docs/limits.mdx` and as a `<Note>` on `docs/idempotency.mdx`. ## Test plan - [x] 15 schema unit tests added (`packages/core/src/v3/schemas/idempotencyKey.test.ts`, `apps/webapp/test/routes/triggerHeaders.test.ts`) — rejection-with-message + boundary acceptance for each capped schema. The webapp test exercises the extracted `TriggerHeadersSchema` directly with no mocks. - [x] `pnpm run build --filter @trigger.dev/core` - [x] `pnpm run typecheck --filter webapp` - [x] End-to-end verified locally: baseline (small key) → 200; 3000-char high-entropy header → 400 with the expected Zod error; same key at the 2048 boundary → 200; same key with the cap reverted → the database rejected the insert and the route returned 500 to the caller. Cap restored. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Eric Allam
|
be1a6cf8de |
feat: Sessions primitive — durable run-aware streams + dashboard
Adds Sessions, a durable, run-aware stream primitive that scopes session.in / session.out records to a session (not a single run). Records survive run boundaries; reconnect-from-last-event-id is built in. Server foundation: - New /realtime/v1/sessions/:session/:io/append + /records routes - sessionRunManager + sessionsRepository + clickhouseSessionsRepository - mintRunToken for short-lived per-session tokens - s2Append retry-with-backoff + undici cause diagnostics - /api/v[12]/packets/* exempt from customer rate limits - BackgroundWorker schema gains taskKind enum (TASK, AGENT, SCHEDULED) - TaskRun.taskKind column + clickhouse 029_add_task_kind_to_task_runs_v2 Core types: - new sessionStreams, inputStreams, realtimeStreams packages in @trigger.dev/core - session-streams-api / realtime-streams-api surface Sessions dashboard UI (the primitive's own viewer): - /sessions index + detail routes - SessionsTable, SessionFilters, SessionStatus, CloseSessionDialog - AGENT/SCHEDULED filter in RunFilters + TaskTriggerSource Includes the sessions-primitive changeset. |
||
|
nicktrn
|
8ba067d8b0 |
feat(webapp): preserve admin tabs search query between Users and Organizations (#3609)
Switching between the Users and Organizations tabs in the admin dashboard now keeps the current `?search=` value, so you can flip between the two without re-typing your filter. Other admin tabs don't take `search` and so don't carry it. |
||
|
James Ritchie
|
759214eabd |
fix(webapp): Evict legacy resizable-panel localStorage on client boot (#3564)
## Summary
- Users on production are hitting `QuotaExceededError: Failed to execute
'setItem' on 'Storage'` when navigating runs, because their localStorage
is full of orphaned `panel-group-react-aria<n>-:<rid>:` entries.
- Each entry is a session-unique key written by the resizable panel
library; they accumulated to thousands per user over the last two months
and now block legitimate `setItem` calls (the run-view inspector can no
longer persist its layout, and the page crashes mid-render).
- This PR evicts the legacy entries once on client boot. The leak itself
is already plugged by the v1.1.3 upgrade in #XXXX — this is the cleanup
that recovers the wasted quota on existing users' machines.
## Root cause (already fixed, for context)
In v0.4.1 of the underlying library, `PanelGroupImpl` defaulted
`autosaveStrategy` to `"localStorage"` unconditionally — so *every*
`PanelGroup` wrote to localStorage on every autosave trigger, including
the four in `QueryEditor`, the one in `ReplayRunDialog`, the storybook
routes, etc. Without an `autosaveId`, the key fell back to
`panel-group-${useId()}`, and React Aria's `useId()` produces a new
session-unique prefix each visit. Result: entries accumulated without
bound across sessions.
The condition was introduced when
[#3282](https://github.com/triggerdotdev/trigger.dev/pull/3282) removed
the wrapper's explicit `autosaveStrategy="cookie"` override (to fix HTTP
431 cookie-size errors). That worked, but the library default that took
over silently caused this leak.
The v1.1.3 upgrade in the resizable-panel PR changed the default to
`autosaveStrategy = autosaveId ? "localStorage" : undefined`, so no new
entries are being written. Existing residue still needs to be removed
from users' browsers.
## Changes
- New file
[`apps/webapp/app/clientBeforeFirstRender.ts`](apps/webapp/app/clientBeforeFirstRender.ts)
— exports a `clientBeforeFirstRender()` function that runs
synchronously, before React hydrates. Encapsulates a small cleanup
helper that scans `localStorage` and removes:
- Every key starting with `panel-group-react-aria` (the legacy
auto-generated keys).
- The orphan `panel-run-parent-v2` key from before the autosaveId v2→v3
bump.
- [`apps/webapp/app/entry.client.tsx`](apps/webapp/app/entry.client.tsx)
— imports and invokes `clientBeforeFirstRender()` once, before
`hydrateRoot()`. This guarantees the cleanup completes before any
`ResizablePanelGroup` mounts and tries to write.
The cleanup is wrapped in `try/catch` so private-browsing /
disabled-storage scenarios fail silently. Idempotent: subsequent loads
find no matching keys and exit immediately.
## Test plan
- [x] Locally seed ~50 fake `panel-group-react-aria…` entries plus a
`panel-run-parent-v2` entry via DevTools console, hard reload → legacy
entries gone, real entries (`panel-run-parent-v3`, `panel-run-tree`)
preserved.
- [x] Idempotency: reload a second time, no errors, no state changes.
- [x] Add a control entry (`panel-run-parent-v3-but-different-suffix`) —
confirmed not over-matched.
- [x] Simulate broken `Storage.setItem` throwing — page still renders,
cleanup swallows the error.
- [x] Typecheck clean.
## Notes
- Customer report: `QuotaExceededError: Failed to execute 'setItem' on
'Storage': Setting the value of 'panel-run-parent-v3' exceeded the
quota.`
- The cleanup runs once per page load. Once a user has loaded the app
after this deploys, their localStorage is clean and the function becomes
a no-op forever.
|
||
|
Matt Aitken
|
e4981d1b11 |
feat(webapp): consolidate auth path + add comprehensive auth tests (#3499)
## Summary
Consolidates the webapp's authentication and authorization into a small
set of route helpers, replacing the ad-hoc `requireUser` /
`requireUserId` / `authenticatedEnvironmentForAuthentication` calls
scattered across routes. Same security model, but the per-request flow
(authenticate → authorize → load) now lives in one place per route
family.
Introduces a plugin seam (`@trigger.dev/plugins`) that lets the cloud
build install a richer RBAC implementation without touching webapp code.
The OSS fallback keeps the pre-RBAC permissive behaviour intact, so
self-hosted deployments work unchanged.
Adds a comprehensive end-to-end auth test suite that didn't exist before
— 193 `it()` blocks (vitest reports ~199 after `it.each` expansion)
covering API key, PAT and JWT auth across the public API surface, plus
dashboard session auth for admin pages.
## Changes
### Plugin contract — `@trigger.dev/plugins`
`RoleBaseAccessController` interface authoritative for both OSS
(fallback) and cloud (enterprise plugin):
- `authenticateBearer(request, { allowJWT? })` — API-key / public-JWT
auth, returns env + ability
- `authenticateSession(request, { userId, organizationId?, projectId?
})` — dashboard auth, caller resolves `userId` from the session cookie
and passes it in (no `helpers.getSessionUserId` callback — decouples the
plugin host from session-cookie code)
- `authenticatePat(request, { organizationId?, projectId? })` — PAT
auth, returns identity + `lastAccessedAt` so the host can throttle the
per-request update
- `authenticateAuthorize*` variants for the auth-and-check-in-one-call
cases
- `isUsingPlugin(): Promise<boolean>` — capability flag for UI /
branching where plugin-present-ness matters; replaces the
sentinel-string coupling that had `personalAccessToken.server` matching
`"RBAC plugin not installed"` literally
### Dashboard auth (started, partial rollout)
Admin and settings pages migrated to a unified `dashboardLoader` /
`dashboardAction` helper that authenticates the session, runs an
authorization check, and exposes the result to the route. Other
dashboard routes still on the old pattern; remaining migration tracked
in TRI-8730.
Migrated routes:
- `admin.*` (14 admin / back-office / feature-flags / LLM-models /
notifications / orgs / concurrency pages)
- `_app.orgs.$organizationSlug.settings.team`
- `_app.orgs.$organizationSlug.settings.roles`
### API / realtime / engine auth (complete for the migrated families)
71 routes migrated to a unified `apiBuilder` that centralizes Bearer /
PAT / Public-JWT authentication and applies the per-route authorization
check before the handler runs. Includes:
- `api.v1.*` and `api.v2.*` and `api.v3.*` — tasks, runs, batches,
queues, prompts, deployments, query, sessions, waitpoints, packets,
workers, idempotency keys
- `realtime.v1.*` — runs, batches, sessions, streams
- `engine.v1.*` — dev / worker-action protocols
29 routes still on the legacy `authenticateApiRequest*` helpers —
tracked as a post-deploy follow-up in TRI-9228.
Multi-resource auth direction is now explicit at the call site via
`anyResource(...)` (OR) and `everyResource(...)` (AND). Bare arrays no
longer typecheck — fixes a class of bug where a JWT scoped to one
resource could implicitly access others under OR semantics.
PAT auth path consolidated: was three DB queries per request (legacy
`authenticateApiRequestWithPersonalAccessToken` findFirst +
`rbac.authenticatePat` join + `lastAccessedAt` update). Now one query in
the steady state — plugin returns `lastAccessedAt`, host smart-skips the
update via JS-side throttle when fresh.
Side effect: action aliases preserved historic JWT scope semantics where
the new model is stricter (e.g. a `write:tasks` JWT now also satisfies
`trigger` / `batchTrigger` / `update` actions on the same resource —
matched at the auth boundary, not in the route handler).
### Backwards-compat fixes
The strict-match model regressed several real-world JWT shapes. Each
preserved via explicit `anyResource(...)` entries in the route's authz
block:
- **Batch retrieve routes** (`api.v1.batches.$batchId`, `api.v2.*`,
`realtime.v1.batches.*`) accept `read:runs` JWTs again (pre-RBAC
literal-match superScope behaviour)
- **Runs list routes** (`api.v1.runs`, `realtime.v1.runs`) accept
type-level `read:tasks` / `read:tags` on unfiltered queries (matched the
legacy `Object.keys` iteration semantic)
- **PAT/OAT auth shape** normalized through `toAuthenticated` so all
auth methods return the same slim `AuthenticatedEnvironment` (was:
API-key returned the slim shape but PAT/OAT returned raw Prisma
`Decimal` / no `orgMember`)
- **Scope `:` preservation** in resource ids — `read:tags:env:staging`
now correctly identifies the tag id as `env:staging`, not `env`
### Slim `AuthenticatedEnvironment`
Extracted to `@trigger.dev/core/v3/auth/environment` — a structural
shape independent of `@trigger.dev/database`. The plugin contract
returns this; webapp consumers import from there; the cloud plugin
(Drizzle) returns the same shape without Prisma's `Decimal` class
leaking into the public surface. Lets internal-packages (run-engine,
etc.) refer to `AuthenticatedEnvironment` without pulling Prisma in.
### Auth test suite (new — `*.e2e.full.test.ts`)
193 e2e tests run against a real spawned webapp + Postgres (no mocks).
Coverage matrix:
- **API key auth** — read / write / trigger / batchTrigger / deploy
actions across runs, batches, deployments, prompts, queues, query,
sessions, input-streams, waitpoints, tasks, idempotency keys; multi-key
resources (a run carries batch / tag / task identifiers — auth must
accept any matching scope)
- **Personal Access Token auth** — comprehensive matrix: scope match,
scope mismatch, missing scope, expired token, malformed token
- **Public JWT auth** — sub-vs-URL environment resolution, expired JWTs,
signature verification, scope checking, otu (one-time-use) token
semantics, branch-environment signing-key fallback
- **Dashboard session auth** — admin-only pages reject non-admins;
per-action gating
- **Cross-cutting edge cases** — revoked API key grace window, JWT
cross-environment isolation, MissingResource branch behaviour
### Hygiene cleanups
- Deleted dead `app/services/authorization.server.ts` (legacy
`checkAuthorization` + types — no live consumers post-migration) and its
orphaned test
- Dropped the never-populated `scopes` field from
`ApiAuthenticationResultSuccess`
- `scheduleEmail` moved out of `email.server.ts` into its own module —
breaks a `commonWorker → marqs/V1` import chain that was poisoning the
auth test graph
- OSS Roles page shows a deployment-aware empty state ("Roles aren't
available in this self-hosted deployment" vs the plan-upsell copy) via
`rbac.isUsingPlugin()`
- Team action handler: explicit per-intent ability gates
(`manage:billing` for purchase-seats, `manage:members` for set-role +
remove-member with self-leave carve-out)
### Cross-repo coordination
All public-package contract changes paired in `triggerdotdev/cloud#763`
(rbac-packages branch) — the enterprise plugin implements the same
`RoleBaseAccessController` interface against Drizzle.
## Test plan
- [x] `pnpm run typecheck --filter webapp` clean
- [x] `pnpm --filter webapp exec vitest run --config
vitest.e2e.full.config.ts` — 193/193 pass (requires Docker for
testcontainers)
- [x] Spot-check an authed API endpoint with a valid + invalid API key
against a local stack
- [x] Spot-check the migrated admin pages render and gate non-admins
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
James Ritchie
|
1e4b896c30 |
Fix(webapp): Notification style updates (#3553)
### Style updates to the notifications - Tightened up the typography - Brighter background to make it stand out a bit more - A bit more padding to make it more readable - Show the close button on hover instead - Turned the notification into a separate component as it's shared on the admin page modal - Minor tweaks to the behavior of toggling the notification beween open/closed side menu states ### Before <img width="224" height="313" alt="before" src="https://github.com/user-attachments/assets/c9a9377c-4a3b-4477-921a-3c86385d3f0b" /> ### After (with image) <img width="239" height="284" alt="CleanShot 2026-05-11 at 17 22 01" src="https://github.com/user-attachments/assets/311b4dbc-4853-4e6c-9f83-8173b38bd466" /> ### After (no image) <img width="239" height="189" alt="after" src="https://github.com/user-attachments/assets/884e062b-3608-4cb3-a462-d50597257753" /> --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> |
||
|
Iss
|
2b845455b0 |
feat(webapp): admin back-office editors for org max projects and batch rate limit (#3475)
## Summary - Adds admin-only editors on the back-office org page for `Organization.maximumProjectCount` and `Organization.batchRateLimitConfig`, alongside the existing API rate limit editor. - Splits the back-office org page into per-section components (`ApiRateLimitSection`, `BatchRateLimitSection`, `MaxProjectsSection`) so each tool is self-contained — adding new sections later doesn't bloat the route. - Generalizes the rate-limit form into a reusable `RateLimitSection` component + `RateLimitDomain` server config so API and batch share the same UI, validation, and action handler. Each domain only owns its env defaults, DB column, and logger key. - "Saved." banner and validation errors are scoped to the section that submitted, not the page. Heads-up: the API rate-limit log key was renamed `admin.backOffice.rateLimit` → `admin.backOffice.apiRateLimit` for symmetry with the new `admin.backOffice.batchRateLimit`. ## Test plan - [ ] As an admin, visit `/admin/back-office/orgs/:orgId` and confirm all three sections render with the org's current values (or system defaults). - [ ] Edit and save each section; confirm only that section shows the "Saved." banner. - [ ] Submit invalid input (e.g. `0` tokens, malformed interval); confirm errors render in the offending form only and the other sections stay closed. - [ ] Confirm a non-admin user is redirected away from the route. - [ ] After saving a rate-limit override, hit the org with traffic and confirm the new limit is enforced (API rate limit + batch rate limit code paths read the column at request time). |
||
|
Eric Allam
|
567e2a2c32 |
feat(webapp,redis): handle READONLY / LOADING during ElastiCache failover (#3548)
## Summary
During an ElastiCache role swap (failover) or node-type change (vertical
scale), the ioredis TCP/TLS connection stays open but the server starts
answering with `READONLY` (the client is talking to a node that became a
replica) or `LOADING` (node still loading data from disk). Without an
explicit hook, those errors surface to caller code as `ReplyError`
instances — every write op on the affected connection fails until the
cluster fully cuts over.
This PR adds `reconnectOnError` to every prod ioredis client so the
disconnect + reconnect + retry cycle absorbs these errors and caller
code never sees them.
## Fix
```ts
export function defaultReconnectOnError(err: Error): boolean | 1 | 2 {
const msg = err.message ?? "";
if (msg.startsWith("READONLY") || msg.startsWith("LOADING")) return 2;
return false;
}
```
Returning `2` tells ioredis to disconnect, reconnect, and re-issue the
failed command. After reconnect, DNS / SG state routes the new socket to
a writable node.
The helper lives in `@internal/redis` and is wired into both the shared
`createRedisClient` (which covers RunQueue, schedule-engine,
redis-worker, and every other internal-package consumer) and the direct
`new Redis(...)` call sites in the webapp.
V1-only marqs files are intentionally not migrated.
## Test plan
- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm run typecheck --filter @internal/run-engine`
- [x] Verified end-to-end against a live ElastiCache vertical-scale
event — caller-surfaced errors went from tens of thousands during the
cutover window down to a handful per ioredis client
- [ ] Confirm steady-state behavior unchanged after deploy
|
||
|
James Ritchie
|
6cdd8814a3 |
fix(webapp): Fix for resizable side panel getting stuck at its min-size (#3538)
## Summary - Run-view inspector panel was glitching out on Firefox: visual flicker on close, locking up at min size, and intermittent `panelHasSpace` invariant errors. Root cause is the underlying `react-window-splitter` library's collapse animation, which uses `@react-spring/rafz` and interacts poorly with Firefox. - Disabled the library's collapse animation on Firefox only, app-wide (every consumer of `RESIZABLE_PANEL_ANIMATION`). Chromium and Safari behaviour is unchanged. ## Changes - **Firefox animation skip** in `RESIZABLE_PANEL_ANIMATION` — UA-detected at module load, resolves to `undefined` for Firefox so the library's animation actor completes in one frame instead of running its rAF loop. - **Inspector min raised 50px → 250px** so dragging can't shrink the panel into a near-useless width. - **`autosaveId` bumped `v2` → `v3`** to invalidate stale persisted snapshots (the library has a `// TODO` branch that ignores prop changes for already-registered panels, so existing users would otherwise still see the old 50px min). - **`react-window-splitter` pinned** to exact `0.4.1` to protect the patch from drifting if line offsets change in a patch release. - **Two hunks added to the existing `@window-splitter/state` patch:** - Removed the library's auto-collapse-on-drag block entirely. Every collapsible panel in the app is parent-controlled, and that block was triggering state-machine deadlocks when handlers were no-ops. Drag-to-collapse is now disabled across the app; collapse is only triggered explicitly (close button, ESC, URL change, etc.). - In `getDeltaForEvent`, fall back to the panel's `default` before its `min` when expanding — so the first ever click on a span opens the inspector at 500px, not 250px. ## Local testing confirmed - [x] Firefox: open a run, click various spans → panel opens instantly at 500px, drags freely between 250px and max, closes instantly to 0. No console errors. - [x] Chrome/Chromium: same flow, but with smooth open/close animation as before. - [x] Safari: same as Chrome. - [x] Reload mid-session → panel restores cleanly to the dragged size. - [x] Other resizable panels in the app (logs, deployments, schedules, batches, bulk-actions, runs index) still animate on Chromium/Safari. ## Notes - Linear: TRI-8584 - Branch contains intermediate commits exploring an unsuccessful snapshot-validator approach; they're reverted by the final commit. Cumulative diff is 6 files. Squash on merge if you'd prefer a clean history. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Eric Allam
|
ead1e5a53d |
feat(webapp): reload LLM pricing registry on Redis pub/sub (#3534)
## Summary Adds a Redis pub/sub reload path to the webapp's in-memory LLM pricing registry. When enabled on a process, the registry reloads from the database whenever a publish lands on the configured channel — instead of waiting for the existing 5-minute interval. Lets pricing/model changes propagate to cost enrichment within seconds. Subscription is **off by default** and opt-in per process. Only OTel-ingesting services need real-time freshness; dashboard and worker services run fine on the periodic interval and shouldn't pile onto each publish with a full-table reload. ## Design When `LLM_PRICING_RELOAD_PUBSUB_ENABLED=true`, subscribes via `createRedisClient` against `COMMON_WORKER_REDIS_*` and listens on `LLM_PRICING_RELOAD_CHANNEL` (default `llm-registry:reload`). The 5-minute periodic reload stays as a backstop, and a SIGTERM/SIGINT handler closes the subscription cleanly. The publisher side lives outside this PR — any process running in the same Redis namespace can trigger a reload by `PUBLISH llm-registry:reload <anything>`. Includes a `.server-changes/` note for the changelog. ### Debounced reload Bursts of publishes are coalesced. The first publish schedules a reload at T+`LLM_PRICING_RELOAD_DEBOUNCE_MS` (default 1s); subsequent publishes during that window are no-ops because the trailing reload picks up everything when it queries the DB. Bounds reload rate to at most 1 per debounce window regardless of publisher chattiness, so a runaway upstream publisher can't fan out into a flood of full-table-scan reloads. ## Test plan - [ ] With `LLM_PRICING_RELOAD_PUBSUB_ENABLED=false` (default): `redis-cli PUBSUB NUMSUB llm-registry:reload` returns `0` while the webapp is up - [ ] With it set to `true`: returns `>= 1` - [ ] `redis-cli PUBLISH llm-registry:reload test` returns `1` (one subscriber received) on a subscribed process - [ ] Mutate an `LlmModel` row externally, publish on the channel, observe the registry's match() picks up the change without waiting for the 5-min tick - [ ] Publish 100x in rapid succession; confirm only one reload fires within the debounce window |
||
|
James Ritchie
|
f7a2bc7c96 |
Feat(webapp) filters UX update (#3451)
## Lots of filter UX improvements across lots of routes ### General - Promoted important filters out of the "More filters" so they're always visible - SearchInput primitive is now reusable and Esc now clears the field (AI filter input also clears with Esc) - Tooltips + keyboard shortcuts on every primary filter button - Brighter text on selected filter items / queue items - Filter dropdowns reordered for better hierarchy - Removed debounce on Tasks page search for faster filtering ### Tasks page search - Esc now clears the field - ENTER submits a search to improve performance when you have lots of tasks https://github.com/user-attachments/assets/4b30521e-dbc4-4468-b2af-8c85bdfb9002 ### Runs filters - Moves Status and Tasks out of the More filters menu - "Root only" toggle is set to false when you filter for a Task. This state isn't stored and flips back to the stored value if filters are cleared <img width="1690" height="986" alt="CleanShot 2026-04-26 at 19 24 08@2x" src="https://github.com/user-attachments/assets/b07da73c-140e-451f-a7bf-c32129317f63" /> ### Batches filters - General consistency improvements <img width="1429" height="948" alt="CleanShot 2026-05-08 at 09 50 35" src="https://github.com/user-attachments/assets/e5ec267f-2aa3-43ef-991e-93bf01bdaea5" /> ### Schedules - General consistency improvements <img width="1567" height="1141" alt="CleanShot 2026-05-08 at 09 51 11" src="https://github.com/user-attachments/assets/34b7da88-87c6-4e4d-a70f-fe13ea9f87ec" /> ### Queues - General consistency improvements <img width="824" height="416" alt="CleanShot 2026-05-08 at 09 52 02" src="https://github.com/user-attachments/assets/b4adc102-8192-4a68-b199-a175c2645a6c" /> ### Waitpoint tokens - General consistency improvements <img width="941" height="363" alt="CleanShot 2026-05-08 at 09 52 19" src="https://github.com/user-attachments/assets/d43aeb3f-7f80-454d-b183-fd077a4e3ff7" /> ### Models - General consistency improvements <img width="1570" height="509" alt="CleanShot 2026-05-08 at 09 53 17" src="https://github.com/user-attachments/assets/066d7646-4672-4cae-8ec0-e30a82889914" /> ### AI metrics - General consistency improvements <img width="1568" height="624" alt="CleanShot 2026-05-08 at 09 53 43" src="https://github.com/user-attachments/assets/fdfc4806-26fa-458d-a5ed-5c226b3bbc9f" /> ### Logs - General consistency improvements <img width="1267" height="752" alt="CleanShot 2026-05-08 at 09 54 30" src="https://github.com/user-attachments/assets/3e9ba871-b9dd-490e-aded-5d87134fd2bb" /> ### Errors - General consistency improvements <img width="1568" height="670" alt="CleanShot 2026-05-08 at 09 54 50" src="https://github.com/user-attachments/assets/fdda027a-e24f-4804-b4bb-203a6c2db960" /> ### Query - General consistency improvements - History, Scope, Triggered (date) filters all have shortcut tooltips - Scope filter now reuses the metrics ScopeFilter component <img width="1566" height="716" alt="CleanShot 2026-05-08 at 09 55 22" src="https://github.com/user-attachments/assets/0130b4a2-9daf-4edc-bada-3380aff4022a" /> ### Dashboards - General consistency improvements - Scope filter gets nicer icons and a shortcut - Nice icons for the Scope menu items <img width="1567" height="769" alt="CleanShot 2026-05-08 at 09 56 10" src="https://github.com/user-attachments/assets/7bea25f7-6c33-4d4a-a36d-3a1cb56afe09" /> ### Custom dashboard - General consistency improvements - Add chart, Add title, and the kebab menu now have tooltips + shortcuts <img width="1566" height="782" alt="CleanShot 2026-05-08 at 09 58 11" src="https://github.com/user-attachments/assets/9df4db25-b2c0-43a2-b92f-00256337d5a9" /> ### Environment variables - General consistency improvements <img width="1569" height="930" alt="CleanShot 2026-05-08 at 09 58 55" src="https://github.com/user-attachments/assets/26e614b4-88e7-400b-aa6d-a96bad488fb8" /> ### Preview branches - General consistency improvements <img width="1570" height="986" alt="CleanShot 2026-05-08 at 09 59 17" src="https://github.com/user-attachments/assets/57a2b939-3670-4252-ab2c-d6dc65bdda1b" /> |
||
|
Iss
|
f8ddb766fa | feat: Plain customer cards (#2933) | ||
|
Daniel Sutton
|
61ae67cc02 |
fix(webapp): stop leaking exception messages on 5xx API responses (#3536)
When a webapp API route's catch-all 500 branch handles a non-typed
exception, it returns the raw `error.message` to the caller. If the
exception originates from an internal subsystem (the ORM client, an
infra dependency, etc.) the server-side error string is surfaced
verbatim in the response body — exposing implementation details the API
surface shouldn't carry.
The leak shows up in three shapes across the routes:
- `return json({ error: error.message }, { status: 500 })`
- `return json({ error: error instanceof Error ? error.message :
"Internal Server Error" }, { status: 500 })`
- ``return json({ error: `Internal server error: ${error.message}` }, {
status: 500 })``
(plus a couple of analogous neverthrow-Result variants on admin routes.)
## Fix
Across 19 webapp routes, replace each leaking branch with a generic body
(`"Something went wrong"` / `"Internal Server Error"` to match the
file's existing fallback) and add `logger.error(...)` so full visibility
is preserved server-side. Catch blocks that branch on typed user-input
errors (`ServiceValidationError`, `EngineServiceValidationError`,
`OutOfEntitlementError`, `PrismaClientKnownRequestError`) are left
intact — those messages are constructed deliberately and intended to be
customer-facing.
## Test plan
- [x] `pnpm run typecheck --filter webapp`
- [x] Per-route manual probe: inject a synthetic `Error` at the top of
the catch'd `try` block (or fake the wrapped call's rejection / Result
error), curl the route with the dev API key, confirm the response body
changed from the synthetic message verbatim → generic body. 21/21 leak
sites verified end-to-end.
- [x] 4xx-typed-error paths spot-checked: throwing
`ServiceValidationError` from inside the catch'd try still surfaces its
message at 422 as intended.
|
||
|
Daniel Sutton
|
749dc467f1 |
feat(webapp): link Sentry events to OTel traces via trace_id (#3531)
## Summary Stamps the active OpenTelemetry `trace_id` and `span_id` onto every Sentry event captured from the webapp, so engineers can copy a `trace_id` from a Sentry issue and search for the corresponding trace in any OTel-aware backend. Also adds an `otel_sampled` tag to indicate whether the trace was head-sampled — a cheap signal for whether the link will resolve to span data or hit a missing trace. ## Why Sentry and OTel were OTel-disconnected: `apps/webapp/sentry.server.ts` initialised Sentry with `skipOpenTelemetrySetup: true`, and no error-capture site (`logger.server.ts`, the Remix-wrapped `handleError`, the root `ErrorBoundary`) attached OTel context to the event. With many spans/sec across services, getting from a Sentry issue to its trace was guesswork. ## Approach Single global Sentry event processor, registered immediately after `Sentry.init`. On each event it reads `trace.getActiveSpan()?.spanContext()` via `@opentelemetry/api`, then writes: - `event.contexts.trace.trace_id` and `event.contexts.trace.span_id` (Sentry's native trace context fields) - `event.tags.otel_sampled` = `"true"` | `"false"` (derived from `traceFlags`) If no active span (module-load errors, scheduled timers without a context, primary cluster process), the processor returns the event unmodified — Sentry's default propagation context fills in. Implementation is co-located in `apps/webapp/sentry.server.ts` (no separate helper module — `sentry.server.ts` is built standalone by esbuild and a separate import would have required a new bundling step). Helper functions are exported so the unit tests can reach them without re-running `Sentry.init`. ## Non-goals (deliberate) - No sample rate change. ~95% of Sentry events will carry a `trace_id` that returns no spans in the tracing backend (head-sampled out). The `otel_sampled` tag makes that obvious at a glance. Raising find-rate is a separate conversation with cost trade-offs. - No user/org tags or `Sentry.setUser` (would need auth-helper + per-request scope wiring across multiple worker entrypoints — separate ticket). - Webapp image only. No changes to supervisor or CLI workers. ## Test plan - [x] Unit tests in `apps/webapp/test/sentryTraceContext.server.test.ts` — 9 tests covering: helper returns \`undefined\` with no active span; returns \`traceId\`/\`spanId\`/\`sampled=true\` for a recording span; returns \`sampled=false\` for a non-recording span; processor leaves the event unchanged with no active span; processor stamps \`trace_id\`/\`span_id\` onto \`contexts.trace\`; preserves existing \`contexts.trace\` fields; tags \`otel_sampled\` correctly for both sampled and non-sampled cases; never throws if \`@opentelemetry/api\` access throws. - [x] \`pnpm run typecheck --filter webapp\` passes. - [x] Manually verified end-to-end against a sandboxed Sentry project: confirmed both sampled and non-sampled traces correctly populate \`contexts.trace.trace_id\` matching the OTel ids logged from the loader, and the \`otel_sampled\` tag appears with the expected value. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Eric Allam
|
31999afcaa |
perf(webapp): trim BackgroundWorker.metadata to the schedule slice on create (#3525)
Large deploys (projects with many tasks or source files) blocked the
webapp event loop for several seconds inside Prisma's client-side
serializer on `BackgroundWorker.create`, tail-latencying every other
in-flight request on the same Node process. The `metadata` JSON column
was being written with the full deploy manifest — every task's config,
every queue and prompt, and the full source of every file — all of which
already live on dedicated columns or in dedicated tables.
Fix: project the manifest to `{ packageVersion, contentHash, tasks: [{
id, filePath, schedule }] }` on insert. The only post-write read site is
`changeCurrentDeployment`, which feeds `tasks[].schedule` into
`syncDeclarativeSchedules` at deploy promotion. The retained top-level
keys and per-task `filePath` are kept solely so
`BackgroundWorkerMetadata.safeParse` still succeeds on read.
## Test plan
- [ ] Deploy a project with declarative schedules; verify schedules are
created on first deploy
- [ ] Modify / remove schedules across subsequent deploys; verify sync
- [ ] Roll back to a previous deploy; verify `changeCurrentDeployment`
re-syncs schedules
- [ ] Inspect `BackgroundWorker.metadata` on a fresh deploy — should be
a small object, not the full manifest
|
||
|
Daniel Sutton
|
14920ce2c4 |
fix(webapp): downgrade expected user-input error logs to warn (#3523)
`dac9c83bd` added `ignoreErrors: /^ServiceValidationError(?::|$)/` in
`apps/webapp/sentry.server.ts` to drop SVEs before they reach Sentry.
The
filter only matches when the captured event's *type* is
`ServiceValidationError`, but nine call sites in the webapp catch SVE
(and
analogous user-input error types — `OutOfEntitlementError`,
`CreateDeclarativeScheduleError`, `QueryError`) and call
`logger.error("wrapper message", { error: e })` *before* the type check.
The captured event is then titled with the wrapper message, with the
inner
error buried in `extra.error` — invisible to the SDK filter. Result: a
steady stream of expected user-input failures escalating as
`error`-level
events when they should be `warn`.
Each catch block now type-discriminates first, logs expected types at
`warn`,
and keeps unknown-error fall-throughs at `error`. For service sites that
wrap into SVE (`createBackgroundWorker`,
`createDeploymentBackgroundWorkerV4`),
the inner error is logged at `error` before wrapping — mirrors the
`waitpointCompletionPacket.server.ts` pattern from `dac9c83bd`.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Eric Allam
|
386b4f65ff |
feat(webapp): per-org S2 basin migration (#3516)
## Summary Move from a single shared S2 basin to **per-org basins** with retention tied to the org's billing plan. Stops S2 from deleting streams out from under live chat sessions when basin retention fires before the chat ends, and unlocks per-org cost attribution. OSS / s2-lite installs are unaffected: provisioning is gated by `REALTIME_STREAMS_PER_ORG_BASINS_ENABLED` (default `false`), and the read precedence falls back to the global basin env var when an entity has no stamped basin. ``` basin = run.streamBasinName ?? session.streamBasinName ?? env.REALTIME_STREAMS_S2_BASIN ``` ## Design Three nullable `streamBasinName` columns (`Organization`, `TaskRun`, `Session`) plus a provisioner that idempotently creates the basin and reconfigures retention on plan changes. The trigger and session-create paths stamp the org's basin onto new rows; the realtime read path picks the basin from the entity context. Admin routes back-fill existing orgs and force-reconfigure a single org. ## Test plan - [x] `pnpm run typecheck --filter webapp --filter @internal/run-engine` - [x] Backfill admin route end-to-end (provision + DB stamp + S2 basin config). - [x] Reconfigure on plan change (all retention tiers). - [x] chat.agent multi-turn drives streams into the per-org basin. - [x] Legacy fallback when entity has no stamped basin. - [x] Provisioner is a no-op when the flag is off. |
||
|
James Ritchie
|
45ec23cc73 |
feat(webapp): app auto session logout (#3473)
<img width="2284" height="2028" alt="CleanShot 2026-05-01 at 18 53 50@2x" src="https://github.com/user-attachments/assets/4f58cbb1-0168-40fb-a523-017f2ba625a1" /> ## Performance - **Per-request DB hit**: `getUserId` runs `getEffectiveSessionDuration` (User lookup + Org `aggregate`) on *every* authenticated request, including each fetcher poll. Consider caching the effective duration in the session cookie with a short TTL (e.g. 60s) and revalidating in the background. - **Double session commit in `root.tsx`**: `getUser` already runs the expiry check; then `commitAuthenticatedSessionLazy` commits the cookie again. Fine, but doubles `Set-Cookie` headers on every page load — worth a quick perf check. ## Correctness / Edge cases - **Lazy backfill assumes a root.tsx hit first**: users whose first post-deploy request is a fetcher/API route (`/resources/*`) skip the backfill until they navigate to a page. Not a security hole, but `getUserId` could backfill itself for completeness. - **No upper bound on `Organization.maxSessionDuration`**: admin API accepts `1` second, which would instant-logout every member on next request. Add a `min(60)` (or `min(300)` to match the lowest user option) to the Zod schema. - **No clock-skew tolerance**: `isSessionExpired` is exact-millisecond. Multi-instance deploys with skewed clocks could log users out a few seconds early/late. Probably fine for the 5-min minimum, but worth noting. ## Security - **Auto-logout audit log lacks IP/orgId**: HIPAA forensics typically wants source IP and which org context. Currently logs only `userId` + path. IP isn't PII for audit purposes; orgIds help correlate. Add both. - **Cookie `Max-Age` is 1 year regardless of user's setting**: intentional (server-side `issuedAt` is the source of truth), but reviewers will ask. Add a one-line comment on the cookie config explaining why. ## API surface - **`maxSessionDuration` is admin-PAT only**: no in-app UI for org owners to set/change their own cap. If this is "Trigger staff sets it during HIPAA onboarding", say so in the PR description; otherwise add an org-settings UI. - **Auto-submit dropdown has no confirmation**: misclicking "5 minutes" immediately shortens the user's session window with no undo. Consider a save button or 3-sec undo toast. ## Schema / migration - **`User.sessionDuration NOT NULL DEFAULT 31556952`**: instant on PG 11+ (metadata-only), but call out in the PR description so reviewers don't worry about a table rewrite on the User table. - **No DB-level constraint matching `SESSION_DURATION_OPTIONS`**: if the option list changes, existing users keep orphaned values. The dropdown's tag-along behaviour hides this — fine for now, but if you ever drop an option you'll need a backfill. ## UX - **Session expiry only fires on next request**: an idle authenticated tab keeps showing UI past the cap (until SSE/polling catches it, ~60s). Add a client-side timer based on the user's effective duration that triggers a fetcher to `/account` or `/logout` at expiry. - **No "you were signed out" message on logout**: users hitting their cap are bounced to `/` with no explanation. Was intentionally reverted in this PR — call that out so reviewers don't request it. ## Tests - Unit coverage on `sessionDuration.server.ts` is solid (215 lines). Missing: integration test for `getUserId` → expired session → redirect to `/logout`, and one for the loader's clamping fix (the most recent bug). Add at least the second one to lock in the regression. --------- Co-authored-by: Matt Aitken <matt@mattaitken.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
nicktrn
|
706a0b88c9 |
chore: upgrade pnpm to 10.33.2 with security hardening (#3489)
## Summary - Upgrade pnpm from 10.23.0 → 10.33.2 (latest minor) - Enable `blockExoticSubdeps: true` for supply-chain defense - Update all version references across the repo ## Security improvements in 10.28.2+ - Path traversal protection in `directories.bin` - Symlink-escape protection for `file:/git:` dependencies (prevents reading `/etc/passwd`, `~/.ssh/...`) - https://pnpm.io/settings#blockexoticsubdeps ## Files updated - `package.json` — `packageManager` field - `docker/Dockerfile` — 5 `corepack prepare` calls - `apps/supervisor/Containerfile` — 1 `corepack prepare` call - `pnpm-workspace.yaml` — added `blockExoticSubdeps: true` - `CLAUDE.md`, `AGENTS.md`, `CONTRIBUTING.md`, `ai/references/repo.md` — version references ## Verification - `pnpm install --frozen-lockfile` succeeds (no lockfile regen needed) - `pnpm install` (plain) produces zero lockfile diff - All CI checks pass Slack thread: https://triggerdotdev.slack.com/archives/C061L2MHW93/p1777625600974279?thread_ts=1777622248.762639&cid=C061L2MHW93 https://claude.ai/code/session_01G759MUqmjsPh9k1qDxbdjG --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Eric Allam
|
04bdf4b90b |
perf(webapp): throttle PAT + OAT lastAccessedAt writes to once per 5 min (#3493)
## Summary
Each successful PAT (`PersonalAccessToken`) or OAT
(`OrganizationAccessToken`) authentication issues a `prisma.X.update({
lastAccessedAt: new Date() })` to bump the timestamp. For tokens used at
high frequency (CLI clients, integrations) this generates a per-request
DB write that is mostly redundant — the `lastAccessedAt` field is only
surfaced on the settings page so users can decide which tokens to
revoke, and "within the last 5 minutes" is plenty of granularity for
that.
## Design
Replace each unconditional `update` with a conditional `updateMany`
whose `WHERE` requires the existing `lastAccessedAt` to be `NULL` or
strictly older than 5 minutes:
```ts
await prisma.personalAccessToken.updateMany({
where: {
id: personalAccessToken.id,
OR: [
{ lastAccessedAt: null },
{ lastAccessedAt: { lt: new Date(Date.now() - PAT_LAST_ACCESSED_THROTTLE_MS) } },
],
},
data: { lastAccessedAt: new Date() },
});
```
The conditional runs inside the SQL `UPDATE`, so concurrent auths can't
race into a double-write.
No schema change. No migration. No new infrastructure. Throttle is a
hardcoded constant (`5 * 60 * 1000`) — easy to revisit.
## Test plan
- [x] `pnpm run typecheck --filter webapp`
- [x] `pnpm vitest run ./test/services/personalAccessToken.test.ts
./test/services/organizationAccessToken.test.ts` — 6/6 pass, verifying
the throttle `WHERE` clause is constructed correctly and the `update` is
skipped on token-not-found / wrong-prefix paths
|
||
|
devin-ai-integration[bot]
|
30bd567d48 |
fix: sync declarative schedules on deployment rollback (#3468)
## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing - Reviewed the code flow for deployment rollback (`ChangeCurrentDeploymentService`) and confirmed it was missing schedule sync - Verified all 4 callers of `ChangeCurrentDeploymentService` (UI rollback, UI promote, API promote, finalize deployment) are now covered - Ran `pnpm run typecheck --filter webapp` — passes cleanly --- ## Changelog When rolling back (or manually promoting) a deployment, declarative schedules were not being synced to match the target deployment's worker metadata. Schedules remained as configured by the most recent deployment rather than reflecting the target version's schedule configuration. This fix adds a call to `syncDeclarativeSchedules` in `ChangeCurrentDeploymentService` after the deployment promotion is updated. It parses the target deployment's stored `BackgroundWorkerMetadata` to restore the correct schedule state. This covers both rollback and promote paths (UI and API). Errors are handled gracefully so they don't block the deployment change itself. --- ## Screenshots N/A — backend-only change. 💯 Link to Devin session: https://app.devin.ai/sessions/0debf012b58c4132be778f8ea88cd2b6 --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com> |
||
|
nicktrn
|
ee3887a321 |
feat(webapp): configurable deploy template machine presets (#3492)
The webapp's compute template creation hardcoded a single machine preset (`small-1x`) at deploy time, regardless of which presets a project actually uses. Tasks running on any other preset paid full cold-snapshot creation cost on first run. Two new env vars: - `COMPUTE_TEMPLATE_MACHINE_PRESETS` - CSV of preset names to build boot snapshots for during deploy. Defaults to `small-1x` so existing deploys don't change behavior. - `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED` - CSV of presets whose failure fails a required-mode deploy. Defaults to the full `PRESETS` list. Optional preset failures are logged but don't block the deploy. The compute client now sends the multi-config request shape; the service evaluates per-preset outcomes against the required set and surfaces a combined failure message when a required preset fails. Both env vars are validated at boot via the env schema - unknown preset names or `_REQUIRED` entries that aren't a subset of `_PRESETS` fail loudly at startup rather than silently per-deploy. |
||
|
DKP
|
1dfd595986 |
fix(webapp): invalid HTML nesting in errors Activity tooltip (#3488)
The Activity peak count tooltip in the errors list rendered a `<button>`
(from `SimpleTooltip`'s default `TooltipTrigger`) inside the row's `<a>`
link (`TableCell to={errorPath}`). Interactive content nested inside
other interactive content is invalid HTML and triggers accessibility
warnings. Adding `asChild` to `SimpleTooltip` makes the existing
`<span>` the trigger directly, removing the nested `<button>`.
|
||
|
Eric Allam
|
ac7177d61f |
feat(schedule-engine): stop persisting per-tick schedule state (#3476)
## Summary
Each scheduled-task tick previously issued **3 Prisma `UPDATE`s**
against
`TaskSchedule.lastRunTriggeredAt`,
`TaskScheduleInstance.lastScheduledTimestamp`,
and `TaskScheduleInstance.nextScheduledTimestamp`. All three were pure
denormalization — every value can be derived without persisting.
After this PR `TaskSchedule` and `TaskScheduleInstance` become **near
read-only**:
writes happen only on schedule create / update / delete (rare admin
actions),
so the per-tick autovacuum churn on these hot tables disappears.
## Design
The previous fire time travels forward through the **schedule worker
payload**,
not through the database. Concretely:
- The `schedule.triggerScheduledTask` worker payload gains an optional
`lastScheduleTime: z.coerce.date().optional()` field.
- When the engine fires a schedule, it re-enqueues the next tick with
`lastScheduleTime = scheduleTimestamp` (the just-fired time).
- When the next tick dequeues, `payload.lastTimestamp` is sourced from
`params.lastScheduleTime` directly. No DB round-trip, no cron-derivation
drift across DST boundaries, no caveats around recently-edited cron
expressions.
`payload.lastTimestamp` keeps its `Date | undefined` SDK shape.
First-ever
fires still report `undefined`, so customer `if
(!payload.lastTimestamp)`
first-run patterns keep working.
For Redis jobs that were enqueued **before** this change (which lack
`lastScheduleTime` in their payload), the engine falls back to
`instance.lastScheduledTimestamp` once. Once those drain, the column is
never read again. Revert is code-only; the columns stay in place and can
be dropped in a follow-up once the rollout is stable.
## Files
- `internal-packages/schedule-engine/*` — engine refactor,
`workerCatalog`
schema field, `TriggerScheduleParams` extension, tests updated to assert
on the worker-payload flow rather than DB readbacks.
- `internal-packages/database/prisma/schema.prisma` — `/// @deprecated`
triple-slash docstrings on the three columns. No migration.
- `apps/webapp/app/presenters/v3/ScheduleListPresenter.server.ts` —
drops
the `lastRunTriggeredAt` Prisma select; "Last run" cell is approximated
from the cron expression's previous slot, gated on `schedule.createdAt`
so brand-new schedules show "–". UI is best-effort; the runs page is the
source of truth.
- `apps/webapp/app/v3/utils/calculateNextSchedule.server.ts` — adds a
`previousScheduledTimestamp` helper for the UI cell above. Public API
responses (`api.v1.schedules.*`) already compute `nextRun` from cron and
don't expose `lastTimestamp` — no public API change.
- `references/scheduled-tasks/` — new reference project with declarative
schedules at multiple cadences and three throw-on-fail validators
(`first-fire-detector`, `interval-validator`, `upcoming-validator`) for
E2E-verifying the worker-payload flow.
Refs TRI-8891
## Test plan
- [x] `pnpm run typecheck --filter @internal/schedule-engine --filter
webapp`
- [x] `pnpm run build --filter @trigger.dev/core`
- [x] `pnpm run test --filter @internal/schedule-engine` — integration
test
asserts first-fire `lastTimestamp === undefined`, second fire carries
the previous fire's timestamp exactly.
- [x] E2E against local webapp via `references/scheduled-tasks`:
- Fresh schedules attached → all three deprecated columns stay `NULL`
after
multiple fires.
- Redis payload at second fire contains
`"lastScheduleTime":"<previous fire timestamp>"`.
- `TaskRun.payload` and the every-minute task's returned output both
confirm
`lastTimestamp = null` on first fire and `lastTimestamp = <prev fire>`
on
second fire, exactly 60s apart.
- All three throw-on-FAIL validators completed successfully on every
non-first fire.
- [x] Schedules REST API end-to-end (`POST` / `GET` / `PUT` / `activate`
/
`deactivate` / `DELETE`) — `nextRun` recomputed live from cron + tz on
every response, no reads of deprecated columns.
|
||
|
DKP
|
19c16759f6 |
feat(webapp): errors page polish and GA rollout (#3477)
## What this does Polish + bug-fix pass on the Errors page so it can ship to everyone. Touches the Slack alert config UX, errors list, error detail page, and unhides the SideMenu entry for non-admins. ## Decisions **"No channel" item over standalone Remove button** Chose pinning a `<XMarkIcon /> No channel` `SelectItem` above the channel list. Rejected the standalone "Remove channel" link in a `<Hint>` — color/hover behaviour clashed with the sibling `<TextLink>`, and "channel selection" is the right context for clearing. Server action already deletes the channel when `slackChannel=""` is submitted. **Slack `<!date^>` token over per-user TZ field for alerts** Chose Slack's native `<!date^TS^…>` token so each viewer sees timestamps in their own timezone (UTC fallback). Rejected per-user/per-org TZ schema work — works for multi-region channels for free. Email/dashboard TZ source-of-truth filed as TRI-8885 / TRI-8886. **Make errors GA** |
||
|
Eric Allam
|
04b4d85f50 |
fix(webapp): allow JWT auth on POST /api/v1/sessions (#3474)
## Summary
`POST /api/v1/sessions` was secret-key-only because the customer browser
flow runs through `chat.createStartSessionAction` (server-side, holds
the secret key). But the `cli-v3` MCP `start_agent_chat` tool is itself
a server-side surface — developer's CLI/IDE acting as their own server —
and only holds a JWT minted from the user's PAT. Without JWT support on
this route the entire MCP agent toolkit (`start_agent_chat`,
`send_agent_message`, `close_agent_chat`) is blocked at session
creation.
Add `allowJWT: true` plus an `authorization` block requiring the
`write:sessions` (or `admin`) super-scope.
## Why a wildcard `sessions` resource
Resource scoping by `taskIdentifier` isn't possible at auth-resolve time
— action routes don't pass `body` to the `resource` callback, and the
task name only lives in the body. So the resource is `sessions: "*"` and
the super-scope does the actual gating. The JWT-issuer (cli-v3 MCP,
customer servers wrapping their own auth helpers, etc.) decides which
scopes to mint, which is where per-task narrowing lives.
## Test plan
- [x] Verified end-to-end against local:
`mcp__trigger__start_agent_chat` → `send_agent_message("pong")` →
`send_agent_message("echo")` → `close_agent_chat` all succeed. Two
assistant turns reuse the same runId (continuation in the idle window).
- [ ] Browser-mediated `chat.createStartSessionAction` flow continues to
work unchanged (still uses secret-key path under the hood).
- [ ] Loader (GET) and other session routes — unchanged, no scope drift.
## Notes
This unblocks T17 in the [ai-chat e2e smoke
catalog](https://github.com/triggerdotdev/trigger.dev/blob/feature/tri-7532-ai-sdk-chat-transport-and-chat-task-system/.claude/skills/ai-chat-e2e/SMOKE-TESTS.md)
(which lives in the feature branch's skill catalog, not this repo).
Pairs with the cli-v3 MCP fix on the feature branch (`feat: AI SDK
custom useChat transport & chat.task harness`, PR #3173) — that PR's
`agentChat.ts` change makes the call shape correct (`taskIdentifier` +
`triggerConfig`); this PR opens the door for the JWT to actually pass.
|
||
|
ThullyoCunha
|
f1736595cd |
feat(webapp): apply default repository policy on ECR repo creation (#3467)
🚀 Publish Trigger.dev Docker / units (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
## Summary
Self-hosters that operate the webapp's ECR account separately from the
account running the EKS workers (e.g., a shared platform account that
hosts the registry plus per-team accounts that host clusters) currently
hit a 403 Forbidden the first time **any** project is deployed:
```
Failed to pull image "<acct-A>.dkr.ecr.<region>.amazonaws.com/<namespace>/proj_…:…":
unexpected status from HEAD request to .../v2/.../manifests/sha256:…: 403 Forbidden
```
`ensureEcrRepositoryExists` in
`apps/webapp/app/v3/getDeploymentImageRef.server.ts` calls
`CreateRepository` and `PutLifecyclePolicy`, but never
`SetRepositoryPolicy` — so the new repo inherits the AWS default (only
the registry-owner account can read/pull). Workers in the cluster
account get 403 every single deploy. The only workarounds today are
running a one-off post-create script or pre-creating every repo by hand.
## Proposed change
Add an optional env var:
```
DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY (V4 mirror: V4_DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY)
```
Raw IAM policy JSON. When set, the webapp calls `SetRepositoryPolicy`
immediately after `CreateRepository` so every new repo carries that
policy from creation. Operators control the principal/actions; we don't
bake in any opinions about cross-account boundaries.
Example value (for the typical self-host case — grant pull to the
cluster account):
```json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "AllowClusterAccountPull",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::<cluster-account-id>:root"},
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
]
}]
}
```
## Why env var (not a chart-level field)
- Mirrors the shape of the sibling vars (`DEPLOY_REGISTRY_ECR_TAGS`,
`DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN`, etc.) which are already
operator-supplied via `webapp.extraEnvVars` in self-host setups.
- Cloud is unaffected — the env var is optional, unset by default;
existing behavior unchanged.
- Existing repos are unaffected — only newly-created repos get the
policy.
- `RepositoryCreationTemplate` from the AWS provider isn't an
alternative here: it only applies to repos created via
pull-through-cache or replication, not to `ecr:CreateRepository` API
calls.
## Implementation
- `apps/webapp/app/env.server.ts` — declare
`DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` and the V4 fallback.
- `apps/webapp/app/v3/registryConfig.server.ts` — propagate
`ecrDefaultRepositoryPolicy` to `RegistryConfig`.
- `apps/webapp/app/v3/getDeploymentImageRef.server.ts` —
`createEcrRepository` accepts the policy; if set, calls
`SetRepositoryPolicy` after `PutLifecyclePolicy`.
- `docs/self-hosting/env/webapp.mdx` — documentation row added under
**Deploy & Registry**.
## Verification
Verified end-to-end against a self-hosted Trigger.dev on EKS where the
ECR account is separate from the cluster account:
- **Without the env var** (current `main`): the new project's first run
pod stays in `ImagePullBackOff` with `403 Forbidden`.
- **With the env var set** to a JSON granting
`ecr:BatchGetImage`/`GetDownloadUrlForLayer`/`BatchCheckLayerAvailability`
to the cluster account: a fresh `trigger.dev deploy --env prod` followed
by a `hello-world` run completes in ~5s end-to-end on the first try.
Manually also confirmed that existing repos are untouched (the call only
fires inside `createEcrRepository`, which only runs when
`DescribeRepositories` returned `RepositoryNotFoundException`).
## Out of scope
- Chart values surface for this — operators already pass the existing
ECR vars via `webapp.extraEnvVars`, so this follows the same pattern.
Happy to add a first-class chart field in a follow-up if that's the
preferred direction.
- IAM-policy validation in the webapp — we forward the JSON verbatim to
AWS and surface AWS's error messages on misuse, matching how
`DEPLOY_REGISTRY_ECR_TAGS` is handled today.
This is a draft pending CI / CodeRabbit pass — happy to iterate on
direction (e.g., split into per-action env vars, or extend the chart
values schema) if any of the above choices feels off.
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
|
||
|
nicktrn
|
226b93edf9 |
fix(webapp): preserve filters on queues page action redirects (#3471)
Queues page action handler was rebuilding the redirect URL with only `?page=`, so any pause/resume/override modal confirmation wiped the user's search query. With hundreds of queues filtered down to a handful, every confirmation dropped you back to the unfiltered list - and pagination still pointed at the previous numeric page, so you'd land on a different slice than you came from. Swap the manual rebuild for `url.search` so the full querystring (including any future filter params) flows through. Drops the now-unused `SearchParamsSchema.parse` call inside `action`; the loader still validates on the way back. |