v.docker.4.4.5
4296 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 nicktrn
|
ee3887a321 |
feat(webapp): configurable deploy template machine presets (#3492)
The webapp's compute template creation hardcoded a single machine preset (`small-1x`) at deploy time, regardless of which presets a project actually uses. Tasks running on any other preset paid full cold-snapshot creation cost on first run. Two new env vars: - `COMPUTE_TEMPLATE_MACHINE_PRESETS` - CSV of preset names to build boot snapshots for during deploy. Defaults to `small-1x` so existing deploys don't change behavior. - `COMPUTE_TEMPLATE_MACHINE_PRESETS_REQUIRED` - CSV of presets whose failure fails a required-mode deploy. Defaults to the full `PRESETS` list. Optional preset failures are logged but don't block the deploy. The compute client now sends the multi-config request shape; the service evaluates per-preset outcomes against the required set and surfaces a combined failure message when a required preset fails. Both env vars are validated at boot via the env schema - unknown preset names or `_REQUIRED` entries that aren't a subset of `_PRESETS` fail loudly at startup rather than silently per-deploy. |
||
|
DKP
|
1dfd595986 |
fix(webapp): invalid HTML nesting in errors Activity tooltip (#3488)
The Activity peak count tooltip in the errors list rendered a `<button>`
(from `SimpleTooltip`'s default `TooltipTrigger`) inside the row's `<a>`
link (`TableCell to={errorPath}`). Interactive content nested inside
other interactive content is invalid HTML and triggers accessibility
warnings. Adding `asChild` to `SimpleTooltip` makes the existing
`<span>` the trigger directly, removing the nested `<button>`.
|
||
|
Eric Allam
|
ac7177d61f |
feat(schedule-engine): stop persisting per-tick schedule state (#3476)
## Summary
Each scheduled-task tick previously issued **3 Prisma `UPDATE`s**
against
`TaskSchedule.lastRunTriggeredAt`,
`TaskScheduleInstance.lastScheduledTimestamp`,
and `TaskScheduleInstance.nextScheduledTimestamp`. All three were pure
denormalization — every value can be derived without persisting.
After this PR `TaskSchedule` and `TaskScheduleInstance` become **near
read-only**:
writes happen only on schedule create / update / delete (rare admin
actions),
so the per-tick autovacuum churn on these hot tables disappears.
## Design
The previous fire time travels forward through the **schedule worker
payload**,
not through the database. Concretely:
- The `schedule.triggerScheduledTask` worker payload gains an optional
`lastScheduleTime: z.coerce.date().optional()` field.
- When the engine fires a schedule, it re-enqueues the next tick with
`lastScheduleTime = scheduleTimestamp` (the just-fired time).
- When the next tick dequeues, `payload.lastTimestamp` is sourced from
`params.lastScheduleTime` directly. No DB round-trip, no cron-derivation
drift across DST boundaries, no caveats around recently-edited cron
expressions.
`payload.lastTimestamp` keeps its `Date | undefined` SDK shape.
First-ever
fires still report `undefined`, so customer `if
(!payload.lastTimestamp)`
first-run patterns keep working.
For Redis jobs that were enqueued **before** this change (which lack
`lastScheduleTime` in their payload), the engine falls back to
`instance.lastScheduledTimestamp` once. Once those drain, the column is
never read again. Revert is code-only; the columns stay in place and can
be dropped in a follow-up once the rollout is stable.
## Files
- `internal-packages/schedule-engine/*` — engine refactor,
`workerCatalog`
schema field, `TriggerScheduleParams` extension, tests updated to assert
on the worker-payload flow rather than DB readbacks.
- `internal-packages/database/prisma/schema.prisma` — `/// @deprecated`
triple-slash docstrings on the three columns. No migration.
- `apps/webapp/app/presenters/v3/ScheduleListPresenter.server.ts` —
drops
the `lastRunTriggeredAt` Prisma select; "Last run" cell is approximated
from the cron expression's previous slot, gated on `schedule.createdAt`
so brand-new schedules show "–". UI is best-effort; the runs page is the
source of truth.
- `apps/webapp/app/v3/utils/calculateNextSchedule.server.ts` — adds a
`previousScheduledTimestamp` helper for the UI cell above. Public API
responses (`api.v1.schedules.*`) already compute `nextRun` from cron and
don't expose `lastTimestamp` — no public API change.
- `references/scheduled-tasks/` — new reference project with declarative
schedules at multiple cadences and three throw-on-fail validators
(`first-fire-detector`, `interval-validator`, `upcoming-validator`) for
E2E-verifying the worker-payload flow.
Refs TRI-8891
## Test plan
- [x] `pnpm run typecheck --filter @internal/schedule-engine --filter
webapp`
- [x] `pnpm run build --filter @trigger.dev/core`
- [x] `pnpm run test --filter @internal/schedule-engine` — integration
test
asserts first-fire `lastTimestamp === undefined`, second fire carries
the previous fire's timestamp exactly.
- [x] E2E against local webapp via `references/scheduled-tasks`:
- Fresh schedules attached → all three deprecated columns stay `NULL`
after
multiple fires.
- Redis payload at second fire contains
`"lastScheduleTime":"<previous fire timestamp>"`.
- `TaskRun.payload` and the every-minute task's returned output both
confirm
`lastTimestamp = null` on first fire and `lastTimestamp = <prev fire>`
on
second fire, exactly 60s apart.
- All three throw-on-FAIL validators completed successfully on every
non-first fire.
- [x] Schedules REST API end-to-end (`POST` / `GET` / `PUT` / `activate`
/
`deactivate` / `DELETE`) — `nextRun` recomputed live from cron + tz on
every response, no reads of deprecated columns.
|
||
|
DKP
|
19c16759f6 |
feat(webapp): errors page polish and GA rollout (#3477)
## What this does Polish + bug-fix pass on the Errors page so it can ship to everyone. Touches the Slack alert config UX, errors list, error detail page, and unhides the SideMenu entry for non-admins. ## Decisions **"No channel" item over standalone Remove button** Chose pinning a `<XMarkIcon /> No channel` `SelectItem` above the channel list. Rejected the standalone "Remove channel" link in a `<Hint>` — color/hover behaviour clashed with the sibling `<TextLink>`, and "channel selection" is the right context for clearing. Server action already deletes the channel when `slackChannel=""` is submitted. **Slack `<!date^>` token over per-user TZ field for alerts** Chose Slack's native `<!date^TS^…>` token so each viewer sees timestamps in their own timezone (UTC fallback). Rejected per-user/per-org TZ schema work — works for multi-region channels for free. Email/dashboard TZ source-of-truth filed as TRI-8885 / TRI-8886. **Make errors GA** |
||
|
Eric Allam
|
04b4d85f50 |
fix(webapp): allow JWT auth on POST /api/v1/sessions (#3474)
## Summary
`POST /api/v1/sessions` was secret-key-only because the customer browser
flow runs through `chat.createStartSessionAction` (server-side, holds
the secret key). But the `cli-v3` MCP `start_agent_chat` tool is itself
a server-side surface — developer's CLI/IDE acting as their own server —
and only holds a JWT minted from the user's PAT. Without JWT support on
this route the entire MCP agent toolkit (`start_agent_chat`,
`send_agent_message`, `close_agent_chat`) is blocked at session
creation.
Add `allowJWT: true` plus an `authorization` block requiring the
`write:sessions` (or `admin`) super-scope.
## Why a wildcard `sessions` resource
Resource scoping by `taskIdentifier` isn't possible at auth-resolve time
— action routes don't pass `body` to the `resource` callback, and the
task name only lives in the body. So the resource is `sessions: "*"` and
the super-scope does the actual gating. The JWT-issuer (cli-v3 MCP,
customer servers wrapping their own auth helpers, etc.) decides which
scopes to mint, which is where per-task narrowing lives.
## Test plan
- [x] Verified end-to-end against local:
`mcp__trigger__start_agent_chat` → `send_agent_message("pong")` →
`send_agent_message("echo")` → `close_agent_chat` all succeed. Two
assistant turns reuse the same runId (continuation in the idle window).
- [ ] Browser-mediated `chat.createStartSessionAction` flow continues to
work unchanged (still uses secret-key path under the hood).
- [ ] Loader (GET) and other session routes — unchanged, no scope drift.
## Notes
This unblocks T17 in the [ai-chat e2e smoke
catalog](https://github.com/triggerdotdev/trigger.dev/blob/feature/tri-7532-ai-sdk-chat-transport-and-chat-task-system/.claude/skills/ai-chat-e2e/SMOKE-TESTS.md)
(which lives in the feature branch's skill catalog, not this repo).
Pairs with the cli-v3 MCP fix on the feature branch (`feat: AI SDK
custom useChat transport & chat.task harness`, PR #3173) — that PR's
`agentChat.ts` change makes the call shape correct (`taskIdentifier` +
`triggerConfig`); this PR opens the door for the JWT to actually pass.
|
||
|
ThullyoCunha
|
f1736595cd |
feat(webapp): apply default repository policy on ECR repo creation (#3467)
🚀 Publish Trigger.dev Docker / units (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / typecheck (push) Failing after 13m3s
🚀 Publish Trigger.dev Docker / publish-webapp (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker (push) Has been skipped
🚀 Publish Trigger.dev Docker / publish-worker-v4 (push) Has been skipped
## Summary
Self-hosters that operate the webapp's ECR account separately from the
account running the EKS workers (e.g., a shared platform account that
hosts the registry plus per-team accounts that host clusters) currently
hit a 403 Forbidden the first time **any** project is deployed:
```
Failed to pull image "<acct-A>.dkr.ecr.<region>.amazonaws.com/<namespace>/proj_…:…":
unexpected status from HEAD request to .../v2/.../manifests/sha256:…: 403 Forbidden
```
`ensureEcrRepositoryExists` in
`apps/webapp/app/v3/getDeploymentImageRef.server.ts` calls
`CreateRepository` and `PutLifecyclePolicy`, but never
`SetRepositoryPolicy` — so the new repo inherits the AWS default (only
the registry-owner account can read/pull). Workers in the cluster
account get 403 every single deploy. The only workarounds today are
running a one-off post-create script or pre-creating every repo by hand.
## Proposed change
Add an optional env var:
```
DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY (V4 mirror: V4_DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY)
```
Raw IAM policy JSON. When set, the webapp calls `SetRepositoryPolicy`
immediately after `CreateRepository` so every new repo carries that
policy from creation. Operators control the principal/actions; we don't
bake in any opinions about cross-account boundaries.
Example value (for the typical self-host case — grant pull to the
cluster account):
```json
{
"Version": "2012-10-17",
"Statement": [{
"Sid": "AllowClusterAccountPull",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::<cluster-account-id>:root"},
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
]
}]
}
```
## Why env var (not a chart-level field)
- Mirrors the shape of the sibling vars (`DEPLOY_REGISTRY_ECR_TAGS`,
`DEPLOY_REGISTRY_ECR_ASSUME_ROLE_ARN`, etc.) which are already
operator-supplied via `webapp.extraEnvVars` in self-host setups.
- Cloud is unaffected — the env var is optional, unset by default;
existing behavior unchanged.
- Existing repos are unaffected — only newly-created repos get the
policy.
- `RepositoryCreationTemplate` from the AWS provider isn't an
alternative here: it only applies to repos created via
pull-through-cache or replication, not to `ecr:CreateRepository` API
calls.
## Implementation
- `apps/webapp/app/env.server.ts` — declare
`DEPLOY_REGISTRY_ECR_DEFAULT_REPOSITORY_POLICY` and the V4 fallback.
- `apps/webapp/app/v3/registryConfig.server.ts` — propagate
`ecrDefaultRepositoryPolicy` to `RegistryConfig`.
- `apps/webapp/app/v3/getDeploymentImageRef.server.ts` —
`createEcrRepository` accepts the policy; if set, calls
`SetRepositoryPolicy` after `PutLifecyclePolicy`.
- `docs/self-hosting/env/webapp.mdx` — documentation row added under
**Deploy & Registry**.
## Verification
Verified end-to-end against a self-hosted Trigger.dev on EKS where the
ECR account is separate from the cluster account:
- **Without the env var** (current `main`): the new project's first run
pod stays in `ImagePullBackOff` with `403 Forbidden`.
- **With the env var set** to a JSON granting
`ecr:BatchGetImage`/`GetDownloadUrlForLayer`/`BatchCheckLayerAvailability`
to the cluster account: a fresh `trigger.dev deploy --env prod` followed
by a `hello-world` run completes in ~5s end-to-end on the first try.
Manually also confirmed that existing repos are untouched (the call only
fires inside `createEcrRepository`, which only runs when
`DescribeRepositories` returned `RepositoryNotFoundException`).
## Out of scope
- Chart values surface for this — operators already pass the existing
ECR vars via `webapp.extraEnvVars`, so this follows the same pattern.
Happy to add a first-class chart field in a follow-up if that's the
preferred direction.
- IAM-policy validation in the webapp — we forward the JSON verbatim to
AWS and surface AWS's error messages on misuse, matching how
`DEPLOY_REGISTRY_ECR_TAGS` is handled today.
This is a draft pending CI / CodeRabbit pass — happy to iterate on
direction (e.g., split into per-action env vars, or extend the chart
values schema) if any of the above choices feels off.
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
|
||
|
nicktrn
|
226b93edf9 |
fix(webapp): preserve filters on queues page action redirects (#3471)
Queues page action handler was rebuilding the redirect URL with only `?page=`, so any pause/resume/override modal confirmation wiped the user's search query. With hundreds of queues filtered down to a handful, every confirmation dropped you back to the unfiltered list - and pagination still pointed at the previous numeric page, so you'd land on a different slice than you came from. Swap the manual rebuild for `url.search` so the full querystring (including any future filter params) flows through. Drops the now-unused `SearchParamsSchema.parse` call inside `action`; the loader still validates on the way back. |
||
|
devin-ai-integration[bot]
|
b0131352f6 |
fix(webapp): constrain usage chart height to 320px (#3469)
## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing Ran the webapp locally with `CLOUD_ENV=development` and verified the usage page chart height at different viewport sizes. The chart now renders at a fixed 320px height instead of expanding to fill the viewport. --- ## Changelog Fix the "Usage by day" chart on the usage settings page taking up 100% of the viewport height. The regression was introduced in PR #2905 when the `UsageChart` was migrated from using `ChartContainer` directly (with `max-h-96 min-h-40 w-full`) to the new `Chart.Root` compound component. The `ChartContainer` base class includes `aspect-video` (16:9 ratio), and the `max-h-96` constraint was lost during migration, causing the chart to scale its height based on viewport width. Fix: wrap `Chart.Root` in a fixed-height container (`h-80` = 320px) and use the `fillContainer` prop, which applies `!aspect-auto` to override the `aspect-video` ratio. --- ## Screenshots Before (chart fills entire viewport):  After (chart constrained to 320px):  💯 Link to Devin session: https://app.devin.ai/sessions/6e5ed40516d3448db85950feb1115ab3 Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com> |
||
|
Eric Allam
|
99dfee3a57 |
fix(webapp): honor RevokedApiKey grace window for public access tokens (#3464)
## Summary Follow-up to #3420. PATs (public access tokens) minted before an API key rotation 401'd immediately on the realtime stream endpoints, even though the rotation flow advertises a 24h overlap. This fixes the gap. ## Root cause PATs are JWTs signed with the env's `apiKey` at mint time. When that secret is rotated, `validatePublicJwtKey` (`apps/webapp/app/services/realtime/jwtAuth.server.ts`) only verifies the signature against `environment.parentEnvironment?.apiKey ?? environment.apiKey` — i.e. the env's *current* canonical key. Any PAT in the wild signed with the previous key fails signature verification → 401, even within the grace window. #3420 wired up the grace-window fallback in two places — `findEnvironmentByApiKey` (raw secret-key auth) and `api.v1.auth.jwt.ts` (signs new JWTs with the canonical key when minting from an old one) — but the *verify* path for already-issued PATs was never updated. In a typical app, `POST /api/v1/tasks/.../trigger` (Bearer secret) keeps working through rotation because that path has the fallback, but `GET /realtime/v1/streams/run_*/...` and `POST /realtime/v1/streams/run_*/input/...` 401 for runs that were already in flight when the rotation happened. ## Fix After the primary `validateJWT` against the env's current `apiKey`, fall back to non-expired `RevokedApiKey` rows for the signing env (parent env when the request is against a child) — but **only on the failure path**, so the hot success path is unchanged. Uses `$replica` to match the rest of the auth path. Symmetrical to the `findEnvironmentByApiKey` two-step from #3420. ## Changes - `apps/webapp/app/services/realtime/jwtAuth.server.ts` — `validateAgainstRevokedApiKeys` helper invoked only on `!result.ok` - `apps/webapp/app/models/runtimeEnvironment.server.ts` — `findEnvironmentById` also selects `parentEnvironment.id` so we can scope the revoked-keys lookup to the correct env ## Test plan E2E verified locally via curl against `GET /realtime/v1/runs/{runId}` (PAT-authenticated): - [x] Pre-rotation, PAT signed with K1 → **200** with run body - [x] Simulate rotation (insert `RevokedApiKey` row + flip env `apiKey` to K2 in a single transaction, mirroring `regenerateApiKey`) - [x] Same PAT (K1) within grace window → **200** with run body — fallback hits - [x] Fresh PAT signed with K2 → **200** — current key still works - [x] Set `RevokedApiKey.expiresAt` to past → **401** — fallback finds no live row - [x] Bogus signature (no rotation) → **401** - [x] Cleanup verified: env `apiKey` restored, `RevokedApiKey` row deleted - [x] `pnpm run typecheck --filter webapp` passes |
||
|
Eric Allam
|
dac9c83bdc |
chore(webapp,run-engine): downgrade boundary log noise to warn (#3462)
## Summary
Several boundary catches and customer-input validation paths were
logging at `error` level for failures the system already handles
gracefully — disconnect on auth failure, return undefined, skip retries,
etc. This batch routes them to `warn` (which stays in stdout) or counts
them as OTel metrics, so visibility is preserved without surfacing them
as alerts.
## Changes
**New helper / pattern:**
- `apiBuilder.server.ts` — `logBoundaryError(message, error, url)`
inspects the inner error type at loader/action boundary catches;
downgrades to `warn` for `AbortError`, `ServiceValidationError`, and
`EngineServiceValidationError`.
- `platform.v3.server.ts` — `platform_client.failures_total` OTel
counter with `{function, kind}` labels; helper
`recordPlatformFailure(fn, kind)` replaces the previous error-level
logging across all `BillingClient` wrappers.
**Log-level downgrades:**
- `handleSocketIo.server.ts` — `Worker authentication failed` → warn
(system disconnects on failure; refs TRI-8863)
- `waitpointSystem.ts` — when `runStatus === "CANCELED"` in the
suspended-without-checkpoint branch, skip the throw and warn instead
(benign cancel-vs-resume race, nothing to resume)
- `runAttemptSystem.ts` — `flushedMetadata` parse/validate failures →
warn (customer-side data shape, system returns gracefully)
- `batch-queue/index.ts` — final-attempt failures with
`result.skipRetries` → warn (callbacks already opted out of retry, e.g.
queue size limit hit)
- `queryPerformanceMonitor.server.ts` — slow queries → warn
(observability signal, not an application error)
- `timeoutDeployment.server.ts` — deployment-state mismatch in the
timeout job → warn (timeout-vs-completion race)
**Inner error preservation:**
- `waitpointCompletionPacket.server.ts` — `logger.error(uploadError)`
before throwing the `ServiceValidationError` wrapper, so the underlying
upload error stays visible.
## Why
The pattern across all of these is the same: a boundary log treated any
thrown/returned error as `error` regardless of cause, even when the
cause was an expected, system-handled condition (client disconnect,
customer quota, race condition, schema validation of customer data).
That made the logs noisy and made it harder to spot real bugs.
Where the underlying signal is still useful operationally (slow queries,
billing call failures), we route it to OTel metrics with low-cardinality
labels so dashboards and alerts can be tuned independently of error
logs.
## Test plan
- [ ] `pnpm run typecheck --filter webapp`
- [ ] `pnpm run build --filter @internal/run-engine`
- [ ] Trigger a run on hello-world and verify task lifecycle is
unaffected
- [ ] Cancel a suspended run and verify the cancel-while-suspended
branch in `waitpointSystem.ts` returns `{status: "skipped"}` instead of
throwing
- [ ] Confirm `platform_client.failures_total` counter shows up in
metrics with `{function, kind}` labels when the billing client errors
|
||
|
Oskar Otwinowski
|
5fe72eefa5 | feat(webapp): Private Links setup wizard UI tweaks (#3465) | ||
|
Eric Allam
|
c69e939c34 |
feat: Sessions - bidirectional durable agent streams (#3417)
> ⚠️ **Not released yet.** This PR is the server-side foundation only. The SDK changes that customers will actually use (`chat.agent` migration, `chat.createStartSessionAction`, `useTriggerChatTransport` updates) live on a separate branch and ship together in an upcoming `@trigger.dev/sdk` prerelease. Until that prerelease is published, this surface is reachable only via direct HTTP. ## What this gives Trigger.dev users A new first-class primitive, **Session**, for durable, task-bound, bidirectional I/O that outlives any single run. Sessions are the run manager for `chat.agent` going forward, and they unblock anything else that needs "one identifier, many runs over time" with a stable channel pair the client can write to and subscribe to. ### Use cases unblocked - **Chat agents that persist across many runs.** One session per chat (keyed on your own `chatId` via `externalId`), turns 1..N attach to the same Session, the UI subscribes once and keeps receiving output as new runs take over. - **Approval loops and long-running tasks with user feedback.** The task waits on `.in`, the client writes to `.in`, the server enforces no-writes-after-close. - **Workflow progress streams that live past the run.** Subscribe to `.out` after the task finishes to replay history. - **Resume-next-day flows.** A session is a durable row, not a transient stream. Send a message a day later and the server triggers a fresh run on the same session. ### How it works (Session-as-run-manager) A Session row is task-bound (`taskIdentifier` + `triggerConfig` are required) and owns its current run via `currentRunId` + `currentRunVersion` for optimistic claim. Three trigger paths: 1. **Session create** — `POST /api/v1/sessions` creates the row and triggers the first run synchronously. 2. **Append-time probe** — `POST /realtime/v1/sessions/:session/in/append` checks if the current run is alive; if it has terminated (idle exit, crash, etc.), the server triggers a new run before processing the append. 3. **End-and-continue handoff** — `POST /api/v1/sessions/:session/end-and-continue`, called by the running agent, triggers a fresh run and atomically swaps `currentRunId`. Used by `chat.requestUpgrade()` for version handoffs. Every triggered run is recorded in the `SessionRun` audit table with a reason (`initial`, `continuation`, `upgrade`, `manual`). ## Public API surface ### Control plane - `POST /api/v1/sessions` — create. Idempotent on `(env, externalId)`. Triggers the first run, returns the session and a session-scoped public access token. Returns 409 if the upserted row is already closed. - `GET /api/v1/sessions/:session` — retrieve by friendlyId (`session_abc...`) or by your own externalId (server disambiguates by prefix). - `GET /api/v1/sessions` — list with filters (`type`, `tag`, `taskIdentifier`, `externalId`, derived `status` ACTIVE/CLOSED/EXPIRED, created-at range) and cursor pagination. Backed by ClickHouse. - `PATCH /api/v1/sessions/:session` — update tags / metadata / externalId. - `POST /api/v1/sessions/:session/close` — terminate. Idempotent, hard-blocks new server-brokered writes. - `POST /api/v1/sessions/:session/end-and-continue` — agent-only handoff to a fresh run. ### Realtime - `PUT /realtime/v1/sessions/:session/:io` — initialize a channel. Returns S2 credentials in headers so high-throughput clients can write direct to S2. - `GET /realtime/v1/sessions/:session/:io` — SSE subscribe. Supports Last-Event-ID resume and an opt-in `X-Peek-Settled: 1` header that fast-closes the stream when the upstream is already settled (`trigger:turn-complete`), eliminating long-poll wait on reconnect-on-reload paths. - `POST /realtime/v1/sessions/:session/:io/append` — server-side appends. - `POST /api/v1/runs/:runFriendlyId/session-streams/wait` — runs wait on a session stream as a waitpoint, with a race-check to avoid suspending if data already landed. ### Auth scopes `sessions` is a new resource type. `read:sessions:{id}`, `write:sessions:{id}`, `admin:sessions:{id}` flow through the existing JWT validator. Session-scoped public access tokens minted by the server replace browser-held trigger-task tokens for chat-style flows — the browser never sees a run identifier or a run-scoped token in steady state. ## What's coming after this PR - **SDK + chat.agent migration**: separate branch, separate PR, ships in the next `@trigger.dev/sdk` prerelease alongside this server deploy. Customers using the prerelease `chat.agent` will follow the [upgrade guide](https://github.com/triggerdotdev/trigger.dev/blob/docs/tri-7532-ai-sdk-chat-transport-and-chat-task-system/docs/ai-chat/upgrade-guide.mdx). - **Dashboard surfaces**: dedicated agent list, agent playground, agent view on the run dashboard. Tracking separately. ## Implementation notes - **Postgres `Session` table**: scalar scoping columns (`projectId`, `runtimeEnvironmentId`, `environmentType`, `organizationId`) without FKs, matching the January TaskRun FK-removal decision. Point-lookup indexes only — list queries go to ClickHouse. Terminal markers (`closedAt`, `expiresAt`) are write-once. - **ClickHouse `sessions_v1`**: ReplacingMergeTree, partitioned by month, ordered by `(org_id, project_id, environment_id, created_at, session_id)`. Tags indexed via `tokenbf_v1` skip index. - **`SessionsReplicationService`**: mirrors `RunsReplicationService` exactly — leader-locked logical replication consumer, `ConcurrentFlushScheduler`, retry with exponential backoff + jitter, identical metric shape. Dedicated slot + publication so the two consume independently. - **S2 keys**: `sessions/{addressingKey}/{out|in}`. The existing `runs/{runId}/{streamId}` key format for run-scoped streams is untouched. - **Optimistic claim**: `ensureRunForSession` triggers a run upfront (cheap to cancel if it loses the race), then attempts an `updateMany` keyed on `currentRunVersion`. Loser cancels its triggered run and reuses the winner's. No DB lock held across the trigger. ### What did NOT change Run-scoped `streams.pipe` / `streams.input` and the existing `/realtime/v1/streams/{runId}/...` routes are unchanged. Sessions are net-new — not a reshaping of the current streams API. ## Deploy notes - Set `SESSION_REPLICATION_CLICKHOUSE_URL` and `SESSION_REPLICATION_ENABLED=1` to enable the replication consumer. - The `Session` table needs `REPLICA IDENTITY FULL` set on the prod source DB before the publication is created (same one-time DDL we did for `TaskRun`). Required for delete events to carry full column values. - Cross-form authorization on the `GET /api/v1/sessions/:session` loader (a JWT minted for either form authorizes both URL forms). Action routes are URL-form-specific, matching how the SDK mints PATs. ## Verification - Webapp typecheck clean (10/10). - `apps/webapp/test/sessionsReplicationService.test.ts` — round-trip tests for insert/update/delete through Postgres logical replication into ClickHouse via testcontainers. - Live end-to-end against local dev: create + retrieve (both forms) + update + close, `.out.initialize` + `.out.append` x2 + `.in.send` + `.out.subscribe` over SSE, list with all filter combinations + pagination, `end-and-continue` swap, `X-Peek-Settled` fast-close (verified in browser via reconnect-on-reload and via curl). Replicated row lands in ClickHouse within ~1s. - Multi-round Devin + CodeRabbit review feedback addressed (read-after-write paths use `prisma` writer, info-leak on auth-routes masked as 403, peek-settled discriminator parsing fix, etc.). ## Test plan - [ ] `pnpm run typecheck --filter webapp` - [ ] `pnpm run test --filter webapp ./test/sessionsReplicationService.test.ts --run` - [ ] Start the webapp with `SESSION_REPLICATION_CLICKHOUSE_URL` and `SESSION_REPLICATION_ENABLED=1`. Confirm the slot and publication auto-create on boot. - [ ] `POST /api/v1/sessions` and verify the row replicates to `trigger_dev.sessions_v1` within a couple of seconds. - [ ] `POST /api/v1/sessions/:id/close`, then confirm `POST /realtime/v1/sessions/:id/out/append` returns 400. - [ ] Reuse a closed session's `externalId` on `POST /api/v1/sessions` and confirm 409. - [ ] `GET /realtime/v1/sessions/:id/out` with `X-Peek-Settled: 1` after a turn completes and confirm `X-Session-Settled: true` response header + immediate close. |
||
|
Eric Allam
|
e134da7306 |
fix(run-engine): debounce hot-key lock contention and 5xx feedback loop (#3453)
## Changes
Three changes in
`internal-packages/run-engine/src/engine/systems/debounceSystem.ts`, in
order of impact:
1. **Fast-path skip before the lock.** In `handleExistingRun`, do an
unlocked read of `delayUntil` (and `createdAt` for the max-duration
check) from the run row before entering `runLock.lock("handleDebounce",
...)`. If `newDelayUntil <= currentDelayUntil` and the run is still
within its max-duration window, return the existing run immediately
without taking the lock. Safe because debounce is monotonic-forward only
— a stale read either matches reality or undershoots, both of which
decay correctly (re-checked properly inside the lock by whichever caller
is actually pushing forward). Trailing-mode triggers carrying
`updateData` still take the lock so the data update is applied.
2. **Quantize `newDelayUntil`.** Round the computed `newDelayUntil` to
1-second buckets (configurable via `quantizeNewDelayUntilMs`, set to 0
to disable). Without quantization, every call has a slightly larger
`newDelayUntil` than the last and they all pass the fast-path check.
With it, concurrent callers on the same key share a target time and ~95%
short-circuit. User-visible effect: a debounced run might fire up to 1s
earlier than the strict spec — non-issue for typical debounce use cases
(chat summarization, batched notifications, etc.).
3. **Graceful lock-contention fallback.** Wrap the `runLock.lock(...)`
call so `LockAcquisitionTimeoutError` and Redlock `ExecutionError` /
`ResourceLockedError` return the existing run id with success instead of
propagating a 5xx. Debounce is best-effort: if we can't take the lock,
the herd is already updating it for us; fall in line. This kills the 5xx
→ SDK-retry feedback loop. With (1)+(2) this rarely fires; without them
it's the difference between 5xx and 200.
Defaults preserve current behaviour aside from quantization (1s) and
fast-path (on). Both are configurable via `RunEngineOptions.debounce`.
## ✅ Checklist
- [x] I have followed every step in the [contributing
guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md)
- [x] The PR title follows the convention.
- [x] I ran and tested the code works
---
## Changelog
Reduce 5xx feedback loops on hot debounce keys by quantizing
`delayUntil`, adding an unlocked fast-path skip before the redlock, and
gracefully handling redlock contention in `handleDebounce` so the SDK no
longer retries into a herd.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
||
|
devin-ai-integration[bot]
|
4b28080ed4 |
feat: add isReplay to run context (#3454)
## Summary Adds `isReplay` boolean to the run context (`ctx.run.isReplay`), following the same pattern as the existing `isTest`. The value is derived from the existing `replayedFromTaskRunFriendlyId` database field, so no schema migration is needed. ## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Testing - Verified `@trigger.dev/core` builds successfully - Verified `webapp` typechecks successfully - All new fields use `default(false)` for backwards compatibility --- ## Changelog - Added `isReplay` to `TaskRun` and `V3TaskRun` schemas in `common.ts` - Added `RUN_IS_REPLAY` semantic attribute and wired it in `taskContext` - Propagated `isReplay` through the dequeue system, run attempt system, and all execution context construction paths (V1 + V2) - Added `isReplay` to `DequeuedMessage` and `TaskRunExecutionLazyAttemptPayload` schemas - Added patch changeset for `@trigger.dev/core` - Updated docs: added `isReplay` to context reference, added "Detecting replays" section to replaying page --- 💯 Link to Devin session: https://app.devin.ai/sessions/1d6f1b3cc39a4623b72d05bf00f2d70c --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: nick <55853254+nicktrn@users.noreply.github.com> |
||
|
nicktrn
|
91fd8a8a03 |
chore(security): close dependabot alerts q2 (#3456)
Closes ~80 dependabot alerts (3 critical, ~25 high, ~31 medium) by
bumping direct deps where possible and narrowly overriding the rest.
Cloud uses `resend` email transport and Node 20 - all bumps are safe for
both cloud and self-hosters.
## Direct upgrades
| Package | Where | From | To | Why |
|---|---|---|---|---|
| `vite` | root devDeps | ^5.4.21 | *(removed)* | dead pin; vitest pulls
vite transitively |
| `dompurify` | apps/webapp | ^3.2.6 | ^3.4.1 | XSS CVEs |
| `effect` | apps/webapp | ^3.11.7 | ^3.21.2 | AsyncLocalStorage CVE in
Effect fibers |
| `nodemailer` | internal-packages/emails | ^7.0.11 | ^8.0.6 | SMTP CRLF
injection (only affects self-hosters w/ smtp/aws-ses transport) |
| `uuid` | apps/webapp | ^9.0.0 | ^14.0.0 | buffer bounds check;
ESM-only but bundled by Remix |
| `uuid` + `@types/uuid` | packages/trigger-sdk | ^9.0.0 | *(removed)* |
dead deps, no usage |
| `@types/uuid` | apps/webapp | ^9.0.0 | *(removed)* | uuid 14 ships its
own types |
| `tar` | packages/cli-v3 | ^7.5.4 | ^7.5.13 | path traversal CVEs |
| `testcontainers` + `@testcontainers/postgresql` +
`@testcontainers/redis` | internal-packages/testcontainers | ^10.28.0 |
^11.14.0 | dev/test cleanup; one-line API fix for
`RedisContainer(image)` |
| `rimraf` | webapp + 6 packages | ^3.0.2 / ^5.0.7 | ^6.0.1 | dev/build
tool consolidation |
## Scoped overrides
All bound by both `>=` and `<` to avoid major-version yanks.
| Override | Closes |
|---|---|
| `tar@>=7 <7.5.11` → `^7.5.11` | supervisor's `@kubernetes/client-node
1.0.0` chain |
| `axios@>=1.0.0 <1.15.0` → `^1.15.0` | replaces older 1.9.0 pin |
| `systeminformation@>=5.0.0 <5.31.0` → `^5.31.0` | bumps existing
5.27.14 pin |
| `lodash@>=4.0.0 <4.18.0` → `^4.18.0` | bumps existing 4.17.23 pin |
| `lodash-es@>=4.0.0 <4.18.0` → `^4.18.0` | new (mirrors lodash) |
| `dompurify@>=3 <3.4.0` → `^3.4.1` | catches transitive dompurify via
mermaid |
| `vite@>=5.0.0 <6.4.2` → `^6.4.2` | path traversal; vite 5 has no patch
|
| `rollup@>=4 <4.59.0` → `^4.59.0` | path traversal in vite/vitest chain
|
| `flatted@>=3 <3.4.2` → `^3.4.2` | prototype pollution in eslint
flat-cache |
| `picomatch@>=2 <2.3.2` → `^2.3.2` | ReDoS in 2.x branch (transitive) |
| `picomatch@>=4 <4.0.4` → `^4.0.4` | ReDoS in 4.x branch
(vitest/tinyglobby) |
| `minimatch@>=3 <3.1.3` → `^3.1.3` | ReDoS in eslint 8 chain |
| `protobufjs@>=7 <7.5.5` → `^7.5.5` | **critical** RCE via
@opentelemetry/otlp-transformer |
| `fast-xml-parser@>=4 <4.5.5` → `^4.5.5` | DOCTYPE bypass + others (4.x
branch via aws-sdk in supervisor) |
| `fast-xml-parser@>=5 <5.7.0` → `^5.7.0` | **critical** + others (5.x
branch via aws-sdk in webapp) |
| `path-to-regexp@>=0.1 <0.1.13` → `^0.1.13` | ReDoS in express 4 /
@remix-run/express |
| `ajv@>=8 <8.18.0` → `^8.18.0` | DoS |
| `socket.io-parser@>=4 <4.2.6` → `^4.2.6` | DoS in @trigger.dev/core's
socket.io |
| `postcss@>=8 <8.5.10` → `^8.5.10` | XSS via stringify |
| `yaml@>=2 <2.8.3` → `^2.8.3` | DoS |
| `semver@>=5 <5.7.2` → `^5.7.2` | ReDoS in 5.x |
| `defu@>=6 <6.1.5` → `^6.1.5` | prototype pollution via __proto__ in
@prisma/config c12 chain |
## Dismissed (~47)
| Reason | Cluster | Count |
|---|---|---|
| `not_used` | langsmith + next 15.x in references/* | 10 |
| `not_used` | minimatch 8.x via prisma-generator-ts-enums
(references/prisma-6) | 3 |
| `not_used` | basic-ftp via puppeteer in references/hello-world +
references/seed | 2 |
| `not_used` | hono / @hono/node-server / express-rate-limit /
path-to-regexp 8.x / @modelcontextprotocol/sdk - all via mcp-sdk chain
(dormant in webapp; dev-only localhost in cli-v3) | 22 |
| `not_used` | fastify / @fastify/static / file-type via evalite devDep
| 5 |
| `tolerable_risk` | rollup 3 + minimatch 5/8/9/10 dev/build tooling |
13 |
## Notes
- **mcp-sdk chain**: `@vercel/sdk` in webapp imports `Vercel` API client
only; `mcp-server/*` subpath isn't loaded at runtime. cli-v3's MCP
server runs only via `trigger mcp` on developer machines. Bumping
`@modelcontextprotocol/sdk` to latest (1.29.0) wouldn't close these
alerts anyway - it ships hono ^4.11.4 which is still vulnerable - so
dismissal is the cleaner call.
- **References ignore list**: confirmed with current dependabot ignore
config; added `references/seed/package.json` (only gap).
- **undici** alerts (CVE-2026-1527, 4 alerts) will auto-close: lockfile
already at 6.25.0 > patched 6.24.0; just needs Dependabot rescan.
- **Effect 3.20 fix** is a runtime-only scheduler fix, no public API
changes - verified with research agent against our four `effect/*`
imports.
- **uuid 14** is ESM-only; we only call `validate`/`version` (no crypto
needed) so Node 20 requirement isn't load-bearing for us.
## Public packages (`packages/*`)
Minimal surface, deliberately. None of these change published runtime
behaviour - all changesets-worthy public package changes are deferred to
a regular release pass.
| Package | Change | Runtime impact |
|---|---|---|
| `packages/trigger-sdk` | Removed dead `uuid` dep (no source imports) |
None - dep was unused |
| `packages/cli-v3` | `tar` ^7.5.4 → ^7.5.13 | Patch bump within
already-allowed 7.x range; nothing CLI consumers see |
| `packages/core` / `packages/build` / `packages/python` /
`packages/rsc` / `packages/react-hooks` / `packages/schema-to-json` |
`rimraf` ^3.0.2 → ^6.0.1 in devDeps | Build-time only, no runtime change
|
No changeset added because nothing in these packages affects what
published consumers run.
## Validation
- Webapp typecheck (forced, no cache) passes after every commit
- Smoke-tested testcontainers v11 changes via real `postgresTest` +
`redisTest` (sync.test.ts, releaseConcurrency.test.ts) - both pass
- Webapp built + verified `require("uuid")` no longer in CJS server
output (now bundled inline)
- Test env webapp deployed at `dependabot-q2.rc0` (cloud#740) - no
issues observed
- Test suite run with package prerelease passed
|
||
|
Eric Allam
|
5693b62cfb |
fix(webapp): propagate abort signal through realtime proxy fetch (#3442)
## Summary
Fixes an RSS-only memory leak in the three realtime proxy routes
(`/realtime/v1/runs`, `/realtime/v1/runs/:id`,
`/realtime/v1/batches/:id`). Client disconnects during an in-flight
long-poll would leave the upstream fetch to Electric running with no way
to abort it, so undici kept the socket open and buffered response chunks
that would never be consumed.
## Root cause
All three routes flow through
`RealtimeClient.streamRun/streamRuns/streamBatch` → `#streamRunsWhere` →
`#performElectricRequest` → `longPollingFetch(url, { signal })`. The
chain was already signal-aware, but `#streamRunsWhere` hardcoded
`signal=undefined` when calling `#performElectricRequest`, so no signal
ever reached `longPollingFetch`.
When a downstream client aborts a long-poll mid-flight:
1. Express tears down the downstream response socket.
2. The `longPollingFetch` promise has already resolved (it returns as
soon as upstream headers arrive) and handed back `new
Response(upstream.body, {...})`.
3. `undici` keeps the upstream socket open and continues buffering
chunks into the `ReadableStream` that nothing will ever read from.
4. The upstream connection is eventually closed by Electric's own poll
timeout (~20s). During that window the per-request buffers stay in
native memory.
These buffers live below V8's accounting — no `heapUsed` or `external`
growth, no sign in heap snapshots, only RSS. An isolated standalone
reproducer (`fetch` against a slow-streaming upstream, discard the
`Response` before consuming its body) measures **~44 KB retained per
leaked request** after GC. That's consistent with the undici socket +
receive buffer + HTTP parser state for a long-lived chunked response.
The pattern is the shape documented in
[nodejs/undici#1108](https://github.com/nodejs/undici/issues/1108) and
[#2143](https://github.com/nodejs/undici/issues/2143).
## What changed
- **`realtimeClient.server.ts`** — add optional `signal` parameter to
`streamRun`, `streamRuns`, `streamBatch`, and the shared
`#streamRunsWhere`; thread it through to `#performElectricRequest`
instead of hardcoding `undefined`.
- **`realtime.v1.runs.$runId.ts`, `realtime.v1.runs.ts`,
`realtime.v1.batches.$batchId.ts`** — pass `getRequestAbortSignal()`
(from `httpAsyncStorage.server.ts`) at the call site. This is the signal
wired to `res.on('close')` and fires reliably on downstream disconnect.
- **`longPollingFetch.ts`** — belt-and-suspenders: cancel the upstream
body explicitly in the error path, and treat `AbortError` as a clean
`499` instead of a `500`. This both releases undici's buffers
deterministically on error and avoids spurious 500s in request logs when
a client legitimately walks away.
## Verification
Standalone reproducer: slow upstream server streams 32 KB chunks every
100 ms for 5 seconds per request. The proxy does `fetch(url)` with
varying signal/cancel strategies, creates `new Response(upstream.body,
...)`, and discards it without consuming the body (simulating the leak
path).
Results from 1 000 parallel fetches per variant, measured post-GC:
| variant | Δ heap | Δ external | Δ RSS |
| --- | --- | --- | --- |
| A. no signal, body never consumed (the bug) | +0.3 MB | 0 MB | **+59.4
MB** |
| B. signal propagated, aborted after headers (this fix) | −0.1 MB | 0
MB | +15.4 MB |
| C. no signal, explicit `res.body.cancel()` | 0 MB | 0 MB | −25.4 MB |
10-round sustained test of variant B to distinguish accumulating
retention from one-time allocator overhead:
```
round 1/10 Δ=+3.2 MB round 6/10 Δ=-12.5 MB
round 2/10 Δ=-7.6 MB round 7/10 Δ=-11.9 MB
round 3/10 Δ=-11.7 MB round 8/10 Δ=-2.6 MB
round 4/10 Δ=+3.2 MB round 9/10 Δ=-8.0 MB
round 5/10 Δ=-1.2 MB round 10/10 Δ=-12.6 MB
```
RSS oscillates in a 49-65 MB band with no upward trend — signal
propagation fully releases the buffers.
## Risk
- Behavior change only on aborted long-polls: the upstream fetch now
cancels promptly instead of running to its natural timeout. This saves
both memory and outbound traffic to Electric.
- `AbortError` now surfaces as `499` rather than `500`. Any dashboard or
alert that counts 500s in request logs will see slightly fewer of them;
this is the intended behavior.
- Signal-aware parameter is optional on
`RealtimeClient.streamRun/streamRuns/streamBatch`, so callers that don't
opt in get the previous behavior.
## Test plan
- [ ] Existing realtime integration tests pass
- [ ] Dashboard realtime views (runs list, batch details) continue
working normally across tab open/close cycles
- [ ] Under a burst of aborted long-polls, server RSS returns to
baseline rather than climbing
|
||
|
Matt Aitken
|
8aa1e55588 |
test: e2e auth baseline tests + webapp testcontainer infrastructure (#3438)
Adds a minimal end-to-end test harness that spawns the compiled webapp as a child process against a throwaway Postgres container, plus a baseline of 8 auth-behaviour tests. These tests will be used as a regression check before and after the upcoming apiBuilder RBAC migration to confirm auth behaviour is unchanged. ## What's included **`internal-packages/testcontainers/src/webapp.ts`** (new) Spawns `build/server.js` with a dynamically allocated port, polls `/healthcheck`, and exposes `WebappInstance` and `startTestServer()` (postgres container + webapp + PrismaClient in one call). Key details: - Uses `process.execPath` so the correct Node binary is found in forked test processes - Sets `NODE_PATH` to `node_modules/.pnpm/node_modules` so pnpm-hoisted transitive deps (e.g. `eventsource-parser`) resolve correctly inside the subprocess - Overrides both `PORT` and `REMIX_APP_PORT` so Vite's automatic `.env` loading doesn't override the dynamically allocated port **`internal-packages/testcontainers/package.json`** Adds `./webapp` sub-path export so tests can `import from "@internal/testcontainers/webapp"`. **`internal-packages/testcontainers/src/index.ts`** Exports `createPostgresContainer` (used internally by `webapp.ts`). **`apps/webapp/test/helpers/seedTestEnvironment.ts`** (new) Creates a minimal org → project → environment row set with random suffixes. **`apps/webapp/test/api-auth.e2e.test.ts`** (new) 8 tests across two suites: - API-key bearer: valid key (auth passes, 404), missing header (401), invalid key (401), error body shape - JWT bearer: valid JWT on JWT-enabled route (passes), valid JWT on non-JWT route (401), empty-scope JWT (403), wrong signing key (401) ## How to run ```bash # Build required first (one-time) pnpm run build --filter webapp cd apps/webapp && pnpm exec vitest run test/api-auth.e2e.test.ts ``` ## Test plan - [x] All 8 tests pass against the current webapp build - [x] Webapp healthcheck returns 200 on startup - [ ] CI passes --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Saadi Myftija
|
496ac78484 |
feat(supervisor): optional ndots override for runner pods (#3441)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
Adds `KUBERNETES_POD_DNS_NDOTS_OVERRIDE_ENABLED` flag (off by default) that overrides the cluster default and sets `dnsConfig.options.ndots` on runner pods (defaulting to 2, configurable via `KUBERNETES_POD_DNS_NDOTS`). Kubernetes defaults pods to `ndots: 5`, so any name with fewer than 5 dots, including typical external domains like `api.example.com`, is first walked through every entry in the cluster search list (`<ns>.svc.cluster.local`, `svc.cluster.local`, `cluster.local`) before being tried as-is, turning one resolution into 4+ CoreDNS queries (×2 with A+AAAA). Using a lower `ndots` value reduces DNS query amplification in the `cluster.local` zone. |
||
|
Eric Allam
|
ca399565ba |
feat(webapp): add per-worker Node.js heap metrics (#3437)
## Summary Adds direct V8 heap and process-memory gauges to the webapp's OpenTelemetry meter. The webapp already exports per-cluster-worker Node.js runtime metrics (event-loop lag / utilization, active handles, active requests, libuv threadpool size) via a custom meter under the `trigger.dev` scope. Heap and memory were missing; this PR adds them alongside, in the same observable-batch pattern. ## New gauges | Metric | Source | Unit | | --- | --- | --- | | `nodejs.memory.heap.used` | `process.memoryUsage().heapUsed` | bytes | | `nodejs.memory.heap.total` | `process.memoryUsage().heapTotal` | bytes | | `nodejs.memory.heap.limit` | `v8.getHeapStatistics().heap_size_limit` | bytes | | `nodejs.memory.external` | `process.memoryUsage().external` | bytes | | `nodejs.memory.array_buffers` | `process.memoryUsage().arrayBuffers` | bytes | | `nodejs.memory.rss` | `process.memoryUsage().rss` | bytes | Gated by the existing `INTERNAL_OTEL_NODEJS_METRICS_ENABLED` flag, same as the adjacent event-loop / handle gauges. Zero overhead when disabled. ## Why `@opentelemetry/host-metrics` publishes `process.memory.usage`, which is RSS only. RSS is the sum of V8 heap, external memory (Buffers, etc.), native code, and thread stacks. Without a direct heap metric it is not possible to size the V8 heap cap (`--max-old-space-size`) from metrics alone, because RSS overstates heap by the external + native footprint. A worker can have a 4 GB RSS with a 2.5 GB heap and 1.5 GB of buffers; the former constrains `--max-old-space-size`, the latter does not. `nodejs.memory.heap.limit` also surfaces the configured `--max-old-space-size` (read from `v8.getHeapStatistics().heap_size_limit`), so operators can see the current limit in the same dashboard as actual usage rather than cross-referencing container environment variables. ## Risk Minimal. Observable gauges are sampled at the configured metric-export interval. `v8.getHeapStatistics()` and `process.memoryUsage()` are each microsecond-level calls, and six gauges are added to the same batch callback that already reads ~20 other Node.js runtime values per sample. Same registration pattern as the existing event-loop metrics in the file. ## Test plan - [ ] Deploy and confirm the six new gauges appear at the configured exporter - [ ] In cluster mode, confirm per-worker granularity (one series per cluster worker, tagged by `process.executable.name` / `service.instance.id`) - [ ] Confirm `nodejs.memory.heap.limit` reports the configured `--max-old-space-size` value in bytes |
||
|
Iss
|
41434b536b |
feat(webapp): admin Back Office tab with org API rate limit editor (#3434)
## Summary - New **Back office** tab at `/admin`, per-org detail page at `/admin/back-office/orgs/:orgId` designed to host future per-org admin actions (project count, delete account, YC deals). - First action: edit an organization's API rate limit — tokenBucket override (refill rate, interval, max tokens), with a live plain-English preview (e.g. *"1,500 requests per minute · 750 request burst allowance"*). Writes are audit-logged via the server logger. - Cleanup: removed unused `v2?` / `v3?` columns from the admin orgs list (display only — Prisma select untouched). ## Test plan - [ ] Back office tab visible in admin nav and highlighted when on a sub-route - [ ] `/admin/orgs` shows a Back office "Open" link per row; no v2/v3 columns - [ ] Empty state at `/admin/back-office` links back to `/admin/orgs` - [ ] Detail page renders the effective rate limit in view mode; Edit reveals the form - [ ] Save writes `Organization.apiRateLimiterConfig`, returns to view mode, shows "Rate limit saved." banner - [ ] Invalid values surface inline field errors and keep edit mode - [ ] Non-admins hitting any new route are redirected to `/` - [ ] Server logs show `admin.backOffice.rateLimit` info line per mutation |
||
|
Eric Allam
|
486f49791d |
fix(webapp): eliminate SSE abort-signal memory leak (#3430)
## Summary Fixes a server-side memory leak in the webapp's SSE helper. Every aborted SSE connection (client tab close, navigation, timeout) was pinning its full request/response graph indefinitely on Node 20, so any long-running webapp process accumulated retained memory proportional to streaming-request churn. ## Root cause `apps/webapp/app/utils/sse.ts` combined four abort signals via `AbortSignal.any([requestAbortSignal, timeoutSignal, internalController.signal])`. The composite signal tracks its source signals in an internal `Set<WeakRef>` registered against a `FinalizationRegistry`; under sustained traffic those entries accumulate faster than they're cleaned up, pinning every source signal (and its listeners, and anything those listeners close over) until the parent signal itself is GC'd or aborts. This is a long-standing Node issue with multiple open reports: - [nodejs/node#54614](https://github.com/nodejs/node/issues/54614) — original report, still open. A [follow-up from ChainSafe](https://github.com/nodejs/node/issues/54614#issuecomment-4055656572) describes the exact same shape in a Lodestar production workload (req + timeout signals composed per request accumulating in long-running worker) and the same mitigation: drop `AbortSignal.any`, compose manually. - [nodejs/node#55351](https://github.com/nodejs/node/issues/55351) — mechanism confirmed by Node member @jasnell: *"the set of dependent signals known to the AbortSignal are kept in an internal Set using WeakRefs. The AbortSignals are being properly gc'd but the Set is never cleaned out of the WeakRefs making those leak."* Partially fixed by [PR #55354](https://github.com/nodejs/node/pull/55354), shipped in Node 22.12.0 — but only covers the tight-loop case, not long-lived parent signals. - [nodejs/node#57584](https://github.com/nodejs/node/issues/57584) — circular-dependency variant, still open. - [nodejs/node#62363](https://github.com/nodejs/node/issues/62363) — regression in Node 24/25 from an unrelated V8 change ("Don't pretenure WeakCells"). Different root cause, same symptom. A separate issue in `apps/webapp/app/entry.server.tsx` — `setTimeout(abort, ABORT_DELAY)` with no `clearTimeout` on success paths — kept the React render tree + `remixContext` alive for 30s per successful HTML request. Same pattern fixed upstream in React Router templates ([react-router#14200](https://github.com/remix-run/react-router/pull/14200)), never backported to Remix v2. ## What changed - **`apps/webapp/app/utils/sse.ts`** — single-signal abort chain. `AbortSignal.any` removed; `AbortSignal.timeout` replaced by a plain `setTimeout` cleared when the controller aborts; named sentinel constants used as stackless abort reasons; request-abort handler explicitly removed on cleanup. - **`apps/webapp/app/entry.server.tsx`** — clears the `setTimeout(abort, ABORT_DELAY)` timer in `onShellReady` / `onAllReady` / `onShellError`. - **`apps/webapp/app/v3/tracer.server.ts` + `env.server.ts`** — gates OpenTelemetry `HttpInstrumentation` and `ExpressInstrumentation` behind `DISABLE_HTTP_INSTRUMENTATION=true` as an escape hatch for future OTel-listener retention patterns. Defaults to enabled. - **`apps/webapp/app/presenters/v3/RunStreamPresenter.server.ts`** — uses the shared `ABORT_REASON_SEND_ERROR` sentinel. ## Verification ### Full-app reproduction (memlab) Isolated local harness, 500 abrupt SSE disconnects against a dev-presence route, GC between passes, heap snapshot diff with [memlab](https://facebook.github.io/memlab/): | Run | Heap delta after 500 conns + GC | memlab retained leaks | | --- | --- | --- | | Before | +16.0 MB (linear with request count) | 158 clusters; 250 `ServerResponse`, 1000 `AbortController`, 250 `SpanImpl` retained | | After | **+3.3 MB (noise)** | **0 app-code leaks** | ### Standalone mechanism isolation To confirm *which* axis of the change is load-bearing, a separate standalone Node script (`/tmp/abort-leak-test.mjs`) ran 2000 requests × 200 KB payload per variant: | Variant | Heap delta after GC | | --- | --- | | baseline (no signal machinery) | 0 MB | | V1: `AbortSignal.any` + string abort reason | **+9.1 MB** | | V2: `AbortSignal.any` only (no reason) | **+10.8 MB** | | V3: string reason only (no `AbortSignal.any`) | 0 MB | | V4: neither (the fix) | 0 MB | | V5: `AbortSignal.any` with no listener on the composite | **+10.2 MB** | This proves `AbortSignal.any` is the sole mechanism. The reason type (`.abort()` vs `.abort("string")`) is irrelevant for retention — V3 is clean, V5 leaks even without a listener on the composite. ## Risk - `sse.ts` is used by the dev-presence routes. Behaviour is equivalent — timeouts and client disconnects still abort the stream. `signal.reason` is now a named string sentinel (`"timeout"`, `"request_aborted"`, etc.) instead of the previous string arg or default `AbortError`. No in-tree reader of `signal.reason` exists. - `entry.server.tsx` change is a standard cleanup of an abort timer, matches upstream React Router guidance. - `tracer.server.ts` change is env-gated and defaults to current behaviour. - Three other webapp `AbortSignal.timeout()` callsites (alert delivery, remote-build status) are fire-and-forget passed directly to `fetch` — not composed with anything long-lived, no retention risk, untouched. ## Test plan - [ ] Existing SSE integration tests pass - [ ] Dev-presence SSE behaves normally across tab open/close cycles - [ ] No heap growth under sustained aborted-connection traffic (heap snapshot diff) ## Follow-up The same `AbortSignal.any([userSignal, internalSignal])` pattern exists in several SDK/core callsites that ship to customers (`packages/core/src/v3/realtimeStreams/manager.ts`, `packages/trigger-sdk/src/v3/{ai,chat,chat-client,sessions}.ts`, `packages/core/src/v3/workers/warmStartClient.ts`). Whether those leak in practice depends on the user passing a long-lived signal. Tracked separately. |
||
|
Matt Aitken
|
fc71e7dd75 |
fix: handle fast-completion race in batch streaming seal check (#3427)
## Problem When `batchTrigger()` is called with large payloads, each item's payload is uploaded to R2 server-side during the streaming loop before being enqueued. This makes the loop slow — around 3 seconds per item. Workers pick up and execute each item as it's enqueued, running concurrently with the ongoing stream. For the last item in the batch, a race exists between the streaming loop finishing and the batch completion cleanup: 1. The loop enqueues the last item and returns from `enqueueBatchItem()` 2. A waiting worker picks up the item almost instantly and executes it 3. `recordSuccess()` fires, `processedCount` hits the expected total, `finalizeBatch()` runs 4. `cleanup()` deletes all Redis keys for the batch, including `enqueuedItemsKey` 5. The streaming loop exits and calls `getBatchEnqueuedCount()` — reads the now-deleted key — returns 0 The count check finds `enqueuedCount (0) !== batch.runCount`, falls through to a Postgres fallback, but the fallback only checked `sealed`. The BatchQueue completion path sets `status = COMPLETED` in Postgres without setting `sealed = true` (that's the streaming endpoint's job), so the fallback misses it too. This causes the endpoint to return `sealed: false`. The SDK treats this as retryable and retries up to 5 times with exponential backoff. Each retry calls `enqueueBatchItem()`, which reads the batch meta key from Redis — also deleted by `cleanup()` — and throws "Batch not found or not initialized" (500). The final retry gets a 422 because the batch is already COMPLETED, which the SDK does not retry, causing an `ApiError` to be thrown from `await batchTrigger()` in the parent run — even though all child runs completed successfully. ## Fix In the Postgres fallback inside `StreamBatchItemsService`, also check `status === "COMPLETED"` alongside `sealed`. This covers the fast-completion path where the BatchQueue finishes all runs before the streaming endpoint gets to seal the batch normally. Also switches `findUnique` to `findFirst` per webapp convention. --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Oskar Otwinowski
|
8eb596f3fe | fix(vercel): Fix vercel settings page (#3424) | ||
|
Eric Allam
|
2d3b2e82e6 |
feat(run-engine): flag to route getSnapshotsSince through read replica (#3423)
## Summary Adds `RUN_ENGINE_READ_REPLICA_SNAPSHOTS_SINCE_ENABLED` (default `"0"`). When enabled, the Prisma reads inside `RunEngine.getSnapshotsSince` run against the read-only replica client instead of the primary. Offloads the snapshot-polling queries fired by every running task runner off the writer. ## Why `getSnapshotsSince` is called from the managed runner's fetch-and-process loop (once per poll interval, plus on every snapshot-change notification). It runs four sequential reads per call — one `findFirst` by snapshot id, one `findMany` on snapshots with `createdAt > X`, one raw SQL against `_completedWaitpoints`, and chunked `findMany` on `waitpoint`. Per concurrent run, every few seconds. It's read-only, tolerates a small amount of staleness, and is an obvious candidate for the replica. ## Replica-lag considerations - **Step 1 "since snapshot not found"**: if the runner just received a snapshot id from the primary and asks the replica before it replicates, the function throws and the caller treats the response as an error (runner falls back to a metadata refresh). Self-correcting, not silent. - **Step 2 missing newly-created snapshots**: the next poll's `createdAt > sinceSnapshot.createdAt` filter still picks them up once the replica catches up. - **Waitpoint junction race**: the riskiest path — if a latest snapshot is replicated but its `_completedWaitpoints` join rows aren't yet, the runner could advance past that snapshot with `completedWaitpoints: []`. WAL/storage-level replication replays commits in order, so in practice both should appear atomically on the reader, but the race window is why the flag ships disabled. Aurora reader shrinks all three windows to single-digit ms in typical conditions, and its storage-level replication gives atomic visibility of committed transactions on the reader. ## Test plan - [ ] Flip the flag on in a non-prod environment, confirm snapshot polling behaves normally and `getSnapshotsSince` errors in Sentry stay flat. - [ ] Verify writer query volume drops and reader query volume rises on the snapshot-polling queries. - [ ] Keep an eye on `AuroraReplicaLag` (or equivalent) during rollout. |
||
|
Eric Allam
|
7c95ee498e |
feat(webapp): tag Prisma spans with db.datasource attribute (#3422)
## Summary
Stamp every Prisma span with `db.datasource: "writer" | "replica"` so
traces can distinguish which client the query went through.
Both `PrismaClient` instances share the same global
`@prisma/instrumentation`, so their spans come out with identical names
and attributes today. This makes them trivially filterable.
## How
Two pieces in `apps/webapp/app/`:
1. **`v3/tracer.server.ts`** — a `DatasourceAttributeSpanProcessor`
reads an OTel context key in `onStart` and calls
`span.setAttribute("db.datasource", value)`. Registered as the first
span processor.
2. **`db.server.ts`** — `tagDatasource(datasource, client)` wraps each
`PrismaClient` with `$extends({ query: { $allOperations } })`. The
middleware sets the context key around the query and directly tags the
active span (to catch `prisma:client:operation`, which Prisma creates
before the middleware fires).
### Context-propagation gotcha
`PrismaPromise` is lazy — `query(args)` returns a thenable that only
starts when someone `.then()`s it. The naive `context.with(ctx, () =>
query(args))` restores ALS synchronously, so when Prisma's internal code
awaits the thenable later, the engine spans fire with the original ALS.
Wrapping as `async () => await query(args)` forces the `.then()` inside
the `context.with` callback, so ALS stays on our context for the engine
spans.
### Coverage
- **Tagged**: all `prisma:engine:*` (`connection`, `db_query`,
`serialize`, `query`, etc.), `prisma:client:operation`,
`prisma:client:serialize`, `prisma:client:connect`
- **Not tagged**: `prisma:client:load_engine` — one-time startup, fires
before any query
Concurrent `Promise.all([writer.x, replica.y])` correctly tags each pool
separately (ALS isolates per-Promise chain).
### Performance
One `context.with` (~200ns) and one `setAttribute` per span (effectively
free per OTel JS benchmarks) per Prisma op. Negligible against a query
path measured in milliseconds.
## Test plan
- [ ] Verify `db.datasource` appears on `prisma:engine:connection` spans
after the webapp is restarted
- [ ] Spot-check a handful of real traces carry the attribute
|
||
|
nicktrn
|
b570586899 |
fix(webapp): allow cancelling runs in DEQUEUED status from the runs list (#3421)
The cancel button was missing from the runs list for runs in `DEQUEUED` status. The runs list gates the button on `run.isCancellable`, which goes through `isCancellableRunStatus` -> `CANCELLABLE_RUN_STATUSES` = `NON_FINAL_RUN_STATUSES`. `DEQUEUED` was never added to that list when it was introduced in the run engine. The single run page uses a separate check (`!run.isFinished`, i.e. the inverse of `FINAL_RUN_STATUSES`), so cancellation already worked there - only the list was affected. Adding `DEQUEUED` to `NON_FINAL_RUN_STATUSES` also flips `isCrashableRunStatus` and `isFailableRunStatus`, but: - The crash path is the right behaviour - a `DEQUEUED` run (worker has claimed but not yet executing) can legitimately crash before `EXECUTING`, same as `PENDING`/`DELAYED` already do. - The fail path (`failedTaskRun.server.ts`) is only reached from V1 code paths (marqs consumers, v1 heartbeat handler). `DEQUEUED` is a V2-engine-only status, so V1 consumers never see it. When cancelling a `DEQUEUED` run the execution snapshot goes to `PENDING_CANCEL` (worker must ack) but `TaskRun.status` flips to `CANCELED` immediately - the UI reflects cancellation without waiting for the worker. Added an integration test in `run-engine/src/engine/tests/cancelling.test.ts` covering the full trigger -> dequeue -> cancel -> worker-ack flow. ## Stall safety The stall recovery path (PENDING_EXECUTING heartbeat miss -> nack-and-requeue -> back to QUEUED) lives entirely inside `@internal/run-engine` and never touches the webapp's `taskStatus.ts` helpers - the engine has zero imports from `~/v3/taskStatus` and doesn't know `CrashTaskRunService` / `FailedTaskRunService` exist. A stalled DEQUEUED run still goes back to the queue for retry; this change cannot cause stalls to crash or fail. The only realistic impact is the intended UI fix - the theoretical V1 crash/fail branches for DEQUEUED are unreachable in practice because V1 runs never have DEQUEUED status. |
||
|
Eric Allam
|
03e4d5fe31 |
feat(webapp,database): API key rotation grace period (#3420)
## Summary
Regenerating a RuntimeEnvironment API key no longer immediately
invalidates the previous one. Rotation is now overlap-based: the old key
keeps working for 24 hours so customers can roll it out in their env
vars without downtime, then stops working.
## Design
- **New `RevokedApiKey` table** (one row per revocation). Holds the
archived `apiKey`, a FK to the env, an `expiresAt`, and a `createdAt`.
Indexed on `apiKey` (high-cardinality equality — single-row hits) and on
`runtimeEnvironmentId`.
- **`regenerateApiKey` wraps both writes in a single `$transaction`:**
insert a `RevokedApiKey` with `expiresAt = now + 24h`, update the env
with the new `apiKey`/`pkApiKey`.
- **`findEnvironmentByApiKey` does a two-step lookup:** primary
unique-index hit on `RuntimeEnvironment.apiKey` first; on miss,
`RevokedApiKey.findFirst({ apiKey, expiresAt: { gt: now } })` with an
`include: { runtimeEnvironment }`. Two-step (not `OR`-join) keeps the
hot path identical to today and puts the fallback cost only on invalid
keys. Both lookups use `$replica`.
- **Admin endpoint** `POST /admin/api/v1/revoked-api-keys/:id` accepts
`{ expiresAt }` and updates the row. Setting to `now` ends the grace
window immediately; setting to the future extends it.
- **Modal copy** on the regenerate dialog updated — previously warned of
downtime, now explains the 24h overlap.
## Why a separate table instead of columns on `RuntimeEnvironment`
- Keeps the hot auth path's primary lookup unchanged — no
OR/nullable-apiKey semantics to reason about.
- Naturally supports multiple in-flight grace windows (regenerate twice
in a day → two old keys valid until their independent expiries).
- FK + cascade cleans up correctly when an env is deleted; nothing to
backfill.
## Test plan
Verified locally against hello-world with dev and prod env keys:
- [x] baseline — current key authenticates (`GET /api/v1/runs`) → `200`
- [x] regenerate via UI — DB shows old key in `RevokedApiKey` with
`expiresAt ≈ now+24h`, env has new key
- [x] grace window — both old and new keys → `200`; bogus key → `401`
- [x] admin endpoint: `expiresAt = now` → old key `401`
- [x] admin endpoint: `expiresAt = +1h` (after early-expire) → old key
`200` again
- [x] admin endpoint: `expiresAt = past` → old key `401`
- [x] admin 400 (invalid body), 404 (unknown id), 401 (missing/non-admin
PAT)
- [x] same flow exercised end-to-end on a PROD-typed env — behavior
identical
- [x] `pnpm run typecheck --filter webapp` passes
|
||
|
Eric Allam
|
881288c615 |
feat(webapp): deprecate v3 CLI deploys server-side (#3415)
## ✅ Checklist - [x] I have followed every step in the [contributing guide](https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md) - [x] The PR title follows the convention. - [x] I ran and tested the code works --- ## Summary Adds a server-side gate that detects deploy attempts from v3 CLI versions (i.e. `trigger.dev@3.x`) at the `POST /api/v1/deployments` entry point and, when enabled, rejects them with a clear upgrade message. v4 CLI deploys are completely unaffected. The last 3.x CLI release was `3.3.7`, which we can't update. This approach short-circuits the deploy before any DB writes, image-ref generation, S2 stream creation, or queue enqueue — no side effects in either mode. ## How v3 vs v4 are distinguished I pulled the published CLI tarballs for `trigger.dev@3.3.7`, `4.0.0`, `4.0.1`, `4.0.5`, `4.1.0`, `4.2.0`, and the current `4.4.4` in the repo. The cleanest, most reliable signal is the request body to `POST /api/v1/deployments`: | Field on initialize | v3.3.7 CLI | v4.x CLI | |---|---|---| | `type` | **never sent** | always sent — `"MANAGED"` (run_engine_v2) or `"V1"` | | `isNativeBuild` / `gitMeta` / `triggeredVia` / `runtime` | not sent | sent | | `registryHost` / `namespace` | sent (v3-only; stripped by current Zod schema) | not sent | Every v4 call site I inspected sets `type: features.run_engine_v2 ? "MANAGED" : "V1"` unconditionally. `payload.type` is `undefined` if and only if the client is a 3.x CLI. ## Behavior - Detection always runs and emits `logger.warn("Detected deploy from deprecated v3 CLI", { environmentId, projectId, organizationId, enforced })`, which lets us watch how many v3 deploys are still happening before enforcement is flipped. - Enforcement is gated behind `DEPRECATE_V3_CLI_DEPLOYS_ENABLED` (default `"0"`, off). When `"1"`, the server returns `400` with: > The trigger.dev CLI v3 is no longer supported for deployments. Please upgrade your project to v4: https://trigger.dev/docs/migrating-from-v3 The v3 CLI surfaces this verbatim as `Failed to start deployment: <message>` because `zodfetch` throws `ApiError` for non-retryable 4xx (400/422) and `deploy.js` in 3.3.7 prints `error.message`. ## Out of scope (intentionally) - `api.v1.deployments.$deploymentId.finalize.ts` / `FinalizeDeploymentService` / `createDeploymentBackgroundWorkerV3.server.ts` are V1-engine paths, not the v3 CLI gate. Leaving them alone per review. - Container-side `createDeploymentBackgroundWorker` call in `managed-index-controller.ts` is still used by v4's in-image indexer. Not touched. - v3 `trigger dev` flow (different code path) — separate deprecation if/when needed. ## Testing - Ran `pnpm run typecheck --filter webapp` locally — passes. - Verified v4 tarballs (4.0.0, 4.0.1, 4.0.5, 4.1.0, 4.2.0, 4.4.4) all include `type:` in the `initializeDeployment` call site, so none will be accidentally blocked. - Verified v3.3.7 tarball's `initializeDeployment` payload has no `type` field. Rollout plan after merge: 1. Deploy with `DEPRECATE_V3_CLI_DEPLOYS_ENABLED` unset → watch `Detected deploy from deprecated v3 CLI` log volume. 2. When comfortable, set `DEPRECATE_V3_CLI_DEPLOYS_ENABLED=1` to enforce. --- ## Changelog Detect v3 CLI deploys on `/api/v1/deployments` and, when `DEPRECATE_V3_CLI_DEPLOYS_ENABLED=1`, reject them with an upgrade message pointing at https://trigger.dev/docs/migrating-from-v3. v4 CLI deploys are unaffected. Link to Devin session: https://app.devin.ai/sessions/b242c11bd86e4099aeec8b59bab62143 Requested by: @ericallam |
||
|
Matt Aitken
|
6e6deb41e1 |
Admin endpoint to set concurrency burst factor (#3412)
Example cURL call using an admin user PAT (replace with a real one): ```sh curl -X PUT https://cloud.trigger.dev/admin/api/v1/environments/<environmentId>/burst-factor \ -H "Authorization: Bearer tr_pat_1234" \ -H "Content-Type: application/json" \ -d '{"burstFactor": 1.5}' ``` |
||
|
nicktrn
|
9a988ab885 |
chore(webapp): clarify admin feature flags are global (#3408)
global flags are global. |
||
|
nicktrn
|
581db83f64 |
feat(webapp): highlight microVM regions on the regions page (#3407)
Adds a `MicroVM` badge next to the region name on the regions page. Uses the existing `small` badge variant for visual consistency with the `Default` badge already on this page. |
||
|
Matt Aitken
|
45ba398c80 |
Error page graph: for a time bucket don't fill zeros for a version with no errors (#3402)
This caused performance issues with large numbers of versions, and bad UX when hovering the graph (showing irrelevant versions) |
||
|
Eric Allam
|
9636e43567 |
fix(webapp): reduce error-level log noise for handled/benign cases (#3403)
Two changes to cut error volume from logs that represent handled
conditions, not real errors (combined ~1600/hr in prod):
1. api.v1.waitpoints.tokens.$waitpointFriendlyId.complete.ts
The route throws `json(..., { status: 404 })` when a waitpoint
isn't found, but the generic catch block caught that Response,
logged it as an error (with an empty {} body because Error fields
are non-enumerable), and rethrew as a 500 — so clients saw a 500
instead of the intended 404, and every stale-waitpoint request
produced a Sentry event.
Fix: re-throw Response objects unchanged so the correct status
propagates and we don't log user 404s as errors. Also serialize
remaining Error instances explicitly (name/message/stack) so the
logs are actionable when we do hit a real error.
2. v3/marqs/sharedQueueConsumer.server.ts:603
"Task run has invalid status for execution. Going to ack" — the
message itself says we're handling it gracefully. Benign race
between dequeue and completion/cancellation. Demote to warn.
|
||
|
Eric Allam
|
67d2025f33 |
feat(webapp): add 60s/60s SWR cache to getEntitlement (#3388)
Wraps getEntitlement in platform.v3.server.ts with the existing
platformCache (LRU memory + Redis) under a new `entitlement` namespace.
Eliminates a synchronous billing-service HTTP round trip on every
trigger.
Cache config: 60s fresh / 60s stale SWR. Cache key is the
organization id. Errors are caught inside the loader and return the
existing permissive { hasAccess: true } fallback, which is also
cached to prevent thundering-herd on billing outages.
Trade-off: plan upgrade/downgrade is now visible after up to ~120s
worst-case (60s fresh + 60s stale revalidation). Acceptable since
the existing limits and usage namespaces use 5min/10min, and the
defensive hasAccess: true fallback already exists.
|
||
|
Eric Allam
|
79b6053e13 |
feat(server): add TaskIdentifier registry to replace expensive distinct query (#3368)
Replace the expensive DISTINCT query for task filter dropdowns with a dedicated TaskIdentifier registry table backed by Redis. Environments migrate automatically on their next deploy, with a transparent fallback to the legacy query for unmigrated environments. Also fixes duplicate dropdown entries when a task changes trigger source, and adds active/archived grouping for removed tasks. Moves BackgroundWorkerTask reads in the trigger hot path to the read replica. |
||
|
Eric Allam
|
94abe97132 | fix(webapp): prevent dashboard crash when span accessory text is not a string (#3400) | ||
|
Eric Allam
|
02d2334c8a |
fix(webapp): fix Redis connection leak in realtime streams and broken abort signal propagation (#3399)
Pool Redis connections for non-blocking ops (ingestData, appendPart,
getLastChunkIndex)
using a shared singleton instead of new Redis() per request. Use
redis.disconnect()
for immediate teardown in streamResponse cleanup. Add 15s inactivity
timeout fallback.
Fix broken request.signal in Remix/Express by wiring Express
res.on('close') to an
AbortController via httpAsyncStorage. All SSE/streaming routes now use
getRequestAbortSignal() which fires reliably on client disconnect,
bypassing the
Node.js undici GC bug (nodejs/node#55428) that severs the signal chain.
|
||
|
nicktrn
|
93f2ca6bf4 |
feat(webapp): extend admin workers endpoint and unify admin api auth (#3390)
Extends the admin worker groups endpoint with a GET loader and more fields on POST (type, hidden, workloadType, cloudProvider, location, staticIPs, enableFastPath), and pulls the PAT + admin check that was inlined or locally duplicated across every admin.api route into a shared helper in personalAccessToken.server.ts. The generic authenticateAdminRequest returns a discriminated result; requireAdminApiRequest is the thin Remix loader/action wrapper that throws. The neverthrow-style route (platform-notifications.ts) now composes the generic helper instead of duplicating the check. Verified locally against GET (listing) and POST (new fields, invalid enum, minimal backwards-compat). |
||
|
devin-ai-integration[bot]
|
7d82041809 |
fix(security): upgrade Remix packages 2.1.0 → 2.17.4 (#3372)
## Summary
Upgrades all `@remix-run/*` packages in `apps/webapp` from **2.1.0 →
2.17.4** to address security vulnerabilities. Recreation of #2951 on a
fresh checkout of `main`.
**Updated packages (`apps/webapp/package.json`):**
- `@remix-run/express`, `@remix-run/node`, `@remix-run/react`,
`@remix-run/serve`, `@remix-run/server-runtime`: 2.1.0 → 2.17.4
- `@remix-run/router`: ^1.15.3 → ^1.23.2
- `@remix-run/dev`, `@remix-run/eslint-config`, `@remix-run/testing`:
2.1.0 → 2.17.4
**Root `package.json` overrides:**
- `@remix-run/dev@2.17.4>tar-fs`: 2.1.3 → 2.1.4
- `testcontainers@10.28.0>tar-fs`: 3.0.9 → 3.1.1
**Documentation:** Updated Remix version references in `CLAUDE.md`,
`apps/webapp/CLAUDE.md`, and `.cursor/rules/webapp.mdc`.
**Server changes:** Added `.server-changes/upgrade-remix-security.md`
for release tracking per `CONTRIBUTING.md`.
No application code changes — only `package.json` files, documentation,
a server-changes entry, and the regenerated `pnpm-lock.yaml`.
### Updates since last revision
Addressed all 3 Devin Review findings:
1. **Missing `.server-changes/` file** — added
`.server-changes/upgrade-remix-security.md` (commit
|
||
|
Oskar Otwinowski
|
097fab0d8c |
feat(webapp): Vercel integration - disable auto promotions (#3376)
<img width="1044" height="709" alt="image" src="https://github.com/user-attachments/assets/7f2cf25c-3b74-46a9-8794-41be077e04bf" /> |
||
|
Eric Allam
|
ed0c3e4f38 |
fix: stop creating TaskRunTag records and join table entries during triggering (#3369)
The TaskRun.runTags string array already stores tag names, making the TaskRunTag M2M relation redundant write overhead. Remove createTags calls, connect: tags, and join table writes from both V1 and V2 trigger paths. Simplify the add-tags API to just push to runTags directly. |
||
|
Eric Allam
|
417ab876e3 | fix(batch-queue): Batch items that hit the environment queue size limit now fast-fail (#3352) | ||
|
Oskar Otwinowski
|
3c9647cb8c |
feat(webapp): Platform notifications admin imporovements (#3324)
- bugfix to show the changelog to the target audience - more functionality for admins, to edit, delete and archive notifications |
||
|
nicktrn
|
e59614a31c |
feat(webapp): gate microvm regions behind compute access feature flag (#3366)
Adds region-level gating so MICROVM regions are only visible and usable by orgs with the `hasComputeAccess` feature flag. Admins and explicit allowlist behavior unchanged. - New shared helper (`regionAccess.server.ts`) with `resolveComputeAccess`, `defaultVisibilityFilter`, and `isComputeRegionAccessible` - `RegionsPresenter` filters out MICROVM regions for non-compute orgs - `SetDefaultRegionService` blocks setting a MICROVM region as default without compute access - `WorkerGroupService` blocks triggering runs in MICROVM regions without compute access - `computeTemplateCreation` refactored to use shared `resolveComputeAccess` - Updated snapshot callback schema |
||
|
nicktrn
|
bd41bb2cbd |
feat(webapp): set application_name on prisma connections (#3348)
Sets `application_name` on the Prisma writer and replica connection strings using the existing `SERVICE_NAME` env var, so DB load can be attributed by service. |
||
|
Matt Aitken
|
def21b26b6 |
fix(batch): retry R2 upload on transient failure in BatchPayloadProcessor (#3331)
⚒️ Publish Worker (v4) / build (supervisor) (push) Has been cancelled
A single "fetch failed" from the object store was aborting the entire batch stream with no retry. Added p-retry (3 attempts, 500ms-2s backoff) around ploadPacketToObjectStore so transient network errors self-heal server-side instead of propagating to the SDK. |
||
|
James Ritchie
|
4f2ff3d9de |
fix(wabapp): Fix for wrapping text on run inspector (#3328)
### Text wrapping fix - Fixes message text not wrapping on the run inspector if there were no spaces in the text - Fixes inspector title truncation - Adds a copy text button for the Message property <img width="468" height="740" alt="CleanShot 2026-04-04 at 10 19 02@2x" src="https://github.com/user-attachments/assets/71e42bf3-d103-44a2-b3b4-937c0b60a4bc" /> |
||
|
James Ritchie
|
cf0fdde3c4 |
Feat(webapp): animated resizable panel (#3319)
This is a small improvement mainly with the UI Skills file: - Animate open and close the Resizable panels - Uses the built in animation hooks from react-window-splitter - Includes a global variable for the animation easing and timing for consistency https://github.com/user-attachments/assets/50ed0019-ed12-4e08-b95c-7c6d1fe5bac0 |
||
|
James Ritchie
|
e31b03eac2 |
fix(webapp): Responsive improvements for the onboarding screens (#3318)
Simple responsive breakpoint changes to the 3 onboarding screens + the login screen to make it mobile friendlier <img width="523" height="872" alt="CleanShot 2026-04-02 at 17 18 55" src="https://github.com/user-attachments/assets/815d19ca-df9b-4b3e-8f6d-e00c81628679" /> |
||
|
Eric Allam
|
f1f1d02f1d | feat(dashboard): a few tweaks to the AI models page (#3315) |